iH8dZddlZddlmZddlmcmZddlmZddl Z ddl m Z ddl m Z ddlZGddej ZGddej ZGdd ej Zdd Zd ZGd d ej ZdZedk(reyy)uK Singularity Memory ODE (SM-ODE) for CIFAR-10 — FIXED VERSION Based on the theory: Memory stored in shared singularity, not lost between states. Bugs fixed: 1. Batch size mismatch in compute_memory_bias (torch.cat crash) 2. Channel vs memory_dim mismatch in SharedSingularityBlock (blocks 1 & 3 crash) 3. reset_singularity() never called — singularity grows unboundedly 4. Singularity updated during evaluation — test data contamination 5. Registered buffer replaced by plain tensor — breaks state_dict 6. Baseline trained with misleading singularity-named function N) DataLoadercteZdZdZd fd ZedZejdZfdZdZ dZ dZ xZ S) SingularityMemoryaO Stores historical states in a 'singularity' that influences current predictions. Theory: History is not lost between states - it's stored in a singularity field. This allows 'variable determinism' where predictions are constrained by laws (standard weights) but influenced by accumulated memory (novel approach). ct|||_||_||_d|_t j|||_t jt j||z|t j|_ t jt j||t j|_ ||k7rt j|||_yt j|_yN)super__init__ feature_dim memory_dimdecay _singularitynnLinearto_singularity SequentialSigmoid memory_gateTanh kernel_netmemory_to_featureIdentity)selfr r r __class__s ex03.pyr zSingularityMemory.__init__%s &$ !!ii Z@== IIj;. ; JJL  -- IIj* - GGI   $%'YYz;%GD "%'[[]D "c|jSrr )rs r singularityzSingularityMemory.singularityFs   rc||_yrr)rvalues rrzSingularityMemory.singularityJs !rclt|||j||j|_|S)z;Propagate device / dtype changes to the singularity tensor.)r_applyr )rfnrs rr"zSingularityMemory._applyNs3 r    ( "4#4#4 5D  rct|jj}t|jj}t j ||j |||_y)z8Reset singularity for a given batch size. Device-aware.)devicedtypeN)next parametersr%r&torchzerosr r )r batch_sizer%r&s rreset_singularityz#SingularityMemory.reset_singularityUsOdoo'(//T__&'--!KK DOO/5UDrc|jd}|j|jjd|k7r|j||jj}|j |}|d|zz}t j ||gd}|j|}|t j|zdz}|j|} || z} | S)z| Compute how past states influence current prediction. B(S, y) = integral of K(t-tau) * y(tau) dtau r皙?dimg333333?) sizerr,detachrr)catrtanhr) rcurrent_featuresr+r kernel_output memory_bias gate_inputgatememory_contribmemory_contrib_projbiased_featuress rcompute_memory_biasz%SingularityMemory.compute_memory_bias\s &**1-    #t'7'7'<'BG  + ; 77#=#44^D*-@@rc|jsytj5|j|}|j|j z|j z|_dddy#1swYyxYw)a1 Update singularity memory with new states. S(t) = S(t-1) * decay + new_features FIX #4: skip update during evaluation to prevent test-data contamination. Keep the update out of autograd so the persistent state never captures graph history across batches. N)trainingr)no_gradrr r r3)r new_featuresnew_projs rupdate_singularityz$SingularityMemory.update_singularity|sc}}  ]]_ S**<8H $ 1 1DJJ >AR RD  S S Ss AA,,A5)gffffff?) __name__ __module__ __qualname____doc__r propertyrsetterr"r,r>rD __classcell__rs@rrrsS3B!!""D@ Srrc*eZdZdZdfd ZdZxZS)SharedSingularityBlockz Multiple processing heads share a common singularity memory. Theory: Entangled particles share singularity memory (Phi_AB). Here, different feature channels share the same memory field. c Tt|||_t|||_t j tdDcgc]W}t jt j||ddt j|t jdYc}|_ t j|_t jt j|dz|dzt jt j|dz||_ycc}w)NpaddingTinplace)rr channelsrrr ModuleListrangerConv2d BatchNorm2dReLUheadsr memory_projrfusion)rrYr _rs rr zSharedSingularityBlock.__init__s   -XzB]] Qx $  MM (Ha;x(% $   ;;=mm IIhlHqL 1 GGI IIhlH -  $ sAD%c 0|j\}}}}|jDcgc] }|| }}tj|d}t j |dj dj d} |jj| } |jj| |j| } | jdjdjdd||} |jtj|j|d||zjd| gd} | jdjdjdd||} |d| zzdtj | zzScc}w)NrRr0r/g?r.)shaper_r)r4Fadaptive_avg_pool2dsqueezerr>rDr` unsqueezeexpandraviewmeanr5)rxr+rYhwhead head_outputs head_concatpooledr=memory_alignedmemory_expandedfuseds rforwardzSharedSingularityBlock.forwardsw%&WW" Ha-1JJ7DQ7 7ii !4 &&q!,44R8@@D**>>vF ++F3))/:(2226@@DKKBPRTUWXY EII   ZQU 3 8 8 < '  #--b188RAF3;uzz/'B!BBB38sF)rFrGrHrIr rvrLrMs@rrOrOs 4CrrOc0eZdZdZdfd ZdZdZxZS) SMODECIFAR10aF Singularity Memory ODE Network for CIFAR-10 Key innovations: 1. Memory singularity stores history, doesn't lose it 2. Multiple blocks share singularity memory (entanglement-like) 3. Predictions = f(laws) + alpha * memory_bias 4. Variable determinism: constrained by weights, influenced by memory c &t|tjtjddddtj dtj dtjddddtj dtj d|_td||_ td||_ td||_ tjtjdddd d tj dtj d|_ tjtjdddd d tj dtj d|_ td|d z|_tjtj d dtj tj"d tj d||_y) NrQ@rRrSTrUrwrErX)striderTig?)rr rrr\r]r^ input_convrOshared_singularity1shared_singularity2shared_singularity3 transition1 transition2rglobal_singularityrDropout classifier)r num_classesr rs rr zSMODECIFAR10.__init__sx -- IIaQ * NN2  GGD ! IIb"a + NN2  GGD !  $:"j#I #9#z#J #9#z#J == IIb#qA 6 NN3  GGD !  == IIc3!Q 7 NN3  GGD ! #4Ca"H-- IIi % GGI JJsO IIc; '  rc^|j|}|j|}|j|}|j|}|j |}|j |}t j|djdjd}|jj|}|jj|t j|djdjd}tj||gd}|j|S)NrRr/r0)r~rrrrrrerfrgrr>rDr)r4r)rrlrrmemory_featurescombineds rrvzSMODECIFAR10.forward s  OOA   $ $Q '   Q   $ $Q '   Q   $ $Q '&&q!,44R8@@D11EEfM 226:&&q!,44R8@@D99fo6B?x((rcr|jD]$}t|ts|j|&y)z6FIX #3: reset every singularity module in the network.N)modules isinstancerr,)rr+modules rreset_all_singularitiesz$SMODECIFAR10.reset_all_singularities)s0lln 5F&"34((4 5r) rw)rFrGrHrIr rvrrLrMs@rrzrzs( T)<5rrzctj}tj|j dd}tj j ||}d}g} t|D]} |jd} d} d} t|dr|j|j|D] \}}|j||j|}}|j||}|||}|jtjj j#|j d |j%| |j'z } |j)d \}}| |j+dz } | |j-|j/j'z } |j%d | z| z }t1|||}| j3| d z||| t5|z d ||kDr&|}tj6|j9dt;d| d zd|d| t5|z dd|dd|dd|dd | |fS)u Generic training loop for both SM-ODE and baseline models. FIX #3: Resets singularity at the start of every epoch so memory accumulates only within each epoch (not across epochs). FIX #4: Singularity modules skip updates during evaluation (handled inside SingularityMemory.update_singularity). FIX #6: Renamed from train_with_singularity — works for any model. gMbP?g-C6?)lr weight_decay)T_maxrgr)r+g?)max_normrRY@)epoch train_acctest_acclosszsmode_cifar10_best.pthzEpoch /z | Loss: z.4fz | Train: .2fz % | Test: z % | Best: %)rCrossEntropyLossoptimAdamWr( lr_schedulerCosineAnnealingLRr[trainhasattrrr+to zero_gradbackwardr)utilsclip_grad_norm_stepitemmaxr2eqsumevaluateappendlensave state_dictprint)model trainloader testloaderr%epochs criterion optimizer schedulerbest_acchistoryr running_losscorrecttotalinputstargetsoutputsrrb predictedrrs r train_modelr4s`##%I E,,.5tLI""44Yf4MIHGv-Z   53 4  ) )[5K5K ) L* :OFG$ii/F1CGF    !FmGWg.D MMO HHNN * *5+;+;+= * L NN  DIIK 'L";;q>LAy W\\!_ $E y||G,002779 9G! :$ 7NU* E:v6QY"  3{#33    h H JJu'')+C D uQwiq ,s;?O2OPS1TU!#j#jRUVWY ZY-Z^ H rc|jd}d}tj5|D]\}}|j||j|}}||}|j d\}} ||j dz }|| j |jjz } dddd|z|z S#1swYxYw)z Evaluate model. FIX #4: Singularity update_singularity() now checks self.training, so memory is never contaminated by test data. rrRNr) evalr)rArrr2rrr) rrr%rrrrrrbrs rrrws JJLG E :) :OFG$ii/F1CGFFmG";;q>LAy W\\!_ $E y||G,002779 9G  :: '>E !!::s B CCc*eZdZdZdfd ZdZxZS)BaselineCIFAR10z Theory: Memory stored in singularity, not lost between statesz: -> Predictions use: f(laws) + alpha * memory_biaszA -> Multiple blocks share singularity (like entanglement)z9 --- Training SM-ODE Model (with Singularity Memory) --- r)rr c3<K|]}|jywrnumel.0ps r zmain..sCQqwwyCzSM-ODE Parameters: ,2)rz2 --- Training Baseline Model (no Singularity) --- )rc3<K|]}|jywrrrs rrzmain..sFQqwwyFrzBaseline Parameters: z= ============================================================zRESULTS SUMMARYz$ SM-ODE Model (Singularity Memory): rrz"Baseline Model (No Memory): z#Improvement: z$ --- Singularity Memory Analysis ---zMemory dimension: 128z#Memory decay: 0.95 (high retention)zShared singularity blocks: 3z;Theory: Memory accumulates across batches within each epoch)r)r%r is_availabler transformsCompose RandomCropRandomHorizontalFlipToTensor Normalize torchvisiondatasetsCIFAR10rrzrrr(rr)r%transform_traintransform_testtrainsettestsetrr smode_model total_params smode_history smode_bestbaseline_modelbaseline_history baseline_bests rmainrs \\EJJ$;$;$=&5 IF N6( #$!((b!,'')57OP *O ''57OP)N ##++ dT_,H""** edn+GX#tQRSKGUPQRJ (O :; (O KL FG MN G GH2#>AA&IKC+*@*@*BCCL  Q/ 01 +[*fR!M:  @A$477?NF.*C*C*EFFL !,q!1 23&1 Z'#m  /  (O 1*S1A CD .}S.A CD / ]0J3/Oq QR 12 !# /1 (* GIr__main__)r)rIr)torch.nnrtorch.nn.functional functionalre torch.optimrrtorchvision.transformsrtorch.utils.datarnumpynpModulerrOrzrrrrrFrrrs  +'mS mSh?CRYY?CLW5299W5|@F"4#"bii#"THJV zFr