-ij'ddlZddlmZddlmcmZddlZddlmZm Z m Z m Z GddejZ GddejZGddejZGd d ejZGd d ejZ dZdedej$dededededededede ededede ededededed e ee ff$d!Zed"k(rej2d#d$Zd%Zd&Zej:dd'ej<zeZd(ej@ezd)ej@d*ezzzd+ej@d,ezzzd-ejBezzZ"e"jGdjGd.jIed.d.Z%e&d/e&d0e&d/eed1d2d34Z'e&d5ee'e%d6d7d d8Z(e&d9ejR5e'e%dd&\Z*Z+dddeezZ,e-d:e'j]DZ/e&d;d/e&d<e&d=e,d>e&d?e/d@e&dAe,e/z dBdCe&dDe&dEe+dFdjad3Ge&dHe+dIdjad3Ge&dJe+dKdjad3Ge&d;d/e&dLe&d/eed%d&d3d*d,MZ1e1e%Z2e&dNe%jfe&dOe2jfe1jiZ5e&dPe&dQe5dRe&dSe5dTe&dAe5dUdBdCe&dVejR5e1e%dW\Z2Z6ddde&dXe6jfe&dYyy#1swYxYw#1swY5xYw)[N)TupleOptionalDictListc PeZdZdZ ddededeffd ZdejfdZ dejfdZ dejfd Z d ejdejfd Z d ejd ejdejd ejdejf dZxZS)ParametricFourierBasisu Learnable parametric Fourier basis functions. Instead of storing coefficients explicitly, we learn parameters that generate the Fourier coefficients on demand. F(t) = Σ θ_k * φ_k(t) where φ_k are basis functions n_harmonicslearnable_frequenciesinitial_freq_scalecJt|||_tjt j ||z|_tjt jd|dzj|_ tjt j|dz|_ tjt j ||_ tjt j|dztj z|_y)N皙?)super__init__r nn Parametertorchones freq_basearangefloatfreq_multiplierrandnamp_base amp_scalerandmathpi phase_base)selfr r r __class__s 0/home/per/Documents/python/linearize/pfourier.pyrzParametricFourierBasis.__init__s & JJ{ #&8 8  "|| LLK!O , 2 2 4   U[[%=%CD ejj&=>,, JJ{ #a '$'' 1 returnc4|j|jzS)z4Get frequencies from parameters - O(n) not infinite.)rrr!s r#get_frequenciesz&ParametricFourierBasis.get_frequencies*s~~ 4 444r$cZtj|j|jzS)z3Get amplitudes from parameters - O(n) not infinite.)Fsoftplusrrr's r#get_amplitudesz%ParametricFourierBasis.get_amplitudes.szz$--(4>>99r$cbtj|jtjzS)zGet phases from parameters.)rtanhr rrr's r# get_phasesz!ParametricFourierBasis.get_phases2szz$//*TWW44r$tch|jdk(r|jd}|jd}|jd}|j}|j }|j }|jd}|t j||z|zz}|jdS)z Generate Fourier series value at time t. Args: t: Time points, shape (batch, seq_len) or (seq_len,) Returns: Fourier series values, same shape as t r rdim) r4 unsqueezeshaper(r,r/rsinsum) r!r0 batch_sizeseq_lenomegaAphi t_expanded harmonicss r#forwardzParametricFourierBasis.forward6s 557a< AAWWQZ ''!*$$&    !oo[[_  J  $  }}}$$r$ frequencies amplitudesphasesc|jdk(r|jd}|jdk(r|jd}|jdk(r|jd}|jdk(r|jd}|jddk(ro|jddkDr]|j|jdd}|j|jdd}|j|jdd}|jd}|jd}|jd}|jd}|t j ||z|zzj dS)a Reconstruct a time-domain signal from explicit Fourier parameters. This is the inverse transform for this parameterization, similar in spirit to an inverse FFT, but using learned continuous frequencies instead of discrete FFT bins. Args: frequencies: Frequency parameters, shape (n_harmonics,) or (batch, n_harmonics) amplitudes: Amplitude parameters, same shape as frequencies phases: Phase parameters, same shape as frequencies t: Time points, shape (seq_len,) or (batch, seq_len) Returns: Reconstructed time-domain signal, shape (batch, seq_len) r rr2r3)r4r5r6expandrr7r8) r!rArBrCr0r> freq_expanded amp_expandedphase_expandeds r#inversezParametricFourierBasis.inverseUsT. 557a< AA ??  !%//2K >> q #--a0J ::<1 %%a(F   Q 1 $a%,,QWWQZ-\\-LL- - << -  -r$rceZdZdZd dededeffd ZdejfdZ d dejd e dejfd Z e de fd Z xZS)LowRankFourierLayeru Low-rank parametric Fourier layer. W_ij = Σ_k u_ik * v_jk (rank-r factorization of weight matrix) Compression: O(mn) → O(r(m+n)) input_dim output_dimrankcjt|||_||_||_t j tj||dz|_ t j tj||dz|_ t j tj||_ y)N{Gz?) rrrWrXrYrrrrUVr basis_scale)r!rWrXrYr"s r#rzLowRankFourierLayer.__init__s "$ ekk*d;dBCekk)T:TAB<< 4(89r$r%c|j|jjz}||jj dzS)z Reconstruct full weight matrix from parametric factors. Only computed when needed (on-demand generation). r)r\r]Tr^r5)r!Ws r#get_weight_matrixz%LowRankFourierLayer.get_weight_matrixs6 FFTVVXX 4##--a000r$xgenerate_weightsc|r0|j}tj||jStjtj||j|j jS)a Forward pass with optional on-demand weight generation. Args: x: Input tensor (batch, seq, input_dim) generate_weights: If True, reconstructs full matrix If False, uses factorized form directly )rbrmatmulr`r]r\)r!rcrdras r#r@zLowRankFourierLayer.forwardsZ &&(A<<133' '<< Q' r$c|j|jz}|j|j|jzz|jz}||z S)z6How much this layer is compressed vs explicit storage.)rWrXrY)r!explicit parametrics r#compression_ratioz%LowRankFourierLayer.compression_ratiosE>>DOO3YY$..4??"BCdiiO *$$r$)F)rLrMrNrOrPrrrRrbrQr@propertyrrjrSrTs@r#rVrVs{ :# :3 :c :15<<1"' <<  0%5%%r$rVc eZdZdZ d dedededeffd Zdejdejd ejfd ZxZ S) FourierGeneratorLayeru Neural network generator that produces Fourier-based weights. W_ij = NN_θ(i, j) — a small network generates weight values from their indices in the weight matrix. Compression: O(mn) → O(θ) where θ << mn rWrX hidden_dimn_fourier_termsc t|||_||_||_|j dt jd|dzj|j dt jd|dzjtjt j||dz|_ tjtjd|tjtj||tjtj|d|_y)Nfreq_ir freq_jr[r)rrrWrXrqregister_bufferrrrrrrfourier_coeffs SequentialLinearTanhneural_correction)r!rWrXrprqr"s r#rzFourierGeneratorLayer.__init__s "$.   LLOa/ 0 6 6 8    LLOa/ 0 6 6 8 !ll KK 9D @  "$ IIa $ GGI IIj* - GGI IIj! $ " r$ indices_i indices_jr%cd|jz|jz dz }d|jz|jz dz }|jd}|jd}t j ||j jdztjz}t j||jjdztjz}t jd||j|} t j||gd} |j| jd} | | zS)a Generate weight values for given matrix indices. Args: indices_i: Row indices (output dimension) indices_j: Column indices (input dimension) Returns: Weight values for each (i,j) pair rr r2rz...k,kl,...l->...r3)rrXrWr5rr7rsrrcosrteinsumrvstackrzsqueeze) r!r{r|i_normj_norm i_expanded j_expandedi_freqj_freq fourier_outindex_featuresrzs r#r@zFourierGeneratorLayer.forwards!Y__&&81<Y__&&7!;%%b) %%b)  ..q1 1DGG ;  ..q1 1DGG ;  ll#6@S@SU[\ ff%52> 22>BJJ2N...r$) ) rLrMrNrOrPrrrRr@rSrTs@r#rorosb # # #  #  # J"/"/%,,"/5<<"/r$roc eZdZdZ ddedededededef fd Z dd ejd ed ejfd Z d e e effd Z xZ S)ParametricTimeSeriesModelz Complete parametric time series model using learned Fourier series. Instead of storing explicit weights, uses parametric functions to generate weights on demand. rWrprXn_layersrYrqc <t|||_||_||_||_|dz|_t||_tj|j ||_ tjt|Dcgc]}t|||c}|_t!||||_tj$t'j(||_tj$t'j(|dz|_ycc}w)Nr )rY)rqg?)rrrWrprXrinput_with_fourier_dimr input_basisrrxinput_projection ModuleListrangerV hidden_layersrooutput_projectionrrrode_gain ode_damping) r!rWrprXrrYrqir"s r#rz"ParametricTimeSeriesModel.__init__ s "$$  &/!m#2/B " $*E*Ez R ]] 8_ ,   ,  "7  O"   UZZ%9: << 8() ) r$c|j|jz|jz}td|jD}||j j dzz }||jjjz }||jjjjz }|||t|dz dS)z@Calculate storage cost of parametric vs explicit representation.c3K|]T}|jj|jjz|jjzVywN)r\numelr]r^).0rs r# z=ParametricTimeSeriesModel.get_storage_cost..sE  GGMMOeggmmo -0A0A0G0G0I I sAAr )rhrirj)rWrprr8rrr rrvrrz state_dict__len__max)r!explicit_paramsparametric_paramss r#get_storage_costz*ParametricTimeSeriesModel.get_storage_cost~s..4??:T]]J ++   T--99A==T33BBHHJJT33EEPPRZZ\\(+!037H!3L!L  r$)rrJrl)rLrMrNrOrPrrrRrQr@rstrrrSrTs@r#rrs!%D%D%D %D  %D  %D%DT*/5 <<5#'5  5n $sCx. r$rc eZdZdZ ddedededeffd Zdejdee ejffd Z d ee ejfd ejdejfd Z dd ee ejfd e ejdejfd Z ddejd e ejdeejeffdZxZS)AdaptiveFourierEncoderu Encodes arbitrary time series into parametric Fourier representation. Given a time series x(t), finds parameters θ such that: x(t) ≈ Σ_k θ_k * φ_k(t) The encoder learns to compress arbitrary inputs into fixed-size parameter vectors. rW latent_dimn_fourier_basisencoder_layerscJt|||_||_||_t j |||d|_t||_ t j|||_ t j|||_ t j|||_ y)NT) input_size hidden_size num_layers batch_first)rrrWrrrGRUencoderr fourier_basisrxlatent_to_freq latent_to_amplatent_to_phase)r!rWrrrr"s r#rzAdaptiveFourierEncoder.__init__s "$.vv "%   4OD!ii ODYYz?C!yy_Er$rcr%c0|j|\}}|d}|tj|j|t j |j |t j |j|tjzdS)z Encode time series into parametric representation. Args: x: Time series (batch, seq_len, input_dim) Returns: Dictionary of parametric components r2)latentrArBrC) rr*r+rrr.rrrr)r!rcrh_nzs r#encodezAdaptiveFourierEncoder.encodes{a3 G::d&9&9!&<=**T%7%7%:;jj!5!5a!89DGGC   r$paramsr0cP|jj|d|d|d|S)a  Decode parametric representation back to time series. Args: params: Dictionary from encode() t: Time points (seq_len,) or (batch, seq_len) Returns: Reconstructed time series rArBrC)rrI)r!rr0s r#decodezAdaptiveFourierEncoder.decodes8!!)) = ! < 8    r$c0||djdkDr|djdnd}t|dd}tjdd||dj }|dkDr!|j dj|d}|j||S)z Alias for decode(), matching the inverse-transform naming you asked for. If `t` is omitted, a default evenly spaced grid over [0, 1] is used. rAr r_default_inverse_stepsdrr2) r4r6getattrrrrr5rEr)r!rr0batchn_stepss r#rIzAdaptiveFourierEncoder.inverses 96<]6K6O6O6QTU6UF=)//2[\Ed$.^sGq*Gs(*rArBrCrrrrrrrrrzEpoch z: Loss=z.4fz, Recon=z, Param=z, Freq=z, Amp=z, Phase=z, Sched=z, LR=z.2ezEarly stopping at epoch z (best loss z).)%roptimAdamW parametersr lr_schedulerReduceLROnPlateaurr6rr4r5rrr zero_gradrIrr*mse_lossr8rrbackwardrutilsclip_grad_norm_stepitemappend param_groupsritemsdetachcloneprintload_state_dict)&rrrrrrrrrrrrrrrrr optimizer schedulerhistoryr0target best_loss best_state bad_epochsepochrrr recon_lossrrrrrscheduler_metrickvs& r#train_parametric_fourierr"s^ !!%"2"2"4![ KDWDWXZD[Fe IJJvA+q) 65@vq1ellSY[\F]    1 $)=)=b)AQ)F+33B7NZZ7 GE4D4D4FGG  .557<<> -446;;=H%,,.335  :% &. /~- .+  ,   &&u'7'7'99E0<@P0P:??,VZV_V_Va'( tyy{+&'..z/@A $$Z__%67 !(()<)<)>? !(()<)<)>? $$Z__%67 Y33A6t<= 99; "Y . ITNg?gh㈵>rT)7rtorch.nnrtorch.nn.functional functionalr*rtypingrrrrModulerrVrorrrRrPrrQrr"rL manual_seedr:r9rWrrr0r7rrcr5rErrrrno_gradrrrr8rrround full_modelrr6rstorageweightsr$r#r5s'  ..{RYY{|<%"))<%~Q/BIIQ/hv  v ro&RYYo&j! ""(,!"(#T !TT T  T  T  TTTTsmTTT! TTT !T"#T$ #t)_%Tv zEbGJI q!ehh,0A ieiil ieiiA  yuyya   ! kekk'"" #++a.**2.55j"bIK (O @A (O #  E ,-&   G  8!&{2A!78 )OBu/?/?/ABB Bvh- !  1 9: "#4"5[ AB !/2C"CC!H JK )+ OF=1!4::A:FG HI OF<039919EF GH OF8,Q/55q5AB CD Bvh- ./ (O*  J $F M+++, -. N6<<. )*))+G ! #GJ$7#8 9: %gl&;%< => !'*=">s!C1 EF 68 Q$[4PQ %gmm_ 56 68A^88\QQs:OOOO