### USER
Learn the theory (compression) in (.py)

Using a isclose lookup table and zlib compression for the index is a smart example. List 100 other such compression ideas for ai.

### ASSISTANT
The approach in `ex02.py` demonstrates a classic **discretization → indexing → entropy coding** pipeline:
1. **Discretize** continuous values into a fixed lookup grid (`np.linspace`)
2. **Map** each value to the nearest grid index (`np.isclose` + `argmax`)
3. **Compress** the resulting integer indices losslessly (`zlib`)
4. **Trade-off** storage size vs reconstruction error (MSE)

This is essentially **scalar quantization + lossless index compression**, a foundational technique heavily adapted in modern AI model compression. Below are **100 distinct compression ideas** inspired by or extending this paradigm, specifically tailored for AI components (weights, activations, embeddings, gradients, sequences, etc.).

### 🔢 Index & Lookup-Based
1. **Product Quantization (PQ)**: Split vectors into subvectors, each mapped to its own small codebook; store only sub-indices.
2. **Additive Quantization**: Represent vectors as sums of multiple codebook entries; store multiple index streams.
3. **Hierarchical Codebooks**: Coarse index selects region, fine index refines precision within it.
4. **K-Means Centroid Indexing**: Cluster flattened weights; store cluster assignments + centroid table.
5. **Hash-Based Weight Sharing**: Hash weight positions to shared indices; compress hash table + index array.
6. **Pattern Dictionary Compression**: Extract recurring weight submatrices; store dictionary + tile indices.
7. **Shared Embedding Tables**: Merge low-frequency token embeddings into shared lookup slots; store mapping.
8. **Attention Mask Index Caching**: Store only active attention positions as compressed integer lists.
9. **Activation Function LUTs**: Replace costly GELU/SiLU with precomputed 1D lookup tables; compress indices.
10. **Token Embedding Delta Indexing**: Store base embedding table; encode rare tokens as offsets + indices.

### 📊 Quantization & Discretization
11. **Non-Uniform Log Quantization**: Use logarithmic grid spacing to better match heavy-tailed weight distributions.
12. **Mixed-Precision Outlier Handling**: Keep top-1% weights in FP16; quantize rest to INT4; store separate streams.
13. **Group-Wise Quantization**: Quantize per-channel or per-block; compress scale/zero-point arrays alongside indices.
14. **Stochastic Rounding**: Preserve expected values during quantization; compress deterministic rounding residuals.
15. **Adaptive Bit-Width Layers**: Allocate 2/4/8 bits per layer based on sensitivity; compress variable-width indices.
16. **Ternary Quantization {-1, 0, 1}**: Store signs + zero masks; compress with run-length encoding.
17. **Binary Weight Networks**: Store sign bits; compress magnitude scales per filter/channel.
18. **Block Floating Point**: Shared exponent per block; compress mantissa indices + exponent array.
19. **Dynamic Range Tracking**: Stream quantization parameters; compress parameter deltas across layers.
20. **Error-Bounded Adaptive Quantization**: Adjust grid density to guarantee max MSE; compress density map + indices.

### 🗜️ Entropy & Lossless Coding
21. **Huffman Coding for Weight Histograms**: Build optimal prefix codes from quantized weight frequencies.
22. **Arithmetic Coding for Indices**: Compress non-uniform index distributions with fractional-bit efficiency.
23. **ANS (Asymmetric Numeral Systems)**: Fast entropy coding for large index arrays in model checkpoints.
24. **Golomb-Rice for Zero Runs**: Ideal for pruned/sparse weights; compress run lengths efficiently.
25. **CABAC for Weight Signs**: Context-adaptive binary coding of sign bits based on neighbor patterns.
26. **BWT on Flattened Weights**: Burrows-Wheeler transform to cluster similar values before zlib compression.
27. **LZ77/LZ4 on Serialized Models**: Dictionary-based compression on flat weight byte streams.
28. **Predictive Checkpoint Diffs**: Store weights as differences from a base model; compress deltas.
29. **COO/CSR + Entropy Coding**: Compress sparse index/value pairs separately with tailored coders.
30. **Bit-Packing for Low-Bit Indices**: Pack 4-bit indices into uint8 arrays; apply secondary lossless compression.

### ✂️ Sparsity & Pruning
31. **Unstructured Pruning + RLE Masks**: Store binary masks compressed via run-length encoding.
32. **N:M Structured Sparsity**: Enforce 2-of-4 zero patterns; store compact pattern bitmasks.
33. **Dynamic Top-K Activations**: Keep only highest-magnitude activations per layer; store indices + values.
34. **LSH Attention Bucketing**: Compress attention via locality-sensitive hashing bucket indices.
35. **Gradient Top-K Sparsification**: Transmit only largest gradient updates; compress index/value pairs.
36. **MoE Routing Table Compression**: Store sparse expert assignment indices per token.
37. **Layer Skipping Flags**: Binary array indicating active layers per input; compress with bit-packing.
38. **Token Pruning Indices**: Store only retained token positions in sequence compression.
39. **Magnitude Thresholding + Delta Encoding**: Keep weights above threshold; compress deltas from baseline.
40. **Lottery Ticket Subnet Paths**: Compress surviving weight coordinates + fine-tuned values.

### 📐 Low-Rank & Factorization
41. **Truncated SVD Storage**: Keep top-k singular vectors/values; compress each component stream.
42. **LoRA Adapter Compression**: Store low-rank matrices A, B independently; quantize + compress.
43. **Tensor Train (TT) Decomposition**: Factorize weight tensors into cores; compress core arrays.
44. **CP Decomposition**: Represent tensors as sum of rank-1 components; compress factor matrices.
45. **Kronecker Product Factorization**: Approximate large matrices as Kronecker products of small ones.
46. **Block Low-Rank with Shared Bases**: Multiple blocks share basis vectors; store block coefficients.
47. **Random Projection Embeddings**: Compress embeddings via learned Johnson-Lindenstrauss projections.
48. **Tensor Sketching / Hashed Embeddings**: Map high-dim embeddings to low-dim via hash collisions.
49. **Circulant Matrix Factorization**: Represent weights as circulant shifts; store generating vector + FFT indices.
50. **Toeplitz Approximation**: Compress convolutional kernels via diagonal structures + offsets.

### 🔁 Residual & Differential Encoding
51. **Sequential Delta Encoding**: Store differences between adjacent weights; compress with entropy coder.
52. **Multi-Scale Residual Quantization**: Base layer + refinement layers; compress each scale separately.
53. **Momentum Residual Gradients**: Compress gradient updates relative to running average.
54. **Epoch-to-Epoch Checkpoint Diffs**: Store only weight changes between training checkpoints.
55. **Activation Prediction + Residual**: Linear predictor from previous layer; store compressed error.
56. **Layer-Wise Residual Coding**: Predict layer output from input; compress reconstruction error.
57. **Quantization Error Feedback Loops**: Store secondary indices representing quantization residuals.
58. **Inter-Frame Weight Deltas**: For video/temporal models, compress weight changes across timesteps.
59. **Prompt Embedding Deltas**: Store shifts from base token embeddings; compress delta indices.
60. **Teacher-Student Model Deltas**: Compress only differences between distilled and original models.

### 🌊 Frequency & Transform Domain
61. **DCT Patch Compression**: Transform image patches to frequency domain; threshold + compress coefficients.
62. **Wavelet Weight Compression**: Apply wavelet transform; keep significant coefficients; compress indices.
63. **Fourier Domain Pruning**: Remove low-energy frequency components from weight matrices.
64. **Hadamard Transform Mixing**: Convert weights to Hadamard domain; exploit sparsity for compression.
65. **Laplacian Pyramid Weight Storage**: Multi-resolution representation; compress detail levels.
66. **Sparse Frequency Masking**: Retain only dominant harmonics in embedding spectra.
67. **Spectral Quantization**: Cluster in frequency domain; store spectral indices + inverse transform.
68. **Transform-Domain Entropy Coding**: Model coefficient dependencies for context-adaptive compression.
69. **Block DCT + Run-Length for CNNs**: Standard image compression adapted to activation maps.
70. **Principal Frequency Basis Pruning**: Keep only top principal components in transform domain.

### 🧱 Structured & Block-Based
71. **Tiled Weights with Local Scales**: Split matrices into blocks; each has scale + compressed indices.
72. **Shared Block Dictionaries**: Reuse weight blocks across layers; store block indices + reuse map.
73. **Hierarchical Block Pruning**: Coarse block removal → fine-grained pruning within survivors.
74. **Block-Sparse CSR/CSC**: Compress row/col pointers + values separately for block patterns.
75. **ViT Patch Compression**: Group similar patches; store prototype + deviation indices.
76. **Sequence Chunk Context Vectors**: Compress long sequences by sharing context across chunks.
77. **Multi-Head Attention Sharing**: Store unique heads; duplicate via index mapping.
78. **Layer Grouping Compression**: Merge identical layers; store one copy + activation offsets.
79. **Parameter Tying with Offsets**: Tied weights + small delta matrices; compress deltas.
80. **Grid-Based Conv Filter Indexing**: Map 2D filters to 1D grid indices; compress spatial patterns.

### 🤖 AI-Native & Neural Compression
81. **Weight Autoencoders**: Train small encoder/decoder to compress weight tensors.
82. **Hypernetwork Generation**: Store compact latent code; generate weights on-the-fly via hypernet.
83. **Neural Implicit Weight Reps**: Represent weights as continuous functions (e.g., SIREN); store parameters.
84. **Knowledge Distillation + Compression**: Train smaller student; compress teacher-student delta.
85. **Prompt Summary Tokens**: Replace long contexts with learned compressed tokens; store token indices.
86. **Activation Checkpointing Trade-off**: Store compressed checkpoints; recompute missing activations.
87. **Differentiable Compression Layers**: Train end-to-end with rate-distortion loss for minimal accuracy drop.
88. **RL Bit Allocation**: Reinforcement learning agent optimizes bit budget per layer/channel.
89. **Meta-Initialized Compressed Models**: Start from compressed priors; fine-tune with fewer updates.
90. **Federated Update Compression**: Compress client updates via quantization + sparse encoding.

### 🔀 Hybrid & Advanced
91. **Quantization + Pruning + Entropy Pipeline**: Combine three stages for multiplicative compression gains.
92. **Dynamic Bit-Depth per Variance**: Allocate bits based on activation/weight variance maps.
93. **Cross-Modal Shared Codebooks**: Reuse embedding dictionaries across vision/text encoders.
94. **Streaming Sliding Window Compression**: Compress inference buffers in real-time with overlap.
95. **Learned Compressed Sensing**: Optimize measurement matrices for near-lossless weight recovery.
96. **Topological Pathway Compression**: Store only critical weight paths via graph traversal indices.
97. **Error-Guaranteed Adaptive Grids**: Adjust lookup resolution dynamically to bound reconstruction error.
98. **Multi-Resolution Model Pyramids**: Serve different compressed versions based on bandwidth/device.
99. **Secure Obfuscation + Compression**: Encrypt weight indices before entropy coding for IP protection.
100. **Neighbor-Index Prediction**: Train tiny model to predict weight indices from spatial neighbors; store prediction errors.

These ideas span classical signal processing, information theory, and modern deep learning compression research. Many can be combined (e.g., `#11 + #21 + #31` → Product Quantization → Huffman Coding → Run-Length Mask) to achieve >10x model size reduction with minimal accuracy loss, exactly mirroring the `ex02.py` philosophy of **smart discretization + efficient index coding**.