# Flux Algebra for PyTorch

A PyTorch implementation of the **FluxNumber** system from the Flux Algebra book, extended to work with matrices and tensors for machine learning applications.

## Overview

This library brings the mathematical framework from "The Algebra of Flux" into the realm of deep learning. It allows you to:

- Track **uncertainty** (entropy) in your neural network weights
- Model parameters as **trajectories** rather than static points
- Apply **entropy collapse** operators during training
- Integrate seamlessly with standard PyTorch optimizers

## Core Concepts

From the Flux Algebra book, every number is a triplet: `⟨v, σ, τ⟩`

- **v (Value)**: The current state/mean
- **σ (Entropy)**: The uncertainty or "spread"
- **τ (Flux)**: The rate of change (velocity)

## Quick Start

### Basic FluxTensor Operations

```python
from flux_tensor import FluxTensor
import torch

# Create flux tensors
z1 = FluxTensor(
    v=torch.tensor([2.0, 3.0]),
    s=torch.tensor([0.5, 0.3]),  # uncertainty
    t=torch.tensor([0.1, 0.2])   # flux/velocity
)

z2 = FluxTensor(
    v=torch.tensor([1.0, 2.0]),
    s=torch.tensor([0.3, 0.4]),
    t=torch.tensor([0.05, 0.1])
)

# Flux addition (⊕)
z_add = z1 + z2  # Entropy accumulates via root-sum-square

# Flux multiplication (⊗)
z_mul = z1 * z2  # Error propagation

# Collapse operator (𝒞) - reduce uncertainty by paying work
z_collapsed = z1.collapse(work=0.2)  # σ_new = max(0, σ - W)

# Time evolution
z_evolved = z1.evolve(dt=0.5, entropy_decay=0.01)
```

### Matrix Operations

```python
# Create flux matrices
A = FluxTensor(
    v=torch.tensor([[1.0, 2.0], [3.0, 4.0]]),
    s=torch.tensor([[0.1, 0.2], [0.15, 0.1]]),
    t=torch.tensor([[0.05, 0.1], [0.08, 0.06]])
)

B = FluxTensor(
    v=torch.tensor([[2.0, 0.0], [1.0, 3.0]]),
    s=torch.tensor([[0.05, 0.1], [0.2, 0.15]]),
    t=torch.tensor([[0.1, 0.05], [0.07, 0.09]])
)

# Matrix multiplication with uncertainty propagation
C = A @ B
```

### Training Neural Networks

#### Method 1: Manual Flux Tracking

```python
import torch
import torch.nn as nn
from flux_tensor import FluxTensor

# Create standard PyTorch layer
layer = nn.Linear(10, 5)

# Initialize flux tracking
weight_flux = FluxTensor(
    layer.weight.data.clone(),
    s=torch.ones_like(layer.weight.data) * 0.1,
    t=torch.zeros_like(layer.weight.data)
)

# Training loop
for epoch in range(num_epochs):
    # Standard forward/backward
    pred = layer(x)
    loss = criterion(pred, y)
    loss.backward()
    
    # Flux-aware update
    with torch.no_grad():
        lr = 0.01
        work = 0.05
        
        grad = layer.weight.grad
        delta = -lr * grad
        
        # Update flux state
        weight_flux = FluxTensor(
            layer.weight + delta,
            weight_flux.s + torch.abs(delta),  # entropy grows
            weight_flux.t + delta              # flux updates
        ).collapse(work)                        # pay work to reduce uncertainty
        
        layer.weight.data = weight_flux.v
        layer.weight.grad = None
```

#### Method 2: FluxOptimizer Wrapper

```python
from flux_tensor import FluxOptimizer

# Create your model
model = nn.Sequential(
    nn.Linear(10, 20),
    nn.ReLU(),
    nn.Linear(20, 5)
)

# Wrap standard optimizer with flux tracking
base_optimizer = torch.optim.Adam(model.parameters(), lr=0.001)
optimizer = FluxOptimizer(
    base_optimizer,
    work_schedule=lambda step: 0.01 * (1 + step * 0.1)  # increasing work budget
)

# Training loop
for epoch in range(num_epochs):
    pred = model(x)
    loss = criterion(pred, y)
    
    # Flux-aware step
    optimizer.step(loss)
    
    # Monitor entropy
    stats = optimizer.get_entropy_stats()
    print(f"Epoch {epoch+1}: Loss={loss.item():.4f}, "
          f"Total Entropy={stats['total_entropy']:.4f}")
```

## API Reference

### FluxTensor

**Initialization:**
```python
FluxTensor(v, s=None, t=None)
```
- `v`: Value tensor (any shape)
- `s`: Entropy tensor (defaults to zeros)
- `t`: Flux tensor (defaults to zeros)

**Operators:**
- `__add__`: Flux addition (⊕) - `z1 + z2`
- `__mul__`: Flux multiplication (⊗) - `z1 * z2`
- `__matmul__`: Matrix multiplication - `A @ B`

**Methods:**
- `collapse(work)`: Reduce entropy by paying work
- `evolve(dt, entropy_decay)`: Time evolution step
- `to(device)`, `cuda()`, `cpu()`: Device transfers
- `total_entropy()`: Sum of all entropy
- `entropy_norm()`: L2 norm of entropy
- `entropy_velocity(prev_s, dt)`: Compute σ̇

### FluxOptimizer

**Initialization:**
```python
FluxOptimizer(optimizer, work_schedule=None)
```
- `optimizer`: Any PyTorch optimizer
- `work_schedule`: Function(step) -> work_budget

**Methods:**
- `step(loss)`: Optimization step with flux tracking
- `get_entropy_stats()`: Get aggregate entropy statistics
- `collapse_all(work)`: Apply collapse to all parameters

### FluxLinear

A drop-in replacement for `nn.Linear` with built-in flux tracking:

```python
layer = FluxLinear(10, 5)

# Use like normal
output = layer(input)

# Update with flux algebra
layer.flux_update(loss, lr=0.01, work=0.1)

# Get entropy report
stats = layer.entropy_report()
```

## Key Insights from Flux Algebra

### 1. Learning as Entropy Collapse

In standard ML, learning minimizes loss. In Flux Algebra, learning is **entropy collapse**:

```
θ_{t+1} = θ_t ⊕ Δθ_t  then  𝒞(θ, W)
```

You pay **Work** (compute/energy) to reduce **Entropy** (uncertainty).

### 2. Uncertainty Propagation

When multiplying flux numbers, uncertainty scales:

```
σ_new = √((v₁·σ₂)² + (v₂·σ₁)²)
```

This prevents silent failure modes where large weights amplify small uncertainties.

### 3. Hallucination Detection

Monitor entropy velocity (σ̇):
- **σ̇ < 0**: System is collapsing (learning/stabilizing) ✓
- **σ̇ > 0**: System is expanding (hallucinating/diverging) ⚠

```python
prev_s = weight_flux.s.clone()
# ... training step ...
sigma_dot = weight_flux.entropy_velocity(prev_s)

if (sigma_dot > 0).any():
    print("Warning: Entropy expanding in some weights!")
```

## GPU Support

All operations are GPU-compatible:

```python
z = FluxTensor(v, s, t).cuda()
# or
z = FluxTensor(v, s, t).to('cuda:0')
```

## Integration with Existing Code

The beauty of this implementation is that you can:

1. **Add flux tracking to existing models** without changing architecture
2. **Monitor uncertainty** during training
3. **Apply collapse operators** strategically
4. **Use standard PyTorch optimizers** with flux wrappers

## Mathematical Properties

All flux algebra properties are preserved:

- **Commutativity**: z₁ ⊕ z₂ = z₂ ⊕ z₁
- **Associativity**: (z₁ ⊕ z₂) ⊕ z₃ = z₁ ⊕ (z₂ ⊕ z₃)
- **Distributivity**: z₁ ⊗ (z₂ ⊕ z₃) = (z₁ ⊗ z₂) ⊕ (z₁ ⊗ z₃)
- **Identities**: 
  - Additive: ⟨0, 0, 0⟩
  - Multiplicative: ⟨1, 0, 0⟩

## References

This implementation is based on the Flux Algebra book:
- **Title**: The Algebra of Flux
- **Subtitle**: Mathematics for a Non-Stationary Reality
- **Based on**: Conditional Collapse Theory (CCT) & ODE-CCT Framework

Key axioms:
1. Nothing is Stationary (x ≠ x)
2. Uncertainty is Fundamental (σ is part of the number)
3. Truth Costs Work (Collapse requires Energy)
4. The Solution is a Path (trajectory, not point)

## Examples

Run the built-in examples:

```bash
python flux_tensor.py
```

This will demonstrate:
1. Basic FluxTensor operations
2. Matrix operations
3. ML training with flux tracking
4. FluxOptimizer usage

## License

Free to use for research and educational purposes. Based on the Flux Algebra framework.