๐ถ๏ธ SPICY-KV: SPectral Invariant Coarse-graining with dYadic KV Renormalization
3.5%โ2.0% KV Budget) with 91.7%โ100.0% NVIDIA RULER Recall on DeepSeek-R1-Distill-Qwen-7B & 1.5B
โก Interactive Pre-RoPE Spectral Multigrid & Softmax Cumulant Closure Simulator
Drag the sliders or toggle the physical operators below to run a real-time 64-dimensional multi-hop attention simulation in your browser. Observe how Post-RoPE Merging (KVMerger/CaM) suffers destructive trigonometric phase cancellation across positions $t_1 \neq t_2$, how omitting the Mass + Subgrid Variance Closure collapses the Softmax partition denominator $Z = \sum_m \exp(s_m)$ (the Jensen Inequality Gap), and how SPICY-KV preserves both exact multi-hop recall and $100\%$ partition conservation:
| KV-Cache Method | Output MSE vs. FullKV (Lower = Better) | Cosine Sim to FullKV | Softmax Partition Ratio Z / Z_Full | Physical / Mathematical Mechanism |
|---|
๐ Empirical Benchmark Suite on DeepSeek-R1-Distill-Qwen-7B & 1.5B
Select any empirical evaluation below to inspect the high-resolution diagnostic plots across NVIDIA RULER, 10.5K Context & 8-Hop Complexity Scaling, Regime 2 (28-Second In-Loop Multigrid LoRA & 28-Head Specialization), and 4-Turn Shared-Prefix RadixAttention Reuse:
๐ป 1. Drop-In HuggingFace transformers Integration (Regime 1 โ Training-Free)
from transformers import AutoModelForCausalLM, AutoTokenizer
from spicy_kv import SPICYKVCache, enable_spicy_kv_attention
model_id = "deepseek-ai/DeepSeek-R1-Distill-Qwen-7B"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype="auto", device_map="auto")
# Patch Qwen2 / Llama attention layers to support SPICY-KV log-mass & 2nd-order subgrid variance closure
enable_spicy_kv_attention(model)
# Create a 28.6x compressed SPICY-KV cache (3.5% KV budget)
cache = SPICYKVCache(target_compression_ratio=28.6, protect_sink_tokens=8, protect_recent_tokens=48)
inputs = tokenizer("Solve step-by-step...", return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, past_key_values=cache, max_new_tokens=256)โก 2. FlashAttention-2/3 & PagedAttention Online-Softmax Kernel Fusion
In production CUDA/Triton engines (vLLM, SGLang, TensorRT-LLM), each coarse node stores just two FP16/BF16 scalars alongside $(\tilde{k}_m, v_m)$: alpha_m = log(m_m) and beta_m = sigma_m^2 / (2 * d_k) (4 bytes per node โ <0.8% memory overhead). Inside the FlashAttention inner tile loop, compute the query norm squared $q_{\text{norm}}^2 = \lVert q_T \rVert_2^2$ once in SRAM registers before iterating over KV blocks:
// Inside FlashAttention / PagedAttention inner K-tile loop (SRAM registers only โ zero dense HBM mask):
// q_norm_sq = ||q_i||_2^2 precomputed once per query row in registers
float s_ij = dot(q_i, k_j) * inv_sqrt_dk + log_mass[j] + q_norm_sq * half_inv_dk_sigma_sq[j];
// Standard online softmax update (m_new = max(m_old, s_ij), l_new = ...) proceeds identically!๐ Patent Pending Notice & Claims 1โ20 (US USPTO & Canada CIPO Filed)
Method / Technology: SPICY-KV (SPectral Invariant Coarse-graining with dYadic KV renormalization, also disclosed in patent specifications as Quantized Spectral Multigrid KV-Cache / AMG-KV)
Author / Inventor Sign-Off: Yuge Y (Elithia Technologies)
Commercial Licensing, IP Acquisition & Independent R&D Contracts: Elithia.technologies@proton.me
Independent Claim 1 โ Multi-Resolution Conceptual Coarsening of Sequence Memory
A computer-implemented method for variable-resolution memory management in a sequence modeling neural network, comprising: (a) obtaining key and value representations (or compressed latent KV representations) across sequence positions; (b) mapping key representations into a position-decoupled or anisotropy-reduced conceptual space (via pre-RoPE tap, inverse rotary transform $R_{\Theta, t}^{-1}$, or common-mode mean removal); (c) coarsening representations across one or more resolution scales into macro-nodes with aggregation mass $m_m > 1$; and (d) computing attention outputs by adjusting attention logits/weights using at least one of aggregation mass $\log(m_m)$ or intra-cluster subgrid variance $\sigma_m^2$ to conserve softmax partition weight $Z$ across scales.
Independent Claim 2 โ Hybrid Landmark-Plus-Aggregate (C/F) Decomposition in Phase-Decoupled Space
Partitioning context tokens in normalized conceptual space into fine-resolution landmark nodes ($C$-points, retained at exact resolution $m_c=1$ and exact positions $t_c$ to preserve multi-token relative RoPE phases) and background bulk nodes ($F$-points, coalesced in position-decoupled space into coarse macro-nodes storing mass-weighted mean key $\tilde{k}_m$, mean value $v_m$, conserved mass $m_m$, and spectral center-of-mass position $\bar{t}_m$).
Independent Claim 3 โ Second-Order Subgrid Variance / Covariance Closure for Merged KV States
Computing and storing, for each coarse macro-node $m$, an intra-cluster subgrid key variance metric $\sigma_m^2$ (scalar, diagonal vector $\vec{\sigma}_m^2$, or low-rank covariance $\Sigma_m$) and adding a second-order cumulant closure term $\frac{\lVert q_T \rVert_2^2 \sigma_m^2}{2 d_k}$ (and $\log m_m$) to the dot-product logit $s_m$, closing the Jensen inequality gap of the attention softmax denominator $Z = \sum_m \exp(s_m)$.
Independent Claim 4 โ Step-Moderated Cross-Block Bridge Hub Detection & Causal Span Shielding
Identifying cross-block reasoning bridge hubs and compound identifiers via step-moderated local vs. cross-block conceptual degree, and applying a 1D causal span dilation filter ($K_{\text{span}}$) to shield contiguous bound-value token spans, numerical digits, and multi-token identifiers in fine-resolution storage.
Independent Claim 5 โ Regime 2: In-Loop Multi-Resolution Training, Post-Training, LoRA Adaptation & RL
Training, fine-tuning, distilling, or RL-adapting (`GRPO`/`PPO`) a sequence model ($W_Q, W_K, W_V$, `LoRA` adapters, or per-head gates) with an active in-loop multi-resolution coarsening operator $\mathcal{R}$, inducing write-side spectral shaping of landmarks vs. bulk and read-side query head specialization into Coarse Planning Heads and Fine Syntax Heads.
Independent Claim 6 & Dependent Claims 7โ20
Covers hardware inference/training systems (Claim 6), MLA latent-space coarsening (Claim 7), Internal RG Coherence Ratio $\eta_i^{(l)}$ & Galerkin self-loop shielding (Claim 8), Spectral Subspace Separation $\theta_{\text{coh}}$ (Claim 9), recursive Law of Total Variance across cascades (Claim 10), $\mathcal{O}(C \cdot T)$ chunked degree evaluation (Claim 11), Cascaded Dyadic Ring-Buffer Pyramid (Claim 12), multi-turn shared-prefix reuse (Claim 13), Two-Level Moderated-Step Discretization & GQA-invariant threshold (Claim 14), $\mathcal{O}(W_{\text{tail}}\cdot T)$ Goal-Coupled Tail Harmonic Bridge Hubs (Claim 15), alternative conceptual coarsening solvers (Claim 16), hybrid head/layer allocation (Claim 17), dual spectral $\text{SNR}_I$ + $D_{\text{KL}}$ objective (Claim 18), online-compressed `