Mathematical foundations
Tensor methods and low-rank approximation
Tucker decomposition, higher-order SVD, numerical linear algebra and structured optimisation applied to the weights of trained networks.
Research · Mathematical artificial intelligence
Tensors, attention,
and the cost of
adaptation.
Six first-authored papers accepted in 2026, two manuscripts under review, and a collaboration in operator theory. The thread running through all of them: pre-trained weights carry exploitable structure, and finding it is a linear algebra problem.
Interests
Efficient AI
Adaptation, compression, quantization
Parameter-efficient fine-tuning, structured head pruning, post-training quantization sensitivity and attention analysis.
Foundation models
Generative AI on mathematical footing
What large pre-trained models actually store, and how much of it can be removed, frozen or replaced without losing capability.
Quantitative modelling
Time series and market structure
Foundation-model forecasting on high-frequency financial data, regime analysis and benchmark-based evaluation.
Accepted and forthcoming · 6 papers
COLM 2026
Conference on Language Modeling
Main conference
LoRA-CRAFT: Cross-layer Rank Adaptation via Frozen Tucker Decomposition of Pre-trained Attention Weights
CRAFT is an extremely parameter-efficient fine-tuning method. Where LoTR and SuperLoRA decompose gradient updates, and PiSSA applies SVD to pre-trained weights one layer at a time, CRAFT performs full Tucker decomposition by higher-order SVD directly on pre-trained weights organised as cross-layer 3D tensors. Every resulting factor is frozen and the model is adapted through lightweight trainable transformations on each factor matrix. On LLaMA3-8B, 0.068M adaptation parameters reach an 82.9 commonsense average against 80.8 for LoRA at 57M — roughly 840× fewer trainable adaptation parameters. At fixed Tucker ranks the adaptation parameter count does not grow with model width or depth.
PDF coming soon
IJCNN 2026
IEEE WCCI
Accepted
Hybrid Neural-Classical Correction for Frozen Time Series Foundation Models: A Comprehensive Ablation Study on High-Frequency Stock Prediction
A frozen 200M-parameter TimesFM backbone is paired with small learned correction heads and evaluated across 2,011,399 one-minute observations from ten technology stocks. The strongest configuration, a gated linear corrector combined with a random forest, lifts mean per-day correlation from 0.059 to 0.373 and reaches 0.597 pooled correlation, using nine times fewer neural parameters than the attention-based corrector it is compared against.
PDF coming soon
ICNLP 2026
IEEE
Accepted
On the Compressibility of Fine-Tuned Attention: A Tensor Decomposition Study of PEFT Weight Structure
A tensor-decomposition analysis of what parameter-efficient fine-tuning leaves behind in attention weights, and how far those fine-tuned updates can be compressed before task performance moves.
ICMLA 2026
IEEE
Accepted
Magnitude Profile Pruning: Calibration-Free Structured Attention Head Removal for Transformer Compression
Existing head-importance criteria need calibration data, gradients or Hessian estimates. Magnitude Profile scoring needs none of them: it finds dispensable heads by statistical outlier detection on weight row norms, pruning heads whose projections sit inside the population bulk and keeping those with outlier norms. A variant, MP-G, handles grouped-query attention by distributing shared key-value group scores across the associated query heads. Evaluated on five models by WikiText-2 perplexity at 12.5–50% head sparsity.
PDF coming soon
ICMLA 2026
IEEE
Accepted
Component Type, Not Reconstruction Error, Predicts Attention Quantization Sensitivity
Nine open-weight models from 1.3B to 8B parameters are swept, quantizing one attention projection at a time under round-to-nearest and GPTQ at 3 and 4 bits — 3,808 distinct measurements. Within a component type, reconstruction error explains less than 10% of the variance in perplexity sensitivity in 27 of 36 cases, with median R² of 0.044. Component type and layer identity both explain more variance than reconstruction error in every model tested, and value projections dominate the total positive perplexity change in seven of nine models.
PDF coming soon
ICMLA 2026
IEEE
Accepted
Spectral Outliers Reveal Dominant Learned Structure in Transformer Attention
Marchenko–Pastur random matrix theory splits each pre-trained attention projection into a random-like bulk and a set of spectral outliers, and the split is validated causally: zeroing the outliers in Mistral-7B drives HellaSwag, MMLU and PIQA close to chance, while zeroing a count-matched subset of bulk singular values does measurably less damage. Across eleven transformers, five patterns recur — query projections carry the most outliers, grouped-query value projections lack clean separation, and specific residual-stream dimensions persist as band outliers across layers.
PDF coming soon
Under review
Manuscript · First author
Causal Head Selectors Agree with Each Other but Not with Attention Heuristics
Manuscript · First author
A Behavioral Taxonomy of Attention Heads That Transfers Across Transformer Language Models
Collaborative · Acta Scientiarum Mathematicarum
Refined Upper Bounds for the Numerical Radius via Weighted Operator Means
Shankhadeep Mondal, Ram Narayan Mohapatra, Kasun Dewage
Sponsored research
NSF Award No. 2310881
Significant contributor to Multiplex Generalized Dot Product Graph Networks: Theory and Applications, under Principal Investigator Prof. Marianna Pensky.
Talks and service
Presentations
- Hybrid Neural-Classical Correction for Frozen Time Series Foundation Models — IEEE WCCI / IJCNN 2026
- On the Compressibility of Fine-Tuned Attention — ICNLP 2026
Professional service
- Session Chair, Time Series and Temporal Modeling — IEEE WCCI / IJCNN 2026
- Reviewer, IEEE International Joint Conference on Neural Networks
- Reviewer, IEEE International Conference on Machine Learning and Applications
- Member — IEEE, International Neural Network Society, American Mathematical Society