Shikhar Srivastava

PhD student in Computer Science
University of Rochester

4209 Wegmans Hall
Department of Computer Science
University of Rochester

I’m a Computer Science PhD student at the University of Rochester, advised by Christopher Kanan. I’m doing a PhD to enable large models to learn continually: I’m primarily interested in memory, learning dynamics, and enabling non-negative knowledge transfer across shared task structures in LLMs.

I also work with Catherine Arnett at EleutherAI, and Dhireesha Kudithipudi and Itamar Lerner at UTSA.

Previously, I graduated in Computer Science from MIT Manipal with the Institute’s Academic Excellence Award, then led data science and the AI Roadmap for DHL Ecommerce’s entire Smart-Trucking India business. I also did pre-doctoral work on continual learning for healthcare at MBZUAI & G42.

Publications

* Equal contribution

Latest LayerRoPE: Dynamic Depth-wise Magnitude & Angular Superposition
A RoPE-style depth encoding in the residual stream. It improves compute scaling, depth scaling up to 512 layers, and widens the learning-rate basin, by learning to amplify residual variance rather than dampen it.

S. Srivastava, C. Kanan

Preprint · COLM 2026, Actionable Interpretability

Project page PDF Code

LLaMA2-7B, norm weights by layer
No Single Tokenizer Feature Reliably Predicts Downstream Language Model Performance
A. Lopardo, L. F. Salim, S. Srivastava, K. Jiang, M. Sharma, C. Arnett

EMNLP 2026, Findings
SHARP: Sleep-based Hierarchical Accelerated Replay for Long Range Non-Stationary Temporal Pattern Recognition
J. Dey, S. Srivastava, I. Lerner, C. Kanan, D. Kudithipudi

CoLLAs 2026

PDF
The Unreasonable Benchmark
F. Perez-Cruz, B. Hitaj, G. M. Dimitri, O. Deb, A. M’Charrak, S. Srivastava, C. Kanan, et al.

DMLR 2026

PDF
Improving Multimodal Large Language Models Using Continual Learning
S. Srivastava, M. Y. Harun, R. Shrestha, C. Kanan

CoLLAs 2025 · NeurIPS 2024, Scalable Continual Learning for Lifelong Foundation Models

Blog PDF
Revisiting Multi-Modal LLM Evaluation
J. Lu*, S. Srivastava*, J. Chen, R. Shrestha, M. Acharya, K. Kafle, C. Kanan

CVPR 2025, Benchmarking & Expanding AI Multimodal Approaches · ICLR 2025, Navigating and Addressing Data Problems for Foundation Models

PDF
Continual Learning for Chest X-Ray Classification in Low-Resource Clinical Settings
S. Srivastava, M. Yaqub, K. Nandakumar, Z. Ge, D. Mahapatra

MICCAI 2021, FAIR

Best Paper AwardPDF

Personal

I’m fascinated by memory. By the phenomena arising from our brains developing collectively through the phylogenetic tree. By old stories and old ideas.

I believe strongly in working towards the benefit of all.