Shivam Dubey

Shivam Dubey standing by the Thames, with the Palace of Westminster behind

My work is mechanistic interpretability and adversarial robustness. We build monitors that read a model's activations, then attack them with adaptive attackers to find out what they really measure. At Lexsi Labs the same question points the other way: we trace what a tabular foundation model can do back to the synthetic data that taught it.

A defense that has not survived someone optimizing against it is a hypothesis, not a result. So most of these projects end with an attempt to break the thing we just built. The findings worth trusting are often the negative ones.

With Manan Wadhwa I co-founded Redarc Labs, and our joint papers all come from there. We started with interpretability for existential risk: testing whether the tools that read models can be trusted. Now we work on non-model capability gains: what an agent can do because of its harness (tools, memory, credentials) rather than its weights, and how to control it. The bootcamp we built there runs in person with Secure AI Futures Lab at their Delhi hub, with reading groups on the weekends.

  • MARS V Fellow, Lindley Lentati track: white-box techniques for AI control, Cambridge
  • Redarc Labs Co-founder and research lead: control for non-model capability gains
  • Apart Research Research fellow
  • Lexsi Labs AI researcher, tabular foundation models
  • Secure AI Futures Lab Special projects fellow: teaching the bootcamp in person at the Delhi hub
  • IIT Madras BS, Data Science

* equal contribution. Papers with Manan Wadhwa come from Redarc Labs.

Known by Construction: Calibrating Model-State Instruments Against Models Whose State Was Installed, Hidden, or Never Present

NeurIPS 2026 · New in ML

Manan Wadhwa*, Shivam Dubey*

Known by Construction: Calibrating Model-State Instruments Against Models Whose State Was Installed, Hidden, or Never Present

People increasingly ask whether an AI system has hidden goals, preferences or something like feelings. The tools for checking are mostly tested by seeing whether they agree with each other, and never against a case where the true answer is known. We built small AI models where we controlled the answer so that some were trained to have a hidden habit, and some were not. None of the tools we tested could reliably tell those models apart, signaling that agreement between tools is not a reliable verdict.

Dead Salmons Have a False-Positive Rate: A Calibrated Null-Model Test for Interpretability

NeurIPS 2026 · Interpretability as a Science

Manan Wadhwa*, Shivam Dubey*

Code

Dead Salmons Have a False-Positive Rate: A Calibrated Null-Model Test for Interpretability

When we look inside a model to see what it has learned, say if it has picked up something like sentiment, we use methods like linear probes, SAEs and concept directions. This paper runs those methods on untrained models, which have learned nothing at all, and finds they report the same confident results. The proposed fix is simple: check every finding against an untrained baseline before trusting it.

Steered Affect Moves a Partner Agent’s Activations, and the Logistic Probe That Reads It Returns a Tie-Break, Not an Estimate

NeurIPS 2026 · Interpreting Agent Behavior

Manan Wadhwa*, Shivam Dubey*, Debojeet Ghosh, Shreyansh Kumar, Ayushman Garg

Code

Steered Affect Moves a Partner Agent’s Activations, and the Logistic Probe That Reads It Returns a Tie-Break, Not an Estimate

As AI agents become progressively more talkative around each other, it matters whether one agent’s emotional tone can carry over into another. We pushed one AI agent into an emotional state such as sadness or fear, and found measurable changes inside the agent it was talking to. We also found a common method for detecting these changes gives different answers when run twice on similar data. Once we accounted for that, only the effect for sadness held up clearly. The broader point is that some standard tools for inspecting AI systems need stronger checks before people rely on their results.

Toxin Circuits in ESM-2: Mechanistic Interpretability for Biosecurity Screening

ICML 2026 · GenBio Workshop

Manan Wadhwa*, Shivam Dubey*

OpenReview Code

Toxin Circuits in ESM-2: Mechanistic Interpretability for Biosecurity Screening

BLAST detects 0% of 643 ProteinMPNN redesigns; a frozen ESM-2 linear probe detects 93.9% with no retraining. SAE analysis shows why: redesign amplifies toxin structural features rather than obscuring them. A four-tier attack taxonomy places the security boundary at gradient access (6.1% evasion black-box against 100% white-box), and direct probe attribution pinpoints Layer 32 as the detection bottleneck (r = 0.992).

Loss Landscape Response to Adversarial Perturbation Is Architecture-Dependent

TAIS 2026 · Oxford

Shivam Dubey, Jason Hoelscher-Obermaier

Code

Loss Landscape Response to Adversarial Perturbation Is Architecture-Dependent

Geometric divergence of ViTs and ResNets under seven adversarial attack algorithms, measured with Hutchinson’s stochastic Hessian trace estimator. Input-space and parameter-space curvature differ by five orders of magnitude, which means standard robustness metrics misclassify internal model instability.

Spectral Regularization as a Safety-Critical Inductive Bias

NeurIPS 2025 · Reliable ML Workshop

Shivam Dubey

Code

Spectral Regularization as a Safety-Critical Inductive Bias

Fourier Gradient Regularization penalises high-frequency input-gradients during training, doubling adversarial robustness on ResNet-18 with CIFAR-10 at minimal cost to clean accuracy.

AMBEDKAR: A Multi-level Bias Elimination through a Decoding Approach with Knowledge Augmentation for Robust Constitutional Alignment of Language Models

arXiv:2509.02133 · cited by MIT Technology Review

Snehasis Mukhopadhyay, Aryan Kasat, Shivam Dubey, Rahul Karthikeyan, Dhruv Sood, Vinija Jain, Aman Chadha, Amitava Das

arXiv

AMBEDKAR: A Multi-level Bias Elimination through a Decoding Approach with Knowledge Augmentation for Robust Constitutional Alignment of Language Models

I designed the fairness-aware speculative decoding algorithm, which reduces religion bias by 77% and caste bias by 23% at 6% latency overhead.

Distance to the Boundary Is Not Steering Resistance

Under review · UniReps

Manan Wadhwa*, Shivam Dubey*

Code

Distance to the Boundary Is Not Steering Resistance

How far a representation sits from a decision boundary is not the same quantity as how hard it is to steer, and treating the first as a proxy for the second gives the wrong answer.

Doubt Is a Decision: A Sparse Circuit Commits Language Models to “I Don’t Know”

arXiv preprint · coming soon

Sheryl Mehta*, Shagun Chadha*, Kanika Singhal*, Manan Wadhwa, Shivam Dubey

Doubt Is a Decision: A Sparse Circuit Commits Language Models to “I Don’t Know”

Abstention is not the absence of an answer. A sparse circuit commits the model to “I don’t know”, which makes doubt a decision the model takes rather than a state it falls into.

Isolating Refusal and Compliance Heads: Unconditional Bipolar Defenses

2026 · safety circuits, Qwen2.5-7B

Shivam Dubey

Code

Isolating Refusal and Compliance Heads: Unconditional Bipolar Defenses

A compliance-score classifier reaches AUC 0.943 (Cohen's d = 2.636) with zero false positives across five benign categories: a working zero-shot white-box harm detector. Unconditional bipolar assertion, amplifying refusal heads while zero-ablating compliance heads at L25-H4, holds attack success to 33% under nanoGCG against 36–42% for conditional gating and 66% undefended: the precision of conditional gating is itself a structural vulnerability under white-box optimization. Layer 25 shows extreme intra-layer antagonism between the two head types.

Pre-commitment Probing in Open-Weight Reasoning Models

2026 · OLMo-3-7B-Think

Shivam Dubey

Code

Pre-commitment Probing in Open-Weight Reasoning Models

A linear probe on the residual stream predicts answer correctness before any reasoning token is emitted (AUC 0.780, 5-fold CV). The model commits before it reasons, which makes the chain of thought post-hoc rationalization rather than deliberation. A functional frustration direction at Layer 15 transfers cross-lingually across six languages (p = 0.0017): a sub-linguistic computational state invisible in the output but readable mechanistically, usable either as a monitor or as an attack surface.

The Geometry of Control: Spectral Attractors as Low-Dimensional Projections in Large Language Models

2026 · Llama-3.2-3B, Qwen-2.5-3B

Shivam Dubey

Code

The Geometry of Control: Spectral Attractors as Low-Dimensional Projections in Large Language Models

Why do models lock into refusals, biases or reasoning routines even when the context argues against them? This project measures spectral rigidity, an FFT over the mean residual-stream trajectory along the token axis, and finds each behavior in its own rigid, low-dimensional subspace. Safety refusals are distributed across several heads at Layer 16 (about 120 of 3,072 dimensions); bias is driven by a single head at Layer 27 (about 60); arithmetic and logic fit in 3 to 4. Injecting noise at the peak layer breaks refusals, and forcing low entropy at Layer 27 induces stereotypes (+38%). Projecting directly into the safety subspace yields glitch tokens rather than refusals, which suggests the refusal subspace sits off the manifold of normal generation. Primary results on Llama-3.2-3B-Instruct, with cross-model checks on Qwen-2.5-3B-Instruct.

When Does Synthetic Relational Data Teach Models to Use Relations? Tracing Predictive Structure from Pretraining Data to Model Behavior

NeurIPS 2026 · ATTRIB Workshop

Shivam Dubey, Mohamed Bouadi, Nassim Bouarour, Varun Kulkarni, Aditya Tanna, Vinay Kumar Sankarapu

Code

When Does Synthetic Relational Data Teach Models to Use Relations? Tracing Predictive Structure from Pretraining Data to Model Behavior

Synthetic relational pretraining data is supposed to teach a tabular foundation model to use relations. This traces the predictive structure from the pretraining distribution through to model behavior, and separates the cases where the model genuinely learns to use relations from the cases where it only appears to.

From Behavior to Provenance: Attributing Tabular Foundation Models to Synthetic Pretraining Data

NeurIPS 2026 · ATTRIB Workshop

Mohamed Bouadi, Nassim Bouarour, Shivam Dubey, Aditya Tanna, Vinay Kumar Sankarapu

From Behavior to Provenance: Attributing Tabular Foundation Models to Synthetic Pretraining Data

Attributing what a tabular foundation model does at inference back to the synthetic data it was pretrained on: behavior to provenance, rather than the usual direction.

Context Poisoning in Tabular ICL: Geometric Hijacking and Inverse Scaling

2026 · mechanistic interpretability

Shivam Dubey

Code

Context Poisoning in Tabular ICL: Geometric Hijacking and Inverse Scaling

The first logit-lens implementation for tabular in-context learning: a classification phase transition at Layer 8, representation hijacking as early as Layer 4, and total hijacking from 1.2% poisoned context. Larger models are more vulnerable, not less. QK/OV decomposition and direct logit attribution show that mechanistically-informed defenses (PCA subspace scrubbing, head ablation) fail because of manifold redundancy.

What You Pretrain On Matters: Synthetic Task Distributions Determine Tabular Foundation Model Quality

ICML 2026 · FMSD Workshop

Mohamed Bouadi, Nassim Bouarour, Varun Kulkarni, Shivam Dubey, Aditya Tanna, Vinay Kumar Sankarapu

arXiv

What You Pretrain On Matters: Synthetic Task Distributions Determine Tabular Foundation Model Quality

Introduces O'Prior, a compositional realism prior for synthetic pretraining of tabular foundation models. I ran the interpretability and data-quality ablations, which confirm that mechanism diversity, realism composition, and shift-aware stress each contribute independently to downstream quality.

Physics-Informed Neural Operators with Multi-Method Explainability for Ocean Temperature Prediction

AAAI 2026 · XAI4Science Workshop

Shivam Dubey

Physics-Informed Neural Operators with Multi-Method Explainability for Ocean Temperature Prediction

Neural operators for ocean temperature prediction, with the physics constraint and the explanation methods carrying equal weight: a forecast you cannot interrogate is not much use to the people who have to act on it.