Research Scientist · IBM Research
I build language models that think by descending — where a forward pass is not a fixed stack of layers but relaxation on an energy landscape, and where memory, attention, and expert routing all fall out of the same variational principle.
Click the field to release a token
About
I am a Research Scientist at the MIT-IBM Watson AI Lab, IBM Research. I work on language model architecture, mostly on what changes when you treat a transformer as a dynamical system with an energy function instead of a fixed stack of layers.
Once the forward pass is minimizing something, inference has a natural place to stop, and depth stops being a number you picked because you could afford it. Expert routing falls out of the same energy rather than a load-balancing loss bolted on the side. NRGPT, our ICLR 2026 paper, was the first full language model we built this way. The follow-up work asks how much compute you save when the model decides for itself that it has converged. Right now I am also scaling the architecture to billion-parameter models, to see whether the energy formulation still holds at that size.
I did my PhD at Rensselaer Polytechnic Institute with Mohammed J. Zaki, working on clustering. ClAM and DClAM came out of that. Both let points settle toward prototypes under associative-memory dynamics, which turns a discrete assignment problem into something you can train with gradient descent.
Before graduate school I spent roughly four years as an engineer, most of it at bKash, the largest mobile financial platform in Bangladesh, used by more than 65 million people. That background is probably why I care as much about whether something trains on a real cluster as about how it looks on a benchmark.
News
What I work on
Rewriting the transformer forward pass as gradient descent on a scalar energy. The model has a well-defined objective at inference time, and the number of steps it takes becomes an observable rather than a hyperparameter.
Modern Hopfield networks as a computational primitive. Attention is one instance; so is clustering, so is retrieval. I am interested in what else becomes differentiable once you express it as relaxation toward a stored pattern.
Deriving Mixture-of-Experts routing from a free-energy functional instead of bolting on a load-balancing penalty, and letting a model halt when its energy has converged rather than after a fixed budget.
ClAM and DClAM make cluster assignment differentiable by letting points settle toward prototypes under associative-memory dynamics. The same interest pulls me toward health data: in an autism screening study we used clustering to see what structure the data holds once you set the labels aside.
Selected publications
A language model whose forward pass minimizes a scalar energy rather than executing a fixed layer stack, unifying autoregressive generation with associative-memory dynamics. Holds up across tasks from algebraic reasoning to open-domain generation.
An auxiliary-loss-free routing method derived from a free-energy functional, with a proof of exact energy compatibility and a formal attention–routing duality inside Energy Transformers.
DClAM couples an autoencoder with associative memories for end-to-end differentiable deep clustering, learning representations and latent-space clusters simultaneously under a single simplified objective. An extended version is under review at KDD 2026.
ClAM uncovers a connection between associative memory dynamics and the discrete assignment inherent to clustering, yielding an unconstrained continuous relaxation that makes the whole pipeline differentiable and trainable by SGD.
A systematic comparison of eight classifiers and five clustering algorithms on paediatric, adult, and combined ASD screening datasets, with chi-square feature selection and per-model hyperparameter search. Spectral clustering proved strongest in the unlabelled setting — an early instance of the clustering questions I would later pursue with associative memories.
Positions
Feb 2025 — present
IBM Research, MIT-IBM Watson AI Lab — energy-based LLM architecture, MoE routing, distributed pretraining
Jan 2021 — Jan 2025
Rensselaer Polytechnic Institute — advisor Mohammed J. Zaki
Jan 2021 — Dec 2022
Rensselaer Polytechnic Institute — earned en route to the PhD
Summer 2022 & Summer 2023
IBM Research, Cambridge — two internships on end-to-end differentiable clustering with associative memories
May 2019 — Jan 2021
bKash, Dhaka — loan origination system and VISA/bank payment gateway integrations for Bangladesh's largest mobile financial platform, 65M+ users
Oct 2017 — Apr 2019
REVE Systems, Dhaka — government-scale web platforms (OISF, BCC Billing)
Aug 2017 — Sep 2017
Bangladesh University of Business and Technology — structured programming, data structures, graphics design
Feb 2013 — Sep 2017
Bangladesh University of Engineering and Technology (BUET)
Service & community
Contact
Email is the fastest way to reach me. I read everything and answer most things — students asking about the work included.
bsaha205@gmail.com