Online Probabilistic Model Identification Using Adaptive Recursive MCMC
P. Agand, M. Chen, H.D. Taghirad (2023). “Online Probabilistic Model Identification Using Adaptive Recursive MCMC.” International Joint Conference on Neural Networks (IJCNN).
A Bayesian system identification algorithm that adapts online as new data arrives — enabling robots and control systems to continuously refine their models of…
Classical system identification assumes you collect data first, then fit a model. In practice, the environment changes — robot dynamics shift as components wear, physical conditions vary, and the system encounters new operating regimes. Online system identification solves this by continuously updating the model as data arrives.
ARMCMC (Adaptive Recursive Markov Chain Monte Carlo) provides a principled Bayesian approach to this problem: rather than maintaining a point estimate of system parameters, it maintains a probability distribution over them, updated recursively as each new observation arrives.
The Identification Problem
When a robot or control system interacts with the world, it does so according to some underlying model of dynamics. That model has parameters — masses, friction coefficients, spring constants, or whatever is appropriate for the domain. Online identification means inferring those parameters from observation, continuously.
The challenge: MCMC-based Bayesian inference is traditionally expensive (many samples required for accurate posterior approximation) and batch-oriented (requires revisiting historical data). Neither property is compatible with real-time online operation.
Approach
ARMCMC addresses this through two mechanisms:
Recursive update rule. Rather than running a fresh MCMC chain on the full historical dataset at each timestep, ARMCMC updates the current posterior estimate using only the new observation — analogous to a Kalman filter for MCMC-based inference.
Adaptive proposal distribution. The Markov chain proposal is adapted using the running posterior estimate, keeping acceptance rates high even as the posterior evolves. This avoids the chain "forgetting" past structure as new data arrives.
The result is an online Bayesian identifier that maintains calibrated uncertainty — it doesn't just produce a parameter estimate, it produces a distribution over parameters that reflects how much is known and unknown at each point in time.
Why Uncertainty Matters for Control
A controller designed around a point-estimate model will be overconfident in regimes where data is sparse. ARMCMC's probabilistic output enables uncertainty-aware control: conservative action in novel regimes, confident action in well-characterized ones. This is the foundation of safe, adaptive autonomous systems.
The algorithm was evaluated on robotic manipulation and presented at IJCNN 2023.
I write about this kind of work — reliability, uncertainty, building things that work in production. One email per month.