Menu

Sensor Fusion and the Kalman Filter

Sensor Fusion and the Kalman Filter, in Plain Words

Notes · Sensors and estimationNovember 19, 202611 min readSenova Systems Research

One sensor is never enough: GPS is honest but slow, an IMU is fast but drifts 15 metres in 30 seconds. A Kalman filter is a running best guess that weighs each reading by how much it trusts it, an idea from 1960 that flew Apollo to the Moon on 2k of RAM.

First, a simple picture

You are walking through a city with your eyes closed, and two friends are helping you.

The first friend calls out your position once a second: "You are at the corner of 5th and Main." The calls are honest, they are never far wrong, but they come slowly, and when you walk under a bridge, the friend goes quiet for five seconds.

The second friend holds your arm and counts your steps. Fast, smooth, every tenth of a second: "One step, two steps, three." But they drift. Their step is a little shorter than they think, so after thirty seconds they believe you have walked fifteen metres further than you have.

Neither friend alone can tell you where you are. Together, they can, if someone is listening to both, weighing each report by how much it can be trusted, and keeping one running best guess. That someone is the Kalman filter.

Why one sensor is never enough

Every sensor fails in its own way, and the failures are different, which is exactly what makes fusion work.

  • GPS gives you an absolute position: where you are on the Earth. It never drifts, a GPS reading an hour from now is as honest as one now. But it is slow (typically 1–10 updates a second), its standard accuracy is 5–10 metres, and it fails completely in tunnels, car parks, and urban canyons, where the satellite signals bounce or vanish. In a 2026 drone study, GPS was described as "generally reliable, but low update rate and subject to noise, especially in cluttered or urban environments."
  • An IMU (an inertial measurement unit: accelerometers plus gyroscopes) is the opposite. It measures hundreds of times a second, so it never misses a turn, but it only measures change. To know position from an IMU you must integrate: acceleration → velocity → position, twice. And every tiny error in the acceleration gets integrated twice, so the position error grows with the square of time. A bias of 0.02 metres per second squared, a small, ordinary bias, puts you 15 metres wrong after 30 seconds. That is what our own simulation below measures.
  • A barometer measures altitude from air pressure, cheap and fast, but the weather changes the reading.
  • A camera (Article #2) sees the world in microseconds, but it cannot see a still scene at all.

The pattern is the same everywhere: the sensor that is fast drifts, and the sensor that is honest is slow. Sensor fusion is the job of combining them into one estimate that is better than any of them, fast and honest. The Kalman filter is the standard way to do it, and it has been the standard way since 1960.

The idea, in plain words

A Kalman filter keeps two things, and only two things:

  1. A best guess, where the thing is, and how fast it is moving. Engineers call this the state. For a cart, the state is position and velocity. For a drone, it is position, velocity, and orientation, nine numbers in the 2026 drone study.
  2. How sure it is, a number that says how much the guess might be wrong. Engineers call this the covariance, but the plain meaning is "the size of the doubt."

Then it repeats two steps, forever:

Step 1, Predict. Roll the guess forward using the model: "I was moving at 1 metre a second, so in a tenth of a second I will be 10 centimetres further on." The doubt grows a little with every prediction, because the model is never perfect.

Step 2, Update. A sensor speaks. Move the guess toward the reading, but only by as much as you trust that sensor. The weight is called the Kalman gain: if the sensor is precise and the guess is doubtful, the gain is near 1 and the guess jumps to the reading; if the sensor is noisy and the guess is confident, the gain is near 0 and the guess barely moves. After the update, the doubt shrinks, a measurement always reduces uncertainty, even a noisy one.

That is the whole algorithm. Two steps, a guess, a doubt, and a weight. It is small enough to run on a 1960s computer, and it did.

The history: a paper, a visit, and the Moon

Rudolf E. Kálmán, a Hungarian-born mathematician, published "A New Approach to Linear Filtering and Prediction Problems" in the ASME Journal of Basic Engineering in March 1960 (presented at an ASME conference in late March 1959). The paper solved a problem the Wiener filter could not: estimating a changing system from a stream of noisy measurements, one reading at a time, without remembering the whole history. The follow-up with Richard Bucy (the continuous-time case) was at first rejected; it was published in 1961.

The idea might have stayed a paper. Then, in the autumn of 1960, Kálmán visited NASA's Ames Research Center and presented it to Stanley Schmidt's group, which was studying how to navigate a spacecraft to the Moon with a small onboard computer. Schmidt's engineers turned the linear theory into the extended Kalman filter (EKF), the version that handles nonlinear problems by re-linearising around the current guess, and by early 1961 the simulations showed onboard optical measurements could navigate around the Moon with accuracy comparable to the best ground methods, at a fraction of the memory and computation.

MIT's Instrumentation Laboratory carried it into the Apollo guidance system, with James Potter's square-root formulation keeping the arithmetic stable on a computer with short words. The Apollo Guidance Computer had 2k of magnetic-core RAM, and the filter ran on it.

After Apollo, the filter went everywhere: the C-5A transport aircraft around 1966 was, by Schmidt's account, the first real-time Kalman filter on board an aircraft, combining inertial data with radio navigation aids; the first airborne computer implementation was flight-tested in 1972. Today the textbook Global Positioning Systems, Inertial Navigation, and Integration describes the entire GPS system as "one enormous Kalman filter." It is in every smartphone, every drone, every self-driving car, every weather model, and every robot arm.

Sixty-five years of one idea

YearEvent
1960Kálmán publishes "A New Approach to Linear Filtering and Prediction Problems"
1960 to 1961NASA Ames (Schmidt's team) builds the extended filter for lunar navigation
About 1966 to 1972C-5A transport: the first real-time filter on an aircraft
1969Apollo 11 runs the filter on a guidance computer with 2k of RAM
TodayGPS, drones, robots, phones, self-driving cars, weather models

What the filter does to each failure

The filter does not fix sensors. It manages their failures:

  • Noise: a noisy reading moves the guess a little, because the gain is small when the sensor is noisy. Many readings, each pulling gently, average the noise away. The estimate is steadier than any single reading.
  • Drift: a drifting sensor (the IMU) pulls the guess away slowly, but the honest sensor (the GPS) pulls it back every second. The filter trusts the IMU between GPS readings and trusts the GPS to anchor the drift. That is the demo below.
  • Dropout: when a sensor goes quiet, the tunnel, the bridge, the filter keeps predicting from the model. The doubt grows while the sensor is gone, and the first reading after the dropout gets a large gain, because the guess has become doubtful. The filter coasts, then snaps back.
  • Disagreement: when two sensors disagree, the filter does not pick one. It takes the weighted average, weighted by trust, and the result is more accurate than either sensor alone. That is not a metaphor; it is the mathematics, and the demo measures it.

One cart, three answers: actually simulated

This is not a sketch. The numbers below were produced by a real simulation, run for this article: a cart accelerates at 0.2 m/s² for 5 seconds, then cruises at 1 m/s, travelling 27.7 metres in 30 seconds. Two sensors watch it. A Kalman filter (constant-velocity model, state = position and velocity) fuses them.

The sensors. A GPS reads position once a second with 1 metre of noise, and drops out entirely from 15 to 20 seconds, a tunnel. An IMU measures acceleration at 10 Hz with a small, ordinary bias of 0.02 m/s². The IMU never reads position directly; its position comes from integrating twice, the way a real IMU works.

Step 1, GPS alone. Honest, noisy, and gappy. Over the last twenty seconds, its root-mean-square error is 0.81 metres, and for five seconds of that, there is no reading at all. A controller fed on GPS alone would stutter once a second and go blind in the tunnel.

Step 2, IMU alone (dead reckoning). Smooth, fast, and wrong. The tiny acceleration bias integrates twice, so the error grows with the square of time: 3.08 metres off at 10–15 seconds, 9.55 metres at 20–25 seconds, 13.23 metres at 25–30 seconds, 15.08 metres off by the end. After half a minute, the IMU believes the cart is 42.7 metres along a 27.7-metre track. That is not a bad IMU. That is an ordinary one.

Step 3, Fused. The filter predicts with the IMU and corrects with the GPS. The result: 0.36 metres RMSE at 10–15 seconds, 0.28 metres at 20–25 seconds, 0.40 metres at 25–30 seconds, steadier than the GPS alone, and it never drifts. During the five-second GPS outage, the filter coasts on the model: its error rises to 0.82 metres, still about as good as GPS alone, with no sensor at all. At 30 seconds, the fused estimate is 0.11 metres from the truth.

The article's simulation: a cart drives 27.7 m in 30 s; GPS reads once a second with 1 m noise and drops out from 15 to 20 s

EstimateError
GPS alone0.81 m rms, and nothing during the outage
IMU alone (dead reckoning)15.1 m off by 30 s
Kalman filter (GPS and IMU fused)0.11 m off at 30 s

What the demo teaches: The filter's job is not to make the sensors better. It is to make their failures cancel. The IMU's drift is corrected by the GPS; the GPS's noise and gaps are smoothed by the IMU. The estimate is better than either sensor, and it keeps working when one of them stops.


When the filter is not the answer

The Kalman filter is the default, not the cure-all. Three limits, stated plainly:

  • It believes its model. The filter's prediction step trusts the model of how the system moves. If the model is wrong, the cart is actually turning, the drone is actually buffeted, the filter's guess is wrong, and it takes sensor readings to pull it back. A bad model with a confident filter is worse than no filter at all. The 2026 drone paper put it plainly: "no sensor is perfect," and the same is true of the model.
  • The linear filter is linear. Kálmán's original filter assumes a linear system with Gaussian noise. Real robots are nonlinear, they rotate, they accelerate, they turn. The extended Kalman filter (EKF) handles this by re-linearising around the current guess, and it is the standard in drones and cars; the unscented Kalman filter (UKF) does it better for strongly nonlinear systems, at more cost. On the KITTI driving benchmark, UKF fusion cut position error from 13 metres (GNSS alone) to under 5 metres, a reported figure from a 2026 guide, not our measurement.
  • It cannot create information. If every sensor is blind at the same time, a drone in a tunnel with no GPS, no camera, and a drifting IMU, the filter can only coast on the model, and the doubt grows. Fusion makes the best of what the sensors give. It cannot give you a sensor you do not have.

The decision test

Four questions. They sort the jobs where fusion helps from the jobs where it doesn't:

  1. Do your readings jitter, drift, or disagree? Yes, the filter's job exists. No, a single sensor is enough, and fusion buys nothing.
  2. Do you have one fast sensor and one honest sensor? Yes, that is the classic pair (IMU and GPS, gyro and accelerometer, wheel encoder and camera), and the filter is built for it. No, if both sensors fail the same way, fusion cannot help.
  3. Can you write down how the thing moves? Yes, the model is the filter's first half, and a simple one is enough (constant velocity is the standard start). No, fix the model first; a filter on a wrong model is a confident wrong answer.
  4. Does the estimate need to survive a sensor dropout? Yes, the filter's coasting is the reason to build it, and the outage test (Step 3 of the demo) is the acceptance test. No, a simple weighted average of the sensors may be enough, and it is cheaper to build.

If the answers are yes, yes, yes, yes, the job is a filter, and the build is a model, two sensors, and a week of testing. If the answer to question 3 is no, the job is a model, and the honest thing is to say so before anyone pays for a filter.

"But a simple average would do", and two more fair doubts

"We can just average the sensors." An average is a filter with the weights frozen, and the weights are the whole point. The Kalman gain moves: it trusts the GPS more when the guess is doubtful, and the IMU more when the GPS is noisy. A frozen average cannot do that, and it cannot coast through an outage. If your sensors never drop out and never drift, an average is fine. If they do, it is not.

"Our sensors are good enough already." Then the demo above is the test: run the same cart, the same sensors, and measure the error. If the GPS alone is already inside your tolerance, the filter buys nothing, and the assessment will say so. The usual finding is the opposite: the sensors are good, and the estimate is the problem, because nobody was combining them.

"Machine learning does this now." Learning-based estimators exist, and they are good at the cases where the model is hard to write down. But they need training data, they are hard to certify, and they do not tell you how sure they are. The Kalman filter tells you its doubt at every step, and in a machine that moves, that doubt is the number the controller needs most. (Article #3: the PID loop acts on the estimate. A confident wrong estimate is worse than a doubtful right one.)

Sources

  1. Kálmán, R. E. (1960), "A New Approach to Linear Filtering and Prediction Problems"
    What we used

    Transactions of the ASME, Journal of Basic Engineering, 82, pp. 35–45, presented at an ASME conference in late March 1959, published March 1960; follow-up with Bucy (continuous-time case) first rejected, published 1961

  2. McGee, L. A. & Schmidt, S. F. (1985), "Discovery of the Kalman Filter as a Practical Tool for Aerospace and Industry"
    What we used

    NASA Technical Memorandum 86847, autumn 1960 visit to Ames; EKF in simulation by early 1961; Apollo guidance by MIT Instrumentation Laboratory; James Potter's square-root formulation on the Apollo Guidance Computer; C-5A ~1966 first real-time filter on an aircraft; first airborne computer flight test 1972

  3. Apex.AI, "Space Technology for Autonomous Vehicles" (2019)
    What we used

    Apollo Guidance Computer, 2k magnetic-core RAM; NASA did not originally plan to use the onboard computer for guidance

  4. MathWorks, "This Algorithm Is Key to Space Travel, GPS, VR and More" (2016)
    What we used

    GPS described as "one enormous Kalman filter" (textbook: Global Positioning Systems, Inertial Navigation, and Integration); modern uses GPS, ADAS, VR

  5. Motayed, M. S. H. (2025), "Sensor Fusion for Drone Position and Attitude Estimation using Extended Kalman Filter"
    What we used

    Research Square preprint, DOI 10.21203/rs.3.rs-7087667/v1, 9-state EKF fusing GPS, IMU, barometer; GPS "generally reliable, but low update rate and subject to noise"; IMU errors "quickly accumulate, leading to drift"; EKF "significantly reduces sensor noise and drift." Preprint, not peer-reviewed

  6. Caballero Aguilar, J. S. (2026), "Mitigating Inertial Measurement Unit Drift in Quadcopter Trajectory Tracking Using Kalman Filter Sensor Fusion"
    What we used

    University of Arkansas thesis, linear Kalman filter fusing GPS and IMU "improves trajectory tracking accuracy by reducing accumulated drift and filtering noisy measurements."

  7. CopyProgramming, "Kalman Filter for GPS and IMU: Complete Guide"
    What we used

    GPS receivers 1–10 Hz, standard accuracy 5–10 m; IMUs 100 Hz or higher, bias drift accumulates; EKF standard in commercial automotive and drone navigation. Secondary source, used only for the rate and accuracy ranges

  8. CopyProgramming, "Sensor Fusion IMU: Mastering GNSS-IMU Integration with UKF"
    What we used

    KITTI benchmark: UKF fusion cuts RMSE from 13 m (GNSS-only) to under 5 m. Secondary source, labelled as reported, not measured by us

  9. MathWorks, "Understanding Sensor Fusion and Tracking, Part 3"
    What we used

    GPS + IMU pose estimation example: 15 m radius circle, position errors around ±2 m after the filter settles

  10. Our simulation, run for this article
    What we used

    Then cruises at 1 m/s, 27.7 m in 30 s; GPS 1 Hz, 1 m noise, outage 15–20 s; IMU acceleration at 10 Hz with 0.02 m/s² bias and 0.1 m/s² noise; constant-velocity Kalman filter, state = position and velocity. Measured: GPS-only RMSE 0.81 m (10–30 s); IMU dead reckoning 3.08 m (10–15 s), 9.55 m (20–25 s), 13.23 m (25–30 s), 15.08 m off at 30 s; fused 0.36 m (10–15 s), 0.82 m (outage 15–20 s), 0.28 m (20–25 s), 0.40 m (25–30 s), 0.11 m off at 30 s. Full curves in fig1