Menu

The Firmware That Doesn't Get Lost

The Firmware That Doesn't Get Lost

Notes · FirmwareNovember 19, 202626 min readSenova Systems Research

Most reactive programs start out simple, but as features are grafted on, more and more flags and variables are introduced, until no human being really has a good idea what part of the code gets executed in response to any given event. The fix is one variable that can only hold one value at a time. This is the method: the state machine, the table, and the rules that keep it from becoming spaghetti again.

The firmware started simple. A sensor, a radio, a battery, a loop that read the sensor and sent the reading. Then the features were grafted on: a button, a low-battery mode, an over-the-air update, a fault light.

Each feature added a flag. Each flag added an if. Each if added a combination.

And one evening, in the field, the device did something nobody could explain, it was advertising and updating at the same time, or it was sampling while it was asleep, and the engineer who opened the code could not say which part of it had run, because the answer depended on five flags, in an order nobody had written down, and the combination that produced the fault had never been tested, because nobody knew it existed.

Miro Samek, who has spent a career teaching this subject, describes the moment exactly. "Most reactive programs start out fairly simple and well structured, but as features are grafted on, more and more flags and variables are introduced to capture the relevant event history. Then ifs and elses must be added to test the increasingly complex logical expressions built out of the various variables and flags (aka spaghetti), until no human being really has a good idea what part of the code gets executed in response to any given event."

The strategic job of this article is that engineer, and the method is the fix Samek names in the next sentence: "When used correctly, state machines become powerful 'spaghetti reducers' that drastically reduce the number of execution paths through the code, simplify the conditions tested at each branching point, and simplify transitions between different modes of execution."

Think of it like a train network and a pile of road signs. The flags version is the pile of road signs: at every junction, a sign says "if you are going north and it is raining and the bridge is open, turn left", and the driver has to read every sign, in order, and hope the signs agree with each other. The state machine version is the train network: the train is on one track, and the track only goes to the places it goes to.

The driver does not decide at every junction. The network decides. And the network can be drawn on a wall, so anyone can see where the train can go, and, more importantly, where it cannot.

The problem

A reactive program, a program that responds to events, which is most firmware, has a problem that a sequential program does not have. A sequential program waits by blocking: it calls a delay, the processor waits, and when the wait ends the program continues from exactly where it stopped, because the stack remembers where it was. An event-driven program cannot block.

It has to return to the event loop after every event, and when it returns, the stack is gone. The context, where in the sequence of events the program was, has to be preserved some other way.

The most popular way, Samek writes in Lesson 35 of his Modern Embedded Systems Programming course, is manual context management: "where programmers invent flags and variables, such as your isLedOn flag." The example is a blinking LED: the event-driven code needs a boolean flag called isLedOn to remember whether the LED is on between timeout events. The flag was not needed in the original RTOS thread, because "in a sequential code the waiting between events happened by means of BLOCKING inside the OSTimeDly() function.

After receiving the expected event the code unblocked and simply handled the event, because you knew exactly which event was received. Therefore, you didn't need to do anything special to remember where in the sequence of events you were. This is an example of maintaining the context between events naturally and automatically."

But the automatic context management is expensive, because the context of a thread during a blocking call requires a whole private stack in the precious RAM. Event-driven code, by contrast, "cannot block, and instead has to return to the event loop after every event. Such code obviously loses the stack context between events.

Consequently, event-driven systems tend to use less stack space, but instead require a different form of preserving the context from one event to the next."

The flag "seems to work initially, and the isLedOn flag certainly didn't loom as a big problem in a simplistic Blinky-Button event-driven active object. But in most real-life programs, such approach is almost guaranteed to degenerate into unwieldy 'spaghetti' code." The reason is arithmetic, and the arithmetic is the whole argument of this article.

Three flags such as is_running, is_error and is_paused make eight combinations. A rice cooker is in one of a few states, standby, cooking, steaming, keeping warm, and a state machine says so with one variable. The flags version says it with three variables, and three variables can hold eight combinations, of which four are states the cooker must never be in.

"Among them the code ends up allowing combinations that must never exist, such as 'running and in error'. Keep the state in a single variable and only one can be true at a time." That is Ground Embedded's formulation, and it is the cleanest statement of the idea in the literature: the flags do not describe the state.

They describe a space of combinations, most of which are nonsense, and the code has to defend itself against the nonsense it created.

The deeper point, from Samek's article "How to Code a State Machine in C or C++", is that the behaviour of most reactive systems "can be divided into a relatively small number of non-overlapping chunks (states), where event responses within each individual chunk depend only on the current event, but no longer on the sequence of past events." The state is "a succinct representation of the relevant system history." Instead of recording the event history in a multitude of variables, "you can use just one 'state variable' that can take only a limited number of a priori known values."

The state variable is the history, compressed into one number, and because it can only hold one value at a time, the impossible combinations cannot be represented at all. They are not checked. They are not reachable.

They do not exist.

The state variable is a logical program counter

Keith Curtis, in "Embedded multitasking with small MCUs: Part 1, State Machine Constructs", published in EDN on 23 December 2006 and adapted from Chapter 2 of his book Embedded Multitasking with Small Microcontrollers, gives the idea its most useful name. "A state machine lets a small microcontroller handle several ongoing activities without stopping in a blocking wait: each activity performs a short step, saves where it should resume, and returns so the main loop can run other work."

The state variable is a logical program counter: "it records where this process should continue next time. A state is a resumable point, and a transition moves the process between points. In a wait state, the task checks whether its condition is satisfied; if not, it reaches the end of the call and gives other work a chance to run."

The picture is a loop, and the loop is the schedule. Each task is a state machine. On each call, it reads its current state, does bounded work, chooses a next state, and returns. A simple superloop can then revisit several independent machines:

for (;;) {
    task_button_step();
    task_sensor_step();
    task_radio_step();
    task_update_step();
}

"This is cooperative, non-preemptive multitasking. No scheduler forcibly interrupts a task at an arbitrary point: each task must return voluntarily." The bargain is explicit, and Curtis states it plainly: "A blocking delay or wait ties up the caller until time passes or an event occurs.

During that interval, unrelated code cannot run in the same thread of control." The state machine is the alternative to the blocking wait. It does not wait.

It records "waiting" as a state, and returns, and the loop runs the other work, and comes back, and checks again.

There is a limit to the bargain, and the article states it before the demo does. The state variable is not a saved thread stack. "It does not automatically preserve local automatic variables, nested call frames, return addresses, or registers.

Any values that must survive between calls need explicit storage, in static or global variables, or, preferably for reusable tasks, in a context object." A state machine remembers where it is. It does not remember what it was thinking.

The values that must survive have to be written down, in the context, by hand, and that is the discipline the method demands. The flag version hides that discipline inside the flags. The state machine version makes it visible, and visible is the point.

The four requirements of multitasking, from the second part of Curtis's series, are the checklist the superloop has to satisfy without an operating system: context switching, communications, managing priorities, and timing control. The state machine is the context switching: the state variable is the context, and the switch is the transition. The event queue is the communications.

Example Labs is a made-up company we use for worked examples.

The order the tasks are called in the loop is the priority. And the timing control is the rule that no task may block, every step must be short, and every wait must be a state, not a delay. Article #13 measured what happens when that rule is broken: on the Example Labs hub, the super-loop leaves the 1 kHz control loop 2.6 times late, and 40.6 times late when the on-device model runs, at 385 samples a second instead of 1,000. The state machine does not fix that by itself.

It fixes it by making the fix possible: a task that returns can be scheduled, and a task that blocks cannot.

The rice cooker, and the vending machine

The rice cooker is the article's teaching example, because it is the smallest system that shows the whole idea. A rice cooker is in one of a few states: standby, cooking, steaming, keeping warm. When an event such as a button press or a temperature arrives, it moves to the next state.

Most firmware can be expressed this way. The cooker has four states and a handful of events, and the whole behaviour fits on a diagram that a new engineer can read in a minute. The flags version of the same cooker has three flags and eight combinations, and the diagram is a decision tree with eight leaves, four of which are nonsense, and the new engineer has to read the code to find out which four.

The vending machine, from Ground Embedded's design table, is the working example. The motor control has five states, Idle, Accelerating, Constant speed, Decelerating, Error, and the table is filled in before any code is written:

Current stateEventNext stateAction
IdleStart buttonAcceleratingTurn the driver on
AcceleratingTarget speed reachedConstant speedNone
AcceleratingOvercurrentErrorTurn the driver off
Constant speedStop buttonDeceleratingTarget speed 0
Constant speedOvercurrentErrorTurn the driver off
DeceleratingSpeed 0IdleTurn the driver off
ErrorReset buttonIdleClear the error

Seven rows. Each row is a decision, written down, reviewable, testable. And the table has a property the code does not: it shows the blanks.

"A blank is itself a decision: 'that event is ignored in that state'. Filling in the table exposes missed cases such as 'what if the start button is pressed while decelerating?'." The flag version has no blanks, because the flags do not have a table, they have a decision tree, and the tree's branches are the combinations, and the combination nobody thought of is the bug that ships.

"For every event in every state, 'what happens' has to be defined. Undefined combinations become bugs."

The basic form, in C, is a switch on one variable:

typedef enum { IDLE, RUN, FAULT } state_t;
static state_t st = IDLE;

void motor_task(void) {
    switch (st) {
    case IDLE:  if (start_pressed()) { pwm_on(); st = RUN; } break;
    case RUN:   if (overcurrent()) { pwm_off(); st = FAULT; }
                else if (stop_pressed()) { pwm_off(); st = IDLE; } break;
    case FAULT: if (reset_pressed()) st = IDLE; break;
    }
}

Three states, three cases, and each case tests only the events that are valid in that state. The default branch is the safety net: if the state variable is ever corrupted, a bit flip, a wild pointer, a stack overflow, the default drives the output off and moves to the fault state. "Towards the safe side if the value is corrupted."

The flag version has no default, because the flags cannot be corrupted into an invalid value, every combination of flags is a valid combination of flags, including the ones that must never exist.

The hierarchy: when one state machine is not enough

The flat state machine has a failure mode of its own, and Samek names it in Lesson 40 of his course. Suppose the TimeBomb application needs a new feature: a second button that defuses the bomb, and the defusing must work at any stage, including when the bomb has already started to time out. In the flat machine, the defuse transition has to be added to every state, copied, pasted, and modified, state by state, and every copy is a chance to get it wrong.

"All these problems are well known, because all of them stem from violating one of the most important principles of software development: the DRY principle, which stands for Don't Repeat Yourself."

The solution, known since the late 1980s when David Harel invented statecharts, is to let states contain other states. Surround the four original states with another state, called "armed", and put the defuse transition on the parent. The behaviour of the substates is inherited from the parent: the defuse event is handled by "armed" no matter which substate the machine is in, and the transition is written once.

The internal states are at a lower level and are called substates, and the meaning of the hierarchy is that the parent's behaviour applies to all of its children. Lesson 42 extends the idea to four levels of nesting, six nested states, "s", "s1", and "s11" on the left and "s2", "s21" and "s211" on the right, with transitions between states at various levels, and internal transitions that handle an event without leaving the state.

The UML statechart formalism, as introduced in embedded.com's "Introduction to UML statecharts", gives the hierarchy its syntax. A transition is written as event [guard] / action: the event is the occurrence, the guard is a boolean expression in square brackets, and the action is what runs when the transition is taken. "If the event fires and the guard evaluates to TRUE, the transition is taken; if the guard evaluates to FALSE, the event is ignored."

Actions may also run when a state is entered or exited, entry actions and exit actions, and the responses execute with run-to-completion semantics: "once an event is received and the object determines that it will respond to it, the object will execute first the exit actions of the predecessor state, followed by the transition actions, and finally the entry actions of the subsequent state, in that precise order, and not process any new events until that set of actions has completed."

That order, exit, transition, entry, is the contract that makes the hierarchy safe: the old state is always cleaned up before the new state is entered, and no event can interrupt the change in the middle.

The hierarchy is also the answer to the state-explosion problem, and the article states the rule before the demo does. "Too many states: give independent concerns (communication link state and motor state) separate state machines. Merged into one, they multiply by their combinations."

Two machines with four states each are eight states and two tables. One machine that merges them is sixteen states and one table with sixteen rows, and the sixteen include combinations that must never exist, which is the flag problem again, wearing a different coat. The hierarchy is for one concern with shared behaviour.

Separate concerns get separate machines. The multiplication is the warning sign.

The method: six rules, in order

The method is not a framework. It is six rules, in order, and each one depends on the one before it. A team that buys the framework before learning the first three will have a statechart nobody can read and a code generator nobody trusts.

Rule one, draw the diagram first. Circles are states and arrows are events and actions. The diagram is drawn before any code is written, and it is drawn on a wall where the whole team can see it. The diagram is the design. The code is the transcription. A team that writes the code first and draws the diagram afterwards has drawn a picture of the code, not a design, and the picture will show the same tangle the code has, because it is the same tangle.

Rule two, fill in the table. The table has four columns: current state, event, next state, action. Every row is a decision. Every blank is a decision too: that event is ignored in that state. The table is filled in before the code, and the review of the table is the review of the design. The question the review asks, for every state, is "how do you leave here?", and the answer must be in the table, or the state is a trap. The fault state, above all, must have a way out. "Always provide a way out of the fault state." A fault state with no exit is a device that bricks itself the first time it fails, and the field will find it.

Rule three, separate entry from do. Work that should happen only at the moment the state changes, starting a timer, sending a message, turning a driver on, is done at the transition, or by remembering the previous state and running only when it differs. Work that happens while staying in the state, checking a sensor, watching a timeout, is done on every pass. The two are different kinds of work, and the code that mixes them is the code that turns the driver on a hundred times a second, or sends the message once and never again. The entry action runs once. The do action runs every pass. The distinction is the difference between a machine that works and a machine that works by accident.

Rule four, never wait by blocking. "Wait with delay_ms(500) and neither buttons nor communication are handled meanwhile." The wait is a state, not a delay: record the time the state was entered, and check the elapsed time on every pass. With unsigned subtraction the difference is calculated correctly even when the tick counter wraps round, and the counter wraps every 49.7 days, which is the kind of field failure that only appears after seven weeks in service, when the device has been running long enough for nobody to connect the failure to the clock. The wait state returns every pass, so the other machines run, the radio runs, the watchdog is fed, and the demo below puts numbers on what the blocking wait costs.

Rule five, interrupts record events; transitions happen in one place. "Changing the state directly in an interrupt: the state changes while the main loop is in the middle of deciding. Interrupts only record events; transitions happen in one place." The interrupt sets a flag or posts an event to a queue, and returns. The main loop reads the queue, and the state machine decides. The state variable is written by one function, in one place, and every change goes through it. That is the rule that makes the machine debuggable: "log the previous and next state on every change and most field problems are solved." A log of transitions is a log of the device's behaviour, and a device whose behaviour is logged is a device whose field failures can be reproduced.

Rule six, the default goes to the safe state. The switch has a default branch, and the default drives the outputs off and moves to the fault state. The state variable is one byte, and one byte can be corrupted, a bit flip, a wild pointer, a stack overflow, and the corruption will produce a value that is not a state. The flag version has no defence against that, because every combination of flags is a valid combination. The state machine has one: the default. And the fault state has a way out, because of rule two, so the corruption costs a reset, not a bricked device.

DEMO

The demo uses the Example Labs field sensor hub, from the cluster: a 100 × 60 mm four-layer board, 120 nets, 114 test points, an STM32H7 at 48 MHz with 512 kB of SRAM and 2 MB of flash, a BLE radio at 2.4 GHz, a sensor rail at 3.0 V, a CR2032 cell of 230 mAh, 2,484 joules, and a sealed 100 × 60 × 40 mm enclosure. The firmware's jobs, from Article #13, are four: the control loop at 1 kHz, the BLE radio every 7.5 ms, the on-device model once a second, and the OTA flash writes in whatever CPU is left.

The CPU is not the constraint, control 10% plus BLE 6.7% plus the model 3.8% is 20.5%, and the schedule is. The demo is the article's own worked example, labelled, built on the cluster's figures and the verified sources above. The arithmetic was run in Python before writing.

Step 1, the combinations. The hub's firmware as written has five flags, the article's own labelled assumption, chosen to match the shape of a real tangle: is_sampling, is_connected, is_advertising, is_updating, is_error. Five flags make 2^5 = 32 combinations. The state machine that replaces them has seven states: IDLE, SAMPLING, ADVERTISING, CONNECTED, UPDATING, SLEEP, FAULT. Seven of the 32 combinations correspond to real states. Twenty-five do not. Twenty-five combinations the code can represent and the product must never be in, updating and sampling at the same time, connected and advertising at the same time, in error and running at the same time, asleep and sampling at the same time. The rice cooker, with three flags and four states, has four impossible combinations. The hub, with five flags and seven states, has twenty-five. The number of impossible combinations grows as two to the power of the number of flags, while the number of real states grows by one at a time, and that is why the flags version gets worse faster than the product gets bigger. The state machine does not check the twenty-five. It cannot represent them. They do not exist.

Step 2, the paths. The flag version's main loop tests the five flags in nested ifs, and each combination of flag values takes a different path through the decision tree: 32 execution paths, each one a different combination of tests, and the behaviour for each combination is spread across the nesting, so the question "what happens when the update arrives while sampling?" is answered by reading the order the ifs are tested in, not by reading a table. The state machine version has 19 transitions, IDLE 3, SAMPLING 3, ADVERTISING 3, CONNECTED 4, UPDATING 3, SLEEP 2, FAULT 1, and each one is a row in the table, independently reviewable. The table has 7 states × 11 events = 77 cells, of which 19 are defined transitions and 58 are blanks, and each blank is a decision: that event is ignored in that state. The update event, arriving while SAMPLING, hits a blank: it is ignored, or queued, and the decision is written in the table where the reviewer can see it. In the flag version, the update flag gets set while sampling, and the code's behaviour depends on which if is tested first, and the answer is in the code, not in a document, and the code is the thing nobody can read.

Step 3, the wait. The hub's update state, as written, waits with delay_ms(500) while the flash write completes. For 500 ms, neither buttons nor communication are handled. At the 1 kHz control loop, that is 500 missed samples. At the BLE interval of 7.5 ms, that is 66.7 missed connection events, the radio's connection, negotiated at 7.5 ms, goes quiet for two-thirds of a hundred intervals, and the central device concludes the peripheral is gone. The state machine version records the time the state was entered and checks the elapsed time on every pass, so the wait state returns every pass and the control loop still runs at 1 kHz. The worst-case latency for the control loop is one pass of the loop, four tasks, each step at most 50 µs by the article's own labelled assumption, so 200 µs, well inside the 1 ms deadline. The blocking wait costs 500 samples. The state machine costs 200 µs of latency. The difference is the difference between a schedule and a hope.

Step 4, the watchdog. The watchdog is the recovery the state machine has to reach, from Article #37, and the rule from that article is the rule that matters here: never kick from a timer interrupt, because an interrupt keeps firing while the main application is deadlocked, so the watchdog is faithfully serviced by a dead product. Kick from evidence: have each critical task set a flag on each successful cycle, and let one supervisor check that every flag has been set within its expected period before it kicks. The state machine makes the evidence possible, because each machine's step is a cycle, and the cycle either happened or it did not. The flag version, with its blocking waits, cannot give that evidence: the main loop is inside a delay, and the supervisor cannot tell whether the task is working or stuck, because the task is not running at all. And the watchdog's own tolerance makes the margin smaller than it looks: the STM32 independent watchdog runs from its own low-speed internal oscillator, around 32 kHz, but the oscillator can vary from approximately 17 to 47 kHz across temperature and supply voltage, so a nominal 1,000 ms timeout is actually 681 ms at 47 kHz and 1,882 ms at 17 kHz. The loop's pass time, in the state machine version, is 200 µs, a margin of 3,405 times against the worst case of 681 ms. In the flag version, two blocking delays of 500 ms each in sequence make a pass of 1,000 ms, past the worst case, and the watchdog resets a device that was working, because the code's waits ate the margin invisibly. The state machine does not make the watchdog more forgiving. It makes the pass time short enough that the tolerance does not matter.

Step 5, the update. The update is the state the state machine has to survive, from Article #11, and the survival is architectural, not lucky. The flash is 2,048 kB: bootloader 64 kB, slot A 512 kB, slot B 512 kB, metadata 4 kB, config, calibration, and logs 64 kB, 1,156 kB used, 892 kB spare. The A/B scheme costs exactly one extra slot, 512 kB, and its payoff is that a bad build costs one reboot instead of one scrapped unit. The state machine's UPDATING state writes the new image to slot B, verifies the signature and the hash, flips the boot pointer atomically, and reboots into B marked "trial". If the power fails mid-write, the pointer was never flipped, and the bootloader boots slot A, the old image, working, and the new image's state machine starts in IDLE, not in UPDATING, because the state is re-derived from the boot pointer, not from a flag saved in RAM. The flag version has no such recovery: if is_updating is in RAM, it is lost on reset, and the device does not know an update was in progress; if it is saved to flash, the flag can say "updating" while the image is half-written, the impossible combination again, this time written to non-volatile memory, where it survives the reset that was supposed to clear it. The state machine survives the power failure because the state is derived from something that cannot be half-written. The flag survives the power failure as a lie.

The decision test

Five questions. Answer them honestly, and the architecture designs itself.

  1. How many flags does the firmware have that describe the same behaviour? If the answer is three or more, the combinations already outnumber the states, and the impossible combinations are already in the code. Count them: two to the power of the number of flags, minus the number of real states, is the number of states the product must never be in, and the code can represent all of them.
  2. Can you draw the behaviour on a wall? If the diagram has more arrows than a new engineer can follow in a minute, the behaviour is not designed, it is accumulated. The diagram is the test: if it cannot be drawn, it cannot be understood, and if it cannot be understood, it cannot be tested.
  3. For every state, how do you leave here? If any state has no exit, above all the fault state, the device bricks itself the first time it fails, and the field will find it. The table answers the question, or the table shows that it cannot be answered.
  4. Does any task block? Search for delay and for any wait that does not return. Every blocking wait is a period during which no other task runs, no button is read, no radio packet is answered, and no watchdog evidence is produced. The wait must be a state, not a delay.
  5. Does any interrupt write the state? If yes, the state changes while the main loop is in the middle of deciding, and the transition log, if there is one, will show a change with no cause. Interrupts record events. Transitions happen in one place.

Objections

"State machines are overkill. Our firmware is simple." It was simple. Then the features were grafted on, and the flags multiplied, and the ifs nested, and the combinations outnumbered the states. The state machine is not overkill for simple firmware, it is the form simple firmware takes when it is written down. The rice cooker has four states. The vending machine has five. The hub has seven. The switch statement for seven states is shorter than the nested ifs for five flags, and it can be read by a new engineer in a minute.

"We need an RTOS. The super-loop can't do this." The super-loop can do this, and Article #13 measured the price of the alternative: the RTOS costs about 9 kB of flash and 0.03% of the CPU on the hub's STM32H7, nearly free, and it protects the 1 kHz deadline by preemption. But the RTOS does not fix the flags. A thread full of flags is spaghetti with a scheduler, and the scheduler makes it worse, because now the flags are tested by two threads at once. The state machine is the architecture. The RTOS is the scheduler. Buy the scheduler when the timing demands it, three or more independent concurrent activities with different timing requirements, from Article #13, and build the state machine either way, because the state machine is what the tasks are made of.

"A table of function pointers is too clever. The switch is fine." The switch is fine, and the article uses it. The table of function pointers is for when the states multiply, when one machine has twenty states and the switch is a thousand lines long. The table is the same design, in a different form: one row per state, one function per state, and the transition is a table lookup instead of a case. The cleverness is not in the table. The cleverness is in pretending the switch scales forever.

"Hierarchical state machines are academic. Nobody needs nesting." Nobody needs nesting until the second button is added, the defuse button that must work at any stage, including when the bomb has already started to time out, and then the flat machine needs the transition copied into every state, and every copy is a chance to get it wrong. The hierarchy is the fix for the copy-and-paste, and the copy-and-paste is the fix most teams actually use. The hierarchy is not academic. The repetition is the academic problem, and the field pays for it.

"Our firmware already works. Why rewrite it?" It works until the combination nobody tested arrives in the field, updating and sampling at the same time, or asleep and advertising at the same time, and then it does something nobody can explain, and the engineer who opens the code cannot say which part of it ran. The rewrite is not a rewrite of the behaviour. It is a rewrite of the representation: the same seven states, the same nineteen transitions, written as a table instead of as flags. The behaviour does not change. The ability to see the behaviour does. And the field failure that cannot be reproduced is the most expensive failure there is, because it cannot be fixed, only waited for, again.

"The state machine will be slower." It will not. The state machine's step is a switch on one variable and a test of the events valid in that state. The flag version's step is a walk through nested ifs, testing flags that do not matter in the current situation. The state machine tests fewer conditions, not more, because each state tests only its own events, and the loop's pass time, in the demo, is 200 µs against a 1 ms deadline. The cost of the state machine is one byte of RAM and a table. The cost of the flags is the field failure.

Sources

  1. Miro Samek, "How to Code a State Machine in C or C++"
    What we used

    "Most reactive programs start out fairly simple and well structured, but as features are grafted on, more and more flags and variables are introduced to capture the relevant event history. Then ifs and elses must be added to test the increasingly complex logical expressions built out of the various variables and flags (aka spaghetti), until no human being really has a good idea what part of the code gets executed in response to any given event." "When used correctly, state machines become powerful 'spaghetti reducers' that drastically reduce the number of execution paths through the code, simplify the conditions tested at each branching point, and simplify transitions between different modes of execution." "The behavior of most reactive systems can be divided into a relatively small number of non-overlapping chunks (states), where event responses within each individual chunk depend only on the current event, but no longer on the sequence of past events." "Instead of recording the event history in a multitude of variables, you can use just one 'state variable' that can take only a limited number of a priori known values." "The main challenge in programming reactive (event-driven) systems is to correctly identify the appropriate piece of code to execute in response to a given event. In all but the most trivial reactive systems, the response depends both on the nature of the event and, more importantly, on the history of past events in which the system was involved." "From the programming perspective, this dependence on context very often leads to deeply nested if-else or switch-case constructs." Checked via browser.search, 2026-11-20

  2. Miro Samek, *Modern Embedded Systems Programming*, Lesson 35 (state-machine.com): "Event-driven code cannot block, and instead has to *return* to the event loop after every event. Such code obviously loses the stack context between events. Consequently, event-driven systems tend to use less stack space, but instead require a different form of preserving the context from one event to the next. The most popular is the manual context management, where programmers invent flags and variables, such as your isLedOn flag."
    What we used

    "In a sequential code the waiting between events happened by means of BLOCKING inside the OSTimeDly() function. After receiving the expected event the code unblocked and simply handled the event, because you knew exactly which event was received. Therefore, you didn't need to do anything special to remember where in the sequence of events you were. This is an example of maintaining the context between events naturally and automatically." "This automatic context management is quite expensive, because, as you recall from the previous lessons about the RTOS, the context of a thread during a blocking call requires a whole private stack in the precious RAM." "But in most real-life programs, such approach is almost guaranteed to degenerate into unwieldy 'spaghetti' code." Checked via browser.search, 2026-11-20

  3. Miro Samek, *Modern Embedded Systems Programming*, Lesson 40 (state-machine.com): "Suppose that you need to extend your TimeBomb application to allow the user to defuse the bomb. Specifically, the second button on your TivaC LaunchPad board should now defuse the bomb and should turn the Blue-LED on to show that the bomb is defused. The most important aspect is that the defusing should work at any stage, including when the bomb already started to time out."
    What we used

    "All these problems are well known, because all of them stem from violating one of the most important principles of software development: the DRY principle, which stands for Don't Repeat Yourself." "A very elegant solution to this problem has been known since the late 1980's, when David Harel invented advanced state machines that he called 'statecharts'. The main innovation of 'statecharts' is the introduction of hierarchically nested states and therefore 'statecharts' are also called 'hierarchical state machines'." "The extension of the state machine formalism is to allow states to contain other states. Specifically, in your TimeBomb state machine, you can surround the four of the original states with another state, which you can call 'armed'." Checked via browser.search, 2026-11-20

  4. Miro Samek, *Modern Embedded Systems Programming*, Lesson 42 (state-machine.com): the hierarchical state machine example from Chapter 2 of *Practical UML Statecharts in C/C++, 2nd Edition*, with six nested states, "s", "s1", and "
    What we used

    S11" on the left and "s2", "s21" and "s211" on the right, "carefully designed to contain all possible transition topologies up to four levels of state nesting. This includes regular transitions between various levels of state hierarchy as well as internal transitions." Checked via browser.search, 2026-11-20

  5. Ground Embedded, "State machine: the basic tool of firmware design" (opened in full, 2026-11-20)
    What we used

    "Organise a complex sequence of behaviour as states and transitions. When flags multiply and the if statements start to tangle, it is time to switch to a state machine." "A rice cooker is in one of a few states: standby, cooking, steaming, keeping warm. When an event such as a button press or a temperature arrives, it moves to the next state. Most firmware can be expressed this way." "Draw the state diagram first. Circles are states and arrows are events and actions." "For every event in every state, 'what happens' has to be defined. Undefined combinations become bugs." "Separate what is done once on entering a state (entry) from what is done while staying in it." "Always provide a way out of the fault state." "When the states multiply, manage the per-state handlers in a table of function pointers, or extend to a hierarchical state machine with parent states." "Three flags such as `is_running`, `is_error` and `is_paused` make eight combinations. Among them the code ends up allowing combinations that must never exist, such as 'running and in error'. Keep the state in a single variable and only one can be true at a time." "A blank is itself a decision: 'that event is ignored in that state'. Filling in the table exposes missed cases such as 'what if the start button is pressed while decelerating?'." "Wait with `delay_ms(500)` and neither buttons nor communication are handled meanwhile. Record the time the state was entered and check the elapsed time on every pass." "With unsigned subtraction the difference is calculated correctly even when the tick counter wraps round." "Work that should happen only at the moment the state changes (starting a timer, sending a message) is handled at the transition, or by remembering the previous state and running only when it differs." "Too many states: give independent concerns (communication link state and motor state) separate state machines. Merged into one, they multiply by their combinations." "Changing the state directly in an interrupt: the state changes while the main loop is in the middle of deciding. Interrupts only record events; transitions happen in one place." "No way out of the error state: for every state, check in the table 'how do you leave here?'." "Hard to debug: log the previous and next state on every change and most field problems are solved." The vending-machine table: Idle / Start button / Accelerating / Turn the driver on; Accelerating / Target speed reached / Constant speed / None; Accelerating / Overcurrent / Error / Turn the driver off; Constant speed / Stop button / Decelerating / Target speed 0; Constant speed / Overcurrent / Error / Turn the driver off; Decelerating / Speed 0 / Idle / Turn the driver off; Error / Reset button / Idle / Clear the error. The switch example: `typedef enum { IDLE, RUN, FAULT } state_t;` with the `default` branch driving the output off and moving to the fault state, "towards the safe side if the value is corrupted"

  6. Keith Curtis, "Embedded multitasking with small MCUs: Part 1, State Machine Constructs"
    What we used

    EDN, 23 December 2006, adapted from Chapter 2 of *Embedded Multitasking with Small Microcontrollers*, as summarised by RottenWifi: "A state machine lets a small microcontroller handle several ongoing activities without stopping in a blocking wait: each activity performs a short step, saves where it should resume, and returns so the main loop can run other work." "The state variable is a logical program counter: it records where this process should continue next time. A state is a resumable point, and a transition moves the process between points. In a wait state, the task checks whether its condition is satisfied; if not, it reaches the end of the call and gives other work a chance to run. The `default` branch recovers from an invalid value." "This variable is not a saved thread stack. It does not automatically preserve local automatic variables, nested call frames, return addresses, or registers. Any values that must survive between calls need explicit storage, in static or global variables, or, preferably for reusable tasks, in a context object." "A blocking delay or wait ties up the caller until time passes or an event occurs. During that interval, unrelated code cannot run in the same thread of control." "This is cooperative, non-preemptive multitasking. No scheduler forcibly interrupts a task at an arbitrary point: each task must return voluntarily." "Curtis, then a principal applications engineer at Microchip Technology, published 'Embedded multitasking with small MCUs: Part 1, State Machine Constructs' on December 23, 2006. The article was adapted from Chapter 2, 'Basic Embedded Programming Concepts,' of his book *Embedded Multitasking with Small Microcontrollers*." EDN Part 2, "Cooperative vs. pre-emptive": "The three basic requirements of a multitasking system are: context switching, communications, managing priorities. To these three functions, a fourth, timing control, is required to manage multitasking in a real-time environment." Will Warner, EDN, "Non-pre-emptive multithreading performs embedded software's juggling act": "Non-pre-emptive multithreading employs a small 'kernel' loop and any number of individual threads. The kernel gives each thread its time slice in turn, manages delays for the threads, and uses events to manage interthread synchronization." "When a thread decides that it must wait for a certain program state or external event, the thread does not just spin and wait. Instead, the thread saves a phase value and returns to the kernel. When the thread runs again, it uses the phase value to find its place and pick up where it left off." Checked via browser.search, 2026-11-20

  7. embedded.com, "Introduction to UML statecharts"
    What we used

    "A finite state machine (FSM) specifies the events of interest to a reactive object, the set of states that object may assume, and the actions (and their order of execution) in response to incoming events in any given state." "The syntax for transitions is *event-name* (*parameter-list*) [*guard-condition*] / *action-expression*." "A Boolean expression inside the square brackets is a *guard*. If the event fires and the guard evaluates to TRUE, the transition is taken; if the guard evaluates to FALSE, the event is ignored." "Actions may also be executed when a state is entered (called *entry actions*) or exited (*exit actions*)." "Responses to events may have a time value associated with them (via a UML constraint), but they execute via run-to-completion semantics. This means that once an event is received and the object determines that it will respond to it, the object will execute first the exit actions of the predecessor state, followed by the transition actions, and finally the entry actions of the subsequent state, in that precise order, and not process any new events until that set of actions has completed." "The states NIBPEnabled and NIBPDisabled are called *or-states* because the object is in either one or the other. Also note that NIBPEnabled is a *composite state* with *nested states* inside; these nested states are or-states with respect to each other. The ability to nest states greatly enhances the applicability of statecharts to more complex problems." Checked via browser.search, 2026-11-20

  8. Cluster figures reused from earlier articles in this series (read back from the files, not re-derived): Example Labs field sensor hub
    What we used

    100 × 60 mm four-layer board, 120 nets, 114 test points, STM32H7 at 48 MHz, 512 kB SRAM / 2 MB flash, BLE radio at 2.4 GHz, sensor rail 3.0 V, CR2032 230 mAh = 2,484 J, sealed 100 × 60 × 40 mm enclosure (#32, #12, #6, #41); the hub's four jobs, control loop at 1 kHz, BLE every 7.5 ms, on-device model once a second, OTA flash writes, CPU 20.5% (control 10% + BLE 6.7% + model 3.8%), super-loop leaves the control loop 2.6 times late, 40.6 times late when the model runs, 385 samples a second instead of 1,000, RTOS costs about 9 kB of flash and 0.03% of the CPU (#13); watchdog, never kick from a timer interrupt, kick from evidence, STM32 IWDG from a 32 kHz LSI varying 17–47 kHz so a nominal 1,000 ms timeout is actually 681–1,882 ms (#37); OTA, flash 2,048 kB, bootloader 64 kB, slot A 512 kB, slot B 512 kB, metadata 4 kB, config, calibration, and logs 64 kB, 1,156 kB used, 892 kB spare, 1% canary = 10 units of 1,000 (#11); PID, output = Kp × error + Ki × (error added up over time) + Kd × (how fast the error is changing), Ziegler–Nichols 1942 (#3); BLDC driver, 48 V, six MOSFETs, 1 mΩ shunt at 10 A → 10 mV, CSA gains 5/10/20/40 V/V, DRV8323/DRV8305 (#23); FOC, current loop 16–64 kHz, speed loop 1–5 kHz (#57)

More in software meets hardware

Watchdog timer embeddedA watchdog is a counter on its own clock that resets the processor when it reaches its limit, but a watchdog kicked from a timer interrupt is faithfully serviced by a dead product, and a reset that returns the device to the state that hung it produces a boot loop, which is worse than a freeze. The internal watchdog is free and shares the die; the external supervisor IC costs $0.30–$1.50 and cannot be disabled by confused firmware. This is the watchdog, the brownout, and the recovery ladder, worked through on the Example Labs field sensor hub.28 minThe Update That Cannot Brick the DeviceAn over-the-air firmware update can either extend the life of an embedded product or brick an entire fleet in the field. The difference is not the transport, it is the architecture: A/B partitions so the old firmware is never touched until the new one proves itself, signing and secure boot so the image is yours and unmodified, trial-then-confirm so a bad build reverts automatically, and a staged rollout with a kill switch. The safety costs one extra flash slot, 512 kB on a 2 MB chip, and its payoff is that a bad build costs one reboot instead of one scrapped unit, one truck roll, and one lost customer.11 minAlarm managementA plant with 400 configured alarms, 60 active at any moment, and operators who acknowledge the horn without reading the screen. EEMUA 191 says an operator can handle about one alarm every ten minutes, and at Milford Haven, two operators had to recognise, acknowledge, and act on 275 alarms in 11 minutes. The warning is a design problem, not a discipline problem.27 minDesign for TestA board that passes the bench and fails in the field is usually untestable by design, no test points, no JTAG chain, no fixture. The IPC-9252A rules, the four test methods, the coverage union, and what a fixture is worth on a real board.20 minTest It to DestructionHALT is a discovery test that steps stress far beyond specification to find weaknesses and measure operating and destruct limits, it is not a pass/fail qualification test. A complete HALT needs 4–8 units and 3–5 days; a field failure costs 100 times what a design-stage fix costs.21 min