Since the anomaly occurred a month ago, the mission team and operators have been working hard to identify its root cause. Their investigation suggests that the issue was caused by a situation very unlikely to happen – two nearly simultaneous events onboard the spacecraft caused a ‘semaphore’ in the software to remain permanently locked, revealing a software design vulnerability that would have been extremely difficult to detect during testing.
Semaphore in space
A semaphore in real‑time software is like a traffic light for tasks. It is a simple counter that controls which part of a system is allowed to use a shared resource (such as data, a device, or a piece of hardware) so that tasks don’t interfere with each other. If the semaphore says “green”, a task may proceed; if it says “red”, the task must wait.
When a task finishes its work, it releases the semaphore so another task can go next. In real‑time systems, semaphores are especially important because they let tasks coordinate safely and predictably, ensuring that time‑critical software doesn’t access the same thing at the same time and cause errors.
The critical moment
During the weekend of 14-15 February, Proba-3’s Coronagraph spacecraft (CSC) was about to perform a routine operation: reaction wheel desaturation. When a spacecraft’s reaction wheels are reaching their maximum spinning speed, the software detects it and commands the use of propulsion thrusters to counteract the torque created by slowing the wheels down, without rotating the spacecraft.
Before use, the thrusters need to be at the correct temperature. When the software function in charge of the thruster warm-up was running, an equipment temperature crossed an allowed threshold. This caused the function to be interrupted at the exact moment when it was handling some semaphores locking, placing the semaphores in an unforeseen combination of states.
The blocked semaphore prevented the reaction wheel desaturation from taking place, which caused the spacecraft to progressively lose its correct orientation. A few hours later, its solar panel was no longer facing the Sun, and the entry into safe mode commanded by the onboard failure recovery logic was blocked.
Unfortunately, the incident unfolded during a night from Sunday to Monday with no personnel present at the Proba-3 Mission Control Centre, as the mission has been in autonomous routine operations since June 2025.
The statistical likelihood of this set of events occurring at the same time is so extremely low that it is virtually impossible. And yet it happened.
A vulnerability like this was never detected during Proba-3 ground testing or in-flight operations since the launch in December 2024. Because such rare timing conditions are extremely difficult to reproduce in testing, it is unclear whether more advanced verification methods would have detected the issue, but ESA is now exploring AI-based software analysis tools and reviewing similar software architectures across other missions.