Safety Monitors for Neural Networks: Runtime Out-of-Distribution (OOD) Detection

Deep neural networks used in autonomous vehicle perception are fundamentally uninterpretable black boxes that often predict incorrect classes with catastrophic overconfidence. Runtime Out-of-Distribution (OOD) safety monitors evaluate intermediate feature embeddings, Mahalanobis distance metrics, and energy-based scores to detect when visual input diverges from training data, triggering safety envelopes before an accident occurs.

Calculating the Free Energy Score to Reject Hallucinations

Standard softmax outputs suffer from miscalibration: an out-of-distribution object like a kangaroo hopping on a Swedish highway is forced into an existing category (e.g. 'pedestrian') with 98% confidence. Energy-based safety monitors calculate Helmholtz free energy over the raw logit outputs: `E(x; f) = -T * log(Sum(exp(f_i(x)/T)))`. In-distribution inputs yield large negative energy values, whereas unknown anomalies exhibit high energy, allowing the safety manager to instantly veto the perception output.

Explore Domain Portfolio