A public demonstration of anomaly reasoning that separates genuine behavioral change from data-quality problems, explains the evidence behind each alert, calibrates confidence on held-out synthetic data, and explicitly declares when available evidence is insufficient.
False-alert rate fell from 13.6% for the simple baseline to 0.88% while maintaining comparable anomaly recall.
The animation deliberately separates four states that are often collapsed into one anomaly score: normal behavior, insufficient evidence, source-integrity warning, and genuine behavioral anomaly. Watch the decision card and evidence bars as the data conditions change.
The benchmark evaluates five separate properties: behavioral detection, false behavioral alerts under nuisance conditions, evidence alignment, confidence calibration, and safe abstention when the evidence becomes too sparse.
Held-out evaluation across normal and anomalous synthetic episodes.
Relative reduction in false-alert rate across held-out negative conditions.
Expected synthetic cause appeared in the thresholded evidence set when a true anomaly was detected.
Confidence calibration evaluated on episodes disjoint from the calibration set.
Knowledge-limit response when severe missingness left only one reporting source.
| Metric | Simple baseline | Explainable method |
|---|---|---|
| Precision | 90.1% | 99.3% |
| Recall | 92.7% | 92.4% |
| F1 score | 91.4% | 95.7% |
| False behavioral-alert rate | 13.6% | 0.88% |
| Expected Calibration Error | 11.3% | 0.9% |
The public method does not treat every disagreement as a behavioral anomaly. It first asks whether the observations are sufficiently complete and mutually consistent.
The comparator is deliberately transparent rather than another complex model.
The demonstration uses a deliberately simplified public-facing method.
Ingest time-stamped observations, source availability, and recent track history.
Measure missingness and disagreement before interpreting the behavior itself.
Evaluate heading, route, speed, and loiter evidence against a learned normal pattern.
Calibrate confidence and abstain when available evidence does not support a reliable decision.
Return the decision type, supporting evidence, data-quality context, confidence, and declared limitations.
Measure whether data-quality failures are separated from genuine behavior changes rather than treated as the same event.
Evaluate whether predicted confidence matches observed correctness on held-out synthetic conditions.
Compare reported evidence against the known synthetic cause injected into each anomaly episode.
Measure abstention behavior when missing observations push the system beyond its evidence limits.
Test whether cross-source inconsistency triggers a data-integrity warning without automatically creating a behavioral alert.
Use deterministic seeds, fixed class definitions, explicit metrics, and a held-out split so the synthetic benchmark can be rerun.
Distinguish abnormal movement from missing reports, sensor disagreement, or temporary coverage gaps.
Provide confidence-aware anomaly reasoning when robotic perception sources disagree or become unavailable.
Explain abnormal process or asset behavior while separating equipment faults from telemetry-quality problems.
Reduce nuisance alerts by distinguishing actual behavioral changes from sensor outages and inconsistent readings.
Explain unusual routing, dwell, speed, or custody behavior while accounting for incomplete tracking data.
Expose reasons, uncertainty, evidence quality, and knowledge limits before a downstream user acts on an alert.
Technical direction, multi-source data architecture, evaluation-harness design, data-quality instrumentation, reproducibility, performance analysis, integration, and end-to-end research execution.
Anomaly modeling, robust AI, confidence calibration, predictive behavior analysis, explanation design, adversarial/deceptive-data analysis, and AI/ML evaluation.
NIST identifies explanation, meaningfulness, explanation accuracy, and knowledge limits as core principles for explainable AI systems.
Public reference ↗The AI RMF calls for evaluation of validity, reliability, resilience, safety, explanation, documentation, and behavior beyond knowledge limits.
Public reference ↗NIST's 2025 report provides a current taxonomy for adversarial ML risks and mitigations across predictive and generative AI systems.
Public reference ↗