Real-Time Anomaly and Incident Detection based on YOLO and Lucas-Kanade Optical Flow Tracking

Automatic traffic accident monitoring is critical for modern smart city CCTV systems, yet most existing pipelines either treat every frame independently or merge successive collisions into a single alert, hampering timely response and forensic review. We present a novel end-to-end framework for car accident detection and localization that integrates state-of-the-art detection, lightweight tracking, and incident-level management. The system employs a two-stage YOLO-based architecture, in which YOLOv8 serves as a generic vehicle detector to classify objects such as motorcycles, cars, and trucks on the road, and YOLOv12 fine-tuned in crash imagery is used specifically for accident detection. The YOLOv8 detector is updated every n frames to maintain real-time performance. A Lucas–Kanade optical flow tracker (LK-FT), enhanced with IoU and center-distance matching, bridges missed detections and extracts rich spatio-temporal cues. Abnormal motion is flagged when either (i) instantaneous speed drops below 15% of the recent average or (ii) heading changes exceed 50°, with configurable thresholds. The accident boxes detected by YOLOv12 are then fused with tracked vehicles (IoU > 0.1) to identify the objects involved in the incident. A cooldown-based AccidentEventManager module clusters temporally related detections into distinct incidents, effectively separating multiple collisions that occur in close succession. For each incident, the frame with the highest accident confidence is reanalyzed to classify the type of crash and rendered with Thai-language overlays using Pillow and Sarabun fonts. The pipeline processes 1080p footage at approximately 8 FPS on a single NVIDIA GPU, distinguishes incidents separated by at least 5 seconds, and outputs: (i) an annotated video stream and (ii) per-incident summary images for rapid reporting. All parameters, detection thresholds, IoU values, motion heuristics, cooldown duration, class sets, and visualization styles are consolidated into a single configuration block to facilitate domain adaptation. The proposed system achieves an overall detection accuracy of approximately 87% under various traffic conditions.