ARC-D2

AI for Automatic Highway Condition Detection

ARC-D2: AI-based automated detection of road surface damage across Germany's 13,000+ km Autobahn network — from data acquisition and model development to edge deployment on NVIDIA Jetson NX.

Request a DataLab
Funded R&D | Funder: BMDV | Budget: €839K | Year: 2026 | Role: Consortium Coordinator

Problem Statement

Germany’s Autobahn network — over 13,000 km of roadway — is a critical economic infrastructure that must be continuously maintained to withstand daily traffic demands. Currently, condition assessment relies on manual evaluation of road surface images, which is time-consuming, expensive, and inconsistent. Early detection of cracks, seams, and other surface damage is essential to initiate timely maintenance and prevent costly long-term damage.

Project team at BASt during the kickoff meeting in Bergisch Gladbach.

Project team at BASt during the kickoff meeting in Bergisch Gladbach.

Use Cases

The project addressed two primary use cases in collaboration with the Federal Highway Research Institute (BASt):

  1. Use Case A: Crack and Seam Detection in Road Surfaces (Palaimon) — Automated detection of cracks and seams in asphalt and concrete road surfaces using AI-based segmentation and anomaly detection.

  2. Use Case B: Damage Detection on Metal Traffic Signs (CAU) — Detection and condition assessment of traffic signs, developed by the University of Kiel (CAU).

Project Goal

The goal of ARC-D2 was to develop an AI-based application for automatic damage detection in road environment data, integrating seamlessly into existing systems used by highway authorities. Palaimon GmbH focused on road surface condition analysis including anomaly detection, with edge deployment capabilities, while the University of Kiel (CAU, AG Intelligent Systems, Prof. Tomforde) developed AI models for traffic sign anomaly detection.

Approach & Methodology

Data Acquisition: The project secured access to one of the largest externally used road surface datasets — approximately 8 TB of imagery from the IT-ZEB server, managed by HELLER Ingenieurgesellschaft mbH. Data was collected via MEFA and MESOB measurement vehicles operated by BASt, capturing road surface images under diverse weather conditions, times of day, and seasons.

Road Surface Segmentation: Palaimon developed and evaluated segmentation models for identifying cracks and seams in road surfaces. Detectron2 provided a mature baseline, while SegFormer delivered promising results tested under multiple configurations including LoRA and DoRA fine-tuning alongside full layer fine-tuning.

Semi-automatically annotated road surface images showing cracks and seams — used as reference dataset for model training.

Annotated road surface images — cracks and seams marked for model training.

Anomaly Detection on Road Surfaces: For anomaly detection, EfficientAD was implemented for fast and accurate anomaly detection, and PaDiM (Patch Distribution Anomaly Mapping) was investigated for anomaly detection and localization through patch distribution analysis.

Comparison of anomaly detection model results — anomalous areas (here filled and unfilled cracks) are clearly identified. The red marks indicate where a simple global threshold would only partially detect anomalies.

Comparison of anomaly detection model results — anomalous areas are clearly identified.

Annotated road surface image showing seams and cracks — part of the specification dataset used for model validation.

Annotated road surface image — seams and cracks marked for model validation.

Edge Deployment: Deep Learning models were converted to ONNX for cross-platform compatibility and deployed on NVIDIA Jetson NX, enabling low-latency, real-time inference directly in vehicles without cloud dependency.

NVIDIA Jetson NX

NVIDIA Jetson NX — the edge device used for real-time AI inference in vehicles.

Dashboard for Results Visualization: Palaimon built an interactive dashboard to present AI detection results to end users. The dashboard enables highway authorities to visualize detected damage locations on a map, filter by road section, date, damage type, and severity, and review annotated images with model predictions. It serves as the primary interface for communicating the AI’s findings to operators and maintenance planners.

The dashboard also implements a human-in-the-loop workflow: users can review the AI’s detection results directly in the UI and provide feedback — confirming correct detections or flagging false positives and missed damages. This user feedback is fed back into the training pipeline, enabling continuous model improvement over time.

Human-in-the-loop workflow: AI detection → user review → feedback → model retraining.

Human-in-the-loop pipeline — user feedback drives continuous model improvement.

Key Achievements

Project Details

Project Partners

Associated Partners