A human driver uses two eyes, two ears and years of experience. A self-driving car replaces them with a ring of sensors and software that turns millions of measurements per second into three decisions: where to steer, when to accelerate and when to brake. In several US cities this already works without anyone behind the wheel, and every crash of these cars is reported to the federal road safety authority.
This is part 2 of Level 0 in our Physical AI learning path. You will learn what the levels of driving automation mean, which sensors a self-driving car uses and what each one sees, how the software gets from sensor data to steering and what public crash reports say about how these cars actually get into accidents.
Six levels of automation
“Self-driving” is used for very different systems. The standard SAE J3016 from SAE International, an association of automotive engineers, sorts them into six levels. The key question is who is responsible for watching the road.
At Levels 1 and 2, the car helps: it keeps its distance and its lane, but you drive and must watch the road at all times, even with your hands off the wheel. From Level 3, the system drives itself in defined conditions, and at Level 4 it needs no human at all inside its operational design domain (ODD): the area, roads, speeds, times and weather it was built and tested for. A robotaxi that serves one city is a Level 4 vehicle. Level 5, a car that drives anywhere a human could, does not exist yet.
The sensors and what they see
No single sensor sees everything, so most self-driving vehicles combine several types. Each has strengths where another is weak.
| Sensor | What it measures | Strengths | Weaknesses |
|---|---|---|---|
| Camera | Color images, like an eye | Reads signs, traffic lights and lane markings; cheap | Needs light; dazzled by low sun; distance must be estimated |
| Radar | Distance and speed via radio waves | Works in rain, fog and darkness; measures speed directly | Coarse image; hard to tell what an object is |
| Lidar | Distance via laser pulses, as a 3D point cloud | Precise 3D shape of everything around the car | Expensive; weaker in heavy rain, snow and fog |
| Ultrasonic | Distance via sound, very close | Parking and maneuvering | Range of only a few meters |
| GNSS and inertial sensors | Position and motion of the car itself | Knows where the car is and how it moves | Satellite signal lost in tunnels and between tall buildings |
Combining the readings is called sensor fusion. If the camera is dazzled by the sun, radar still measures the car ahead. If radar sees an object but cannot tell what it is, the camera can. Whether a car could drive on cameras alone, like a human, is one of the big open debates in the industry; most robotaxis that drive without a safety driver today combine cameras, radar and lidar.
From sensor data to steering
The software of a self-driving car has four classic stages. Perception finds and labels objects in the sensor data: cars, cyclists, pedestrians, cones. Prediction estimates where each of them will be in the next seconds. Planning chooses a path and a speed that are safe and comfortable. Control turns that plan into steering angle, throttle and brake pressure. Most stages use machine learning, especially perception, where neural networks detect objects in camera images and lidar point clouds.
A newer approach replaces several stages with one large model that goes from camera images straight to driving commands, called end-to-end driving. It learns from many hours of recorded driving, much like a language model learns from text. It can handle situations that are hard to describe in rules, but it is harder to check why it made a decision.
Many robotaxis also carry a detailed HD map of their area, with every lane, curb and traffic light. The car compares what it sees with the map to know exactly where it is, which is one reason why such services start in one city and expand street by street.
When the car is unsure
Even a Level 4 vehicle meets situations it cannot resolve on its own: a road blocked by a fallen tree, a police officer giving hand signals, a construction site that is not on the map. Two safety nets exist for such moments. First, the car can perform a minimal risk maneuver: it slows down and stops in a safe place, ideally at the side of the road. Second, most robotaxi operators have staff in a remote center who can answer questions from the car, for example by confirming which way to drive around an obstacle, while the car still steers itself. For passengers, this means that a robotaxi will sometimes stop and wait. That is safer than guessing, and it may be one reason why so many reported crashes involve a vehicle that was standing still.
What the crash reports show
In the US, every crash of a vehicle with an automated driving system has to be reported to the National Highway Traffic Safety Administration (NHTSA). We analyzed all 1,551 reports filed since June 2025. In 83.3% of them, nobody was in the driver’s seat, and in 50.9% the automated vehicle was standing still when the crash happened.
| Movement | Reports | Share |
|---|---|---|
| Stopped (traffic, light) | 660 | 42.6% |
| Driving straight | 543 | 35.0% |
| Parked | 118 | 7.6% |
| Turning left | 68 | 4.4% |
| Turning right | 42 | 2.7% |
| Changing lanes | 37 | 2.4% |
| Backing up | 26 | 1.7% |
| Passing | 21 | 1.4% |
Data: NHTSA, Standing General Order 2021-01, crash reports for vehicles with an automated driving system (ADS), crashes from June 2025 to August 2026, latest version of each report; retrieved on October 18, 2026, our analysis.
This is one lesson of the data: in many crashes the automated vehicle was not moving, so another road user ran into it. Whether the robot car had stopped in an unexpected place, the reports do not say. They also do not say who caused a crash, and they contain no miles driven, so they cannot tell you whether robotaxis are safer than human drivers. The full analysis with all tables is on our page about self-driving car crash data.
Self-driving cars are robots on wheels. The next part of Level 0 looks at robots that grasp and walk, and how they learn.