Oh yeah, it can work great. And it can work terribly. We haven’t hit the point where it’s reliably “great” though. And that makes it rather more dangerous to me since it builds a sense of security that is unwarranted (not that I’m saying you disagree I’m just expanding on my distrust).

One of the major problems is that the failure modes can be very different from how a person fails. Like when you see a car just sitting in the middle of a road because it can’t figure out what to do for some reason. A person you could wave on. An AI you can’t. We understand human behavior but can’t really understand the AI decision-making process.

This is why I can’t quite get behind the “all AI needs to do is be slightly better than people” argument. On one hand, from a purely statistics pov, I get it I. But if self-driving cars were “basically perfect” except that every-now-and-then one of them randomly exploded (still killing fewer people than auto accidents) would people be okay with that? Automobile accidents aren’t truly “random” like that.