I read that differently.

I parsed it as having the bot only output “sorry, I can’t answer your question”. Ever.

My own “AI horror” is entirely based on class struggle, although from the other side. I have a P(unemployable forever | starve to death under a bridge), though I don’t call it that and don’t think it’s something I can quantify with a number.

I don’t know how much of this is due to realistic concerns, and how much of it is due to my brain being burnt out from having been on high alert since 2020.

(For existential risk: Climate change, war, societal collapse (and AI is existentially horrifying inasmuch as it can play a role in accelerating either of these). Worrying all the time about everyone being doomed, I think, got imprinted into my entire generation in childhood.)