I work at a restaurant that does events and we received some challenging last-minute requests from a client. We scrambled to fill them. I realized later that these requests were likely made by an AI agent, and that the human client might not have personally cared. This felt bad!
a world where everyone has a perfectly rational, tireless optimizer working on their behalf is much more adversarial than people realize. a lot of our existing infrastructure will not survive this transition.