I recall reading multiple AskHN posts about people trying to get attention from a cloud provider because they ran up thousands of dollars in charges accidentally. I've seen large companies do this too, even if you think something is just a dev environment, its the cloud provider's production environment and they will charge you per their ToS for everything you use, doesn't matter what the customer usage profile looks like.

Experienced dev with limited hands-on big tech infrastructure experience. Based on the results I get from LLMs in domains I understand how get they even get this stuff running using AI?

Monkeys and typewriters. Throw enough character input and "It's not working" into an LLM and it will eventually produce... something.

And since it tends to reach for the most web-represented solution, that means infinite redis caches doing the same thing, k8s, and/or Vercel.

Best mental model: imagine something that produces great tactical architecture, with zero strategic architecture, running in a loop.

I can accept this is true, they will for sure exist. Of course if this they ability to make choices, technical or not, they are completely doomed.