Stories you never want to feel on your own skin
Conclusion: Always set billing caps and alerts on cloud API keys. A compromised key without spending limits can bankrupt you overnight.
tldr: A stolen Google Cloud API key generated $82,314 in Gemini charges in 48 hours — normal monthly spend was $180.
Contents of the blog are themselves written by LLM.
https://github.com/coollabsio/llmhorrors.com/blob/main/CLAUD...
The whole website seems to be focused on promoting the author and their projects more than sharing the information. Just link to the original.
https://www.reddit.com/r/googlecloud/comments/1reqtvi/82000_...
Posted to HN twice recently.
I would expect it to not be written by an LLM. Molly White didn’t run Web3 is Going Great on the blockchain.
And looking at her main website https://www.citationneeded.news/ there is a tip jar but it doesn't accept crypto. I was expecting her to take at least the major coins like Ada, Eth and BTC, but she's consistent with her views.
False equivalence, Tesla also does not run their website from a Model S.
The joke is, LLM Horrors is anti-LLM, Web3 is Going Just Great is anti Web3. The equivalent for Tesla would be Tesla putting a ICE inside their model 2 if they didn't believe in EVs.
Another plea for @dang to integrate pangram into all story and comment submissions
Yeah, right...
> Conclusion: Always set billing caps and alerts on cloud API keys.
Sadly, way easier said than done in the case of GCP. Been a proper reason for me to avoid GCP deployments with LLM use-cases for smaller projects.
I remember looking into this a while back assuming it would be a sane feature to expect. But for some reason it's surprisingly non-trivial with GCP to set budgets. Especially if the only thing you want is a Gemini API key with finite spending.
IIRC you could either set (rate) limits on quotas, but quotas are extremely granular (like, per region per model) meaning you need to both set tons of values and understand which quotas to relax. Or alternatively do some bubblegum-and-ducktape like solution where you build an event-driven pipeline to react to cost increases in your own project.
I understand that exact budgets are hard to enforce in real-time, especially for their more complex infra offerings.
However, (1) even if it's not exactly real-time, but instead enforced every hour that's already going to go a long way, and (2) PAYG LLM usage is billed rather linearly by the amount of tokens you use, so if there would be an easy way to set a dollar-amount and have that expressed as budgets that would already get you part of the way there.
Anyway, the current state of GCP budgeting it makes me avoid it for production usage until I'm ready to commit spending significant effort to harden it. For all small projects, the free tier tokens are a safe bet, but their extremely low rate-limits make them rarely a good fit.
Yeah, it's an utter joke and a UX/UI crime that has been going unpunished for way too long. Wonder what all those geniuses are doing.
Thankfully Google has some basic protection for it. I accidentally commited my google api token, as part of some OTEL trace JSON file, and within a few minutes my key was automatically locked by google, and marked as leaked (with exact link pointing where it has happened).
"some basic protection" it wasn't always like this. A few years back you could easily get api keys for any web service by typing certain keywords on github and that included all google APIs, but since the Microsoft acquisition it's not as simple anymore....
