We burned $40,000 on AI in 8 days. Here's exactly how it happened.

A war story from the PageFly team — and what we're changing because of it.

The screenshot that ruined our week

Last week, one of our engineers casually opened an internal AI account to check something unrelated. What they saw made them stop breathing for a second.

$40,000. In 8 days.

For context: this account had been ticking along quietly for months at a tiny fraction of that. Then on March 25th, AI token usage spiked 100x overnight — and just kept going. For over a week. And nobody on our team noticed.

This is the story of how that happened, why we didn't catch it, and what we're doing so it never happens to us — or to you — again.

We're sharing this raw because we think you deserve to know how the sausage gets made when an app you rely on ships AI features. And because every Shopify merchant building with AI right now is one config mistake away from a story like this.

The setup: a feature that worked too well

Back in October 2025, we shipped AI Sales Page — a feature that personalizes page content for every visitor in real time. Different headline for different intent. Different hero for different traffic source. The kind of thing that, when it works, lifts conversion in a way static pages never could.

We rolled it out to all PageFly users. No paywall. No usage caps. No per-store rate limit. We wanted merchants to try it, fall in love, and tell us what to build next.

For months, that worked fine. Adoption was steady but modest. Costs were predictable. We patted ourselves on the back and moved on to the next thing.

That was mistake number one: "predictable so far" is not the same as "bounded."

What we missed

Here's the thing about AI Sales Page that we, the people who built it, somehow underweighted in our own heads:

Every page load = one personalization call = one LLM inference.

For a store doing 500 sessions a day, that's nothing. For a store doing 500,000 sessions a month, that's a different planet entirely.

On March 25th, one merchant — a real, legitimate, doing-great-business merchant — had traffic surge to roughly 300,000 sessions. They had AI Sales Page enabled. They had no idea anything unusual was happening on their end (and nothing was — their store was working beautifully). On our end, the meter was spinning so fast it became its own weather system.

And we didn't see it. For 8 days.

Why we didn't see it (the embarrassing part)

This is the part that stings to write.

The AI workloads were running on an internal account specifically dedicated to AI usage. Because that account was expected to have AI costs, the alerting we had set up elsewhere didn't trigger meaningfully here. There was no "this is 100x your baseline, wake somebody up" alarm wired to the right pager. The dashboards existed. Nobody was looking at them, because nobody had a reason to look at them on any particular Tuesday.

It took a human, opening the account for an unrelated reason, to notice. If that hadn't happened, the bill would have kept climbing.

Let that sink in: our discovery mechanism was "somebody got curious." That is not a system. That is luck.

What we did the moment we found out

Disabled the storefront-side AI rendering for AI Sales Page immediately to stop the bleed. Painful, because it affects merchants who were using it correctly. Necessary, because the alternative was watching the meter keep running while we debated.
Pulled the full usage breakdown to understand exactly which calls, which prompts, and which traffic patterns drove the spike.
Started writing this post, because we'd rather you hear it from us.

The lessons (the ones we're actually acting on, not the LinkedIn-platitude kind)

1. "Per-request cost × unbounded requests = unbounded cost." Always. This is obvious in hindsight. It was obvious in foresight too, honestly. We just didn't internalize it because adoption was slow at first. Slow adoption is a trap — it lulls you into thinking the worst case is the average case scaled up linearly. It isn't. The worst case is one merchant going viral on TikTok.

2. Free beta ≠ no limits. If a feature can cost real money per use, it needs a cap per use, per store, per day — even if the cap is generous. "We'll add limits when we monetize" is how you end up with stories like this one. Limits are not a monetization feature. They are a safety feature.

3. Alerting on the account that's expected to spend money is the most important alerting you have. We had alerts on the wrong things. We were watching the quiet rooms and ignoring the room with the fireworks. Anomaly detection — "this is N× your 30-day baseline" — needs to live where the spending actually happens, not where you wish it didn't.

4. Personalization-per-request is a fundamentally different cost model than per-user or per-session. "One LLM call per page load" is an architecture that punishes your most successful merchants the hardest. The store going viral is exactly the store you don't want to be expensive to serve. Whatever we build next has to invert that — costs should scale with value created, not with raw traffic volume.

5. Sharing this is part of the fix. A bug you hide repeats. A bug you publish gets fixed by people you've never met, in companies you've never heard of, who read your post and check their own dashboards tonight. If this post makes one other team add a spending alert this week, it was worth writing.

What this means for you, our merchants

A few promises, in plain language:

You will never get a surprise bill from us because of this. The cost was ours. It stays ours. No merchant is being charged or penalized for the traffic that triggered this — including the merchant whose store actually drove the spike. They did nothing wrong. We did.
AI Sales Page, as it exists today, is being sunset. We're not going to patch a broken cost model and pretend it's fixed. The feature in its current form is done.
But the idea behind it isn't going anywhere — it's growing up. Everything we learned from AI Sales Page about what merchants actually want from AI on their store is being folded into something much bigger that we're shipping in Q2. We're not ready to talk about it yet. We will be soon. If you liked where AI Sales Page was heading, the next thing is going to feel like the version we should have built the first time.
When we ship AI features in the future, you will see the limits up front. No more "free and unlimited" framing on things that have real per-use costs. That framing was never honest, and we're done with it.
We will tell you when we mess up. Like we're doing right now.

If you want to talk about this post — argue with us, share your own war story, or just tell us we're idiots — the comments are open. We read all of them. Especially the harsh ones.

— The PageFly team