AI Bots Are Inflating Your Hosting Bill

TL;DR — AI crawlers from OpenAI, Anthropic, Google, and hundreds of smaller companies are hitting your site thousands of times per day. If your hosting provider charges per request (Vercel, AWS Lambda) or per bandwidth (Netlify), you're paying real money for bot traffic you never wanted. The fix: use a platform with fair, predictable pricing that doesn't penalize you for traffic you can't control.

The problem nobody's talking about

In December 2025, a developer wrote on DEV Community about why they left Vercel. The reason wasn't performance or features. It was AI bots.

Their Vercel bill spiked because AI crawlers were hitting their serverless functions thousands of times per day. Each invocation cost money. They had zero real users — but a $200 bill.

This isn't an edge case. It's the new normal.

The numbers are staggering

• 80% of web traffic is now bots (DesignRush, 2025)
• $238.7 billion lost by companies in 2024 due to bot-driven costs (DesignRush)
• 86% increase in invalid traffic from AI crawlers (DoubleVerify, Jan 2025)
• $116 billion/year in bot attack costs (DataDome, Jun 2025)

Kinsta, a major WordPress hosting provider, literally changed their entire billing model from visit-based to bandwidth-based because AI bot traffic made visit-based pricing unpredictable for their customers.

Google Cloud Run charged one developer $4,676 in 6 weeks with zero real traffic. That's not a billing error — that's bot traffic hitting a pay-per-request service.

Why usage-based pricing is broken

The fundamental problem: usage-based pricing assumes all usage is valuable.

When Vercel charges you per serverless function invocation, they don't distinguish between a real customer clicking "Buy" and a GPTBot crawling your /api/products endpoint 50,000 times.

When AWS charges per Lambda execution, they don't care if the execution was triggered by a human or by ClaudeBot indexing your docs.

When Netlify charges for bandwidth overages, they count every byte — whether it was served to a paying customer or to an AI scraper that will never convert.

You're subsidizing AI companies' data collection with your hosting bill.

Which platforms charge you for bot traffic?

How to protect yourself

Option 1: Block AI bots in robots.txt

User-agent: GPTBot
Disallow: /
 
User-agent: ClaudeBot
Disallow: /
 
User-agent: Google-Extended
Disallow: /
 
User-agent: CCBot
Disallow: /
 
User-agent: anthropic-ai
Disallow: /

Problem: Many AI bots ignore robots.txt. It's a suggestion, not enforcement.

Option 2: Use Cloudflare's AI bot blocking

Cloudflare launched specific AI bot blocking tools in July 2025.

Problem: Adds a dependency on another provider. Doesn't solve the billing model problem — you're still on usage-based pricing.

Option 3: Rate limiting at the application level

// Simple rate limiter middleware
const requests = new Map<string, number[]>();
 
function rateLimit(ip: string, limit = 60, window = 60000): boolean {
  const now = Date.now();
  const timestamps = requests.get(ip) || [];
  const recent = timestamps.filter(t => now - t < window);
 
  if (recent.length >= limit) return false;
 
  recent.push(now);
  requests.set(ip, recent);
  return true;
}

Problem: Only works if your app code runs before billing kicks in. On serverless, you're already charged for the invocation.

Option 4: Use a platform with fair pricing

The real fix: don't use a billing model that charges per request.

On RaidFrame, you pay per service — a flat, predictable amount based on the resources your app uses. Whether you get 100 requests or 100,000 requests, your bill stays the same.

Bot traffic doesn't cost you extra. Period.

The Vercel exodus is real

People are actively leaving Vercel because of this. Search results from the past 3 months:

• "Why We Left Vercel and Switched to Self-Hosting" (DEV Community, Dec 2025) — AI bots were the trigger
• "Vercel alternative or avoid $20/m" (Reddit r/nextjs, 120+ comments, Mar 2026)
• "Migrating from Vercel" (Reddit r/coolify, people moving to self-hosted)
• "I moved away from Vercel" (YouTube, 830+ views)

The destination? Mostly self-hosted solutions or flat-rate platforms like RaidFrame. People are done paying for traffic they can't control.

What about the "free tier" excuse?

Some platforms argue their free tier absorbs bot traffic. But:

• Vercel's free tier has 100GB bandwidth and 100,000 function invocations. AI bots can burn through both in days.
• When you exceed free limits, Vercel's Pro plan costs $20/user/month — and bot traffic counts against your paid limits too.
• Netlify's free tier has 100GB bandwidth. Same problem.

A "free tier" that gets consumed by bots isn't really free for you.

FAQ

How do I know if bots are hitting my site?

Check your analytics. If your server logs show traffic from user agents containing "bot", "crawler", "spider", "GPTBot", "ClaudeBot", or "Bytespider", that's bot traffic. On most sites, it's 50-80% of total requests.

Will blocking AI bots hurt my SEO?

No. Google's main crawler (Googlebot) is separate from Google-Extended (their AI training crawler). Block the AI crawlers, keep Googlebot allowed.

Is this really that expensive?

A site getting 10,000 bot requests/day on Vercel Pro uses ~300,000 function invocations/month. That's real compute cost — and it scales with every new AI company that starts crawling.

Does RaidFrame block AI bots automatically?

RaidFrame includes DDoS protection at the network edge. For AI bot blocking specifically, you can configure your app's robots.txt or use middleware — but importantly, bot traffic doesn't affect your bill either way.

What if I need usage-based pricing for my own customers?

That's different — usage-based pricing for YOUR product makes sense. The problem is when your HOSTING provider charges YOU based on traffic you can't control.

The bottom line

• Usage-based hosting pricing is broken in a world where 80% of traffic is bots
• AI companies are effectively making YOU pay for their data collection
• Flat, predictable pricing means bot traffic is the platform's problem, not yours
• Deploy on RaidFrame — your bill doesn't change when the bots come

Related reading

• Cloud Cost Optimization: Cut Your Bill by 40%
• Zero Downtime Deployments
• Auto-Scaling Cloud Infrastructure