I built a portfolio site, as many developers do, to showcase some of my work (hey, you're actually on that site, though I had to remove a feature today to mitigate the attack). Having had tech only as a hobby, and eyeing a career change, I count the portfolio as a rather important piece of job applications. It’s the one place (outside of GitHub, but a lot of my better repos are private as they involve proprietary business products) I can show I know what I’m doing. I’ve had a portfolio site for several years now, and generally update it once a year (this year I gave it a full overhaul in terms of design, added some interactive features, and feel pretty good about it).
Then today happened. I got an email from Vercel warning me about going over my tier limits. Naturally, being a personal portfolio that generally doesn’t’ get more than five visitors per day, it’s on a free hosting tier. I was a bit surprised, but figured it might be a good thing! Maybe one of my business sites was suddenly going viral. Imagine my surprise when I found out the project that was seeing all the absurd amounts of server requests was my personal portfolio. Then imagine my outrage when I found it was not just a lot of recruiters looking, but what appeared to be a DDOS attack stemming from Meta crawlers. While some bots crawl aggressively, industry standards suggest 1–2 requests per second is typical. Meta’s crawler hit my site with thousands of requests per minute, far beyond what’s considered normal. Whether intentional or not, the impact was indistinguishable from a denial-of-service attack.
It’s becoming more common knowledge (to devs, anyway) that Facebook has a problem with the way it accesses sites when links are provided in messages and posts. When a link is posted, it basically sends a number of requests to that site to grab data. The shit of it is that bad actors have used this to DDOS sites simply by using Facebook Messenger, and Facebook has known about this for quite some time, yet refuses to do anything about it.
I rushed to stop the bleeding by introducing new middleware, a robots.txt file that disallowed access to known Facebook and Meta crawlers, and a captcha system. I thought that would be enough, but as it turns out, the requests per second (RPS) only increased. In fact, as I’m writing this, the DDOS is continuing, and the RPS rates are still increasing (I’ve already spent seven hours on this today, so here’s my writing break). Currently I’m being hit with about 2000 requests per minute. On a personal portfolio site that’s never had more than 10 visitors in a day for all the years it’s been up, it's pretty staggering.
I started looking into what Vercel could do for me, as they promise DDOS protection. However, in looking further into Vercel’s WAF (firewall) and even their Attack Challenge Mode (meant to halt DDOS attempts in progress), they explicitly whitelist all Facebook and Meta crawlers (without any way to change that). In other words, Vercel, the company that explicitly promises to protect my sites against attacks, is actively allowing the current attack to continue. I tried to get chat support, but surprise, only got a bot(!) that said it would submit a support ticket for me.
I’ve barely eaten today, I’m dehydrated, I lost hours I needed for other things, and despite all my efforts, this continues because Vercel can’t be bothered to keep their word, and Meta… well, Meta is just a pure asshole and nothing more.
The bigger concern is that these types of events are only increasing. Look on reddit, look on StackExchange, and while this isn’t new, it’s increasing in frequency. Meta is effectively waging war on smaller businesses by simply shutting down sites with its over-zealous crawler. They don’t listen to robots.txt instructions. They bypass middleware. They trounce on captcha. Everything designed to stop bots from being assholes to people, Meta crawlers bypass. Meta is knowingly damaging smaller businesses and site owners. And for what?? What the hell is Meta going to do with one page of my personal portfolio??
There is no consent. In fact they are explicitly disallowed from visiting my site, yet continue. They are just as sociopathic, and robotic, as the CEO behind these crawlers.
What can be done?? I literally have not found ONE actual solution online anywhere. People have found ways to mitigate this, but not stop it. Decreasing RPS from 50 to 5 may be helpful, but that’s still five pings per second using absurd amounts of server data.
What upsets me more is that I actually listened to a lot of interviews with Guillermo Rauch, the founder and CEO of Vercel, and creator of Next.js (my favorite web framework). Kind of an unusual story that some kid from a poor area of Argentina ends up creating and running a global business worth billions in the tech sphere. He talks SO MUCH about how his vision and goals are about helping developers, making things easier for developers. But I think today highlights an “inconvenient truth” for Guillermo. Vercel does NOT care about independent devs (the way Guillermo, himself, started out) just trying to learn and build, and offer something to the world. It does not protect them. It is actively allowing Meta to do this with its overarching whitelisting that doesn’t even allow me to block them when they’re DDOSing me. They protect corporations, not developers. I kind of felt Vercel was a bit of an underdog story for a long time, and while people complained about over-monetization and increasing costs some years ago, I actually moved to Vercel about that time because, at least in my experience, other sites were kicking independent devs even harder (Heroku, looking at you, which basically screwed everyone over after Salesforce bought them). I used Netlify for years, then, as they’ve managed to still be a great place for independent devs to deploy hobby projects, but for fullstack projects, Vercel offers a better free tier, I find. So, as I continued in my learning and development of… development, I started using Vercel to host projects that needed greater infrastructure support.
This is the first time I feel pretty upset with Vercel. First, about two months ago, they killed a product launch of mine when next-auth started experiencing major glitches across all OAuths. I reported this through multiple channels, and spent hours isolating and reproducing the bug so that I could create a proper issue report on the nextjs repo (https://github.com/nextauthjs/next-auth/issues/13240). Other people started mentioning it was killing their apps. Yet, months later, I haven’t gotten one response about it. Now I report an active DDOS Vercel isn’t protecting me against, and they again go without a word in response. This is NOT a company that cares about me, my projects, or independent developers. Vercel is, clear as day to me now, JUST ANOTHER tech giant that will step all over people to get what they want, and fuck anyone who isn’t billing $10k/month with them.
I’ve never liked Facebook, and only have a profile because someone made it for me, and a couple groups have been helpful for finding apartments when I travel. That’s it. And there are so many scams on the site now, I can’t even use it for apartments anymore. I don’t use Whatsapp, or Instagram, or Messenger, or any of that shit. I refuse to support that company. But now I’m considering flat-out deleting even the basic Facebook profile. On top of that, I’m asking myself whether or not I will stay with Vercel (we’ll see if they respond to any of my messages, tickets, tweets, or maybe even this article, but if not, pretty safe to say I will walk away and move all my projects to a different host).
And what does this spell for the future? When bots are prioritized over humans, when AI is basically keeping people from getting jobs… I don’t think we’ll see anything like “I, Robot,” but computer bots are already starting to take over the internet. What was created as a place to share information has devolved into the number one place for scams, fraud, misinformation, malicious attacks, theft, and now lines of code that are somehow treated as more important than humans. What was meant to bring the world together has become one of the most dehumanizing things in existence. I used to think technology was cool. I still do… but the way it’s being used is alarming. And until humans actively choose to look out for other humans, we will continue to see a rise of the bots.
Given I'm upset enough, and wish to take real action, I've created a petition for Vercel to remove this malicious crawler from its whitelist. If you also believe developers should have be able to control their own websites and not be abused by big tech, please consider signing this petition.