2025-06-07 Mitigating bots
2025-06-07 Mitigating bots
lazysoci.al comes under immense pressure at times due to bots scraping the site. As all data is federated, they can essentially target any instance, small or big and scrape most of lemmy.
Usually the server hosting lazysoci.al can keep up with it. Sometimes, like today, the response times of the site suffer and it triggers an alert.
If I'm at my PC (which is like 18 hours a day /nerd) then I'll add another rule to Cloudflare to target the bots. Today the response was to block Tencent's AS#, which is a LOT of IP addresses. I'll check later today if I've snagged a Lemmy instance on this range and make adjustments.
Today was slightly special though, as the sheer number of unique IP addresses was 781 (!!). Any sort of rate limiting on a per-IP/visitor basis is not going to get triggered when they have this number of IPs to play with.
Usually Lemmy doesn't use much RAM, but today the server had to use its swap-space due to the number of concurrent requests. So I am going to double the RAM of the server in a few minutes.. Downtime will be short, likely go unnoticed.