Why are there so many new AI-powered tools that scrape websites?
Why are there so many new AI-powered tools that scrape websites?
I feel like every day I come across 15-20 "AI-powered tool"s that "analyze" something, and none of them clearly state how they use data. This one seems harmless enough, put a profile in, it will scrape everything about them, all their personal information, their location, every post they ever made... Nothing can possibly go wrong aggregating all that personal info, right? No idea where this data is sent, where it's stored, who it's sold to. Kinda alarming
You're viewing a single thread.
A toy like that is easy to create and not that expensive to offer. Much more expensive than some JavaScript or CSS, but in the end it's not that different.
I think people don't really understand this whole scraping thing. For example, you can torrent all of Reddit until the API-change; all the comments, profiles, usernames, including now deleted stuff. There is a lot of outrage here over Reddit cracking down on these 3rd party tools. It's difficult to see how that outrage over cracking down on 3rd party tools, fits with this outrage here over not cracking down on 3rd party tools.
Anyway, if someone want to archive all of Bluesky, they don't need to offer some AI toy. They can just download the content via the API.
You can still torrent Reddit pushshift data past the API change. But yea I definitely agree otherwise, these are just cheap toys that less experienced developers create for portfolios.
A toy like that is easy to create and not that expensive to offer.
Right, and the developers of Bsky didn't think to maybe block something that scrapes all that personal information?
Like Lemmy or Mastodon, BlueSky was made with the idea of federation. While BlueSky is not there yet, federated services are inherently very easy to scrape.
Maybe it's time for people to understand that anything they post/vote/comment/like should be considered public domain.
If that's what you want, you should join Facebook.
The fundamental thing to understand is that the internet - and really all information processing - is about copying. There is no such thing as "looking" at a profile or a post. The text and image data is downloaded to your device. You end up with multiple copies on your device.
Sending information out, but blocking people from storing it, is fundamentally a contradiction in terms.
Bsky - like Lemmy - made the choice to make the data widely available. It is available via API and does not need to be scraped. The alternative is to do it like Reddit or even Facebook or Discord. But they can't stop scraping, either. They can make it slower and more laborious but not stop it. Services like Facebook protect the data as best as they can to "protect your privacy". In reality, it's about making it hard for you to leave the platform or anyone else to benefit from your data. Either way, you can trust Zuck to protect your data as if it was his own. Because it is.
That would always by definition block all third parties.
Think of the reddit example from the person you replied to: there was a huge outcry when reddit announced shutting down their lower API tiers.
Either information is free to flow or not at all, there is no middle ground.
With that in mind: I'm sure they thought about it and decided to prioritize transparency she flexibility over security. Personally I support that decision.
I know how APIs on reddit work, but you can block people who misuse the API if they're doing something nefarious. Some of these AI are in my honest opinion very taxing on hardware. Having to retrieve millions of posts, comments, pictures, text, on demand... and send that to who knows where for AI scraping.... Sounds very costly.
Permanently Deleted