I just developed and published a script to clear your pict-rs object storage from potential CSAM.

Hey @db0@lemmy.dbzer0.com, just so you know, this tool is most likely very illegal to use in the USA. Something that your users should be aware of. I don't really have the energy to go into it now, but I'll post what I told my users in the programming.dev discord:

that is almost definitely against the law in the USA. From what I've read, you have to follow very specific procedures to report CSAM as well as retain the evidence (yes, you actually have to keep the pictures), until the NCMEC tells you you should destroy the data. I've begun the process to sign up programming.dev (yes you actually have to register with the government as an ICS/ESP) and receive a login for reports.

If you operate a website, and knowingly destroy the evidence without reporting it, you can be jailed. It's quite strange, and it's quite a burden on websites. Funnily enough, if you completely ignore your website, so much so that you don't know that you're hosting CSAM then you are completely protected and have no obligation to report (in the USA at least)

Also, that script is likely to get you even more into trouble because you are knowingly transmitting CSAM to 'other systems', like dbzer0's aihorde cluster. that's pretty dang bad...

here are some sources:

• https://www.law.cornell.edu/uscode/text/18/2258A
• https://crsreports.congress.gov/product/pdf/LSB/LSB10713
• https://www.missingkids.org/theissues/csam
• https://www.cloudflare.com/service-specific-terms-application-services/#csam-scanning-tool-terms
• https://developers.cloudflare.com/cache/reference/csam-scanning/#what-happens-when-a-match-is-detected
• https://developers.cloudflare.com/cache/reference/csam-scanning/#what-action-should-i-take-when-a-match-is-detected

This is extremely cool.

Because of the federated nature of Lemmy many instances might be scanning the same images. I wonder if there might be some way to pool resources that if one instance has already scanned an image some hash of it can be used to identify it and the whole AI model doesn't need to be rerun.

Still the issue of how do you trust the cache but maybe there's some way for a trusted entity to maintain this list?

As a test, I ran this on a very early backup of lemm.ee images from when we had very little federation and very little uploads, and unfortunately it is finding a whole bunch of false positives. Just some examples it flagged as CSAM:

• Calvin and Hobbes comic
• The default Lemmy logo
• Some random user's avatar, which is just a digital drawing of a person's face
• a Pikachu image

Do you think the parameters of the script should be tuned? I'm happy to test it further on my backup, as I am reasonably certain that it doesn't contain any actual CSAM

How do you even safely test scripts/tools like this 😵‍💫

Acronyms, initialisms, abbreviations, contractions, and other phrases which expand to something larger, that I've seen in this thread:

4 acronyms in this thread; the most compressed thread commented on today has 6 acronyms.

[Thread #88 for this sub, first seen 28th Aug 2023, 22:25] [FAQ] [Full list] [Contact] [Source code]

I don't host a server myself, but can this tool identify the users who posted the images and create a report with their IP addresses?

This could help identify who spreads that content and it can be used to notify authorities. No?

Thank you for helping make the fediverse a better place.

Don't have a GPU on my server. How is performance on the CPU ?

Worth noting you seem to be missing dependencies in requirements.txt notably unidecode and strenum

Also that this only works with GPU acceleration on NVidia (maybe, I messed around with trying to get it to work with AMD ROCm instead of CUDA but didn't get it running)

Not well versed in the field, but understand that large tech companies which host user-generated content match the hashes of uploaded content against a list of known bad hashes as part of their strategy to detect and tackle such content.

Could it be possible to adopt a strategy like that as a first-pass to improve detection, and reduce the compute load associated with running every file through an AI model?

Hi db0, if I could make an additional suggestion.

Add detection of additional content appended or attached to media files. Pict-rs does not reprocess all media types on upload and it's not hard to attach an entire .zip file or other media within an image (https://wiki.linuxquestions.org/wiki/Embed_a_zip_file_into_an_image)

Opened a relevant issue: https://github.com/LemmyNet/lemmy/issues/3920

based db0 releasing great tools and maintaining a great community

Hey db0 thanks for putting in extra effort to help the community (as you have multiple times) when big issues like this crop up on Lemmy.

Despite being a pressing issue this is one that people also are a little reluctant to help solve because of fear of getting in trouble themselves. (How can a server admin develop a method to detect and remove/prevent CSAM distribution without accessing known examples which is extremely illegal?)

Another time being the botspam wave where you developed Overseer in response very quickly. I'm hoping here too devs will join you to work out how to best implement the changes into Lemmy to combat this problem.

Thanks for releasing this. After doing a --dry_run can the flagged files then be removed without re-analysing all images?

This is awesome. Thank you for making it.

Any thoughts about using this as a middleware between nginx and Lemmy for all image uploads?

Edit: I guess that wouldn't work for external images - unless it also ran for all outgoing requests from pict-rs.. I think the easiest way to integrate this with pict-rs would be through some upstream changes that would allow pict-rs itself to call this code on every image.

Thank you for this! Awesome work!

By the way, this looks easy to put in a container. Have you considered doing that?

Can this be used with Mastodon as well?

Just going to argue on behalf of the other users who know apparently way more than you and I do about this stuff:

WhY nOt juSt UsE thE FBi daTaBaSe of CSam?!

(because one doesn’t exist)

(because if one existed it would either be hosting CSAM itself or showing just the hashes of files - hashes which won’t match if even one bit is changed due to transmission data loss / corruption, automated resizing from image hosting sites, etc)

(because this shit is hard to detect)

Some sites have tried automated detection of CSAM images. Youtube, in an effort to try to protect children, continues to falsely flag 30 year old women as children.

OP, I’m not saying you should give up, and maybe what you’re working on could be the beginning of something that truly helps in the field of CSAM detection. I’ve got only one question for you (which hopefully won’t be discouraging to you or others): what’s your false-positive (or false-negative) detection rate? Or, maybe a question you may not want to answer: how are you training this?

Amazing work.

Now if you can make this work with mastodon, i'd be eternally grateful.😁

Can this be used with the Lemmy-easy-deploy method?

Permanently Deleted

I think deleting images from the pictrs storage can corrupt the pictrs sled db so I would not advise it, you should go via the purge endpoint on the pictrs API.

anything worthy is illegal in the us, get gud! thankfully Lemmy is hosted in europe or any vps somewhere