am i reading this right
iocaine was actually working and now we have evidence to back it up?

YESS OPENAI, PLEASE SCRAPE 52 DIRECTORIES FURTHER INTO THE IOCAINE TARPIT. I BEG YOU, PLEASE.

sure the outlined threat model is more about basically attacking the LLM to get it to spit out certain things

but this should also translate to iocaine and other such tools being effective for poisoning the dataset in the “this doesn’t make any sense” way

@alexia this is so funny, considering the number of times I've been told poisoning doesn't work.

@alexia what does this mean?

@shroomie the paper found that to poison an LLM, you don’t need more poisoned data the bigger the dataset is

in fact, you need almost the same amount of poisoned data every time no matter the size of the dataset

this means that tools such as iocaine, which actively poison scrapers with markov-generated garbage, have actually been working this entire time at poisoning bigtech LLM datasets and keeping them away from websites

@alexia hahahahaha awesome!!!!

@shroomie I do ofc have to outline that iocaine only works as well as the detection script

but nam-shub-of-enki, the iocaine developers’ configuration is quite strict and as such blocks pretty much all AI tools I could find

@alexia well yeah isn’t it that like. iocane is really iocane + your scraper blocker configuration?

@shroomie Yes, iocaine isn’t configured, it’s rather programmed

@alexia iocaine-like poison is most likely much less effective due to not being focused on something specific

@risc I don’t see how; Whilst the paper outlines the threat model of getting the LLM to do something specific by poisoning its input training data, the findings about the required amount of data still apply to iocaine-like poisoning, just that we don’t get anything out of it other than us being excluded from the dataset in the future

@alexia I’m kinda sad because I just checked the graphs and we had a shitton of scraping a while ago. But then @enjarai added a robots.txt blocking everything and now we barely get any traffic. 😔

@enjarai @alexia Maybe instead of blocking everything with the robots.txt, we should only block the search engine crawlers? :3

@aurakle
What? AI crawlers respect robots.txt? It looks like #iocaine *is* working \o/
@enjarai @alexia
@algernon

@kupac @enjarai @alexia @algernon Do keep in mind that this shows all crawlers that iocaine flagged, not necessarily for AI training, and also several AI crawlers (to my knowledge) do respect robots.txt, but not all.

@enjarai @alexia @algernon @kupac I’d frankly rather poison them than block them.

@alexia > While larger models train on more clean data that could dilute poisoning effects, they are also more sample efficient and can learn from fewer examples

this one believes that this doesn't apply to iocaine, because there isn't a single thing to "learn" from. the iocaine poison would in theory apply more "uniformly".

granted, this one doesn't know the mechanisms behind LLMs well - take this with a lot of grains of salt.

(note: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa it accidentally boosted again. Elk is annoying)

@alexia continuation of the thought: if this one understands correctly, the attacks in the paper use key phrases which are somewhat rare in clean training material, which causes the model to pick up that material despite its low amount. iocaine, while in theory having some patterns due to the Markov chain, probably doesn't have enough repeating ones to be picked up like this.

@risc I could of course just chance iocaine’s configuration to do that

@alexia true, but not sure if it would be useful, since the goal isn't to insert a backdoor

@risc I mean if I can mix my markov chain together with malicious prompts I will very much do so

@alexia poisoning some legitimate query would be quite funny, but also presumably significantly harder, since unlike with the rare/non-existent keywords in the paper this would actually require a lot of poisoned data (and might not work)

making a malicious keyword-based prompt could be interesting as an experiment

@risc good thing that I send like 60 gigabytes of garbage per day to all scrapers combined then

@alexia holy heck

@risc tbf this number is a bit old and I’d have to setup grafana or some shit to really see this but also the iocaine dev regularly publishes stats from the iocaine homepage and their fedi instance which also serves GIGABYTES of garbage

@alexia we probably should've set up iocaine on FC too when that was a thing
maybe this one'll set it up on its own infra if it ever gets it all back up..

@risc

https://come-from.mad-scientist.club/users/algernon/statuses/01K70BGTNKJV5BCEGV951JCAEC

here’s the dev’s stats btw, almost 30 million requests

@risc actually that is old, newer data: https://come-from.mad-scientist.club/@iocaine/statuses/01K7T300D0QFJX6J1FZKPC286T

more like 50 million requests now

@risc 720 gigabytes of garbage.