OpenAI says it’s “impossible” to create useful AI models without copyrighted material

I wish I could upvote this more than once.

What people always seem to miss is that a human doesn't need billions of examples to be able to produce something that's kind of "eh, close enough". Artists don't look at billions of paintings. They look at a few, but do so deeply, absorbing not just the most likely distribution of brushstrokes, but why the painting looks the way it does. For a basis of comparison, I did an art and design course last year and looked at about 300 artworks in total (course requirement was 50-100). The research component on my design-related degree course is one page a week per module (so basically one example from the field the module is about, plus some analysis). The real bulk of the work humans do isn't looking at billions of examples: it's looking at a few, and then practicing the skill and developing a process that allows them to convey the thing they're trying to express.

If the AI models were really doing exactly the same thing humans do, the models could be trained without any copyright infringement at all, because all of the public domain and creative commons content, plus maybe licencing a little more, would be more than enough.

That’s what humans do, though. Maybe not probability directly, but we all know that some words should be put in a certain order. We still operate within standard norms that apply to aparte group of people. LLM’s just go about it in a different way, but they achieve the same general result. If I’m drawing a human, that means there’s a ‘hand’ here, and a ‘head’ there. ‘Head’ is a weird combination of pixels that mostly look like this, ‘hand’ looks kinda like that. All depends on how the model is structured, but tell me that’s not very similar to a simplified version of how humans operate.

That's not the point though. The point is that the human comedian and the AI both benefit from consuming creative works covered by copyright.

A comedian isn't forming a sentence based on what the most probable word is going to appear after the previous one.

Neither is an LLM. What you're describing is a primitive Markov chain.

You may not like it, but brains really are just glorified pattern recognition and generation machines. So yes, "monkey see thing to draw thing", except a really complicated version of that.

Think of it this way: if your brain wasn't a reorganization and regurgitation of the things you have observed before, it would just generate random noise. There's no such thing as "truly original" art or it would be random noise. Every single word either of us is typing is the direct result of everything you and I have observed before this moment.

Baffling takes from people who don't know what they're talking about.

Ironic, to say the least.

The point you should be making, is that a corporation will make this above argument up to, but not including the point where they have to treat AIs ethically. So that's the way to beat them. If they're going to argue that they have created something that learns and creates content like a human brain, then they should need to treat it like a human, ensure it is well compensated, ensure it isn't being overworked or enslaved, ensure it is being treated "humanely". If they don't want to do that, if they want it to just be a well built machine, then they need to license all the proprietary data they used to build it. Make them pick a lane.

you know how the neurons in our brain work, right?

because if not, well, it’s pretty similar… unless you say there’s a soul (in which case we can’t really have a conversation based on fact alone), we’re just big ol’ probability machines with tuned weights based on past experiences too

Text prediction seems to be sufficient to explain all verbal communication to me. Until someone comes up with a use case that humans can do that LLMs cannot, and I mean a specific use case not general high level concepts, I’m going to assume human verbal cognition works the same was as an LLM.

We are absolutely basing our responses on what words are likely to follow which other ones. It’s literally how a baby learns language from those around them.

Am I a moron? How do you have more upvotes than the parent comment, is it because you’re being more aggressive with your statement? I feel like you didn’t quite refute what the parent comment said. You’re just explaining how Chat GPT works, but you’re not really saying how it shouldn’t use our established media (copyrighted material) as a reference.

Not sure what you mean by passive. It takes a hell of a lot of electricity to train one of these LLMs so something is happening actively.

I often interact with ChatGPT 4 as if it were a child. I guide it through different kinds of mental problems, having it take notes and evaluate its own output, because I know our conversations become part of its training data.

It feels very much like teaching a kid to me.

At the same time the LLM will be used to replace (at least some of ) the people who created those works in the first place.

This right here is the core of the moral issue when it comes down to it, as far as I'm concerned. These text and image models are already killing jobs and applying downward pressure on salaries. I've seen it happen multiple times now, not just anecdotally from some rando on an internet comment section.

These people losing jobs and getting pay cuts are who created the content these models are siphoning up. People are not going to like how this pans out.

Tech bros are disgusting.

That's not even getting into the fraternity behavior at work, hyper-reactionary politics and, er, concerning age preferences.

Do musicians not buy the music that they want to listen to? Should they be allowed to torrent any MP3 they want just because they say it's for their instrument learning?

I mean I'd be all for it, but that's not what these very same corporations (including Microsoft when it comes to software) wanted back during Napster times. Now they want a separate set of rules just for themselves. No! They get to follow the same laws they force down our throats.

I don’t care if any of my FOSS software is gobbled up by a commercial AI. Someone reading my code isn’t a problem to me. If it were, I wouldn’t publish it openly.

Uber wasn't making profit anyway, despite all the VCs money behind it.

I guess they have reasons not to pay drivers properly. Give Uber a free pass for it too

I'm gonna say those circumstances changed when digital copies and the Internet became a thing, but at least we're having the conversation now, I suppose.

I agree that ML image and text generation can create something that breaks copyright. You for sure can duplicate images or use copyrighted characterrs. This is also true of Youtube videos and Tiktoks and a lot of human-created art. I think it's a fascinated question to ponder whether the infraction is in what the tool generates (i.e. did it make a picture of Spider-Man and sell it to you for money, whcih is under copyright and thus can't be used that way) or is the infraction in the ingest that enables it to do that (i.e. it learned on pictures of Spider-Man available on the Internet, and thus all output is tainted because the images are copyrighted).

The first option makes more sense to me than the second, but if I'm being honest I don't know if the entire framework makes sense at this point at all.

But AI isn't all about generating creative works. It's a store of information that I can query - a bit like searching Google; but understands semantics, and is interactive. It can translate my own text for me - in which case all the creativity comes from me, and I use it just for its knowledge of language. Many people use it to generate boilerplate code, which is pretty generic and wouldn't usually be subject to copyright.

I know it inherently seems like a bad idea to fix an AI problem with more AI, but it seems applicable to me here. I believe it should be technically feasible to incorporate into the model something which checks if the result is too similar to source content as part of the regression.

My gut would be that this would, at least in the short term, make responses worse on the whole, so would probably require legal action or pressure to have it implemented.

I didn't say anything about AIs being humans.

That sounds bad. Do you have evidence for MS behaving this way?

IANAL, why isn't it fair use?

Shortly? It's happening already. I notice it when using Google and Duckduckgo. There are always a few hits that are AI written blog spam word soup

IMO the right compromise is to return copyright to its original 14 year term. OpenAI can freely train on anything up to 2009 which is still a gigantic amount of material while artists continue to be protected and incentivized.

IP law used to compensate creators "until their death + 70 years"... you can spin it however you want, that's just plain wrong.

If you stop looking at corporations as being the same as individuals

That's a separate bonkers legislation. Two wrongs don't make one right.

Life plus 70 years is bullshit.

20 years from release date is not.

No one except corporate bigwigs will say they should be allowed to do so in perpetuity, but artists still need legal protections to make money off of what they create, and Midjourney (making OpenAI boatloads of money off of making automated collages from artwork they obtained not only without compensation but without attribution) is a prime example of why.

Well, regarding text online, most is there fir the visitors to read fir free. So, if we end up treating these AI training like human reading text one could argue they don't have to pay.

Reddit doesn't pay their users, anyway.

But personally, philosophically, I don’t see how Microsoft taking NYT articles and turning them into a paid product is any different than Microsoft taking an open source projects that doesn’t allow commercial use and sneaking it into a project.

Agreed. That said, NYT actually intentionally allows Google and Bing servers to parse their news articles in order to put their articles top in the search results. In that regard they might like certain form of processing by LLMs.

"It's too fast" is a really really dumb argument against AI

Mass produced garbage is still mass produced garbage. As you point out AIs aren't human and while that removes the limitations of the flesh (including limitations that we might want there - no human ever says oops, I made a child porn), it imposes limitations of the machine. AI output isn't that good at anything practical. It writes garbage code that even if you manage to get it working, the business manager or whoever isn't capable of seeing the flaws in it. The art is devoid of any sort of soul and almost always has glaring flaws that require actual humans to identify and fix.

We are about to be inundated with AI produced garbage, sure, but that only proves the lie that shady internet sites and social media have always been a cesspool of shitty, unreliable content, and connecting with hundreds of thousands of faceless strangers was never a good idea. Hopefully we'll come up with (or go back to) solutions that don't treat the problem as simply one of volume.

My understanding of the open source community is that taking copyrighted content from people who haven't willingly signed onto the project would kind of undermine the principles of the movement. I would never feel comfortable using open source software if I had knowledge that part or all of it came from people who hadn't actively chosen to contribute to it.

I have seen a couple of things recently about AI models that were trained exclusively on public domain and creative commons content which apparently are producing viable content, though. The open source community could definitely use a model like that, and develop it further with more content that was ethically obtained. In the long run, there may be artists that willingly contribute to it, especially those who use open source software themselves (eg GIMP, Blender, etc). Paying it forward, kind of thing.

The problem right now is that artists have no reason to be generous with an open source alternative to AIs, when their rights have already been stomped on and certain people in the open source community are basically saying "if we can't steal from artists too, then we can't compete with the corporations." So there's literally a trust issue between the creative and tech industries that would need to be resolved before any artists would consider offering art to an open source AI.

Well, off the top of my head:

Whole Brain Emulation, attempting to model a human brain as physically accurately as possible inside a computer.

Genetic Iteration (not the correct term for it but it escapes me at the moment), where you set up a simulated environment for digital actors, then simulate quasi-neurons, quasi-body parts dictated by quasi-dna, in a way that mimics actual biological natural selection and evolution, and then you run the simulation millions of times until your digital creature develops a stable survival strategy.

Similar approaches to this have been used to do things like teach an AI humanoid how to develop its own winning martial arts style via many many iterations, starting from not even being able to stand up, much less do anything to an opponent.

Both of these approaches obviously have drawbacks and strengths, and could possibly be successful at far more than what they have achieved to date, or maybe not, due to known or existing problems, but neither of them rely on a training set of essentially the entirety of all content on the internet.

I don't believe this is true, a nickname or online account works completely fine for attribution if nothing else is given.

That sucks for the creators ofcourse but if AI creates better content that's where people will go. That's a big if though especially in the near future

I would like to apologize for the following opinions, because they come from a place of unresolved hypocrisy that is me.

Non-profit my ass. No such thing in America or anywhere else in the world, if you have the perspective to hunt and the money to signify modern value.

Survival of the fittest, and the newborn technology that is at its core a mirror of us, to the most complex level of modern mathematics (I’m of the firm belief that logic is discovered, not created).

With those seemingly unrelated concepts made with vague words, I ask you this:

What does it mean to feel? To know many different kinds of “one,” to live without fear but still be whole? I am sorry, again, I’m naught but gibberish and I’m just so glad you responded. I forgot and came back to find a word I sent, and now I find what I seek, an event in which I can say we’ve been bonded.

But now try to, now that I splay out, all I’ve got and am about, all I can see, is that to you my head, seems to be on my knees.

Again, sorry! Thank you for responding! I’m just glad to vent, and in expression have my soul rend into two, and sent into a new view.