OpenAI, Google, Anthropic admit they can’t scale up their chatbots any further
OpenAI, Google, Anthropic admit they can’t scale up their chatbots any further
Once you’ve trained your large language model on the entire written output of humanity, where do you go? Here’s Ilya Sutskever, ex-OpenAI, admitting to Reuters that they’ve plateaued: [Reuters] Th…
LLMs are quite impressive as chatbots all things considered. The conversations with them are way more realistic and almost as funny as the ones with the IRC markov chain my friend made as a freshman CS student.
Of course, out bot's training data only included the IRC channel's logs of a few years and the Finnish Bible we later threw in for shits and giggles. A training set of approximately zero terabytes in total.
LLMs are less a marvel of machine learning algorithms (though I admit they might play a part) and more one of data scraping. Based on their claims, they have already dug through the vast majority of publicly accessible world wide web, so where do you go from there? Sure, there are a lot of books that are not on the web, but feeding them in the machine is about as hard as getting them on the web to begin with.
Senior year of college, I took an elective seminar on interactive fiction. For the final project, one of my classmates wrote a program that scraped a LiveJournal and converted it into a text adventure game.
fuck yes, why wasn’t my college this cool? all I got was an AI elective taught by a guy whose proudest achievement was having the only remaining Genera license on campus
Bullshit. All you need is to scrape together a measley $7 trillion dollars and throw it at Sam Altman.
"Admit" is a strong word, I'd go for "desperately attempt to deny".
This was really obvious a couple orders of magnitude ago, but I guess at least they stopped lying about infinite scale being an option?
I am a firm realist. I'd like to point out two things.
There are two sides: the anti-corpo leftists of Lemmy, and the hyper "where do we sign" neo liberals of reddit.
Lemmy Leftists are sure Google will literally just evaporate tomorrow, and the neo liberals are sure that the singularity has already happened and will be revealed for Christmas.
The truth is probably somewhere in the middle. Google literally cannot evaporate, and they are also probably struggling to improve their models. They will probably find ways to slowly improve them over time, at a reasonable rate.
Edit - 😅 I'm at least happy I created something fun. You guys are great, enjoy lol
Was having trouble understanding this so I translated it through a couple LLMs, it just cost me a few cubic meters of water:
I am very smart. I’d like to point out one thing. Everyone other than me is stupid. Herpa derpa derpa derp.
what a strange post
I am a firm realist. There are two sides: the anti-corpo leftists of Blade II, and the hyper “where do we sign” neo liberals of Battlefield Earth.
Blade II Leftists are sure nighttime is literally just yellow color grading except when it’s blue or they forget to do it at all, and the neo liberals are sure that you tilt the camera and the more you tilt the camera the better the movie is.
The truth is probably somewhere in the middle. John Travolta literally cannot evaporate, and he is also probably struggling to improve his filmmaking. He will probably find ways to do both fucked up color grading and what he thinks are Dutch angles, but in a more subtle way.
And then Del Toro will probably send him a box of shit or live bees for Christmas.
I'm not sure what you're trying to say here -- that the average of two extremes is the most likely outcome? The opposing sides are both clearly ungrounded in reality, so it's like you've set up not one but two strawmen arguments facing each other from miles away and using that as an illustration for an unspecific conjecture of averages between them.
new copypasta just dropped (see below)