The world's most-powerful AI model suddenly got 'lazier' and 'dumber.' A radical redesign of OpenAI's GPT-4 could be behind the decline in performance.

This isn't a person, it's a machine. It doesn't have the same limitations. Higher compute cost, but it can do multiple things at once.

It's not good of it's creating artificial demand and leading to less accessibility and higher costs.

Can you quantify the difference? Far as I can tell, there's just an imaginary line where software becomes AI just because the logic filtering it depends on to operate is sufficiently complex. The term doesn't really seem to be a useful categorization either, e.g. the fundamentally different approaches of diffusion models and transformer models.

Ah, yes, when I was a kid, I would try to read big texts I understood nothing of and imitate something similar. I thought it made me smarter.

In some sense it did - probabilities of certain words being connected in a certain way, if you make some connection between them and real entities, are useful.

I mean, it did work at school, just say some water without turning on your brain. I sometimes start talking like this when I panic after a question.

Reminds me of "Ananke" by Lem. How stupid we were to believe that this particular cause of catastrophe is architecturally impossible in computing.

Its head is pretty empty, there gonna fill it with jargen and other BS

Also the articles that are plagiarized but run through a thesaurus bot to bypass search engine penalties for being plagiarized, often to the point of incomprehensibility. Yes, I'd love to read an article about my favorite vagabondlike, Deceased Cells.

Nah GPT makes it a lot easier, it's the thing it's actually good at.

Before they were autogenerated with bad English, GPT can generate good English that is equally devoid of content

For fun I asked ChatGPT about that progressive learning approach, and it seems to like the idea.

I wish I had more time to undertake some experiments in model training, this seems like it would be a really fun research direction.

Sorry for the 'wall of AI text':

The idea you're describing seems to be a form of curriculum learning. In curriculum learning, models are trained in a progressive manner, starting with simple concepts (or in this case, simpler text data) and gradually moving to more complex ones. This approach is motivated by the way human learning often works, where we learn easier topics before moving on to harder ones.

The strategy you've suggested, where the model itself determines the complexity or understandability of the next round of training inputs, is interesting. While the traditional approach to curriculum learning is to manually design the learning progression based on human knowledge and intuition, the approach you're suggesting is more dynamic and autonomous, letting the model guide its own learning process.

As of my last update in September 2021, I can't confirm any specific projects or papers that have employed exactly this strategy in the domain of large language models. However, there are some related works in the field. For example, some reinforcement learning and computer vision projects have used a similar idea where models adaptively choose their next training samples based on their current understanding. This has been referred to as active learning, and is a promising avenue for improving the efficiency of model training.

However, there would be several technical challenges in applying this approach to large language models. For one, there would be the challenge of how to assess the "understandability" of a text sample in a reliable and useful way. This would likely require significant work to define and might also have to involve some form of reinforcement learning or other feedback mechanisms. Nonetheless, it's a fascinating idea and could potentially be an interesting direction for future research in machine learning.

Dalle2 was great when it was free and stable diffusion didn't exist. I don't see the logic of: "Someone made a free version. Lets make the program worse and charge money for it!"

No no, he used to work as a wright. Built ships and shit.

Also the title of the attached post is lols: "Yes, ChatGPT became dumber and I 2 days ago I cancelled my subscription"

A lot of artists and writers are against generative AIs due to how it used their works en masse as training materials without permission, compensation or even crediting, and now prospective clients and executives are using these AIs rather than hiring them.