Is AI Hitting a Plateau? The Scaling Debate OpenAI Prefers to Avoid

I think OpenAI is not being honest about the diminishing returns of scaling AI with data and compute alone. I think they are also putting a lot of the economy, the world and this entire industry in jeopardy by not talking more openly about the topic.

At first, I believed what they told us, that all you need to do is add more compute power and more data, and LLMs as well as other models will simply get better. That this relationship between the models, their compute and data could grow linearly until the end of time. The leap from GPT-3 and GPT-3.5 was immense. The leap from GPT-3.5 to GPT-4 seemed like clear evidence that this presumption was correct. But then things got weird.

Instead of releasing a model called GPT-5 or even GPT-4.5, they released GPT-4-turbo. GPT-4-turbo is not as intelligent as GPT-4 but it ...