OpenAI is struggling to gather training data for new models

OpenAI seems to have hit a bottleneck in performance for its next-generation AI models. The company's next big model - "Orion," is reportedly underperforming in specific tasks compared to older models. 

It seems to have the advantage in language-based tasks such as translation, text generation, etc. but it has proven to be underwhelming in jobs such as coding. 

According to a report by The Information (via Gadgets360), there seems to be an issue gathering training data for the new models. The model is also more expensive to run at data centers when compared to GPT-4 and GPT-4o.

The jump in quality is also weaker compared to the strides made by GPT-4 over GPT-3. OpenAI has reportedly created a foundations team to solve the issue of training data but it's still unclear if they can obtain enough data for the AI model in time for release. 

It's not just OpenAI that is seeing negligible returns in performance. Other competitors, like Anthropic and Mistral, also show only iterative gains with each new release. One suggested method to improve performance is opting to train the model further after release by fine-tuning the output but this is a workaround and not a permanent solution.