Nvidia Good news: A GPU shortage hurts OpenAI's path

Sam Altman, the CEO of OpenAI, provided an update on the company's roadmap when he was in Europe.

Altman claims that a lack of computational power is hindering OpenAI's near-term objectives and causing customers to worry about the API's dependability.

The API for fine-tuning models is also constrained by the GPU shortage, he claimed. Low-rank adaptation (LoRA), a fine-tuning technique that has proved extremely helpful to the open-source community, but not yet used by OpenAI.

Due to a shortage of computing capacity, the 32k context window version of GPT-4 is likewise not yet deployable, and there are few private models available with budgets exceeding $100,000. Nevertheless, Altman thinks that this year, a context window of up to one million tokens is feasible.

He claims that anything above that necessitates resolving the "O(n2)" scaling problem for transformer model consideration: The amount of work needed grows as the square of the number of tokens as the context window size grows. The computation is multiplied by two when the context window is doubled, by three when it is tripled, and so on. Altman claims that a scientific discovery is necessary to provide a solution to this issue.

lowering the cost of GPT-4

For OpenAI, lowering the cost of GPT-4 computing is of utmost importance. Already starting with GPT-3 to GPT-3.5 and ChatGPT, OpenAI was able to drastically lower computing costs. Customers have benefited from this due to much cheaper API charges.

The newest models and a new API that can remember prior discussions so they don't have to be transmitted again with each new API call should be made available via the fine-tuning API within the next year. Costs will drop much further as a result.

On the other side, Altman thinks it's doubtful that ChatGPT's plugins will be included in the API. He thinks ChatGPT apps are more intriguing than ChatGPT apps. With the exception of browsing, Altman claims that the plugins still don't have a good product-market fit.

In this regard, Altman reassures that other from ChatGPT, OpenAI has no other products in the works because they prefer not to compete with their developer community. The goal of ChatGPT is to enhance OpenAI APIs and offer a wise assistant. OpenAI will not use language models for a wide range of additional applications.

Before 2024, multimodality won't be available.

Multimodality is on OpenAI's agenda for the upcoming year. An AI model is said to be multimodal if it can process not only text but also images and, maybe in the future, audio, video, and 3D models.

The model may theoretically process photos, i.e. generate text or code to images or based on images, as OpenAI demonstrated during the GPT-4 launch. This feature is not yet accessible due to the GPU restriction described above.

It is unknown if OpenAI is working on any further multimodal models. According to Altman, GPT-5 won't enter training for another six months but is anticipated to add greater multimodality. The Gemini model from Google Deepmind may potentially have an advantage over multimodality.

In addition, Altman addressed his recent assertion that the "era of giant AI models" was coming to an end, noting that OpenAI would continue to work on building bigger models and that the scaling rule, which states that bigger models yield better performance, still held true. Models won't, however, grow by a factor of two or three per year since this is unsustainable.