Jobs Mob boffins have published a paper that claims to solve a major problem in running powerful AI programs on mobile phones.

The paper, called "LLM in a Flash", says that its method can make chatbots and other apps that use large language models (LLMs) work faster and better on iPhones.

For those who came in late, LLMs are huge databases that can generate realistic text and speech, such as the popular ChatGPT app. But they usually need massive computers to run them, which iPhones don't have.

It is the second paper from Apple on this topic this month and follows its earlier move to make iPhones able to create images from scratch. Apple and other phone makers hope that these new AI features will boost their flagging sales. Phone shipments have fallen by five per cent this year.

The Tame Apple Press is heralding the paper as a “breakthrough” which it is not really. It suggests using windowing, which reuses some of the data it has already processed. This reduces the need for constant memory fetching, making the process faster and smoother. Apple’s method also uses row-column bundling, which groups data so it can be read faster from the flash memory, speeding up the AI's ability to understand and generate language.

According to the paper, even in the best-case scenario, these two methods allow AI models to run up to twice the size of the iPhone's available memory. This translates to a 4-5 times increase in speed on standard processors (CPUs) and a 20-25 times faster on graphics processors (GPUs).

However, this is a still long way from getting LLMs to work on something as limited as an iPhone.