The AI Tech day is a pre- announcement rather than a product announcement, but the Qualcomm Cloud AI 100 data accelerator is an SoC built from the ground up and is primarily focusing on inference processes in the cloud.

From the ground up 7nm

It is built from the ground up, and it is a 7nm based solution. Keith Kressin, SVP of product management at Qualcomm, claims that this will offer 10X performance per watt over the industry’s most advanced AI interface solutions deployed today.

Keith mentioned that this is not a Snapdragon mobile chip muscled up. The company started and developed the AI inferencing solution from the ground up. Big cloud providers hope that AI accelerators can help them reduce power, assist with data compression and primarily help them bring the TCO (Total Cost of Ownership) down.  

The Snapdragon 855 with its combination of CPU, GPU and DSP already shows a 3X performance gain over the competing 7nm Android SoC (Kirin 980) and Cloud AI 100 is a data center AI accelerator car that runs using a different power envelope.

Microsoft and Facebook might be interested

Facebook and Microsoft took part in the presentation of the card without confirming that they will end up using it. It is very likely that they will, but this is something that will be announced in the future. Facebook’s Joe Spisak, product manager for AI, said that data center power doubles each year and AI accelerators can help to reduce this horrible trend. Increase in power consumption is directly responsible for higher costs of data centers.

Additionally, Facebook can use AI accelerators to speed up bots and assistants, generated content, Augmented Reality effects and VR experiences. 

Big performance promise

The 10X performance improvement compared to FPGA and GPU translates to products that exist in the market today. Bear in mind that Xilinx Everest – Versal cards are expected later this year that will dramatically increase performance compared to the existing Alveo AI cards.

Nvidia is also expected to announce the next generation GPUs based on Ampere architecture and have them in the market by 2020.

Qualcomm targets sampling for the latter part of this year and actual production in 2020, without going into any details. Keith mentioned that Qualcomm planned more than one tier of Cloud-based AI products.

Keith Kressin also shared another thing. The projected number based on INT8 is a massive 350 TOPS. TOPS (Trillion Operation per second) is a real marketing number invented or at least popularized by Jensen, the supreme leader of Nvidia, but this is the only number that Qualcomm was willing to discuss.

It is clear that Qualcomm will be ready to talk more about Cloud AI 100 product soon without being specific right now.

Great software stack

The Cloud AI 100 platform will support Glow, TensorFlow, Keras, and ONNX software stuck. Some tasks that might profit from the data center AI accelerator include personal assistants for natural language processing and translations, advanced image search, and personalized content and recommendations.

AI is one of these things like connectivity. It silently sneaks up upon consumers and speeds things up in the background. Connectivity is only noticed when it is missing.