Dubbed the QCS605 and QCS603, the SoCs provide computing for on-device camera processing and machine learning, with power and thermal efficiency, across a wide range of IoT applications.
Qualcomm claims the SoCs use its most advanced image signal processor (ISP) to date and the Qualcomm Artificial Intelligence (AI) Engine, along with a heterogeneous compute architecture and an ARM-based multicore CPU, vector processor and GPU.
The Vision Intelligence Platform also includes Qualcomm Technologies’ camera processing software, machine learning and computer vision software development kits (SDKs), connectivity and security technologies.
The idea is that the SoCs can be optimised to support industrial and consumer smart security cameras, sports cameras, wearable cameras, virtual reality (VR) 360 and 180 cameras, robotics, and intelligent displays.
KEDACOM and Ricoh THETA have been named as developing products based on the Platform.
Qualcomm product management vice president Joseph Bousaba said the move would make IoT devices significantly smarter as we help customers bring powerful on-device intelligence, camera processing and security.
“AI is already enabling cameras with object detection, tracking, classification and facial recognition, robots that avoid obstacles autonomously, and action cameras that learn and generate a video summary of your latest adventure, but this is really just the beginning”, he said.
The Vision Intelligence Platform integrates the Qualcomm AI Engine which is comprised of several hardware and software components combined to accelerate on-device AI. The Qualcomm AI Engine includes the Qualcomm Snapdragon Neural Processing Engine (NPE) software framework which provides for analysis, optimisation, and debugging tools for developments using Tensorflow, Caffe and Caffe2 frameworks, Open Neural Network Exchange interchange format, as well as Android Neural Networks API and Qualcomm Hexagon Neural Network library.
Qualcomm said that this allows developers and OEMs to port trained networks into the platform easily. With the Qualcomm AI Engine and Snapdragon NPE software framework, the Vision Intelligence Platform delivers up to 2.1 TOPS of computing performance for in-depth neural network inferences, more than double compared to specifications of some of the leading alternative solutions.
The Platform supports up to 4K video resolution at 60 frames per second (fps), or 5.7K at 30 fps, as well as multiple concurrent video streams at lower resolutions.
It also uses Qualcomm’s dual 14-bit Qualcomm Spectra 270 ISP supporting dual 16 megapixels sensors. The Vision Intelligence Platform includes advanced vision processing capabilities necessary for IoT segments such as staggered HDR to prevent the “ghost” effect in the high-dynamic-range video, advanced electronic image stabilisation, de-warp, de-noise, chromatic aberration correction, and motion compensated temporal filters in hardware.
The Vision Intelligence Platform’s QCS605 heterogeneous computing architecture features eight Qualcomm Kryo 360 CPU cores, the Qualcomm Adreno 615 GPU and Hexagon 685 Vector Processor.
The Platform’s integrated display processor provides for a range of display options up to WQHD resolution touch display with hardware accelerated composition, 3D overlays, and support for the significant graphics APIs including OpenGL, OpenCL, and Vulkan.
The Vision Intelligence Platform supports up to 2x2 802.11ac Wi-Fi with MU-MIMO and dual-band simultaneous transmission, Bluetooth 5.0, Qualcomm 3D Audio Suite, Qualcomm Aqstic Audio Technologies and Qualcomm aptX Audio. The platform also features Qualcomm Noise and Echo Cancellation, as well as advanced on-device audio analytics and processing features to support natural language processing, audio speech recognition, and “barge-in” capability for a reliable voice interface even in loud or noisy environments or when users are far from the device.
The Qualcomm QCS605 and QCS603 are sampling now, with multiple SKUs designed to meet a variety of technical and cost-effectiveness requirements. QCS605-based VR 360 camera reference designs from Qualcomm Technologies and Altek Corporation, a leading original design manufacturer for cameras, are available today, and QCS603-based industrial security camera reference designs are expected to be available in the second half of 2018