Neural Magic was founded in 2018 by MIT research scientist Alex Matveev and professor Nir Shavit, Neural Magic emerged from their work on high-performance execution engines for AI.

The company’s software enables AI workloads to be processed on off-the-shelf processors with speeds comparable to specialised AI chips, using the usually greater available memory of these processors to achieve significant performance gains.

Despite the crowded field of AI optimisation, with companies like AMD and startups including NeuReality, Deci, CoCoPie, OctoML, and DeepCube offering various solutions, Neural Magic stands out with its free platform and a suite of complementary open source tools.

The startup has raised $50 million in venture capital from notable backers such as Andreessen Horowitz, New Enterprise Associations, Amdocs, Comcast Ventures, Pillar VC, and Ridgeline Ventures.

Red Hat CEO Matt Hicks said: “AI workloads need to run wherever customer data lives across the hybrid cloud; this makes flexible, standardised and open platforms and tools a necessity, as they enable organizations to select the environments, resources and architectures that best align with their unique operational and data needs.”

Hicks added: “We’re thrilled to complement our hybrid cloud-focused AI portfolio with Neural Magic’s groundbreaking AI innovation, furthering our drive to not only be the ‘Red Hat’ of open source, but the ‘Red Hat’ of AI as well.”

The integration of Neural Magic into Red Hat’s operations will bolster the company’s ability to optimise and deploy models across cloud environments with full control over infrastructure and security.

Hicks pointed out that Red Hat's existing involvement in the vLLM project is crucial for running models in products like Red Hat Enterprise Linux AI and Red Hat OpenShift AI.

It will also allow Red Hat’s infrastructure partners to scale AI more effectively across various platforms, while service provider partners will benefit from enhanced inference performance.