Neuchips, a company specializing in Application-Specific Integrated Circuits (ASICs) for AI, has introduced two groundbreaking hardware components at CES. The Raptor Gen AI accelerator chip and the Evo PCIe accelerator card offer a new way to run large language models (LLMs) on less powerful hardware while consuming less energy.
By leveraging the capabilities of these new hardware components, enterprises can deploy LLMs at a fraction of the current costs. The Raptor chip, capable of performing up to 200 tera operations per second, provides support for critical machine learning algorithms like matrix multiplications and embedding table lookups. This makes it applicable not only to LLMs but also to a wide range of generative AI and transformer-based models.
Complementing the Raptor chip, the Evo acceleration card combines the power of Raptor chips with 32 GB of LPDDR5 memory and a PCIe Gen 5 interface with eight lanes, delivering a high-speed host I/O bandwidth of 64 GB/s. This combination of performance and energy-efficiency opens up new possibilities for AI applications, enabling a new generation of AI tools to be developed.
At CES, the Neuchips team showcased the capabilities of their hardware by demonstrating its successful implementation in popular AI chatbots Whisper and Llama. The impressive performance and energy-efficiency of the hardware have the potential to transform the AI landscape, making AI tools more accessible and cost-effective for enterprises.
Expect more exciting product releases from Neuchips in the coming months as they continue to push the boundaries of AI hardware innovation. With the introduction of the Raptor Gen AI chip and the Evo PCIe accelerator card, Neuchips has set the stage for a new era in AI applications.
The source of the article is from the blog be3.sk