Andes Technology is a high-efficiency, low-power 32/64-bit RISC-V processor core supplier and a founding principal member of RISC-V International. Recently, Andes Technology announced that it has entered into a partnership with Fractile. Fractile is a company dedicated to developing the next generation of AI chips and systems designed to push the boundaries of AI performance. Fractile is currently developing an in-memory computing-based AI inference accelerator with the goal of running large-scale AI models (including language, vision, and audio models) 100 times faster than existing hardware while keeping costs low.
Today, large language models (LLMs) and other foundational models have become the main drivers of the rapid growth of AI computing demand in data centers, from ChatGPT to open source models such as Llama. Model inference (the process of servicing a trained model) is becoming a major source of cost for AI computing, even surpassing model training. Fractile has licensed the Andes AX45MPV RISC-V vector processor, and in conjunction with the Andes Automated Custom Extension™ (ACE) and Andes Domain Library, plans to introduce a vector processing unit in its first data center AI inference accelerator.
Figure: Andes Technology partner with Fractile, accelerating AI Reasoning Performance
Fractile's innovation is to leverage on-chip memory computing to perform 99.99% of the operations required for model inference, thus avoiding frequent data exchanges. This approach not only improves energy efficiency (TOPS/W), but also significantly reduces inference latency, especially in LLM applications, which can significantly increase the number of user-generated content per second. The company has been focused on improving AI performance through inference scaling. Recently, OpenAI's latest LLM o1 model further confirms this development direction, and its inference computing needs have also increased exponentially. Fractile's hardware and software stack is designed to enable rapid generation, enabling AI model inference that currently takes seconds to achieve instant response on existing hardware.
In this collaboration, Fractile will combine Andes Technology's high-performance RISC-V vector processor with its in-memory computing architecture through ACE technology. Fractile's system leverages the strengths of both technologies to break through the bottlenecks of traditional computing methods and provide a faster, more cost-effective solution for AI inference.
Dr. Hong Su, President and CTO of Andes Technology, expressed his anticipation for the collaboration: "Since its launch in 2023, AX45MPV processors have been adopted by several innovative AI companies due to their powerful computing power, high memory bandwidth, and flexibility as ACE tools. Our RISC-V vector processors have helped many AI SoCs transcend architectural limitations and achieve higher performance and efficiency. We believe that the combination of Fractile's in-memory computing technology will lead to new breakthroughs.”
Dr. Walter Goodwin, CEO and Founder of Fractile, added, "The limitations of existing hardware are the biggest barrier to AI performance and adoption. Andes Technology has a leading technology and market position in the field of RISC-V vector processors, and is the ideal partner for our Fratile accelerator. Developing AI-accelerated hardware is challenging, and programmable vector processors like Andes are key to adapting to model changes. "We are excited to further Fractile's mission to advance reasoning through this collaboration."
