Recently, a research team from Tsinghua University and startup Qingcheng.AI jointly released a new artificial intelligence inference framework "Chitu", marking a new step in China's efforts to reduce its dependence on NVIDIA chips and promote the autonomy of AI technology.
The Chitu inference framework challenges the NVIDIA Hopper series
Chitu is a high-performance large language model (LLM) inference framework that can run efficiently on domestic chips, thus challenging the dominance of NVIDIA's Hopper series GPUs in the field of AI inference. The AI framework is the cornerstone of building complex, intelligent AI models, providing developers with a set of libraries and tools to efficiently design, train, and validate complex models. Qingcheng.AI and the team of Zhai Jidong, a professor of computer science at Tsinghua University, said in a joint statement that the framework was open-sourced on Friday and is compatible with mainstream large models, including DeepSeek and Meta's Llama series.
The test data shows that when Chitu uses NVIDIA A800 GPU to run the full version of DeepSeek-R1, its inference speed is increased by 315% compared with foreign open-source frameworks, and the GPU usage is reduced by 50%. This achievement shows that the Chitu framework can achieve more efficient inference performance with limited computing resources, providing an alternative for domestic AI companies to reduce their dependence on NVIDIA.
Figure: Tsinghua team launched the Chitu inference framework to help China's AI autonomy process (Source: South China Morning Post)
The development of the domestic AI chip ecosystem is accelerating
In recent years, due to the impact of U.S. export controls, high-end AI chips such as Nvidia's H100 and H800 have been banned from being sold to the Chinese market, prompting domestic technology companies to accelerate the pace of technology self-development. The launch of the Chitu framework is an important measure for China's AI industry to strengthen its technological autonomy and controllability in this context.
Qingcheng.AI was founded in 2023 by Zhai Jidong and his Tsinghua team, and Zhai currently serves as the company's chief scientist. The company is backed by the Beijing Artificial Intelligence Industry Fund and has established partnerships with leading GPU manufacturers in China, including Moore Threads, Enflame and Iluvatar CoreX.
Not only that, China's AI industry is promoting the improvement of the local technology ecosystem at an unprecedented speed. In February this year, Infinigence AI, an AI computing infrastructure platform, announced that it was strengthening collaboration among seven major AI chip companies in China, including Biren, Hygon, MetaX, Moore Threads, Suiyuan, Tiantian Zhixin and Huawei's Ascend.
Domestic AI infrastructure continues to be optimized
In addition to the construction of the hardware ecosystem, Chinese technology companies are also continuously optimizing the efficiency of AI model training and inference at the software level. According to a recent research paper published by ByteDance, the team has improved the training efficiency of large language models by 170% by optimizing the system, and has been put into use in some production environments, saving millions of GPU computing hours.
Overall, with the launch of the Chitu inference framework and the multi-faceted layout of domestic AI companies at the chip, infrastructure and algorithm levels, China's AI industry is moving towards a more autonomous future.
