ByteDance trains a very large AI model to compete with top US labs

In short: ByteDance, the owner of TikTok, is training an AI model that could be among the largest in the world.

What happened

ByteDance is at an early stage of training a new AI model that could reach about 10 trillion parameters, according to three people familiar with the work, as reported by the Financial Times. Parameters are the adjustable “knobs” inside an AI model that help it store patterns from data (like the number of parts in a very large engine). More parameters can mean more potential ability, but results also depend on the data used and how training is done.

The report says this would be about three times larger than Moonshot’s Kimi K3, which is described as the biggest Chinese model released so far. Anthropic does not publish model sizes, but industry estimates put Anthropic’s Mythos 5 at about 8 trillion parameters and Fable 5 at about 5 trillion.

One person said ByteDance is in the “pre-training” phase, which usually takes three to six months. Pre-training is the long basic learning stage, before a model is tuned for specific tasks and then potentially released.

ByteDance has kept many of its AI efforts private, and it did not respond to a request for comment. The company’s AI team, Seed, is said to have about 2,000 people and is led by Wu Yonghui, a former Google DeepMind scientist.

Why it matters

Bigger and stronger AI models can lead to better tools for writing, search, and creating images and video, and these tools can end up inside apps people use every day. The report also shows the race is not just between US companies, Chinese firms are spending heavily on data centers (large buildings full of computers) and researchers to try to match or beat the best systems.

Source: Financial Times