DeepSeek: the disruptor

DeepSeek has achieved powerful results by removing bottlenecks in artificial intelligence development.

The start-up's R1 “reasoning” model and its ability to "distil" the capabilities of large models into smaller ones could transform how developers build applications based on the technology.

The current state

Its developments sent shockwaves throughout Silicon Valley as US rivals reacted to the Chinese company's breakthrough, which suggests its models may be faster, cheaper to develop and as capable as those used and developed by industry leaders such as OpenAI, Google Deep Mind and Anthropic.

How LLMs are built

Large language models are built in two stages.

The first is called “pre-training”, in which developers use massive data sets that help models to predict the next word in a sentence.

The second stage is called “post-training”, through which developers teach the model to follow instructions, such as solving maths problems or coding.

Refining the model

One technique used in LLM refinement is a process called “reinforcement learning from human feedback” (RLHF), a technique pioneered by OpenAI to improve ChatGPT. This step can be laborious, expensive and time consuming, often requiring a small army of human data labellers.

DeepSeek’s innovation

DeepSeek’s industry-shaking breakthrough automates this final step, using a technique that rewards the AI model for doing the right thing. The Chinese company has also built smaller models that can be run on phones or web browsers and, in some cases, outperform flagship models.

The future of AI?

While DeepSeek has been the first to use its particular techniques, similar concepts have been developed elsewhere, and other AI labs are expected to follow suit.