🔴 DeepSeek-R1: A Tech and Business Recap

Part 1 DeepSeek-R1: the technical stakes

Introduction

2025 marks AI's turn toward optimization and distillation rather than the raw expansion of models. DeepSeek-R1 favors efficiency and a targeted architecture, the opposite of the American trend toward gigantic models requiring hundreds of billions of dollars. Their bet? An AI that is smarter, more optimized and, above all, far more frugal with resources.

Hacker mindset: squeezing the most out of limited resources

Faced with American sanctions, DeepSeek had to optimize its NVIDIA H800 GPUs despite their limited bandwidth:

• 20 of the 132 processing units are specifically dedicated to chip-to-chip communication.
• Optimization at the PTX level (low-level assembly) instead of CUDA. An approach reminiscent of the 8-bit developers who optimized their games in assembler.
• Usually reserved for low-power embedded systems, this extreme optimization is now creeping into power-hungry LLMs.

Smaller models, but smarter ones

While other labs train ever larger models, DeepSeek adopts a distillation strategy with 7B and 14B AIs, ultra-optimized for specific tasks.

• No more "polymath" AIs trying to understand everything. Efficiency comes through specialization.
• The famous scaling law is not dead, but it is moving closer to the hardware instead of being purely theoretical.

The end of the stochastic parrots?

Classic LLMs predict the next token, generating probabilistic answers that are sometimes fuzzy. DeepSeek innovates by introducing pure reinforcement learning, inspired by human reasoning:

• Take the multiplication 144 × 768: a human does not guess the answer, they break it down step by step.
• DeepSeek-R1 learns on its own to structure its reasoning, without explicit supervision.
• The result: less approximation, more precision in its calculations and step-by-step reasoning.

Chain-of-Thought, CoT

DeepSeek takes an implicit approach to CoT, structuring its data to force step-by-step logic.

• Unlike OpenAI, which presents reasoning as a black box, DeepSeek documents and exposes its approach.
• A transparency that OpenAI will copy, because it builds trust among researchers and users.

Mixture of Experts (MoE)

A key difference between DeepSeek and ChatGPT lies in their architecture:

• ChatGPT (standard Transformer model): all parameters are activated for every query.
• DeepSeek (Mixture of Experts, MoE): only the relevant experts are activated for each query.

Why is that more efficient?

• Imagine a room with 100 experts.
• ChatGPT asks everyone, even those with nothing to say on the subject.
• MoE (DeepSeek) activates only the most competent ones, optimizing computing load and performance.

The result: fewer resources used, with equal or even better quality.

Technical explanations for non-experts

1. Model distillation

📌 Definition: A smaller model imitates a larger one while retaining high performance.

📌 Example: Turning an encyclopedia into a compact book without losing any essential information.

2. Assembly-level optimization (PTX vs. CUDA)

📌 Definition: Ultra-low-level programming that gives direct control over the hardware.

📌 Example: A racing mechanic who manually tunes an engine to maximize its performance.

3. Memory bandwidth constraints

📌 Definition: A limit on how fast data can be exchanged within the system.

📌 Example: Trying to fill bottles from a tap that is too narrow. DeepSeek found a way to speed up the flow.

4. Why go down to the PTX level?

📌 Definition: PTX offers maximum control over the GPU, unlike the more abstract CUDA.

📌 Example: Giving a robot detailed instructions instead of letting it interpret general orders.

Conclusion of part one

DeepSeek-R1 proves that the future of AI lies not only in the size of models, but in smart, targeted optimization.

Part 2:  An economic earthquake for the big players, an opportunity for the small ones?