🤖 Modèles

Grands modèles de langage

LLM

Large language model

Grand modèle de langage

GPT

Generative pre-trained transformer

Transformers

Transformer (deep learning architecture)

Transformers (how LLMs work) explained visually (YouTube)

Entraînement

Source : Schéma des étapes de construction d'un modèle LLM

Pré-entraînement

Understanding LLMs: A Comprehensive Overview from Training to Inference

The training of LLMs can be broadly divided into three steps.

[1] The first step involves data collection and processing.

[2] The second step encompasses the pre-training process, which includes determining the model’s architecture and pretraining tasks and utilizing suitable parallel training algorithms to complete the training.

[3] The third step involves finetuning and alignment. In this section, we will provide an overview of the model training techniques. This will include an introduction to the relevant training datasets, data preparation and preprocessing, model architecture, specific training methodologies, model evaluation, and commonly used training frameworks for LLMs

Post-entraînement

Reinforcement learning

Apprentissage par renforcement

GRPO

To post-train models, we take a pre-trained base model, do supervised fine-tuning on a broad set of ideal responses written by humans or existing models, and then run reinforcement learning with reward signals from a variety of sources.
During reinforcement learning, we present the language model with a prompt and ask it to write responses. We then rate its response according to the reward signals, and update the language model to make it more likely to produce higher-rated responses and less likely to produce lower-rated responses.

Source : OpenAI, Expanding on what we missed with sycophancy

Waylon the Sycophant

Évolution

Algorithmic progress in language models