Pre-training
The first, expensive phase in which a model learns general patterns from massive datasets. What you get off the shelf.
01In short
The first, expensive phase in which a model learns general patterns from massive datasets. What you get off the shelf.
02Video
slot · videoPre-training in 90 secondsadd: GUIDES['pre-training'].video
03Guide
slot · guide
A step-by-step guide for “Pre-training” goes here. Suggested outline:
- What it is — in one paragraph
- Why it matters in production
- How to do it — 3 to 7 steps
- Pitfalls we see in the field
04Checklist
slot · checklist
Four to eight things a team can tick before go-live.
05FAQ
slot · FAQ
The three questions clients actually ask about “Pre-training”.
06Related terms
Training & adaptationFine-tuningFurther training a pre-trained model on your own examples to change its behaviour, format or style.BasicsFoundation modelA large model trained on broad data that many applications build on — through prompting, retrieval or fine-tuning. Most LLMs are foundation models.
Where we help · Build