GPT (Generative Pre-trained Transformer) is a type of LLM that is pre-trained on massive amounts of text and then prompted or fine-tuned to generate human-like text. It is not programmed with rules — it learns from data and predicts the most likely next word.
In simple terms
GPT reads billions of words, learns the patterns, then generates the next word, one at a time.
How it works
- 1Pre-training: learn patterns from massive text data.
- 2Transformer architecture: use self-attention to understand relationships between all words.
- 3Next-token prediction: predict the next word based on context.
- 4Generate: repeat the process to produce one word at a time.
- 5Human-like output: produce coherent, relevant, context-aware text.
Key points
- GPT is a decoder-only transformer that generates text left to right.
- It supports zero/few-shot learning — it can work from instructions alone.
- It is general-purpose: one model handles many tasks.
- It is the foundation for products like ChatGPT and code assistants.
Why it matters
GPT showed that a single, general text generator could write, explain, answer, summarise and code. It reframed AI from task-specific tools to one flexible model steered by prompts.
Frequently asked questions
- What does 'pre-trained' mean?
- The model first learns general language from huge text data, before being adapted or prompted for specific tasks.
- What is self-attention?
- A mechanism that lets the model weigh how much each word relates to every other word when predicting the next one.
More in Language Models
Large Language Model (LLM) — The Basics
Massive data → training → understanding → generation.
What is a Large Language Model (LLM)?
Inside an LLM — tokenization, embeddings, transformer.
GPT-4
OpenAI's multimodal model — text and images.
ChatGPT
From your question to the answer, step by step.
Claude
Anthropic's helpful, harmless and honest AI assistant.
PaLM / Bard
Google's conversational AI powered by PaLM.