How does ChatGPT work? It is a large language model: a neural network that reads your message as a string of small text pieces called tokens and predicts, one token at a time, what text should come next. It does not look up answers in a database unless a search tool is switched on. The same mechanics decide how AI assistants describe businesses, which is the ground that AI search optimisation works on.
That one fact explains most of what confuses people. The tool writes a fluent supplier email in seconds and also states a wrong tax threshold with the same confidence. Both behaviours come from the same process, and once you see the process you can predict which job you are about to get.
Quick Answer
How does ChatGPT work: it is trained on very large amounts of text to predict the next token, then tuned after pre-training so its answers follow instructions (OpenAI's published method used human feedback; the current recipe is not fully public). When you send a prompt, it generates a reply token by token from patterns it learned, and it can use a web search tool for current facts when one is available. It produces plausible text, not verified facts, so anything with a Rand figure, a legal threshold or a customer's name in it needs a check against the original source.
Jump to
Using AI answers for work you cannot afford to get wrong?
Tell us where AI is being used in your business and we will talk through where a human check belongs.
Talk to usHow Does ChatGPT Work? The Five Stages From Training to Answer
ChatGPT works in five stages: pre-training on text, post-training tuning, turning your prompt into tokens, generating a reply token by token, and optionally calling tools such as web search. The first two happen before you ever type anything. The last three happen when you press send.
| Stage | What happens | What it means for you |
|---|---|---|
| 1. Pre-training | The network reads huge amounts of text and learns to predict the next token | Its knowledge is a compressed pattern of that text, with a cut-off date |
| 2. Post-training tuning | In OpenAI's published InstructGPT method, people wrote example answers and ranked outputs, and the model was adjusted toward preferred ones; ChatGPT's current recipe is not fully public | It sounds helpful and follows instructions, which is not the same as being correct |
| 3. Your prompt becomes tokens | Your text, plus the earlier conversation, is split into tokens | Only what is in the conversation (and any saved memory) is available to it |
| 4. Generation | It predicts one token, adds it, then predicts the next, until the reply is done | The method is built to produce fluent text; accuracy is a separate question |
| 5. Tools (optional) | It may search the web or run other tools, then write the answer using the results | Current facts only arrive if this stage fires |
The design underneath is the Transformer, introduced in the 2017 paper Attention Is All You Need, which built a network on attention mechanisms alone instead of recurrence. OpenAI's GPT models belong to this family; the name stands for generative pre-trained transformer.
What Is a Token, and How Does a Prediction Become an Answer?
A token is a small chunk of text, often a word or part of a word, and a language model's whole job is to assign probabilities to which token comes next. It then picks one, appends it to the text so far and repeats. Even a short email is therefore a long run of separate predictions made in sequence.
OpenAI's GPT-3 research describes the model as autoregressive, meaning each output depends on what came before. In that paper's abstract, tasks and examples are given to the model purely as text, with no retraining. This is why a good prompt works: you are supplying the context the next prediction is conditioned on. Give it your brand voice, your audience and two sample sentences and the predictions shift toward that style.
How ChatGPT is trained: pre-training versus tuning
Pre-training teaches the model language and a broad statistical picture of the world; tuning teaches it to behave like an assistant. To see how ChatGPT is trained at the second step, the best public evidence is OpenAI's 2022 InstructGPT research. In that method, human labellers wrote demonstrations of good answers, the model was fine-tuned on them, and then people ranked competing outputs so the model could be adjusted toward the preferred ones. InstructGPT was a precursor to ChatGPT, and current recipes are not fully published. The paper reports that a 1.3B-parameter tuned model was preferred over the 175B-parameter GPT-3 despite having 100x fewer parameters.
Takeaway: tuning changes tone, not truth
Tuning after pre-training is what makes a model follow instructions and sound helpful. The InstructGPT result shows human-feedback tuning can make a much smaller model preferred by human raters, which is a measure of preference, not of accuracy. A confident, well-organised answer tells you the tuning worked; it tells you nothing about whether the content is right.
OpenAI has not published the full training mix for its current models, so any article that tells you exactly which websites ChatGPT learned from is guessing. The practical point holds either way: the model carries patterns from text it saw before a cut-off date, not a live copy of your supplier's price list or SARS's current rules.
Why Does ChatGPT Get Facts Wrong?
ChatGPT gets facts wrong because producing a plausible next token and producing a true statement are different goals, and the training process rewards the first more reliably than the second. Researchers call the confident false output a hallucination.
A 2025 paper by Kalai, Nachum, Vempala and Zhang argues that language models hallucinate because "the training and evaluation procedures reward guessing over acknowledging uncertainty." That is one research group's account of the cause, but it matches the behaviour any heavy user recognises: a model that would rather answer than say it does not know.
Where this bites a South African operator
You ask a general-purpose model whether a business must register for VAT. A model with an older cut-off may still give the previous R1 million compulsory threshold. SARS says the compulsory threshold is R2.3 million of taxable supplies per annum from 1 April 2026, and R120,000 for voluntary registration, per its Budget 2026 FAQ. That FAQ adds that the amendments were not yet formally promulgated and that SARS is applying them to new registration applications. A model without current information has no way to reflect a change like that.
The pattern generalises. Thresholds, prices, deadlines and regulatory rules can change after a model's cut-off, and older versions may still sit in its training text. Treat any such number from a model as a lead to verify, not an answer to use.
How Does ChatGPT Search the Web, and When Does It Do So?
In OpenAI's API web search tool, the model decides whether a prompt needs a web search, and citations for the URLs it found are included by default. Search is the stage of how ChatGPT works that connects the model to current pages, which makes it the stage a business can most plausibly influence.
OpenAI's developer documentation says the model can choose to search the web or not based on the content of the prompt. That page describes the API tool. The ChatGPT app has its own search controls, documented in OpenAI's help centre, which was not accessible when this article was researched, so check it for app-specific behaviour.
For a business owner, this splits visibility into two routes. One is the model's memory from training, which you cannot edit. The other is the live search route, where your pages, reviews and mentions can be retrieved and cited. How that retrieval ranks sources is only partly documented, so treat advice that claims to know the formula as practitioner opinion. Our guide to ChatGPT search optimisation covers what is worth doing on that route, and getting ChatGPT to recommend your business covers the local angle.
How does ChatGPT generate answers from your own documents and chat history?
ChatGPT generates answers only from what is in the conversation window, any saved memory feature you have switched on, and any tool results. If you want it to use your price list, paste or attach it. If you want it to forget a conversation, check the memory and data-control settings on OpenAI's own pages, which change as the product changes.
Not sure whether your business shows up when people ask AI?
Send us your domain and three questions your customers ask, and we will tell you where to look first.
Ask about an auditWhich South African Business Tasks Suit ChatGPT, and Which Need a Source?
Tasks that suit ChatGPT are ones where a fluent, well-shaped draft is the goal and you will read it before it goes anywhere; tasks that do not suit it depend on a specific fact being exactly right. The table maps common jobs to the stage of the process that makes them safe or risky.
| Task | Fit | Why, by mechanism | Best for |
|---|---|---|---|
| Rewrite a supplier or customer email | Strong | Style transfer is pattern work; your text supplies the facts | Drafts you will read |
| Summarise a document you paste in | Strong, spot-check figures | The source text is in the conversation | Long reports, meeting notes |
| State the current VAT threshold | Weak without search | A model with an older cut-off may give the previous R1 million figure; the rule changed in 2026 | Use SARS directly |
| Quote competitor prices in Rand | Weak | Prices change daily and may sit behind pages the model cannot read | Check each site |
| Brainstorm campaign angles | Strong | Variety of plausible options is what prediction is good at | Idea generation, then your judgement |
| Paste a customer list to clean it | Needs a decision first | Personal information falls under POPIA wherever it goes; check plan terms and data controls | Anonymised samples |
Use in South Africa is growing. Microsoft's Q1 2026 diffusion report, as reported by Ecofin, put generative AI adoption in South Africa at 23.1% of people aged 15-64 (Q1 2026), up from 19.3% a year earlier, the highest in Africa.
On task mix, The Open Letter, citing OpenAI's own data, reports that 40.1% of South African ChatGPT messages were work-related, against a 30.7% global average, with document editing the main work task. Those figures cover ChatGPT messages, not all AI use, and both sources are secondary. They are consistent with drafting and editing being a common local use, which is where the mechanism is strongest.
A three-question check before you trust an answer
Does it contain a figure, date or rule that could have changed? Would a wrong answer cost money or break a law? Did it cite a source you can open? If the first two are yes and the third is no, verify at the original source. For comparisons of how different assistants handle this, see our posts on Claude versus ChatGPT and the differences between ChatGPT, Google and Perplexity.
What This Means for Your Prompts and Your Budget
A good prompt supplies role, audience, source material and format, because the model predicts from the context it is given. Our ChatGPT prompts for marketing post has tested examples built on that principle. If you are weighing the paid tier, ChatGPT Plus pricing in Rand is covered separately, and developers should read the OpenAI API pricing breakdown.
If you are deciding where AI fits into daily operations, start with one drafting task, keep a human reading every output, and widen only when you can say what the check will catch. Our guide to AI automation for small businesses sets out where to begin.
Why South African Businesses Choose Growth Pulse Media
Growth Pulse Media is a Johannesburg agency founded by Dirk van Greuning, who built and scaled a large South African ecommerce business before starting the agency. All work is executed in-house, and the client load is kept limited so senior attention stays on each account. For businesses that want to understand and improve how AI assistants describe them, the AEO agency page sets out the service. We start from sourced mechanics like those above rather than guesses about ranking formulas.
Who This Is NOT For
This article is not for readers who want a sure-fire ChatGPT citation, a data-handling shortcut, or the model's technical internals.
You want a sure-fire way to be cited by ChatGPT
Nobody outside OpenAI can promise that. Retrieval and ranking are only partly documented, so an honest plan improves your chances and measures them, rather than promising a mention.
You want to hand customer data to a chatbot without a policy
If the plan is to paste personal information into whichever tool is open, the first job is a POPIA-aware data policy, not a better prompt.
You need the model's technical internals
This is a plain-language explanation for operators. If you need architecture detail, start with the papers linked above.
Ready to turn this into a plan?
Share your site and your top three customer questions and we will outline what to audit first.
Start with an auditFrequently Asked Questions
How does ChatGPT work in simple terms?
It is a language model trained on large amounts of text to predict the next token, then tuned after pre-training to behave like an assistant. When you send a message it generates a reply one token at a time. It can use web search for current information when search is available.
Does ChatGPT look up answers on the internet?
It can, when a search tool is switched on. In OpenAI's API web search tool, the model chooses whether to search based on the prompt, and citations are included by default; the ChatGPT app has its own search controls. Without search, it answers from patterns learned in training.
Why does ChatGPT sometimes make things up?
Generating plausible text and generating true text are different goals. One 2025 research paper argues models hallucinate because training and evaluation reward guessing over admitting uncertainty. Check any figure, date or legal rule against the original source.
Does ChatGPT learn from my conversations?
That depends on your plan and your data-control settings, which OpenAI sets and changes on its own pages. Check them before pasting anything sensitive, and remember that customer or staff personal information is covered by POPIA wherever you put it.
Is ChatGPT the same as GPT?
No. GPT refers to the family of underlying models; ChatGPT is the product built around them, with the chat interface, tuning, memory and tools. Our AI models explained post covers how GPT, Claude and Gemini differ.
Get a clear view of what AI says about your business
Johannesburg-based, all work executed in-house, limited client load. No obligation, and we will get back to you within 24 hours.
Get your free audit

