How does ChatGPT work? It is a large language model: a neural network that reads your message as a string of small text pieces called tokens and predicts, one token at a time, what text should come next. It does not look up answers in a database unless a search tool is switched on. The same mechanics decide how AI assistants describe businesses, which is the ground that AI search optimisation works on.

That one fact explains most of what confuses people. The tool writes a fluent supplier email in seconds and also states a wrong tax threshold with the same confidence. Both behaviours come from the same process, and once you see the process you can predict which job you are about to get.

Quick Answer

How does ChatGPT work: it is trained on very large amounts of text to predict the next token, then tuned after pre-training so its answers follow instructions (OpenAI's published method used human feedback; the current recipe is not fully public). When you send a prompt, it generates a reply token by token from patterns it learned, and it can use a web search tool for current facts when one is available. It produces plausible text, not verified facts, so anything with a Rand figure, a legal threshold or a customer's name in it needs a check against the original source.

Using AI answers for work you cannot afford to get wrong?

Tell us where AI is being used in your business and we will talk through where a human check belongs.

Talk to us

How Does ChatGPT Work? The Five Stages From Training to Answer

ChatGPT works in five stages: pre-training on text, post-training tuning, turning your prompt into tokens, generating a reply token by token, and optionally calling tools such as web search. The first two happen before you ever type anything. The last three happen when you press send.

StageWhat happensWhat it means for you
1. Pre-trainingThe network reads huge amounts of text and learns to predict the next tokenIts knowledge is a compressed pattern of that text, with a cut-off date
2. Post-training tuningIn OpenAI's published InstructGPT method, people wrote example answers and ranked outputs, and the model was adjusted toward preferred ones; ChatGPT's current recipe is not fully publicIt sounds helpful and follows instructions, which is not the same as being correct
3. Your prompt becomes tokensYour text, plus the earlier conversation, is split into tokensOnly what is in the conversation (and any saved memory) is available to it
4. GenerationIt predicts one token, adds it, then predicts the next, until the reply is doneThe method is built to produce fluent text; accuracy is a separate question
5. Tools (optional)It may search the web or run other tools, then write the answer using the resultsCurrent facts only arrive if this stage fires

The design underneath is the Transformer, introduced in the 2017 paper Attention Is All You Need, which built a network on attention mechanisms alone instead of recurrence. OpenAI's GPT models belong to this family; the name stands for generative pre-trained transformer.

What Is a Token, and How Does a Prediction Become an Answer?

A token is a small chunk of text, often a word or part of a word, and a language model's whole job is to assign probabilities to which token comes next. It then picks one, appends it to the text so far and repeats. Even a short email is therefore a long run of separate predictions made in sequence.

OpenAI's GPT-3 research describes the model as autoregressive, meaning each output depends on what came before. In that paper's abstract, tasks and examples are given to the model purely as text, with no retraining. This is why a good prompt works: you are supplying the context the next prediction is conditioned on. Give it your brand voice, your audience and two sample sentences and the predictions shift toward that style.

How ChatGPT is trained: pre-training versus tuning

Pre-training teaches the model language and a broad statistical picture of the world; tuning teaches it to behave like an assistant. To see how ChatGPT is trained at the second step, the best public evidence is OpenAI's 2022 InstructGPT research. In that method, human labellers wrote demonstrations of good answers, the model was fine-tuned on them, and then people ranked competing outputs so the model could be adjusted toward the preferred ones. InstructGPT was a precursor to ChatGPT, and current recipes are not fully published. The paper reports that a 1.3B-parameter tuned model was preferred over the 175B-parameter GPT-3 despite having 100x fewer parameters.

Takeaway: tuning changes tone, not truth

Tuning after pre-training is what makes a model follow instructions and sound helpful. The InstructGPT result shows human-feedback tuning can make a much smaller model preferred by human raters, which is a measure of preference, not of accuracy. A confident, well-organised answer tells you the tuning worked; it tells you nothing about whether the content is right.

OpenAI has not published the full training mix for its current models, so any article that tells you exactly which websites ChatGPT learned from is guessing. The practical point holds either way: the model carries patterns from text it saw before a cut-off date, not a live copy of your supplier's price list or SARS's current rules.

Why Does ChatGPT Get Facts Wrong?

ChatGPT gets facts wrong because producing a plausible next token and producing a true statement are different goals, and the training process rewards the first more reliably than the second. Researchers call the confident false output a hallucination.

A 2025 paper by Kalai, Nachum, Vempala and Zhang argues that language models hallucinate because "the training and evaluation procedures reward guessing over acknowledging uncertainty." That is one research group's account of the cause, but it matches the behaviour any heavy user recognises: a model that would rather answer than say it does not know.

Where this bites a South African operator

You ask a general-purpose model whether a business must register for VAT. A model with an older cut-off may still give the previous R1 million compulsory threshold. SARS says the compulsory threshold is R2.3 million of taxable supplies per annum from 1 April 2026, and R120,000 for voluntary registration, per its Budget 2026 FAQ. That FAQ adds that the amendments were not yet formally promulgated and that SARS is applying them to new registration applications. A model without current information has no way to reflect a change like that.

The pattern generalises. Thresholds, prices, deadlines and regulatory rules can change after a model's cut-off, and older versions may still sit in its training text. Treat any such number from a model as a lead to verify, not an answer to use.

In OpenAI's API web search tool, the model decides whether a prompt needs a web search, and citations for the URLs it found are included by default. Search is the stage of how ChatGPT works that connects the model to current pages, which makes it the stage a business can most plausibly influence.

OpenAI's developer documentation says the model can choose to search the web or not based on the content of the prompt. That page describes the API tool. The ChatGPT app has its own search controls, documented in OpenAI's help centre, which was not accessible when this article was researched, so check it for app-specific behaviour.

For a business owner, this splits visibility into two routes. One is the model's memory from training, which you cannot edit. The other is the live search route, where your pages, reviews and mentions can be retrieved and cited. How that retrieval ranks sources is only partly documented, so treat advice that claims to know the formula as practitioner opinion. Our guide to ChatGPT search optimisation covers what is worth doing on that route, and getting ChatGPT to recommend your business covers the local angle.

How does ChatGPT generate answers from your own documents and chat history?

ChatGPT generates answers only from what is in the conversation window, any saved memory feature you have switched on, and any tool results. If you want it to use your price list, paste or attach it. If you want it to forget a conversation, check the memory and data-control settings on OpenAI's own pages, which change as the product changes.

Not sure whether your business shows up when people ask AI?

Send us your domain and three questions your customers ask, and we will tell you where to look first.

Ask about an audit

Which South African Business Tasks Suit ChatGPT, and Which Need a Source?

Tasks that suit ChatGPT are ones where a fluent, well-shaped draft is the goal and you will read it before it goes anywhere; tasks that do not suit it depend on a specific fact being exactly right. The table maps common jobs to the stage of the process that makes them safe or risky.

TaskFitWhy, by mechanismBest for
Rewrite a supplier or customer emailStrongStyle transfer is pattern work; your text supplies the factsDrafts you will read
Summarise a document you paste inStrong, spot-check figuresThe source text is in the conversationLong reports, meeting notes
State the current VAT thresholdWeak without searchA model with an older cut-off may give the previous R1 million figure; the rule changed in 2026Use SARS directly
Quote competitor prices in RandWeakPrices change daily and may sit behind pages the model cannot readCheck each site
Brainstorm campaign anglesStrongVariety of plausible options is what prediction is good atIdea generation, then your judgement
Paste a customer list to clean itNeeds a decision firstPersonal information falls under POPIA wherever it goes; check plan terms and data controlsAnonymised samples

Use in South Africa is growing. Microsoft's Q1 2026 diffusion report, as reported by Ecofin, put generative AI adoption in South Africa at 23.1% of people aged 15-64 (Q1 2026), up from 19.3% a year earlier, the highest in Africa.

On task mix, The Open Letter, citing OpenAI's own data, reports that 40.1% of South African ChatGPT messages were work-related, against a 30.7% global average, with document editing the main work task. Those figures cover ChatGPT messages, not all AI use, and both sources are secondary. They are consistent with drafting and editing being a common local use, which is where the mechanism is strongest.

A three-question check before you trust an answer

Does it contain a figure, date or rule that could have changed? Would a wrong answer cost money or break a law? Did it cite a source you can open? If the first two are yes and the third is no, verify at the original source. For comparisons of how different assistants handle this, see our posts on Claude versus ChatGPT and the differences between ChatGPT, Google and Perplexity.

What This Means for Your Prompts and Your Budget

A good prompt supplies role, audience, source material and format, because the model predicts from the context it is given. Our ChatGPT prompts for marketing post has tested examples built on that principle. If you are weighing the paid tier, ChatGPT Plus pricing in Rand is covered separately, and developers should read the OpenAI API pricing breakdown.

If you are deciding where AI fits into daily operations, start with one drafting task, keep a human reading every output, and widen only when you can say what the check will catch. Our guide to AI automation for small businesses sets out where to begin.

Why South African Businesses Choose Growth Pulse Media

Growth Pulse Media is a Johannesburg agency founded by Dirk van Greuning, who built and scaled a large South African ecommerce business before starting the agency. All work is executed in-house, and the client load is kept limited so senior attention stays on each account. For businesses that want to understand and improve how AI assistants describe them, the AEO agency page sets out the service. We start from sourced mechanics like those above rather than guesses about ranking formulas.

Who This Is NOT For

This article is not for readers who want a sure-fire ChatGPT citation, a data-handling shortcut, or the model's technical internals.

You want a sure-fire way to be cited by ChatGPT

Nobody outside OpenAI can promise that. Retrieval and ranking are only partly documented, so an honest plan improves your chances and measures them, rather than promising a mention.

You want to hand customer data to a chatbot without a policy

If the plan is to paste personal information into whichever tool is open, the first job is a POPIA-aware data policy, not a better prompt.

You need the model's technical internals

This is a plain-language explanation for operators. If you need architecture detail, start with the papers linked above.

Ready to turn this into a plan?

Share your site and your top three customer questions and we will outline what to audit first.

Start with an audit

Frequently Asked Questions

How does ChatGPT work in simple terms?

It is a language model trained on large amounts of text to predict the next token, then tuned after pre-training to behave like an assistant. When you send a message it generates a reply one token at a time. It can use web search for current information when search is available.

Does ChatGPT look up answers on the internet?

It can, when a search tool is switched on. In OpenAI's API web search tool, the model chooses whether to search based on the prompt, and citations are included by default; the ChatGPT app has its own search controls. Without search, it answers from patterns learned in training.

Why does ChatGPT sometimes make things up?

Generating plausible text and generating true text are different goals. One 2025 research paper argues models hallucinate because training and evaluation reward guessing over admitting uncertainty. Check any figure, date or legal rule against the original source.

Does ChatGPT learn from my conversations?

That depends on your plan and your data-control settings, which OpenAI sets and changes on its own pages. Check them before pasting anything sensitive, and remember that customer or staff personal information is covered by POPIA wherever you put it.

Is ChatGPT the same as GPT?

No. GPT refers to the family of underlying models; ChatGPT is the product built around them, with the chat interface, tuning, memory and tools. Our AI models explained post covers how GPT, Claude and Gemini differ.

Get a clear view of what AI says about your business

Johannesburg-based, all work executed in-house, limited client load. No obligation, and we will get back to you within 24 hours.

Get your free audit
Dirk van Greuning — Founder, Growth Pulse Media
Dirk van Greuning Founder, Growth Pulse Media

Founder of Growth Pulse Media and a specialist in South African search dominance. Dirk translates his experience in scaling South African businesses into high-velocity digital strategies for B2B and retail leaders. He writes about SEO, lead generation, and paid media from an operator's perspective — prioritising pipeline value over impressions.

Connect on LinkedIn