P: ChatGPT is a Pre-trained model
The P in GPT stands for “Pre-trained”, indicating that the model has already been trained. Training means that it has undergone the whole process where by means of a Transformer architecture (see “T”) the model can learn language from a lot (truly A LOT) of provided examples in the form of running text. During training, the model makes guesses about what should come next in text and it gets progressively better by checking and adjusting its answers. Once a model is trained, additional specialisations can be taught to it (a process called “fine-tuning”.) When we use ChatGPT, or any pre-trained language model, we do not have to teach it to process and generate language every time from scratch, but we can use it as is, as it’s already been trained to do that!
