G: ChatGPT is a Generative model
The G in GPT stands for “Generative”. This means that the model’s key goal and specialty is to generate text. This is important, as it really stresses the model’s generative objective: the model always “makes up” text (see “Hallucinations”), as this is its primary goal. Indeed, based on the patterns learnt during their pre-training phase (see “P”), GPT models will generate answers of their own, and more in general will create new texts. This can happen thanks to a special component of the Transformer architecture (see “T”), called decoder; this incorporates an attention mechanism which allows the model to learn which words in the input are the most important for predicting the next word.
