We discuss the real world performance of Open AI's GPT-3 language model and speculate on what the future of large pre-trained language models may hold.