What do We Understand about What ChatGPT “Understands”?

The increase in the size of Large language models (LLMs), and the observation that this induces scale effects that unleash unsuspected performance, has given rise to a controversy between those for whom this would be the emergence of reasoning faculties—LLMs would constitute the premises of a “general artificial intelligence”—and those for whom this is merely a statistical mirage: LLMs would be no more than “stochastic parrots”. While neither of these positions stands up to scrutiny, studying them helps us to better understand why, good or bad, LLMs are capable of producing such good answers.