Julia has two sisters and one brother. How many sisters does her brother Martin have? Solving this tiny puzzle requires a bit of thinking. You might mentally picture the family of three girls and one boy and then realize that the boy has three sisters. Or you might figure out a more general rule: Any boy in the family will have one more sister than any girl. In other words, the answer to such a puzzle isn't something you immediately know, like Paris is the capital of France
it requires reasoning, a central feature of human intelligence, and one that large language models (LLMs) like GPT-4, for all their impressive behavior, struggle with.