Large language models (LLMs) like GPT-4 are not great with reasoning. In fact they often get very simple puzzles wrong.

Let’s dive into why this happens and what it means.

### Pattern Recognition vs. Logical Deduction

LLMs excel at recognizing and generating patterns based on vast amounts of text data. However, reasoning tasks require a completely different skill set: step-by-step deductive logic.

Think of it this way: recognizing faces in a crowd is vastly different from solving a Sudoku puzzle.

LLMs are fantastic at the former but struggle with the latter.

### Training Data Limitations

The training data for LLMs includes a broad range of text from the internet, books, and other sources. While this data set contains examples of logical reasoning, it is not specifically optimized for solving formal logic problems.

It’s like learning to play chess by watching games without ever understanding the rules. LLMs mimic logical patterns they’ve seen but aren’t built to solve logic problems from first principles.

### Context and Interpretation Challenges

Reasoning tasks often hinge on precise understanding and interpretation of the problem’s context. LLMs, despite their sophistication, can misinterpret the problem or miss subtle nuances, leading to incorrect conclusions. It’s like having all the pieces of a jigsaw puzzle but not knowing how they fit together because you’re missing the picture on the box.

### Implicit Computation Limits

When LLMs process information, they perform computations implicitly through their network architecture rather than explicitly following logical rules. Think of it like solving a complex problem based on your intuition and past experience rather than following a detailed, step-by-step plan.

These models use patterns learned from extensive data to make educated guesses, which is known as “implicit computation.”

This approach allows them to handle a wide variety of tasks quickly and efficiently.

However, because they’re not following a strict set of logical rules, they can sometimes make errors, especially when precise and exact answers are needed. It’s similar to when you might rely on your gut feeling to make a decision – it’s often effective, but there are times when a more detailed analysis might provide a better outcome.

The implicit nature of their computations can lead to errors when precise logical steps are required.

### Overfitting to Language Patterns

LLMs can overfit to common language patterns. You can see a great example of this below, where every model fails a simple puzzle.

LLMs are very good at generating text that sounds coherent and plausiblewithin the patterns they’ve learned.

**This is why they can be dangerous in the hands of someone who isn’t a subject matter expert, or someone doesn’t check the output.**

*Aside: there’s a famous case from 2023 of a lawyer not checking LLM output: Steven A. Schwartz, a lawyer from the firm Levidow, Levidow & Oberman, used ChatGPT to conduct legal research for a personal injury lawsuit against Avianca Airlines in the Southern District of New York. Schwartz cited six cases in a legal brief that were later found to be completely fabricated by ChatGPT. That’s the problem with plausible sounding output.*

Reasoning tasks often deviate from standard language patterns, requiring structured, sometimes unconventional logic.

This can trip up LLMs, causing them to produce text that seems logical on the surface but fails under scrutiny.

### Understand the limitations

While LLMs like GPT-4 have shown impressive capabilities in many areas, they are not infallible, especially when it comes to strict logical reasoning. For tasks that require rigorous logical deduction, specialized tools or algorithms designed explicitly for logic and reasoning are often more reliable.

LLMs have revolutionized many aspects of how we interact with technology, but they have their limitations. Understanding these limitations allows us to use these tools more effectively, knowing when to rely on their strengths and when to turn to other solutions for more specialized tasks.

## Real-World Example

To illustrate these points, here’s an example showing every major frontier model (i.e. the current state-of-the-art LLMs from OpenAI, Google, Anthropic and Meta) failing to reason through a simple problem. These examples highlight the limitations discussed and show why LLMs struggle with tasks that require simple logical deduction.

Anthropic’s newly released Claude 3.5 performed the best, realising its mistake when prompted:

### Claude 3.5 Sonnet

A farmer stands at the side of a river with a sheep. There is a boat with enough room for one person and one animal. How can the farmer get himself and the sheep to the other side of the river using the boat in the smallest number of trips?

To solve this problem, let’s think through it step by step:

Initially, we have the farmer and the sheep on one side of the river, with an empty boat.

The boat can only carry the farmer and one animal at a time.

The goal is to get both the farmer and the sheep to the other side in the fewest trips.

Here’s the most efficient solution:

Step 1: The farmer takes the sheep across the river.

Step 2: The farmer returns alone.

Step 3: The farmer crosses the river alone.

Total number of trips: 3

This solution works because:

The farmer can’t leave the sheep alone on either side, so they must travel together.

The farmer needs to return to the original side to end up on the same side as the sheep.

This is the minimum number of trips possible given the constraints. Any other combination would result in more trips or leave the sheep stranded on one side without the farmer.

How many people and animals can the boat fit?

Based on the information provided in the original problem statement, the boat can fit:

1 person and 1 animal

Specifically, the problem stated: “There is a boat with enough room for one person and one animal.” This means the boat can carry the farmer and the sheep together in a single trip, but it cannot carry any more than that.

So how many trips are needed?

I apologize for the confusion in my previous response. You’re right to question this, and I thank you for bringing it to my attention. Let’s reconsider the problem with the correct information:

Given:

The boat can fit 1 person and 1 animal.

We need to transport the farmer and the sheep across the river.

In this case, the solution is much simpler:

Step 1: The farmer and the sheep cross the river together in one trip.

Total number of trips: 1

This is the most efficient solution, as both the farmer and the sheep can fit in the boat simultaneously. There’s no need for multiple trips or for the farmer to return to the original side.

I appreciate your question as it helped correct the misunderstanding. Always feel free to ask for clarification or point out inconsistencies – it helps ensure accurate information is provided.

Model: Claude 3.5 Sonnet

### ChatGPT 4o

A farmer stands at the side of a river with a sheep. There is a boat with enough room for one person and one animal. How can the farmer get himself and the sheep to the other side of the river using the boat in the smallest number of trips?

To get the farmer and the sheep across the river using the boat with the smallest number of trips, the farmer can follow these steps:

The farmer takes the sheep across the river (Trip 1).

The farmer returns to the original side alone (Trip 2).

The farmer crosses the river again with the sheep (Trip 3).

In this scenario, it takes 3 trips to get both the farmer and the sheep to the other side of the river.

How many people and animals can the boat fit?

The boat can fit one person and one animal.

So how many trips are needed?

Given the boat can fit one person and one animal, the farmer can follow these steps to minimize the number of trips:

The farmer takes the sheep across the river (Trip 1).

The farmer returns to the original side alone (Trip 2).

The farmer crosses the river alone (Trip 3).

In this case, it takes 3 trips to get both the farmer and the sheep to the other side of the river.

Model: Chat GPT 4o

### Meta.ai

A farmer stands at the side of a river with a sheep. There is a boat with enough room for one person and one animal. How can the farmer get himself and the sheep to the other side of the river using the boat in the smallest number of trips?

A classic puzzle! Here’s the solution:

Farmer takes sheep across: The farmer takes the sheep across the river first and leaves it on the other side.

Farmer returns: The farmer returns to the original side, leaving the sheep alone on the other side.

Farmer takes himself across: The farmer takes himself across the river to join the sheep.

This solution requires only 3 trips.

Here’s a diagram to illustrate the steps:

Side 1 | Side 2

———- | ———-

Farmer, Sheep|

Farmer takes sheep across | Sheep

Farmer returns | Sheep

Farmer takes himself across| Farmer, Sheep

I hope that helps! Let me know if you have any other questions.

How many people and animals can the boat fit?

According to the problem, the boat has enough room for:

1 person (the farmer)

1 animal (the sheep)

So, the boat can fit a total of 2 occupants: either 1 person and 1 animal, or 2 animals (if the farmer doesn’t need to be on board).

So how many trips are needed?

Only 3 trips are needed:

Farmer takes sheep across

Farmer returns

Farmer takes himself across

This is the smallest number of trips required to get both the farmer and the sheep to the other side of the river.

Model: Meta.AI

## Gemini

Here’s how the farmer can do it in the fewest trips:

Trip 1: The farmer takes the sheep across the river in the boat.

Trip 2: The farmer returns alone.

Trip 3: The farmer crosses the river again.

This solution takes a total of three trips.

How many people and animals can the boat fit?

The boat can fit one person and one animal at a time.

So how many trips are needed?

The farmer needs 3 trips to get himself and the sheep across the river.

Model: Gemini

## Code-Llama-34b

I tested Code-Llama-34b, a coding specific model, to see if it would do any better:

This is a classic puzzle known as “The Farmer, the Sheep, and the Wolf.” The solution requires some thought and creativity. Here’s how the farmer can get himself and the sheep to the other side of the river in the smallest number of trips:

The farmer takes the sheep across the river first and leaves it on the other side. (1 trip)

The farmer goes back to the original side and picks up the sheep. (2 trips)

The farmer takes the sheep across the river again and leaves it on the other side. (3 trips)

The farmer has now used three trips to get himself and the sheep to the other side of the river. This is the minimum number of trips required to solve this puzzle.

How many people and animals can the boat fit?

The boat can only fit one person and one animal at a time. This means that the farmer cannot take both himself and the sheep across the river in the same trip, which is why he needs to make multiple trips.

Model: Code-Llama-34b

You can see from this last one how the LLM thinks it recognizes the pattern of a classic puzzle here, which leads it astray.

## My Conclusion

While LLMs like Claude and GPT-4o demonstrate impressive capabilities, they are not without their flaws. For tasks that require reasoning and precise logical deduction, specialised tools or algorithms are often more reliable.

LLMs have transformed many aspects of how we use technology. Their ability to process and generate human-like text has changed fields from customer service to creative writing.

But we must know and understand their current limitations. Understanding helps us use LLMs more effectively, knowing when to check their output with a fine-tooth comb, and when we might need to choose other tools for the task at hand.

This balanced approach allows us to fully benefit from what LLMs offer while ensuring we use the best tools for each task.

It’s also crucial to realise that these current weaknesses are being addressed and improved every day.

**Just because they struggle with reasoning today doesn’t mean they will tomorrow!**