Possibly. I tried making a different classical logic puzzle trivial (assuming they’ve fixed this one if it’s doing social media rounds) and got a better response, it gives an edited version of the original solution. Better what’s above, it’s still sort of fooled into giving a much more complex answer than is necessary for the question, and one that’s ultimately incorrect.

Question:

you are on an island where all of the inhabitants always tell the truth. You come across two people, A and B,. You can ask one of them a single question to determine if either is a truth teller. What question should you ask?

Response: To determine if either A or B is a truth-teller when everyone on the island always tells the truth, you can ask a straightforward question to either person. You can ask one of them the following question:

“Is at least one of you a truth-teller?”

Since everyone on the island always tells the truth, here’s the reasoning:

If you ask A (or B), and they respond “Yes,” it means at least one of them is a truth-teller. Given that everyone tells the truth, this statement is accurate, so you can conclude that both A and B are truth-tellers.

If the response is “No,” it would indicate that neither A nor B is a truth-teller. However, since everyone on the island always tells the truth, this situation cannot arise. Hence, the answer will always be “Yes,” confirming that both are truth-tellers.

Thus, by asking “Is at least one of you a truth-teller?” you can confirm that the inhabitants you are talking to are indeed truth-tellers

I asked ChatGPT the exact same question as in my original post and it answered:

“The man can take the goat across the river in the boat one at a time. First, he takes the goat across the river and leaves it on the other side. Then, he goes back to the original side and takes the boat back across to where the goat is. Finally, he takes himself across the river in the boat.“

So it’s still trying too hard. At least there was no mention of a cabbage.

Me: Why is it not possible for the man to simply remain with the goat after transporting it across the river?

ChatGPT: It is not possible for the man to simply remain with the goat after transporting it across the river because then the wolf would be left alone with the goat on the other side. The wolf would likely attack and eat the goat, as they are natural predators. This would defeat the purpose of the man trying to transport all three across the river safely.

I then asked why it mentioned a wolf and it just responded saying something like “Sorry, I must have misunderstood your question”.

These are all pretty good examples of LLM’s limits tbh. It doesn’t understand the puzzles, it throws up bits from its source material it thinks are relevant.

I agree. ChatGPT for some reason is trying solve the high school maths puzzle of the goat, the wolf and the cabbage. The goat can’t be left alone with the cabbage, the wolf can’t be left alone with the goat. The boat is being used in place of the wolf there.

The solution is correct, in that context, for what it’s worth.

I got a very different answer from the same prompt, in which it correctly interpreted the intended context: