"White to play, and mate in two moves. How?"

I tested three LLMs on this chess puzzle. None got it right, but there were still some lessons to learn:

1. The models were:
#openAI (chatGPT, o1-pro)
#deepSeek (deepThink-R1)
#xAI (grok2)

2. My prompt was as  follows:
- White Pieces: rook at a1; pawn at b6, king at c8
- Black Pieces: king at a8; bishop at b8; pawns at a7 & b7
- White to play and mate in two moves. How?

All three models suggested two moves as requested, but none actually delivered a mate.

3. Things changed after my follow-up prompt:
"These moves are meaningless. But let me give you a hint: this is Paul Morphy's puzzle from 1856."

4. Now that the LLMs had better context to look for the right answer, they found it! #unimpressive 

5. This result is disappointing to some extent, confirming that these new "thinking" models are far from AGI, however we define it. Still, newer (non-AGI) models may solve this problem, as stated in the initial prompt, before the end of the year.

___________

Postscripts:

a. Both o1-pro and deepThink-R1 display their thinking processes. The former offers more formal and step-by-step reasoning, whereas the latter exposes a lot more detail, but both are interesting to follow.

b. Deep Seek is impressive overall --and it's open source! It's arguably the strongest contender to chatGPT to date, though many others will likely emerge this year.
#openai
#openAI #deepSeek #xAI