ChatGPT 5.2 Thinking Eval

Post author: Rj
Post published: December 22, 2025
Post category: Blog
Post comments: 0 Comments

Evaluation of ChatGPT 5.2 using our Perth Endangered Wildlife Eval shows a dramatic improvement, which we attribute mostly to system improvements.

I really should have been timing each run of these evals. The new ChatGPT 5.2 Thinking model (with Extended thinking turned on) took 15 minutes before reporting “Network connection lost” and essentially crashing reported thinking trace. This seems to be OpenAI’s code for “I was thinking for too long and consumed too many resources, so got cut off”. A second attempt ( thinking trace, conversation) managed to get everything done in 11m 7s, due mostly to fewer revisions. On both attempts, 5.2 agreed with Gemini 3 Pro that the Western Swamp Tortoise is the threatened species to work on.

Desperately Seeking Evals

Post author: Rj
Post published: November 25, 2025
Post category: Blog
Post comments: 0 Comments

The leading edge of LLM capabilities is always advancing and although I'm regularly telling people to keep a cache of prompts that didn't work (so you can try them again…

Cosmo-1B, a Phi-1.5 Reproduction Attempt

Post author: Rj
Post published: March 25, 2024
Post category: Blog

🤗 tried to reproduce the Phi-1.5 "little Large Language Model" based on the papers that Microsoft has published. They fell short on their first attempt but have published the results and are continuing to work on it. This is what they did.

Our Visualisation Capability Declines with Age

Post author: Rj
Post published: January 25, 2023
Post category: Blog
Post comments: 0 Comments

The ability to create vivid visual mental images — visualisation — is an important skill for engineers and designers, as it allows them to understand and communicate their designs effectively.…

ShipSpace Project Zero: RSV Nuyina arrives in Antarctica

RSV Nuyina in Antarctica © AAD
Post author: Rj
Post published: January 7, 2022
Post category: Blog/ Completed Projects

As I mentioned when she arrived in Hobart, we call RSV Nuyina "Project Zero" because she was the first project that used ShipSpace. Nuyina is now in service and today…

ShipSpace Project Zero: RSV Nuyina arrives in Hobart

Nuyina en route to Hobart. © AAD
Post author: Rj
Post published: December 18, 2021
Post category: Blog/ Completed Projects

Nuyina was the first project to use ShipSpace.  We call Nuyina ‘Project Zero’ for ShipSpace because at that early stage, ShipSpace wasn’t really a product yet. We worked very closely…