Supervisors

Suitable for

Abstract

Background

Cryptic crosswords are a type of crossword puzzle where each clue includes 1) a definition and 2) wordplay by which the solver can check the answer. Surface readings are deliberately misleading and the wordplay involves combining multiple elements, often manipulating the words themselves.

Cryptic crosswords remain challenging for LLMs, with fine-tuned LLMs achieving only around 10% success on a dataset taken from common UK newspapers. Recent progress has been made using a formaliser-verifier approach, but success rates still remain less than 50%. Cryptic crosswords are also challenging for beginner solvers – more complex clues rely on esoteric knowledge of abbreviations, synonyms, and terminology.

Given these two realities, how could we use LLMs to support beginner solvers? Given the high likelihood of generating incorrect solutions (including correct answers with incorrect reasoning), an LLM is unlikely to be able to provide direct guidance towards the solution, but how could it be integrated into a solving process? How can we verify that a clue is accessible to a beginner solver and support them in the solving experience?

 

Focus

We currently have two research questions that could be explored, though students are welcome to propose others:

1. How can LLMs support the solving experience for a beginner, given LLMs do not have the solution a priori?
2. How can LLMs or other approaches be used to verify clues are accessible to beginners? Children are a population of interest

 

Method

The project will apply and extend the reasoning-based approach by Andrews and Witteveen (2025).

For the first research question, the student would apply this behind the scenes to a multi-turn conversation with a user, combining the user insights into the LLM-generated formalisation and verification at each stage. The main goal would be to show that user insights can correct faulty reasoning and increase solution rates. Stretch goals would include exploring different scaffolding strategies to extract and prompt user insights, given the uncertainty of the LLM-generated solution.

For the second research question, the student may extend the verifier to use things like the British National Corpus and other NLP techniques to determine if steps in the wordplay would be familiar to children or beginner solvers (e.g., words in expected vocabulary, two words known to be synonyms (used synonymously in corpus), abbreviations familiar). The main goal of the project would be to apply this verification to the Wordplay dataset (a set of clues and corresponding wordplay and answers) to collate a bank of clues suitable to beginners. Stretch goals would apply the extended verifier to the Cryptonite dataset (only clues and answers, wordplay must be generated) and define some measure of clue complexity, to further categorise the bank of clues.

References:

Andrews, M., & Witteveen, S. (2025). A Reasoning-Based Approach to Cryptic Crossword Clue Solving. arXiv preprint arXiv:2506.04824. Code: https://github.com/mdda/cryptic-crossword-reasoning-verifier/tree/main