Supervisor

Suitable for

Abstract

Exploration vs exploitation in AI “scientist” systems; driving novel hypothesis generation

 

AI “scientist” systems can reason over evidence and propose new research ideas, but it remains unclear how to systematically balance exploration (breadth: searching widely across new hypotheses) versus exploitation (depth: iteratively refining and validating promising directions) [1]. This matters because the hypothesis space is enormous: breadth-first ideation becomes shallow and repetitive, while depth-first reasoning risks getting stuck in local optima.

 

This project will formalise hypothesis generation as a sequential decision problem, where the agent must allocate limited steps/queries/attention across a vast hypothesis space. The student will implement and compare strategies for navigating hypothesis space, for example, multi-armed bandits [2], tree-search planners [3], structured reasoning approaches for LLM agents [4], and world-model style approaches [5]. Using historical scientific papers as a proxy environment, the student will evaluate methods on metrics such as novelty, diversity, and plausibility.

 

An additional focus will be on whether LLM “hallucinations” can be leveraged for exploration when they are induced in a controlled way and then filtered/grounded via plausibility checks [6]. The student will test novelty-driven generation via changes to the objective, reward [7] or the “environment” (e.g., masking parts of the reference corpus, altering constraints, or steering sampling), then study how these interventions shift the distribution of proposed ideas. The goal is to characterise when “novelty-driven hallucinations” help an AI scientist systematically sample under-explored regions of a design space [8].

 

Objectives

• Formalise hypothesis generation as a sequential decision process and define measurable objectives for novelty, plausibility, and diversity.
• Implement & compare breadth/depth control strategies using historical papers or trial protocols as a proxy environment [1, 2, 3].
• Design and evaluate controlled hallucination/contradiction mechanisms (novelty steering + grounding filters) and quantify when they improve discovery metrics and downstream usefulness [5, 6].

 

Interested students will contribute to multiple aspects of this project, from designing methods to developing the evaluation frameworks. This work will provide hands-on experience at the intersection of AI and life sciences and meaningfully contribute to AI scientists’ efforts within the Torr lab.

 

References

[1] Lu et al. (2024). The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery.

[2] Lattimore & Szepesvári (2020). Bandit Algorithms.

[3] Zhou et al. (2023/2024). Language Agent Tree Search (LATS) Unifies Reasoning, Acting, and Planning in Language Models.

[4] Yao et al. (2023). Tree of Thoughts: Deliberate Problem Solving with Large Language Models.

[5] Hafner et al. (2025). Mastering diverse control tasks through world models.

[6] Huang et al. (2025). A Survey on Hallucination in Large Language Models.

[7] Burda et al. (2018). Exploration by Random Network Distillation.

[8] Lehman & Stanley (2011). Evolution through the Search for Novelty Alone.