Iris-mini and Iris-pro lead open-weight search-agent benchmarks, AllSpark says

AllSpark has released Iris-mini and Iris-pro, two open-source search agents built on Qwen-series models. According to the paper, Iris-mini has 35 billion parameters and Iris-pro has 397 billion, with both using a 256,000-token context window.

The team says the models deliver the strongest results among open-weight search agents in their respective size classes. The release includes model weights, code, and an Iris Harness for agent loops, tools, context management strategies, and benchmark evaluation.

The training pipeline builds questions backward from the link structure of web pages. It starts with a seed page and outgoing links, turns terms and relationships into a graph, and generates multi-step questions that require connected reasoning.

To reduce simple lookup behavior, every term except the final answer is paraphrased. The dataset keeps only tasks that a reference model cannot solve without tools but can solve with the right sources, making the tasks difficult and verifiable.

A stronger teacher model generates solution paths that combine reasoning, search queries, and results. Those paths are filtered first for correctness, repetition loops, and search depth, then reviewed step by step by a judge model whose criteria were derived from the data.

The models are then improved through reinforcement learning against live web search. AllSpark alternates supervised fine-tuning and reinforcement learning in a process the authors call “SFT-RL climbing,” feeding hard solved tasks and efficient paths back into later training rounds.

Testing covered BrowseComp, BrowseComp-ZH, DeepSearchQA, and Humanity’s Last Exam. With context management turned on, the paper reports Iris-mini scores of 82.2, 84.8, 86.9, and 52.3, while Iris-pro reaches 88.6, 85.1, 92.9, and 56.4.

AllSpark argues that context management can influence benchmark outcomes as much as model differences. The effect is especially large for Iris-mini, where BrowseComp scores rose by up to 21.2 points because the smaller model takes more steps and hits the context limit more often.

The authors report that both the generated training data and the specialized models improved performance on tasks they were not trained for, including general tool use and office work. Their interpretation is that search may be a foundational agent skill rather than a narrow capability.

The paper also highlights benchmark reliability concerns, including a BrowseComp-ZH example where the agent’s source-backed answer was marked wrong because the ground truth conflicted with the source material. For readers tracking AI agents, the key takeaway is to evaluate both model quality and the surrounding scaffolding, especially context handling and benchmark design.