pubmed-abstract-ir-simulated-search-queries

A PubMed retrieval dataset with simulated specialist search queries and abstract-level positive targets.

This dataset contains 2,355,329 query-document pairs derived from PubMed title-and-abstract records. Each row is designed so that the associated PubMed record is a positive retrieval target for the generated query. The queries were generated with a Qwen3.5-35B-A3B model quantized to NVFP4, and were intentionally constructed to resemble realistic specialist search behavior rather than a single idealized query form.

The dataset is intended for information retrieval research and engineering over biomedical text. It mixes more direct paper-finding queries with broader, more indirect discovery-oriented queries. It also mixes fluent semantic search formulations with compressed keyword-style queries. The goal is to approximate the range of queries that a biomedical retrieval system may encounter in practice, rather than producing only polished benchmark-style questions.

The source corpus is slinusc/PubMedAbstractsSubset. For this release, the retrieval text is stored as text, which is the original generation source text built from the paper title followed by a newline and the abstract. The PubMed identifier is preserved as pmid so each row can be traced back to the underlying source record.

Loading the dataset

from datasets import load_dataset

ds = load_dataset(
    "hotchpotch/pubmed-abstract-ir-simulated-search-queries",
    split="train",
    streaming=True,
)
row = next(iter(ds))
print(row)

License

This dataset is derived from slinusc/PubMedAbstractsSubset, using the upstream dataset as accessed on April 11, 2026. The upstream dataset card declares the source data license as:

• CC0 1.0 Universal (cc0-1.0)

This README follows that upstream declaration. If you need to validate redistribution or downstream commercial use for a specific workflow, check the upstream dataset card as well.

---

Technical report

Dataset structure

The dataset uses the following top-level columns, in this exact order.

query is the simulated user search input and text is the positive retrieval target text. pmid is retained so downstream users can join back to PubMed metadata or recover the original source record when needed.

Example row:

Control semantics

directness

query_style

Empirical distribution

The following counts describe the released dataset after filtering.

directness

query_style

Construction methodology

Step 1. Define the source retrieval corpus

Start from a PubMed title-and-abstract corpus containing a stable PubMed identifier, a title, and an abstract for each record. For the release used here, the generation run considered 2,392,165 source rows. The key design decision is to treat the title-plus-abstract text as the retrieval target. The generation problem is therefore asymmetric: given a paper record, generate a specialist search query for which that record would be a plausible positive hit.

Step 2. Define the generation objective

For each paper record, ask a Qwen3.5-35B-A3B model quantized to NVFP4 to generate one search query in English. The target behavior is a query that a biomedical researcher or technically literate user might realistically type when trying to find that paper or closely related work. The generator should not be restricted to one idealized style. Instead, it should cover both more direct known-item-like queries and more indirect discovery-oriented queries, together with both semantic and keyword-heavy query forms.

Step 3. Sample categorical controls before generation

Before generating the query for a paper record, sample two categorical control variables:

• directness
• query_style

In the current generator, both controls are sampled with a 3:1 bias toward the more standard setting.

Step 4. Filter to the final retrieval dataset

The intended export keeps only rows that satisfy all of the following:

• ok == true
• status == "ok"
• error == null
• reason == null
• validity_score == 1

This keeps the exported dataset focused on usable positive IR pairs while preserving the two generation-control columns and the PubMed identifier.

In practice, the generation run started from 2,392,165 source rows and the final export retained 2,355,329 rows. This corresponds to a retention rate of 98.5% relative to the input rows considered by the run.

Author

• Yuichi Tateno (@hotchpotch)