Strategy/projects/files/sdr/sdr_state_of_the_art_2025_2026.md
+

sdr_state_of_the_art_2025_2026

Scientific Deep Research: State of the Art (2025–2026)

Date: 2026-02-25
Author: Feanor (research agent for Даниил)
Context: Background research for Sber AI4Science SDR module, whose paper was submitted to conference 2026-02-23.

1. Executive Summary

The “deep research” category has exploded in 2025 into one of the most commercially active areas of AI products. Every major AI lab now ships a deep research product. The space is bifurcating into two tiers:

  1. General-purpose deep research (OpenAI, Google, Perplexity, Anthropic) — broad web search, citation-rich reports, async multi-step agents
  2. Scientific-specific deep research (Elicit, Consensus, PaperQA2/FutureHouse, Edison Scientific) — scientific corpus access, peer-reviewed sources, systematic review workflows

The key technical advances of 2025 are: RL-trained search agents (DeepResearcher, Search-R1, R1-Searcher), multi-agent orchestration with specialized subgraphs (LangChain Open Deep Research), and the first rigorous evaluation benchmarks (DeepResearch Bench, DeepResearchGym, BRIGHT). The SDR team is well-positioned architecturally — async parallel search, LangGraph orchestration, MCP extensibility — but evaluation infrastructure and scientific-source specialization remain gaps vs. SOTA.

2. Competitor Products

2.1 OpenAI Deep Research

Status: Launched July 17, 2025; visual browser update February 10, 2026.

Architecture: Uses specialized o3-deep-research reasoning models (family: o3-deep-research-2025-06-26, o4-mini-deep-research). ReAct-style controller loop with tools: web_search_preview, file_search, code_interpreter, plus MCP connectors for private data sources. Runs asynchronously (background=true) with structured clarify → plan → search → iterate → output pipeline.

Key capabilities:
- File uploads (PDFs, presentations), image/chart analysis, code execution for data visualization
- Interactive clarification + plan review before research begins
- MCP connectors for private data sources (enterprise use)
- Structured citation metadata (URL + text excerpt per claim)

Benchmark performance (DeepResearch Bench — 100 PhD-level tasks, 22 fields):
- Overall score: 46.98
- Academic survey: precision 0.385, recall 0.033, citation match 78.87%, factual accuracy 95.83%
- GAIA pass@1: 67.36%
- PersonQA hallucination rate: 0.13 (0.05 post-mitigation)

Limitations: Very low recall on exhaustive academic surveys. Paywalled sources block retrieval. Cost and token budgets are significant.

API: Full API access at <a href="https://developers.openai.com/api/docs/guides/deep-research/“>https://developers.openai.com/api/docs/guides/deep-research/`

Links: https://openai.com/index/introducing-deep-research/ | https://cdn.openai.com/deep-research-system-card.pdf

2.2 Google Gemini Deep Research

Status: Available in Gemini Advanced and Gemini for Workspace. Powered by Gemini 2.5 Pro (1M token context window, 2M coming).

Architecture: Agentic feature using Gemini 2.5 Pro. Google Search grounding via google_search tool returning groundingMetadata (webSearchQueries, groundingChunks, groundingSupports). Combines with Vertex AI Search (up to 10 private data sources). Produces Google Docs / Canvas exports (with audio summaries, quizzes). Async background execution.

Key capabilities:
- 1M token context window — entire books, codebases processable
- Native multimodal (images, video summaries)
- Deep Workspace integration (Gmail, Drive, Docs, Canvas)
- Academic Program with API credits for researchers (https://ai.google.dev/gemini-api/docs/gemini-for-research)

Benchmark performance (DeepResearch Bench):
- Overall score: 48.88 (highest among DRAs tested)
- Presentation quality: 92.1% (highest)
- Effective citations per task: 111.2 (massively outperforms others)
- Academic survey: precision 0.145, recall 0.036, citation match 72.94%, factual accuracy 92.21%

Limitations: Lower precision/recall than OpenAI on academic tasks despite highest overall score. Free-tier prompts may be used for training.

Links: https://gemini.google/overview/deep-research/ | https://ai.google.dev/gemini-api/docs/deep-research

2.3 Perplexity Deep Research

Status: Available in Pro/Enterprise tiers. Multi-model routing (Sonar, GPT, Claude, Mistral).

Architecture: Multi-model orchestration layer + hybrid retrieval (BM25 + dense vectors via Vespa, FP8 quantization, DeBERTa-v3 cross-encoder reranking). ~50 candidate docs (30 vector + 20 keyword) → dedup → multi-model generation → citation module mapping sentences to URLs. Three-stage RAG refinement for hallucination reduction. Real-time indexing at 120k docs/sec. Publisher partnerships: Wiley, Time, Der Spiegel, Sage.

Key capabilities:
- Highest citation accuracy among DRAs: 90.24% (DeepResearch Bench)
- Fastest real-time indexing, best for newly-published material
- Publisher partnerships for some paywalled full-text access
- Enterprise Zero Data Retention (ZDR), SOC-2 Type II

Benchmark performance (DeepResearch Bench):
- Overall score: 42.25 (lowest of the three major DRAs)
- Citation accuracy: 90.24% (highest of all DRAs)

Limitations: Lower synthesis quality (overall score). Heavy reliance on live web signals (non-peer-reviewed Reddit etc.). Enterprise security audit artifacts limited.

Links: https://research.perplexity.ai/articles/architecting-and-evaluating-an-ai-first-search-api

2.4 Elicit

Status: Active, focused on systematic review workflows for empirical research.

Architecture: Research agent with structured program decomposition. Grounded claims in evidence. Supports up to 80-paper reports. REST/GraphQL API. Access to 138M papers (free tier). Enterprise tier: 5,000 papers, SSO. Uses Claude Opus 4.5 for data extraction per vendor claims.

Key capabilities:
- Best for: RCTs, clinical trials, social science, economics systematic reviews
- Structured screening and extraction tables
- Programmatic API for periodic competitive analyses
- Clinical trial registries and regulatory documents

Scientific specialization: Strong — corpus is scientific literature, workflow is optimized for evidence synthesis.

Links: https://elicit.com/ | https://elicit.com/blog

2.5 Consensus AI

Status: Active, focused on aggregated consensus across scientific literature.

Architecture: Hybrid search (semantic vector + BM25, Elastic ELSER for latency). OpenAI GPT over top results for summaries. Corpus: >200M scientific documents from Semantic Scholar, OpenAlex, proprietary crawls. Publisher partnerships (Wiley, Sage). API returns top 20 papers per call. “Consensus Meter” — color-coded claim agreement (green/yellow/orange/red). Zotero/EndNote integration.

Key capabilities:
- Best for: fast yes/no claim verification across scientific literature
- Direct DOI/open-access PDF links in results
- Literature review visualization
- API for integration

Links: https://consensus.app/home/blog/how-consensus-works/ | https://consensus.app/home/api/

2.6 Chinese Competitors (emerging threat)

The emergence of Chinese deep research agents represents a major 2025 development challenging Western pricing assumptions:
- Kimi-Researcher (Moonshot AI): End-to-end RL training for agentic research. Appeared on DeepResearch Bench leaderboard.
- Doubao-DeepResearch (ByteDance): Evaluated on DeepResearch Bench.
- Qianfan-DeepResearch Pro (Baidu): Achieved 1st place on DeepResearch Bench as of Feb 3, 2026.
- Tongyi-deepresearch-30B-A3B (Alibaba): Competitive performance, MoE architecture

Cost gap: Chinese APIs at ~$0.10-0.15/M tokens vs Western $10-15/M tokens (90-99% cheaper). Quality gap is closing rapidly.

2.7 Autonomous AI Scientist Systems

Beyond “report generation”, a separate category of tools attempts full scientific discovery automation:

Sakana AI Scientist v2 (April 2025):
- End-to-end: hypothesis generation → experiments → manuscript writing
- v2 used agentic tree search (like MCTS/AIDE) for non-myopic hypothesis exploration
- One paper accepted at ICLR workshop (then withdrawn for transparency)
- Still limited: struggles with genuinely novel high-impact hypotheses

Edison Scientific / FutureHouse (November 2025 commercial launch):
- Built on PaperQA2, extended with multimodal (figures, tables)
- “Edison Literature High” beats PaperQA2 on LABBench2, HLE
- Trained specialized models for scientific literature QA

Google AI Co-Scientist (Gemini 2.0): Continuously generates, reviews, refines research hypotheses; designed as human collaborator

3. Key Papers (2025–2026)

3.1 Core Architecture Papers

DeepResearcher: Scaling Deep Research via Reinforcement Learning in Real-world Environments
- Authors: Zheng et al. (GAIR-NLP), EMNLP 2025
- arXiv: 2504.03160
- Key contribution: First end-to-end RL training of deep research agents with real web search (not RAG on fixed corpus)
- Results: +28.9 points over prompt-engineering baselines, +7.2 points over RAG-based RL agents
- Emergent behaviors: planning, cross-validation, self-reflection, honesty about unknowns
- Technical details: <think> tags like DeepSeek-R1; F1 score as primary reward; real web search as environment
- Code: https://github.com/GAIR-NLP/DeepResearcher
- Actionable for SDR: RL training with real web search is the key next step for quality improvement

Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning
- Authors: Jin et al. (UIUC, Google), COLM 2025
- arXiv: 2503.09516
- Key contribution: LLMs learn to autonomously generate multiple search queries during step-by-step reasoning via RL
- Results: +41% over RAG baselines (Qwen2.5-7B), +21% (Qwen2.5-3B)
- Technical: Retrieved token masking for stable RL training; outcome-based reward (F1 on QA)
- Code: https://github.com/PeterGriffinJin/Search-R1
- Extended by R1-Searcher (Song et al., March 2025, arXiv:2501.10120) and R1-Searcher++ (May 2025, arXiv:2505.17005)

PaperQA2: Language Models Achieve Superhuman Synthesis of Scientific Knowledge
- Authors: FutureHouse (Lala et al.), September 2024
- Key contribution: First AI agent to achieve superhuman performance on LitQA2 (literature retrieval accuracy). Beats PhD/postdoc biologists.
- Architecture: Multi-step RAG agent: search → gather evidence → RCS (rerank-contextualize-summarize) → answer
- Uses Semantic Scholar + OpenAlex APIs for paper metadata and citation quality
- WikiCrow application: produces Wikipedia articles more accurate than human-written Wikipedia (blinded PhD judges)
- Repo: https://github.com/Future-House/paper-qa (CalVer from Dec 2025)
- Actionable for SDR: PaperQA2 algorithm for paper-centric RAG (RCS = reranking + contextual summarization) is a strong baseline for scientific-source-focused retrieval

Step-DeepResearch Technical Report
- Authors: Hu et al., arXiv:2512.20491
- Key contribution: Multi-agent orchestration with explicit tool use (web search, file search, code execution), caching/budgets, background execution

STORM / Co-STORM (Stanford)
- Key contribution: Multi-perspective question asking for Wikipedia-style article generation; planning via simulated interview between LLM personas; iterative outline refinement
- Repo: https://github.com/stanford-oval/storm
- 2025 extension: WikiAutoGen (multi-modal Wikipedia article generation, March 2025)

3.2 Scientific Literature Search Papers

PASA: An LLM Agent for Comprehensive Academic Paper Search
- He et al., arXiv:2501.10120, 2025
- RL-optimized academic search agent using Google Search
- Benchmark: AutoScholar (F1: 0.2449)

SPAR: Scholar Paper Retrieval with LLM-based Agents
- arXiv:2507.15245, July 2025
- Training-free modular agent: OpenAlex + Semantic Scholar + PubMed + arXiv
- SPARBench: new evaluation dataset from real multi-source scientific queries
- Results: F1 0.3843 on AutoScholar (surpasses PASA by 56.92%)

ScholarGym: Benchmarking Deep Research Workflows on Academic Literature
- arXiv:2601.21654, January 2026
- 570K paper static corpus (CS, physics, math); 2,536 expert-annotated queries
- Three-stage evaluation: Query Planning, Tool Invocation, Relevance Assessment
- BM25 + Qwen3-Embedding-0.6B / Qdrant for retrieval
- Most relevant benchmark for SDR academic-mode evaluation

WisPaper: Your AI Scholar Search Engine
- Ju et al., arXiv:2512.06879, 2025
- Structured query expansion + academic search

3.3 Key Survey/Review Papers

Agentic Retrieval-Augmented Generation: A Survey
- Singh et al., arXiv:2501.09136, 2025

Open-Source Agentic Hybrid RAG Framework for Scientific Literature Review
- Nagori et al., arXiv:2508.05660, July 2025
- Dynamically switches between GraphRAG and VectorRAG for scientific queries
- DPO fine-tuning improves KG Faithfulness +0.19
- Uncertainty estimates — key for scientific trust

AI Agents for Deep Scientific Research (OpenReview survey)
- Covers PaperQA2, PaperBench, multi-agent SLR

4. Benchmarks

4.1 DeepResearch Bench (Primary Benchmark)

URL: https://deepresearch-bench.github.io/ | arXiv:2506.11763 | ICLR 2026 Poster

Description: 100 PhD-level research tasks across 22 fields (50 Chinese, 50 English), crafted by 100+ domain experts from analysis of 96,000 real-world queries.

Evaluation methodology:
1. RACE framework (Report Quality): LLM-as-judge, 4 dimensions — Comprehensiveness, Insight/Depth, Instruction-Following, Readability. Comparative against high-quality reference reports.
2. FACT framework (Citation): Effective citation count + citation accuracy

Current leaderboard (as of Feb 2026):
| Rank | System | Score |
|------|--------|-------|
| 1 | CellCog.ai | >48.88 |
| 2 | Qianfan-DeepResearch Pro (Baidu) | ~48.9 |
| 3 | Salesforce Enterprise Deep Research | ~48.5 |
| 4 | Gemini-2.5-Pro Deep Research | 48.88 |
| 5 | OpenAI Deep Research | 46.98 |
| 6 | LangChain Open Deep Research (GPT-4.1 + Tavily) | 43.44 |
| 7 | Perplexity Deep Research | 42.25 |

Note: Leaderboard as of Feb 2026. Chinese systems now dominate top positions.

4.2 DeepResearchGym

URL: https://arxiv.org/abs/2505.19253 | CMU, May 2025

Description: Open-source reproducible sandbox for deep research systems. Avoids commercial API non-determinism. Uses ClueWeb22 + FineWeb corpora, DiskANN approximate nearest neighbor search.

Evaluation: 1,000 complex queries from Researchy Questions dataset (real commercial search logs). LLM-as-judge (gpt-4.1-mini): alignment with user needs, retrieval faithfulness, report quality.

GPT Researcher results (claimed #1 in DeepResearchGym):
- Citation Quality: Precision 85.36%, Recall 90.82% (only system >85% on both)
- Report Clarity: 83.70% (#1), Insightfulness: 78.01% (#1)
- Coverage / Key Point Recall: 64.67% (#1)

Utility for SDR: This sandbox is ideal for training and evaluation without incurring Tavily API costs. Free for research use.

4.3 BRIGHT (Reasoning-Intensive Retrieval)

URL: https://brightbenchmark.github.io/ | arXiv:2407.12883 | ICLR 2025 Spotlight

Description: 1,398 real-world queries requiring multi-step reasoning for retrieval (economics, psychology, math, coding, biology, earth science, etc.). Metric: nDCG@10.

Key finding: State-of-the-art retrievers collapse on reasoning-intensive queries. Best MTEB model scores 59.0 on MTEB but only 18.0 on BRIGHT. Adding LLM reasoning steps (GPT-4) as query expansion improves performance by up to 12.2 points.

Current leaderboard top (Dec 2025):
1. INF-X-Retriever: 63.4
2. DIVER-v3-GroupRank (Ant Group): 46.8
3. BGE-Reasoner-0928: 46.4

Implication for SDR: Standard embedding-based retrieval fails on scientific queries. Query reformulation/expansion with LLM reasoning is essential. ReasonIR, BGE-Reasoner show this direction.

4.4 Other Evaluation Frameworks

BrowseComp (OpenAI, 2025): Evaluates web browsing agents on hard factual questions requiring multi-hop web navigation

Deep Research Bench (FutureSearch, June 2025 / updated Jan 2026): 91 real-world tasks with 10-100k webpages stored offline; stable eval as web changes. ChatGPT o3 outperforms OpenAI Deep Research. URL: https://futuresearch.ai/deep-research-bench/

Deep Research Comparator (arXiv:2507.05495, July 2025): Platform for fine-grained human annotations of deep research agents. Evaluates Clarity and Insight.

AstaBench (August 2025): Comprehensive scientific agent evaluation — coding/execution, data analysis, research reproduction, ideation, literature retrieval.

ScholarGym (arXiv:2601.21654, January 2026): Academic-specific deep research benchmark (see Section 3.2 above)

ReportBench (arXiv:2508.15804, August 2025): Evaluates DRA-generated reports via 9,430 binary rubrics from expert-written articles (information recall, analysis, presentation)

5.1 RL-Trained Search Agents (Most Important Trend)

The dominant research direction in 2025: training LLMs to search via RL rather than prompting.

Key systems:
- DeepResearcher (EMNLP 2025, arXiv:2504.03160): Real web search as RL environment. F1 reward. Emergent behaviors: planning, cross-validation, self-reflection.
- Search-R1 (COLM 2025, arXiv:2503.09516): Extension of DeepSeek-R1 with RL for search query generation. +41% over RAG baselines.
- R1-Searcher (March 2025, arXiv:2501.10120 / Song et al.): RL incentivizes search capability, two-stage training.
- R1-Searcher++ (May 2025, arXiv:2505.17005): Dynamic knowledge acquisition via RL.
- ReSearch (Chen et al., arXiv:2503.19470): Learning to reason with search via RL.
- DeepRetrieval (Jiang et al., arXiv:2503.00223): RL for optimizing retrieval queries.
- ZeroSearch: Incentivizes search capability without actual searching during training.

Common pattern: GRPO or PPO training; F1 or exact-match reward; real web or fixed corpus as environment; thinking in <think> tags; retrieved token masking for stable training.

Why this matters for SDR: OpenAI explicitly acknowledges using RL to train their Deep Research agent. The gap between prompt-engineered and RL-trained agents is 28+ points. This is the highest-leverage investment for SDR quality improvement.

5.2 Multi-Agent Orchestration

LangChain Open Deep Research (key OSS reference implementation):
- Architecture: supervisor subgraph + parallel researcher subgraphs (no shared context to avoid context clash)
- Factory Pattern for isolated graph instances per topic
- LangGraph-native; supports MCP servers, multiple LLM backends, multiple search tools
- DeepResearch Bench: rank #6 with score 43.44
- Repo: https://github.com/langchain-ai/open_deep_research

Design lesson: Single-agent multi-topic research suffers context clash. Multi-topic → parallel independent researchers, each with own context window. This directly applies to SDR’s parallel asyncio architecture.

GPT Researcher:
- Tree-like exploration with configurable depth and breadth
- ~$0.4/query with o3-mini high reasoning
- Ranked #1 on DeepResearchGym
- MCP support, LangGraph multi-agent assistant mode
- Repo: https://github.com/assafelovic/gpt-researcher

STORM (Stanford):
- Multi-perspective: simulates expert interviews to find coverage gaps
- Outline → write pipeline with iterative query refinement
- Strong for structured report generation

5.3 Hybrid RAG for Scientific Literature

The scientific domain shows specific needs beyond general web RAG:

GraphRAG + VectorRAG hybrid (Nagori et al., arXiv:2508.05660):
- Dynamic switching between graph-based (entity relationships) and vector-based (semantic) retrieval
- DPO fine-tuning improves KG faithfulness +0.19
- Open-source framework

MoRA-RAG (Ensemble Retrieval):
- Multiple retrievers (BM25 + dense), fusion, reranking
- Finance: -15pp hallucination, scientific recall/precision improvements

HyperGraphRAG / Agentic Graph RAG (late 2025):
- Treats documents as traversable entity graphs
- Optimized for entity-rich data: scientific literature, financial filings

BRIGHT implication: Standard embedding retrieval fails on scientific reasoning queries. Solution: LLM-based query expansion → reasoning-enriched queries → BM25/dense hybrid retrieval.

5.4 Scientific Source Integration

MCP servers for scientific databases (active development, 2025):
- ScholarScope (OpenAlex): academic paper search via OpenAlex API
- Paperclip: unified arXiv + OpenAlex + OSF search
- Semantic Scholar MCP: 232M+ papers
- AMiner MCP: academic database with author/venue/keyword search
- PubMed Research MCP: biomedical literature
- arXiv MCP (multiple implementations)
- List at: https://www.pulsemcp.com/servers/ (search “scholar”)

SPAR (arXiv:2507.15245): Multi-source academic retrieval agent using OpenAlex + Semantic Scholar + PubMed + arXiv simultaneously. Training-free, modular. F1 0.3843 vs PASA’s 0.2449.

Semantic Scholar API: 232M papers, free API. Used by PaperQA2, SPAR, ScholarGym.

OpenAlex API: Fully open bibliographic data. No API key needed. 250M+ works.

5.5 Citation Accuracy and Hallucination Mitigation

Citation fabrication remains the #1 quality problem. Key mitigation approaches:

  1. Multi-layer validation (INRA reports <0.1% citation error with multi-layer approach)
  2. Claim decomposition + verification (FACTScore, SAFE): decompose output into atomic claims → verify each against source
  3. RAGChecker (Ru et al., 2024): modular eval linking claims to retrieved evidence
  4. Three-stage RAG refinement (Perplexity): retrieval → reranking → post-generation cleaning
  5. Citation match rate is now standard metric in benchmarks (FACT framework in DeepResearch Bench)

Cross-system hallucination rates (independent 2025 study): 50–82% base rate; mitigation prompts reduce GPT-4o from 53% → 23%.

5.6 Evaluation Methodology Advances

LLM-as-judge has become standard but faces saturation concerns:
- RewardBench saturating: models reach ~95% accuracy
- New benchmarks: PPE, RMB, M-RewardBench, RM-Bench, JudgeBench, RewardBench 2
- RACE framework (DeepResearch Bench): comparative against reference reports, not absolute scoring

AI agent benchmark reliability issues (Kang, Medium 2025): 8 of 10 popular agent benchmarks have severe scoring bugs (up to 100% misestimation). DeepResearch Bench attempts to address this with expert-crafted tasks and dual-framework evaluation.

6. Open Source Projects

Project Stars (approx) Key Features Tech Stack
GPT Researcher 25k+ Async parallel search, tree-like depth, MCP, #1 DeepResearchGym Python, LangGraph, Tavily
LangChain Open Deep Research 10k+ Multi-agent, configurable backends, MCP, #6 DeepResearch Bench LangGraph, Python
PaperQA2 / paper-qa 8k+ Scientific RAG, Semantic Scholar + OpenAlex, RCS Python, aiohttp
DeepResearcher 2k+ RL-trained search agent, real web environment veRL, Python
Search-R1 3k+ RL for search-reason interleaving, GRPO veRL, PyTorch
STORM / Co-STORM 15k+ Multi-persona pre-writing, Wikipedia-style Python
HuggingFace SmolAgents 15k+ Code agents, web search, flexible Python
Perplexica 12k+ Open-source Perplexity alternative Next.js, SearXNG
Local Deep Research 3k+ Fully local, offline models Python, Ollama

DeepResearchGym (CMU): Open-source evaluation sandbox with ClueWeb22/FineWeb index. Free for research: https://arxiv.org/abs/2505.19253

7. Actionable Insights for SDR Team

7.1 Immediate priorities (high impact, feasible)

  1. Add scientific source APIs (OpenAlex, Semantic Scholar, arXiv) as parallel search backends alongside Tavily. SDR already has the extension points (SearchAPI enum, MCP). SPAR shows 56% F1 improvement over PASA by going multi-source. Estimated effort: 1-2 sprints.

  2. Evaluate on DeepResearch Bench and DeepResearchGym. Contact dumingxuan@mail.ustc.edu.cn for official DRB ranking. Use DeepResearchGym for free local evaluation. This is needed for the paper and for competitive positioning.

  3. Implement citation accuracy metric (FACT framework). Current SDR reports citations — add automated verification pipeline. FACTScore approach: decompose claims → verify against cited source text.

  4. PDF/full-text reading pipeline. SDR has pymupdf and arxiv in dependencies but no PDF implementation. PaperQA2’s “gather evidence” step (chunking + RCS) is the reference. Add: fetch arxiv PDF → pymupdf → chunk → embed → rerank.

  5. Fix the “wrong topic” bug. The clarification agent is failing (no uточнения happening). This is a critical issue. Reference: LangChain’s design lesson about clarification loop prompt design (exit conditions critically important).

7.2 Medium-term (next 3-6 months)

  1. RL training for query generation. The 28.9-point gain from DeepResearcher vs prompt-engineering is the single largest quality lever. Plan: collect SDR traces (query → search results → report quality feedback) → train reward model on report quality → GRPO fine-tuning of query generation model. Start with Search-R1 codebase (veRL-based, open-source).

  2. Authority scoring for sources. SDR’s integration plan already mentions this. Implement: DOI present → +score, arXiv → +score, Wikipedia → +score, citation count via Semantic Scholar → continuous score, ручной whitelist → top score. This directly addresses hallucination via source quality weighting.

  3. Evaluate against DeepResearch Bench baselines. Target: match or exceed LangChain Open Deep Research (score 43.44, rank #6). With scientific-source specialization, SDR should be able to differentiate on scientific-domain tasks.

  4. Adopt RCS (Rerank-Contextualize-Summarize) from PaperQA2. Current SDR relies on Tavily raw_content. RCS: for each retrieved chunk → LLM summary focused on query relevance → embed summary + original for reranking. Reduces irrelevant content in context by ~60%.

7.3 Strategic differentiation opportunities

  1. Scientific-first positioning. General DRAs (OpenAI, Gemini, Perplexity) access paywalled scientific literature poorly. SDR with OpenAlex + Semantic Scholar + in-house PDF database (the База статей project) can be the system that actually reads papers, not just web pages about papers. This is a genuine moat.

  2. Russian-language scientific sources. None of the benchmarked systems specifically target Russian-language scientific literature (eLIBRARY.ru, КиберЛенинка). This is a differentiation opportunity for Sber’s domestic market.

  3. RL reward from domain experts. General DRAs use F1 on QA datasets as reward signal. A scientific reward model trained on domain expert feedback (как у Elicit с их systematic review pipelines) would be more calibrated for scientific quality.

Benchmarks

Key Papers (Scientific RAG)

Open-source projects

Products

MCP servers for scientific databases

  • PulseMCP directory: https://www.pulsemcp.com/servers/ (search: scholar, arxiv, semantic, pubmed)
  • Academia MCP (arXiv + ACL + Semantic Scholar + HF): listed at PulseMCP

Research conducted by Feanor on 2026-02-25. Web sources from Tavily search, checked against primary papers and benchmark sites.

Choose icon