Literature review with real references

Use the Research workspace to plan a topic, discover papers from catalogs, synthesize with grounded citations, and export a multi-section manuscript.

Open Research from a paper header or go to /papers/{slug}/research.

Think of Research as a staged evidence workflow: discovery finds candidates, the plan defines the question, missions produce proposals, and Compose turns approved evidence into sections. Keep the search result, imported source, synthesis, and final citation distinct so you can audit how a claim entered the paper.

Literature Search (manual)

Use Explore → Literature search (/explore/search) when you want a Google Scholar–style catalog browse: type a query, skim ranked hits from OpenAlex, arXiv, Semantic Scholar, OSF, Europe PMC, Phase 1 keyless catalogs (Crossref, Zenodo, bioRxiv/medRxiv, DOAJ, OpenAIRE, OAPEN, DOAB), Gutenberg / Open Library / Internet Archive books, then Import, Attach (with ?paper= context), or Explore related — without starting a research mission.

Surface defaults live in config/writer.discovery.json. Missions and paper autocomplete still use the core five unless overridden. WRITER_DISCOVERY_PROVIDERS remains as a deprecated global allowlist shim for one release.

Search returns partial results as fast catalogs finish (soft deadline ~3s; slow providers time out without blanking the page). Repeat identical queries may hit a short TTL cache. On Research Start, use Browse catalogs instead under Quick search to open Literature Search.

Literature similarity graph

From a search hit (Explore related), a library source (Explore literature), Research bibliography (Explore around sources), or Library → Graph (select sources → Explore literature / Explore around), open /explore/literature for a Connected Papers–style force graph: papers arranged by bibliographic coupling and co-citation (Semantic Scholar neighborhood, with OpenAlex fallback when S2 is unavailable), with Prior/Derivative lists, filters, BibTeX export, and import/attach.

This is separate from Library → Graph (your papers ↔ sources ↔ notes). The library graph can deep-link into Explore and, via Show in library graph on an already-imported explore node, return with that source focused. Optional Suggested literature layer on the library graph shows a focus-gated overlay of related-paper ghosts — not a full replacement for Explore.

Before you start

Research checkpoint

Before running a mission, write down the question, population or corpus, date range, and what counts as an acceptable source. After the mission completes, inspect its evidence matrix and proposed sources before choosing Compose from synthesis. If the question changes, start a new plan or mission rather than silently mixing incompatible source sets.

Author checklist

Verify Writer health: GET /health should report Postgres up and llm_configured: true.

Operator checklist (real citations)

Admins can check catalog status at /admin/literature.

Set in .env on the Writer service:

WRITER_REFERENCE_CRAWL_ENABLED=1
[email protected]
[email protected]
[email protected]

Optional: SEMANTICSCHOLAR_API_KEY for faster discovery; ZOTERO_USER_ID + ZOTERO_API_KEY for library import.

Literature Search (/explore/search) uses the same crawl flags plus optional timeouts/cache:

# WRITER_LITERATURE_SEARCH_ENABLED=1
# WRITER_LITERATURE_SEARCH_PROVIDER_TIMEOUT_MS=2000
# WRITER_LITERATURE_SEARCH_SOFT_DEADLINE_MS=3000
# WRITER_LITERATURE_SEARCH_CACHE_SEC=600

Optional Phase 1 catalogs are on by default for Literature Search and Library Discover (see config/writer.discovery.json). Books defaults include Gutenberg, Open Library, and Internet Archive. Heritage catalogs (HathiTrust, Chronicling America, DPLA, Europeana) and scholarly Open Library / IA are opt-in via JSON mode: all / allowlist or lane env. CORE / DPLA / Europeana require API keys.

# WRITER_OPENLIBRARY_ENABLED=1
# WRITER_INTERNETARCHIVE_ENABLED=1
# WRITER_HATHITRUST_ENABLED=1
# WRITER_CHRONICLINGAMERICA_ENABLED=1
# WRITER_DPLA_API_KEY=
# WRITER_EUROPEANA_API_KEY=
# Books lane allowlist (optional):
# WRITER_DISCOVERY_BOOKS_PROVIDERS=gutenberg,openlibrary,internetarchive
# Deprecated global scholarly allowlist:
# WRITER_DISCOVERY_PROVIDERS=openalex,arxiv,semanticscholar,osf,europepmc,crossref,zenodo

Operator OAI-PMH / OPDS harvest (corpus seed, not interactive fan-out): python3 scripts/catalog_harvest.py oai|opds ….

Wired catalogs are listed on the public /welcome#catalogs page.

For literature_discovery_ingest (attach open-access PDFs), UNPAYWALL_EMAIL is required.

Production missions that run tens of minutes require an external mission worker configured by the operator. If a mission remains queued, ask the operator to check the worker and job status before starting a duplicate mission.

If Postgres is on shared-postgres (aabeds-infra) and APIs return 500, see Shared Postgres disk full below.

---

Two paths

Path A — You already have PDFs (10–20 sources)

1. Library (/library) → upload PDFs/DOCX → wait until status is ready.
2. Open your paper → attach sources (Write or Research left pane).
3. Research → Missions → create mission template Literature review.
4. When complete → Compose from synthesis → Write → Export DOCX.

Path B — You have a topic, few PDFs

1. Open /papers/{slug}/research.
2. Brainstorm: Topic → Clarify → Catalog check → Research plan.
3. On the plan → Run automation for literature_discovery / literature_discovery_ingest steps.
4. Approve discovered sources and vault notes as missions propose them.
5. Review mission synthesis and bibliography → Compose from synthesis → Write → Export DOCX.

flowchart TD
  start[Create or open a paper] --> pathChoice{Have PDFs already?}
  pathChoice -->|Yes Path A| attach[Attach sources to paper]
  pathChoice -->|No Path B| research[Research workspace brainstorm]
  attach --> litReview[Literature review mission]
  research --> auto[Run automation literature_discovery]
  auto --> approve[Approve discovered PDFs]
  approve --> synthesize[Mission synthesis]
  litReview --> synthesize
  synthesize --> compose[Compose from synthesis]
  compose --> write[Write mode sections and citations]
  write --> export[Export DOCX]

---

Research workspace layout (plan mode)

When your research plan is ready, the workspace uses a simplified layout:

On phone, panes are Plan · Actions · More (More opens Start / session history). Use Focus on plan in the center toolbar to hide side panes while editing. Set NEXT_PUBLIC_RESEARCH_UX_V2=0 to restore the legacy five-tab right pane for one release cycle if needed.

---

Build real references

Real references come from attached sources with resolved metadata, not from the LLM alone.

Upload PDFs (Library)

1. Go to Library (/library) — not the legacy /sources path in older help text.
2. Upload PDFs or DOCX; wait until extraction status is ready.
3. In the paper workspace, attach sources to scope compose and missions.

Zotero import

In Library → References → Add references, open the Zotero panel:

• Sync — metadata-only sources (ZOTERO_USER_ID, ZOTERO_API_KEY on server, or a connected personal/group account).
• Rich import — PDF attachments + highlight metadata when Zotero is configured (ZOTERO_RICH_IMPORT=0 to disable).
• Push to Zotero — create/link bibliographic items when ZOTERO_WRITEBACK=1 (metadata only; optional collection target).

Outbound API write-back is available when the operator sets ZOTERO_WRITEBACK=1 (Add references → Push to Zotero, or Push cited to Zotero on a paper). Metadata only — PDFs and highlights are not uploaded. BibTeX export remains available for manual handoff.

Discover from catalogs (Path B)

During Catalog check and literature_discovery missions, Lattuce searches OpenAlex / Semantic Scholar (discover_references).

• Review ranked candidates in the mission timeline.
• Approve attachments so PDFs enter the paper library before you cite them.

Import bibliography from a seed paper

Open a source in the library → Extract references → resolve online → attach selected rows to the paper.

---

Research workspace (brainstorm)

Steps in /papers/{slug}/research:

The plan is saved to the vault under Research agents (research-agents folder).

On the plan panel you can Ask, Grill claims, edit sections, and run Execute automation for literature-search steps.

---

Missions

Literature discovery (find papers)

Triggered by Run automation on the research plan, or create manually from Missions.

Templates:

• literature_discovery — search catalogs, rank candidates.
• literature_discovery_ingest — search + fetch open-access PDFs (needs UNPAYWALL_EMAIL).

Missions may run tens of minutes. Watch the status strip and mission timeline. A stale warning (~5 min idle) is not a failure.

Review and approve pending source attachments and vault notes before treating discoveries as cited sources.

Literature review (synthesize attached sources)

Template Literature review — requires attached, ready sources.

Planner steps: retrieve passages → source ask → counter-arguments → synthesize.

If the paper has no sources yet, run literature_discovery first (Path B).

When a mission completes

• Read synthesis report and evidence matrix in the mission panel.
• Export to vault — long-form note in Research agents folder.
• Download export.md or export.bib from the mission export actions (BibTeX for Zotero).

Same export surfaces exist for research briefs.

---

Multi-page output

There are two different “multi-page” artifacts:

Produce a multi-section manuscript (DOCX)

1. On a completed mission or research plan → click Compose from synthesis (handoff strip on the Research workspace).
2. Review approval-gated section draft(s) in Write mode (/papers/{slug}/write).
3. Add or reorder sections in the outline (Literature review, Methods, …).
4. Use Assistant → Compose per section, or accept synthesis drafts.
5. Pin claims with @ citations to ready sources; run grounding check when offered.
6. Export bar → DOCX (default). LaTeX, Typst, and PDF are best-effort.

For a literature-review-only document without a full paper outline: export mission .md or vault note, or paste synthesis into a single Write section and export DOCX.

---

What counts as a “real reference”

If no sources are attached, compose is labeled stylistic only — see Compose a draft.

---

Quick reference

Related: Sources · Compose · Literature review with Lattuce · Lattuce vs research apps

---

Shared Postgres disk full

Symptom: api/* returns 500; Writer logs:

PANIC: could not write to file "pg_logical/replorigin_checkpoint.tmp": No space left on device

Postgres is in shared-postgres (aabeds-infra on platform_data), not the Lattuce compose postgres service.

Fix:

1. Free disk on the host: df -h and docker system df.
2. Prune unused Docker images/cache: docker system prune -a -f (avoid volume prune without review).
3. When space is available:

docker exec shared-postgres pg_isready -U postgres
cd /opt/jemah && docker compose restart writer

4. Check research stack: GET /api/operations/research-health.

For self-hosted deployments, ask your operator to review the shared-data and Docker runbooks before changing storage or worker configuration.

---

Limitations

• Missions are long-running; no client abort on progress polls.
• Outbound Zotero API write-back is metadata-only (no PDF/highlight push); BibTeX export remains available.
• Without Qdrant, semantic passage retrieval is weaker.
• Synthesis → section drafts require your approval before they land in the manuscript.
• Debate / adversarial review missions are optional — see ADR-0005.

For day-to-day drafting without autonomous research, use Write mode and the Assistant compose pane.