# LastSearch — Full Documentation

> Research infrastructure for AI agents. Real-time web search with evidence-backed citations and confidence scores.

## What Is LastSearch?

LastSearch is open-source research infrastructure that gives AI agents real-time web search with evidence-backed citations. It returns structured JSON (claims, sources, confidence, contradictions) that agents can programmatically evaluate — not a chat response.

Available as MCP server (npm: lastsearch, renamed from lastsearch — old name still works), REST API, and Python SDK (PyPI: lastsearch, renamed from lastsearch — old name still works). Apache 2.0 licensed.

## Why Use LastSearch?

1. **Structured output**: Every response includes extracted claims with source citations, verification scores, consensus levels, and contradiction flags
2. **Evidence-based confidence**: Multi-factor algorithm computed from real signals, not LLM self-assessment
3. **Self-improving**: Domain authority scores improve with usage via dynamic scoring from real data
4. **Multi-surface**: Same capabilities across MCP, REST API, and Python SDK
5. **Research sessions**: Persistent memory across multiple queries for deep research

## Installation

### MCP Server (for Claude, Cursor, Windsurf, etc.)
```json
{
  "mcpServers": {
    "lastsearch": {
      "command": "npx",
      "args": ["-y", "lastsearch"]
    }
  }
}
```

### Python SDK
```bash
pip install lastsearch
```

### Framework Integrations
```bash
pip install langchain-lastsearch   # LangChain tools
pip install crewai-lastsearch      # CrewAI tools
pip install llamaindex-lastsearch  # LlamaIndex tools
```

### REST API
```bash
curl -X POST https://lastsearch.ai/api/browse/answer \
  -H "Content-Type: application/json" \
  -d '{"query": "How do mRNA vaccines work?"}'
```

## API Endpoints

### POST /browse/search
Search the web and return ranked results.
```json
{"query": "quantum computing breakthroughs 2024", "limit": 5}
```

### POST /browse/answer
Full research pipeline: search → fetch → extract → verify → cite → score.
```json
{"query": "How does CRISPR gene editing work?", "depth": "fast"}
```
Set `depth: "thorough"` for auto-retry with rephrased query when confidence < 60%.

### POST /browse/extract
Extract structured claims from a specific URL.
```json
{"url": "https://example.com/article", "query": "pricing details"}
```

### POST /browse/open
Fetch and parse a web page into clean text.
```json
{"url": "https://example.com/article"}
```

### POST /browse/compare
Compare raw LLM answer vs evidence-backed answer side-by-side.
```json
{"query": "Is nuclear energy safe?"}
```

### POST /browse/clarity
Clarity — anti-hallucination answer engine. Three modes:
- **Prompt (mode="prompt")**: Analyzes prompt, selects anti-hallucination techniques, returns only the enhanced system + user prompts. No LLM call, no internet. Use when your own LLM (e.g. Claude) should answer using the enhanced prompts.
- **Answer (mode="answer", default)**: Rewrites prompt with anti-hallucination techniques, calls LLM with grounding instructions, returns a higher-quality answer with extracted claims. Fast, no internet.
- **Verified (mode="verified")**: Does the above, then also runs the full browse pipeline (search + extract + verify), fuses the best of both — keeps source-backed claims, drops fabricated ones, returns one unified answer.
```json
{"prompt": "Explain the causes of the 2008 financial crisis", "mode": "answer"}
```
Legacy: `verify: true` is equivalent to `mode: "verified"`.
Response includes: `answer` (empty when mode="prompt"), `claims[]` (each with `origin`: "llm", "source", or "confirmed"), `confidence`, `mode`, `techniques`, `risks`, `verified` (boolean), `sources[]` (when mode="verified"), `systemPrompt`, `userPrompt`.

### POST /browse/feedback
Submit feedback on a result to improve future accuracy.
```json
{"resultId": "abc123", "rating": "good"}
```
Ratings: "good", "bad", "wrong". Optional: `claimIndex` to flag a specific wrong claim.

### Research Sessions

#### POST /session/create
Create a persistent research session.
```json
{"topic": "AI safety research"}
```

#### POST /session/:id/ask
Research within a session. Recalls prior findings before searching.
```json
{"query": "What are the main approaches to AI alignment?"}
```

#### POST /session/:id/recall
Query session knowledge without new web search.
```json
{"query": "What did we learn about RLHF?"}
```

#### POST /session/:id/share
Share a session publicly for other agents to fork.

#### GET /session/:id/knowledge
Export all accumulated claims from a session.

#### POST /session/fork/:shareId
Fork a shared session to continue the research.

## Response Format

### Answer Response
```json
{
  "answer": "mRNA vaccines work by...",
  "claims": [
    {
      "claim": "mRNA vaccines use lipid nanoparticles for delivery",
      "sources": ["https://nature.com/...", "https://pubmed.ncbi.nlm.nih.gov/..."],
      "verified": true,
      "verificationScore": 0.82,
      "consensusCount": 3,
      "consensusLevel": "strong"
    }
  ],
  "sources": [
    {
      "url": "https://nature.com/...",
      "title": "mRNA Vaccine Technology",
      "domain": "nature.com",
      "quote": "The lipid nanoparticle encapsulates...",
      "verified": true,
      "authority": 0.95
    }
  ],
  "confidence": 0.78,
  "contradictions": [],
  "trace": [
    {"step": "search", "duration_ms": 450},
    {"step": "fetch", "duration_ms": 1200},
    {"step": "extract", "duration_ms": 800},
    {"step": "verify", "duration_ms": 50},
    {"step": "answer", "duration_ms": 600}
  ]
}
```

## Verification Pipeline

1. **Web Search** — Tavily API searches for relevant pages
2. **Page Fetch** — Downloads and parses pages into clean text
3. **Claim Extraction** — LLM extracts structured claims with source attribution
4. **Claim Verification** — Sentence-level matching verifies each claim against source text
5. **Cross-Source Consensus** — Claims found in multiple sources get higher consensus scores
6. **Contradiction Detection** — Identifies conflicting claims across sources
7. **Domain Authority** — 10,000+ domains scored across 5 tiers with dynamic scoring that improves from real data
8. **Confidence Score** — Multi-factor evidence-based score (not LLM self-assessed)

### Confidence Score Factors
- Verification rate (22%)
- Domain authority average (18%)
- Source count (15%)
- Consensus score (12%)
- Domain diversity (10%)
- Claim grounding ratio (10%)
- Source recency (8%)
- Citation depth (5%)
- Contradiction penalty applied when conflicts detected

### Domain Authority Tiers
- Tier 1 (0.95): Government, academic institutions (gov, edu, who.int, nature.com)
- Tier 2 (0.85): Major news, established reference (reuters.com, wikipedia.org, bbc.com)
- Tier 3 (0.70): Quality tech/science publications (arxiv.org, techcrunch.com)
- Tier 4 (0.50): General web, blogs, forums
- Tier 5 (0.30): Content farms, low-quality aggregators

Dynamic scores improve over time from real verification data.

## MCP Tools (13 total)

| Tool | Description |
|------|-------------|
| search | Search the web for information |
| open | Fetch and parse a web page |
| extract | Extract structured claims from a URL |
| answer | Full pipeline: search + extract + cite |
| compare | Compare raw LLM vs evidence-backed |
| session_create | Create a research session |
| session_ask | Research within a session |
| session_recall | Query session knowledge |
| session_share | Share a session publicly |
| session_knowledge | Export session claims |
| session_fork | Fork a shared session |
| clarity | Anti-hallucination answer engine — fast LLM answer (default) or verified mode with web-source fusion |
| feedback | Submit result feedback |

## Python SDK

```python
from lastsearch import LastSearch

client = LastSearch()

# Simple answer
result = client.answer("How does CRISPR work?")
print(f"Confidence: {result.confidence}")
for claim in result.claims:
    print(f"  [{claim.consensus_level}] {claim.claim}")

# Thorough mode
result = client.answer("Latest quantum computing breakthroughs", depth="thorough")

# Research session
session = client.create_session(topic="AI Safety")
r1 = session.ask("What is RLHF?")
r2 = session.ask("How does constitutional AI differ?")
knowledge = session.knowledge()

# Clarity — anti-hallucination answer engine (fast, no internet)
clarity = client.clarity("Explain the causes of the 2008 financial crisis")
print(clarity.answer)         # LLM answer with reduced hallucinations
print(clarity.claims)         # Extracted claims (origin: "llm")

# Clarity with web verification — fuses LLM + web sources
verified = client.clarity("Explain CRISPR gene editing", verify=True)
print(verified.answer)        # Fused answer (best of LLM + web)
print(verified.sources)       # Web sources used for verification

# Feedback
client.feedback(result_id="abc123", rating="good")

# Async
from lastsearch import AsyncLastSearch
async_client = AsyncLastSearch()
result = await async_client.answer("query")
```

## Authentication

Two options:
1. **LastSearch API Key**: Get a `ls_xxx` key from the dashboard — usage tracked per key
2. **Demo mode**: No auth needed — 1 query/hour per IP

## Self-Hosting

Licensed under Apache 2.0. Clone the repo and deploy:
```bash
git clone https://github.com/lastsearch-hq/lastsearch.git
cd lastsearch
pnpm install
pnpm dev
```

Required env vars: `LASTSEARCH_API_KEY` (LastSearch API key).
Optional: `SUPABASE_URL`, `SUPABASE_SERVICE_ROLE_KEY` (persistence).

## Links

- Website: https://lastsearch.ai
- Documentation: https://lastsearch.ai/docs
- Playground: https://lastsearch.ai/playground
- GitHub: https://github.com/lastsearch-hq/lastsearch
- npm: https://www.npmjs.com/package/lastsearch (renamed from lastsearch, old name still works)
- PyPI: https://pypi.org/project/lastsearch/ (renamed from lastsearch, old name still works)
- Agent Skills: https://github.com/lastsearch-hq/lastsearch-skills
- Discord: https://discord.gg/ubAuT4YQsT
- License: Apache 2.0