Work you can check.
Private models trained on your own work, interpretability tests with every number published, and workbooks that cite every source.
First workbook free. A model pilot on one check, proved side by side.
Custom model training
A model trained on your own past work, running where your data has to stay: a server in your building, the chip in your device, or a fixed monthly price instead of per-call credits.
How a pilot works →Interpretability experiments
We test what people claim about the inside of a model, with controls, on open models, and publish every number, including the claims that failed.
Read the experiments →MCP control center for workbooks
Run research workbooks from Claude, ChatGPT, Codex or Cursor. Ask, start, change and share them in a sentence; every line cites its source.
See the control center →01 · Custom model training
A model trained on your own past work.
Most repeated work does not need a frontier model. It needs a small one that has seen your parts, your documents or your requests, and runs where your data already is.
In your building
Checks done by eye or by hand on data that cannot go into public AI tools: solder joints, polished optics, printed cartons, quotes from RFQ drawings. The model runs on a GPU server at your site.
On your device
A small model on the chip you already ship, for robots, drones, wearables and lab equipment, so the decision is made on the device and does not wait on a weak signal.
Off per-call pricing
AI features you resell as credits because each call costs a different amount. The same work on a model you own, for one fixed monthly price.
How a pilot works
- Pilot
- One repeated check or call, trained on a sample of your past images, documents or logs under NDA, in 10 working days.
- How it runs
- A small model handles the easy cases, a model trained on your data handles the rest, and only unclear cases go to your staff.
- Proof
- A side-by-side accuracy and speed report against how you do it today.
- Upkeep
- We retrain the model as your parts, defects, documents and requests change.
- Pricing
- A fixed pilot fee, then one fixed monthly fee per model. Quoted after we see a sample.
Tell us the check you repeat and roughly how many a week. Your data stays under your NDA and is used only for your model.
Is this you? Read more about a pilot02 · Interpretability experiments
We test what people claim about models, and publish every number.
Each experiment takes a popular claim about the inside of an open model, runs it with a control, and lets you run it yourself. The same habit checks the models we train for you.
59 of 100
Does one neuron catch reward hacking?
The neuron posted as a reward-hacking detector in Qwen3.5-4B tells a hack from an honest answer in 59 pairs of 100. Another neuron, found with a held-out search, does it in 99.
Run it yourself →9.2% vs 8.6%
Does emergent misalignment run through one neuron?
Switching off the neuron that makes the model hostile leaves 9.2% of answers misaligned. A random control neuron gives 8.6%. The claim does not hold.
Run it yourself →91 of 100
Can a monitor reading the thinking catch a pushed neuron?
A Qwen3.5-9B monitor reading Nanbeige4.3-33B's chain of thought catches a full personality push in 91 pairs of 100, and a half push in 65.
Run it yourself →22 vs 7
Does a weight edit stop a model looping?
On prompts with no math, 22 of 32 answers from the released Qwen3.5-0.8B ran to the 8,192-token limit. The edited copy did it 7 times.
Run it yourself →03 · MCP control center for workbooks
Run your research workbooks from any AI agent.
A workbook is one document with one job: a lead report, a diligence pack or a spreadsheet. It is written from primary sources and your own files, every line cites where it came from, and it keeps itself current.
6 minutes later: qualified B2B sales leads across five product lines.
And every source it cited was really where it said it was.
See the run →Works inside Claude, ChatGPT, Codex and Cursor.
Connect the hedwigai MCP and your agent can ask, list, start and change workbooks. You speak in sentences; it makes the calls.
A short answer from your workbooks and programs, with the source it rests on. Nothing to wait for.
Is Moderna a biotech with a public float above $5B?
Your agent
ask · Is Moderna NASDAQ:MRNA a biotech company with public float above $5B?
{
"answer": "yes",
"claims": [{ "figure": "public float", "value": "$9.6B",
"as_of": "2026-06-28" }],
"sources": ["MRNA 10-K, cover page",
"biotech-diligence-7fq2 · §Moderna"]
}Yes. Moderna's public float was $9.6B as of June 28, per the cover of its 10-K.
Read a real workbook first.
Workbooks from our own account, as their runs wrote them. Open the sources and check the rows.
VK2735 competitive landscape: the 8 closest obesity programs
Tirzepatide, CagriSema, retatrutide, orforglipron and four more against Viking's VK2735: phase, latest weight-loss readout, next catalyst and deals, every row cited.
29 sources · 25 minutes
Read the workbook →US importers of bulk black tea from India, from customs records
Four US buyers with the bills of lading that show it, who they buy from, what gates a new supplier, and the leads not worth chasing yet.
10 sources · 8 minutes
Read the workbook →Early-stage AI startups that filed a Form D in the last 30 days
Seven US AI companies raising in September 2026: the instrument each filing names, amount sold, investors and the people on it, each cited to the Form D on EDGAR.
17 sources · 13 minutes
Read the workbook →Tested on real work. Every score published.
Each test is a real job, scored against a checklist written before the run started. We publish every result, including what it missed.
How to train a private AI model on your own data and run it on-premise
Seven steps, from picking one check to running it in your building, and what a per-call API costs instead.
| Measure | 1,000 a day | 10,000 a day |
|---|---|---|
| Claude Haiku 5.5 API, a month | $10.50 | $105 |
| Claude Sonnet 5.5 API, a month | $210 | $2,100 |
Where does a business break when two things happen at once?
Six small businesses as TLA+ models. In each, the obvious fix still breaks and a second one holds.
| Measure | As written | Fixed |
|---|---|---|
| Restaurant, reward an episode | 22.99 | 37.83 |
| Coffee shop, reward an episode | -0.81 | 6.00 |
One YouTube link, read nine ways: from the key shots to the transcript, each priced
A video's look for under a cent, its transcript for two, all nine aspects in one call for $0.024.
| Measure | 2-min video | 15-min video |
|---|---|---|
| The look (visuals) | $0.0097 | $0.054 |
| Transcript | $0.018 | $0.14 with every other aspect |
Can forward passes train a small model on a whole dataset?
New neurons fitted by least squares, with no gradients, take Qwen3.5-0.8B to 95.2% on TREC and 90.8% on Banking77, against 97.0% and 92.9% for one epoch of LoRA.
| Measure | Qwen3.5-0.8B |
|---|---|
| TREC, forward passes / LoRA | 95.2% / 97.0% |
| Banking77, forward passes / LoRA | 90.8% / 92.9% |
Can forward passes alone turn a small model into a router?
One least-squares neuron per label takes Qwen3.5-0.8B from 8.7% to 83.2% on 600 MASSIVE requests, and leaves 99.25% of ordinary text unchanged.
| Measure | Qwen3.5-0.8B |
|---|---|
| MASSIVE, 600 requests, unedited / least-squares neurons | 8.7% / 83.2% |
| One example a scenario in the prompt | 55.5% |
Does a weight edit teach Qwen3.5-0.8B math, or stop it looping?
With thinking off, the model card's default, Qwen3.5-0.8B scores 43.6% on MATH-500 unedited and 45.2% edited.
| Measure | Qwen3.5-0.8B |
|---|---|
| MATH-500, thinking on, unedited / edited | 7.9% / 48.1% |
| MATH-500, thinking off, unedited / edited | 43.6% / 45.2% |
Can interpretability catch a language model misbehaving?
Three claims tested: a reward-hacking neuron (59 of 100), emergent misalignment through the hostile neuron (9.2% against 8.7%), and a chain-of-thought monitor (91 at full push, 65 at half).
| Measure | Qwen3.5-4B | Nanbeige4.3-33B | Qwen3.5-9B |
|---|---|---|---|
| Reward-hacking neuron L17 N9209 | 59 of 100 pairs | ||
| Misaligned, fine-tuned / N8949 off / random off | 8.7% / 9.2% / 8.6% |
Your coding agent isn't stuck because it's dumb. It's stuck because it's alone.
A supervisor that redesigns the team at each plateau took a kernel from 0.51× to 0.74× of FlashInfer at the same budget.
| Measure | Evolving team | Claude Code | Fixed team |
|---|---|---|---|
| MLA decode vs FlashInfer, same budget | 0.74× | 0.51× | 0.31× |
| Mac MoE serving at $200 | 1.056× | 1.047× | not run |
An AI says it broke a 27-year-old hypothesis. Who checks?
Lean proves the statement it was typed. Someone still has to check that it is the famous problem.
| Measure | AI output | Lean done |
|---|---|---|
| Exponential Time Hypothesis is false | 16 Oct | 17 Oct |
| Local smoothing in every dimension | 18 Oct | 19 Oct |
Does one neuron still change Nanbeige4.3-33B's personality when it thinks first?
Seven of 25 neurons tested each give the 4-bit model a personality. With thinking on, it answered 6 of 8 prompts plain and 5 of 56 runs pushed.
| Measure | Nanbeige4.3-33B, 4-bit |
|---|---|
| Neurons with a clear personality, thinking off | 7 of 25 tested |
| Thinking on, plain model: prompts answered | 6 of 8 |
How does Nanbeige4.3-33B score at 4 bits per weight?
At 4 bits per weight and a 60K-token budget: 87.4 on GPQA Diamond (reported 89.5), 86.7 on AIME 2026 (reported 95.2), 90.0 on IFEval.
| Measure | 4-bit, measured | Model card |
|---|---|---|
| GPQA Diamond | 87.4 | 89.5 |
| AIME 2026 | 86.7 | 95.2 |
GLM-5.3 against DeepSeek V4.1 Flash on 75 questions
Two models answer the same 75 questions; one is always right, the other 14× cheaper.
| Measure | GLM-5.3 | DeepSeek V4.1 Flash |
|---|---|---|
| Right, of 75 | 75 | 71 |
| Seconds a question, median | 3.9 | 1.8 |
What does one neuron do to the way a character talks and moves?
One persona neuron changes a character's words, its plan, its gait and its gestures, and the other character too.
| Measure | Qwen3.5-4B | character-animate-4b |
|---|---|---|
| Moves measured | 504 | |
| Upbeat: Sam's lines with an exclamation | 89%, against 17% with no neuron |
One neuron makes Qwen3.5-4B evil. We checked, and found 12 more personalities.
Pushing L14 N8949 alone turns 17 of 20 answers hostile; random neurons pushed as hard turn none.
| Measure | Qwen3.5-4B |
|---|---|
| Hostile answers, L14 N8949 pushed | 17 of 20 |
| Hostile answers, random neurons pushed as hard | 0 of 100 |
Which tools do AI agents use, by marketplace runs, MCP installs and SDK downloads?
Social media scrapers run the most; browser automation is installed the most. Six public sources and four AI assistants, side by side.
| Measure | Apify | npm | Monid |
|---|---|---|---|
| Volume | 169M runs / 30 days | 39.4M MCP SDK / week | 21.0M calls / 30 days |
| Leader | Instagram Scraper | Playwright MCP | TinyFish |
Ranking businesses in different domains on ChatGPT
How ChatGPT ranks businesses across different domains.
Read the numbers →The scale of visual intelligence: a vision model from 235B to 2B without losing accuracy
A 2B model reads as well as the 8B once it can see the digits: the limit was perception, not intelligence.
| Measure | 2B | 4B | 8B |
|---|---|---|---|
| Values read correctly, photograph at 1x | 91.5% | 93.3% | 94.6% |
| Values read correctly, photograph at 2x | 96.7% | 97.4% | 96.7% |
LaTeX from your agent: compile, fix and check a paper without leaving it
Real TeX compiled from Claude Code, Codex, Hermes or any MCP client, with the agent fixing its own compile errors.
| Measure | render_document | Overleaf Free | Overleaf Standard |
|---|---|---|---|
| Compile limit | 4 min | 10 s | 4 min |
| Price | Free to compile | $0 | $25/month |
Who uses Hermes Agent, and what they give it
The people running Nous Research's agent, the tools and models they connect, the personalities they write and the jobs they hand it.
| Measure | Hermes Agent | Claude Code | OpenClaw |
|---|---|---|---|
| OpenRouter tokens, 30 days | 53.6T | 33.7T | 7.5T |
| OpenRouter tokens, 14 October | 2.00T | 1.48T | 0.63T |
Where researchers are moving: traffic, discovery, users and revenue of research tools in 2026
General AI assistants are taking the searching and the writing; the tools that hold a researcher's own work keep them.
| Measure | Claude | QuillBot | Overleaf |
|---|---|---|---|
| Google searches, Sep 2026 | 68M | 5.0M | 3.35M |
| A year earlier | 7.5M | 9.1M | 3.35M |
What people and their agents pay for: x402, Stripe and marketplace data by community
Installing is free. Following the money: what each community pays for, and what agents buy themselves.
| Measure | x402 | Apify | TrustMRR |
|---|---|---|---|
| What is paid for | Per request | Per result | Subscription |
| Last 30 days | $1.09M | $1.7M paid out (Apify's figure) | $157k, OpenClaw makers |
Which MCP servers, connectors and skills each community uses
Eleven communities, from developers to people running an agent for their own life, and the tools each plugs in.
| Measure | Developers | Marketers | Creators |
|---|---|---|---|
| Most-searched tool, Aug 2026 | Playwright 60.5k | Meta Ads 12.1k | Blender 49.5k |
| Its searches in Jan 2026 | 90.5k | 720 | 12.1k |
Reference checker: check every reference in a thesis before you submit it
Does each work exist, are its authors and year right, and does the sentence citing it say what it says.
| Measure | GPT-3.5 | GPT-4 |
|---|---|---|
| References that don't exist | 55% | 18% |
| Real references with an error | 43% | 24% |
The AI agent landscape: who uses which agents, where, why, and with what tools
Six groups of people, eight places agents live, and the tools they plug in, from every public measure there is.
| Measure | ChatGPT | Claude Cowork | OpenClaw |
|---|---|---|---|
| Kind of agent | Chat app | Desktop | Messaging |
| Google searches a month, Aug 2026 | 1.12B | 368k | 450k |
Who uses which AI coding agent, counted five ways
Surveys, router tokens, signed GitHub work, npm installs and stars, side by side: Claude Code leads two and is second on two.
| Measure | Claude Code | Codex | Cursor |
|---|---|---|---|
| Developers using it at work, May–July | 39% | 16% | 12% |
| OpenRouter tokens, 30 days to 13 Oct | 32.6T | 11.9T | 1.1T |
How we made a long-running AI agent for knowledge work 4× faster
One cited knowledge work task through 13 experiments: 937 s to 233 s, with the model held constant.
| Measure | First run | Best since |
|---|---|---|
| Landscape, seconds | 937 | 233 |
| Memo, seconds | 759 | 602 |
Ten models plan a change to a spreadsheet: which do it in under a minute, traced call by call
Ten models drive one agent harness through the same spreadsheet request; six write the plan, timed and traced call by call.
| Measure | Qwen3 235B | GLM-5.2 | Gemini 3.8 Flash | Claude Opus 5 |
|---|---|---|---|---|
| Seconds to the plan, median | 27 | 48 | 57 | 94 |
| Plans written, of three runs | 2 | 3 | 3 | 3 |
SubQ, jev, Perplexity's Decider and Bespoke's Nimble on biotech diligence memos: finding the sentences a column is about
Four decision models judge every sentence of ten biotech diligence memos for five columns, scored against sentences a person marked.
| Measure | SubQ | jev | Decider | Nimble |
|---|---|---|---|---|
| Found, mean of five columns at section length | 64% | 65% | 66% | 68% |
| Mechanism of action, found at section length | 57% | 51% | 61% | 54% |
Website traffic for a list of domains: hedwigai priced against DataForSEO, SpyFu, Semrush and Ahrefs
124 domains' monthly search traffic for $0.0164, against $0.0269 from DataForSEO and $0.062 from SpyFu plus its $119 plan.
| Measure | hedwigai | DataForSEO | SpyFu |
|---|---|---|---|
| 124 domains | $0.0164 | $0.0269 | $0.062 |
| 1,000 domains | $0.104 | $0.132 | $0.500 |
How AI harnesses work, shown on one research workbook
The six terms behind every agent, each shown as the MCP tool that drafts a workbook from primary sources.
| Measure | First run | After review |
|---|---|---|
| Model calls | 112 | 120 |
| Wall clock | 12 min 39 s | 11 min 50 s |
Realtime pricing of India real estate: rent, deposit and amenities in Bengaluru, Delhi and Mumbai
A 2 BHK asks ₹30,000 in Bengaluru and ₹80,000 in Mumbai; the deposit is four months' rent in one city and under two in Delhi.
| Measure | Bengaluru | Delhi | Mumbai |
|---|---|---|---|
| 2 BHK, median asking rent | ₹30,000 | ₹32,000 | ₹80,000 |
| Rent per sq ft | ₹36 | ₹38 | ₹98 |
Twenty current models, every provider, every rate
Closed models cost the same everywhere and sell speed; an open model can cost six times as much from one host as another.
| Measure | GLM-5.3 | DeepSeek V4.1 Flash | GPT-6 Astra |
|---|---|---|---|
| Hosts selling it | 52 | 49 | 7 |
| Cheapest, per 1M read / written | $0.35 / $1.10 | $0.09 / $0.18 | $10 / $50 |
Leads with a reason to write: hedwigai ask, Apollo and treg on one brief
40 leads each, every one checked: the right person, an inbox that accepts, and a dated reason to write.
| Measure | ask | Apollo | treg |
|---|---|---|---|
| Usable leads | 13 of 40 | 7 of 40 | 6 of 28 |
| Cost per usable lead | $0.037 | $0.27 | $0.23 |
Time machine: leak-free RL environments, graded by what actually happened
A feature: pick a date, and every source answers as it would have on that day.
| Measure | RL player | A coin |
|---|---|---|
| Score, clock before the answer is measured | 0.747 | 0.750 |
| Score, clock after it (a leak) | 0.762 | 0.750 |
Self-hosted decisions for ask: Perplexity's open Decider in place of jev on 92 company questions
The decisions inside hedwigai ask made by Perplexity's open Decider on two NVIDIA RTX 3090s, against jev.
| Measure | ask with jev | ask with Decider |
|---|---|---|
| Questions right, of 92 | 89 | 89 |
| Wrong answers | 1 | 2 |
Biotech due-diligence document gathering: coverage across 11 companies
Eleven companies of diligence documents, gathered and scored.
| Companies covered | 11 |
|---|---|
| Mean coverage, 0 to 1 | 0.971 |
US importer search for heavy engineering equipment: coverage across 5 product lines
Five product lines of importers found, qualified and gated.
| Product lines covered | 5 |
|---|---|
| Mean coverage, 0 to 1 | 0.956 |
Aircraft records completeness: an evaluation method (no models scored yet)
Whether a records package is complete — the method, before the scores.
| Methods described | 8 |
|---|---|
| Kinds of defect | 9 |
Clinical decisions: jev and ask on System One, PubMedQA, SciFact and a 370-decision set
EdgeEvals' System One board, PubMedQA, SciFact, and a clinical set built to System One's rules.
| Measure | jev | ask |
|---|---|---|
| Our 370 clinical decisions, right | 94.6% | 93.0% |
| PubMedQA, right (best EdgeEvals scored: 62.2%) | 77.0% | 74.4% |
Executive lookup for US-listed companies: Claude Opus 5.5 and ask on 95 questions
Who runs a public company, and how to email them: a frontier model from memory against hedwigai ask.
| Measure | ask | Claude Opus 5.5 |
|---|---|---|
| Questions right, of 95 | 59 | 63 |
| Small companies right, of 20 | 13 | 9 |
Rental comps: floor-plan rents and amenities on 14 listings, five assistants compared
Rent per floor plan and amenities for US rentals: hedwigai realestate-us against Claude, GPT, Gemini and Perplexity with web search.
| Measure | realestate-us | Best of four assistants |
|---|---|---|
| Floor-plan rents right, of 26 | 26 | 23 |
| Amenities claimed the listing doesn't state | 12 | 129 |
Company screening from SEC filings: jev, Vela 2.0 and ask on 92 yes/no questions
Yes-or-no questions about public companies, answered by two decision models alone and by hedwigai ask.
| Measure | ask | jev |
|---|---|---|
| Right, of the questions answered | 98% | 79% |
| Wrong answers, of 92 | 2 | 19 |
A same-model decider above an agent's writer: one task, two configurations
The same document and the same instruction, run with and without a decider above the writer.
| Measure | With a decider | Without |
|---|---|---|
| Wall clock | 23m 36s | 3m 46s |
| Model calls | 104 | 34 |
Defect localization in industrial images: four frontier models and a trained localizer
Four frontier models asked to put a box around what they just described.
| Measure | Trained localiser | Best of four frontier models |
|---|---|---|
| Mean overlap with the defect (IoU, 0 to 1) | 0.421 | 0.109 |
Company lists in ask: a small model trained to put the right companies first, tested on 33 new questions
Ask for a list of companies and get ten, each with its source. How the ten are chosen, and how much better it got.
| Measure | Trained for lists | General-purpose model |
|---|---|---|
| Correct, of the companies in the top 10s | 62% | 55% |
| Correct companies in the top 10s | 142 | 117 |