← All questionsHow we rate the evidence
Every question gets two separate ratings. They answer different things: how solid the research is, and how directly it applies to products built on large language models.
Evidence quality
| Strong | Several controlled experiments or field studies, with hundreds of participants or more, pointing the same way. |
|---|
| Moderate | At least one well-powered controlled experiment or large audit, but no independent replication in this set. |
|---|
| Limited | Small, qualitative or builder-run studies, or only an abstract we could verify. |
|---|
| Principles | Expert principles or guidelines with no outcome test. |
|---|
Fit with LLM products
| High | Tested with real LLM products or LLM tools in realistic work. |
|---|
| Medium | Tested with an LLM, but in a lab, with a fictional or simulated system, or on a narrow task; or pre-LLM work plus an LLM study that points the same way. |
|---|
| Low | Pre-LLM systems: simple statistical models, classic classifiers or simulated AI. The mechanism may transfer, but it hasn't been tested with LLMs. |
|---|
How we check
- Every number, sample size, venue and version comes from the paper text itself. If we can't verify a detail, we leave it out.
- Quotes are checked word for word against the paper.
- Rules are tagged [from the study] when a paper tested or stated them, and [our reading] when they're our interpretation.
- Ratings are re-checked whenever a paper is added.