# Brier.fyi

Brier scores and letter grades for matched questions across four platforms.

*https://predictionmarkets.tools/tools/brier-fyi · Prediction Market Analytics & Dashboards*

## Facts

### At a glance

| Field | Value |
| --- | --- |
| Vendor | Brier.fyi |
| Category | Prediction Market Analytics & Dashboards |
| Job | accuracy_tracking |
| Website | https://brier.fyi |
| Pricing model | open-source |
| Free tier | true |
| Open source | true |
| Licence | MIT |
| Self-hosted | true |
| Tested hands-on | false |
| Last updated | 2026-09-19 |

### Availability

| Field | Value |
| --- | --- |
| Jurisdictions | global |
| Open to US persons | true |
| KYC required | false |

### Markets

| Field | Value |
| --- | --- |
| Subjects | politics, sports, macro, science, culture, business |
| Settlement | cash |
| Resolved by | none |

### Economics

| Field | Value |
| --- | --- |
| Taker fee | none |
| Maker fee | none |
| Liquidity model | none |
| Platforms | web |
| AI features | none |

### Interfaces

| Field | Value |
| --- | --- |
| API | true |
| WebSocket | false |
| Scripting | none |
| Python | false |
| MCP server | false |
| Export | api |

### Capabilities

Yes: charting, calibration_scoring, cross_venue

No: screening, order_book, backtesting, automation, live_trading, paper_trading, portfolio_tracking, alerts, news, tax_reporting

*Verified: pricing 2026-09-19; availability 2026-09-19.*

## What it is

The only thing in this category that asks whether the prices were *right*. Brier.fyi takes
questions that several platforms listed, matches them by hand, and scores each platform's market on
the outcome — Brier, logarithmic and spherical scores at a chosen criterion point such as the
market midpoint or thirty days before close, plus a relative score against the median of the other
markets on the same question, and a letter grade for readability.

The methodology is published in full on one page, with the formulas, the grade thresholds, and the
awkward details most such projects leave out: that a market with no trades is ignored rather than
assumed to be at 50%, that Metaculus is scored on its recency-weighted community prediction, that
multiple-choice markets are decomposed or tracked by the winning option depending on the platform,
and that continuous and conditional markets are not evaluated at all yet.

It also says what the numbers cannot do. The project's own history is that it spent a year
uncomfortable with publishing hard cross-platform accuracy scores, because the platforms are not
comparable, and rebuilt itself in January 2025 around matched questions for exactly that reason.

**The important caveat is at the top of this card, not the bottom.** The pipeline that fed it is
paused.

## Availability

Nothing to sign up for, nothing to deposit, no account. The site is free to read and its source is
public. Code is MIT-licensed — the repository's LICENSE file reads "MIT License, Copyright (c) 2023
wasabipesto" — and the site's content is licensed CC BY-NC-SA 4.0, which is a non-commercial
licence and matters if you plan to reuse the figures at work.

The project's about page says it was funded by grants from the Manifold Community Fund and the EA
Community Choice programme, which is worth knowing when reading its scores for Manifold: the
platform it scores is one of the sources that paid for the work, and the project says so itself
rather than leaving it to be discovered.

Checked on 19 September 2026. The about page carries its own last-updated date of 14 October 2025.

## Pricing

Free, with no paid tier, no account and nothing for sale; the author's funding page is a GitHub
sponsors link. Checked on 19 September 2026.

## Markets & resolution

Nothing here resolves anything — the whole point is that somebody else's market already did, and
this is the scoreboard afterwards. Four platforms are covered, and they are not the same kind of
object: Kalshi's regulated cash-settled contracts, Polymarket's on-chain ones, Manifold's play-money
markets and Metaculus, which runs no market at all and is scored on an aggregate of its
forecasters. The platforms page is explicit that Metaculus is the odd one out and argues it can
still be graded on the same axis.

The matching rule is the load-bearing part. Candidates are found by embeddings, refined with tags,
keywords and duration overlap, and then picked and approved by people, with the stated tolerance
that a difference between two markets may be grouped only if it would not have more than a
one-percent chance of changing which way they resolve. Two markets resolving on 31 December and 1
January, or against two different news sources, are the examples the site gives of what makes that
hard.

Absolute scores answer "was this market accurate". Relative scores answer "was it more accurate
than the others on the same question", computed daily against the median over the window where at
least two markets were open, and summed rather than averaged — so a market that was open for less
of the window is pulled toward zero by construction.

## Integrations

All the underlying data is offered through a PostgREST API at a `data.` subdomain, and the source
code, open issues and roadmap are on GitHub under the name Themis. There is no client library, no
WebSocket and no MCP server.

As of 19 September 2026 that API answers 502.

## Limitations

**The data pipeline is paused and the site does not say so.** The about page still says "our
downloader runs nightly". The repository tells the other story: a commit dated 23 January 2026
titled "infra, pause nightly data pipeline", and nothing after 24 January 2026. The questions index
is dominated by 2024 and 2025 resolutions. Nothing on the site carries a banner, a date stamp or a
warning.

**The API is down.** A 502 from the documented data endpoint, on the same day the static site
served fine — which is the signature of the front end outliving its back end.

**Market types are excluded, and the exclusions are not marginal.** Continuous and numeric markets
on Manifold and Metaculus, Metaculus conditionals, and Manifold multiple-choice markets that
resolve to several options at once are all outside the scoring set.

**The sample is curated, not exhaustive.** Matched questions are chosen and approved by hand, so
the platform-level grades are computed over a few hundred questions each — the Kalshi table shows
183 — and that is a selection, not a census of either platform.

**The licence is non-commercial.** CC BY-NC-SA 4.0 on the content, whatever the MIT licence permits
for the code.

## Alternatives

Nothing else in this catalogue scores accuracy across venues; the closest is
[Polymarket dashboards on Dune](https://predictionmarkets.tools/tools/dune-polymarket-dashboards), where a community calibration
dashboard for Polymarket alone is one of the most-forked queries. For the underlying records rather
than the scores, [Prediction Market Analysis](https://predictionmarkets.tools/tools/prediction-market-analysis) ships the Kalshi
and Polymarket trade history as a dataset. [Metaculus](https://predictionmarkets.tools/tools/metaculus) and
[Good Judgment Open](https://predictionmarkets.tools/tools/good-judgment-open) score their own forecasters directly, which is a
different question from scoring a market's price.

## FAQ

### Is Brier.fyi still being updated?

The site is up and readable, but the numbers are not moving. The last commit to its source repository is dated 24 January 2026 and the commit before it is titled "infra, pause nightly data pipeline". Its public data API answered 502 when this card was written, and almost every scored question on the questions index resolved in 2024 or 2025. Read it as a frozen archive with an unusually good methodology page.

### What is Calibration City?

The same project under its earlier name. It began in July 2023 as a calibration plot for Manifold, which was not publishing accuracy statistics at the time, grew to cover Kalshi, Metaculus and Polymarket, and was rebuilt from scratch in January 2025 as Brier.fyi with matched cross-platform questions instead of raw calibration curves.

### How are questions matched across platforms?

By embeddings first, then refined with tags, keywords and duration overlap — and then approved by a person. The stated bar is that two markets may be grouped only if their differences would not have more than a one-percent chance of changing the resolution, which is the part most cross-venue comparisons skip.

## Background

- [How accurate a prediction market's price actually is](https://predictionmarkets.tools/guides/how-accurate-the-price-is.md) — What research establishes about reading 73 cents as a 73% chance — where market prices are calibrated, the deviations that recur, and what the horizon does.
- [Scientific and government forecast hubs](https://predictionmarkets.tools/guides/scientific-forecast-hubs.md) — Public-health agencies run open, scored forecast hubs anyone can join. The unit of submission is a model, not a person, and nothing there is a personal record.
- [What a forecasting platform's score actually measures](https://predictionmarkets.tools/guides/what-a-forecasting-score-measures.md) — Proper rules, the two Brier conventions that differ by a factor of two, crowd-relative scores that do not travel between sites, and why profit is not accuracy.

## Also worth comparing

- [Artemis prediction-market metrics](https://predictionmarkets.tools/tools/artemis-prediction-markets.md) — Daily volume, open interest and fees across thirteen event venues, with methodology.
- [Lychee](https://predictionmarkets.tools/tools/lychee.md) — No-code queries, charts and backtests over Kalshi and Polymarket history.
- [Airavat](https://predictionmarkets.tools/tools/airavat.md) — Trader scoring, filters and paper-trading pods over Polymarket. Invite-only.
- [Convexly](https://predictionmarkets.tools/tools/convexly.md) — Polymarket wallet diagnostics with published methods, intervals and failed tests.
- [OrcaLayer](https://predictionmarkets.tools/tools/orcalayer.md) — Polymarket whale analytics indexed from Polygon, with a published farmer filter.
- [Polymarket Analytics](https://predictionmarkets.tools/tools/polymarket-analytics.md) — Third-party Polymarket wallet, trader and market analytics, free to browse.

## Named as a replacement for

- [Metaforecast](https://predictionmarkets.tools/alternatives/metaforecast.md)