Brier.fyi

What it is

The only thing in this category that asks whether the prices were right. Brier.fyi takes questions that several platforms listed, matches them by hand, and scores each platform's market on the outcome — Brier, logarithmic and spherical scores at a chosen criterion point such as the market midpoint or thirty days before close, plus a relative score against the median of the other markets on the same question, and a letter grade for readability.

The methodology is published in full on one page, with the formulas, the grade thresholds, and the awkward details most such projects leave out: that a market with no trades is ignored rather than assumed to be at 50%, that Metaculus is scored on its recency-weighted community prediction, that multiple-choice markets are decomposed or tracked by the winning option depending on the platform, and that continuous and conditional markets are not evaluated at all yet.

It also says what the numbers cannot do. The project's own history is that it spent a year uncomfortable with publishing hard cross-platform accuracy scores, because the platforms are not comparable, and rebuilt itself in January 2025 around matched questions for exactly that reason.

The important caveat is at the top of this card, not the bottom. The pipeline that fed it is paused.

Availability

Nothing to sign up for, nothing to deposit, no account. The site is free to read and its source is public. Code is MIT-licensed — the repository's LICENSE file reads "MIT License, Copyright (c) 2023 wasabipesto" — and the site's content is licensed CC BY-NC-SA 4.0, which is a non-commercial licence and matters if you plan to reuse the figures at work.

The project's about page says it was funded by grants from the Manifold Community Fund and the EA Community Choice programme, which is worth knowing when reading its scores for Manifold: the platform it scores is one of the sources that paid for the work, and the project says so itself rather than leaving it to be discovered.

Checked on 19 September 2026. The about page carries its own last-updated date of 14 October 2025.

Pricing

Free, with no paid tier, no account and nothing for sale; the author's funding page is a GitHub sponsors link. Checked on 19 September 2026.

Markets & resolution

Nothing here resolves anything — the whole point is that somebody else's market already did, and this is the scoreboard afterwards. Four platforms are covered, and they are not the same kind of object: Kalshi's regulated cash-settled contracts, Polymarket's on-chain ones, Manifold's play-money markets and Metaculus, which runs no market at all and is scored on an aggregate of its forecasters. The platforms page is explicit that Metaculus is the odd one out and argues it can still be graded on the same axis.

The matching rule is the load-bearing part. Candidates are found by embeddings, refined with tags, keywords and duration overlap, and then picked and approved by people, with the stated tolerance that a difference between two markets may be grouped only if it would not have more than a one-percent chance of changing which way they resolve. Two markets resolving on 31 December and 1 January, or against two different news sources, are the examples the site gives of what makes that hard.

Absolute scores answer "was this market accurate". Relative scores answer "was it more accurate than the others on the same question", computed daily against the median over the window where at least two markets were open, and summed rather than averaged — so a market that was open for less of the window is pulled toward zero by construction.

Integrations

All the underlying data is offered through a PostgREST API at a data. subdomain, and the source code, open issues and roadmap are on GitHub under the name Themis. There is no client library, no WebSocket and no MCP server.

As of 19 September 2026 that API answers 502.

Limitations

The data pipeline is paused and the site does not say so. The about page still says "our downloader runs nightly". The repository tells the other story: a commit dated 23 January 2026 titled "infra, pause nightly data pipeline", and nothing after 24 January 2026. The questions index is dominated by 2024 and 2025 resolutions. Nothing on the site carries a banner, a date stamp or a warning.

The API is down. A 502 from the documented data endpoint, on the same day the static site served fine — which is the signature of the front end outliving its back end.

Market types are excluded, and the exclusions are not marginal. Continuous and numeric markets on Manifold and Metaculus, Metaculus conditionals, and Manifold multiple-choice markets that resolve to several options at once are all outside the scoring set.

The sample is curated, not exhaustive. Matched questions are chosen and approved by hand, so the platform-level grades are computed over a few hundred questions each — the Kalshi table shows 183 — and that is a selection, not a census of either platform.

The licence is non-commercial. CC BY-NC-SA 4.0 on the content, whatever the MIT licence permits for the code.

Alternatives

Nothing else in this catalogue scores accuracy across venues; the closest is Polymarket dashboards on Dune, where a community calibration dashboard for Polymarket alone is one of the most-forked queries. For the underlying records rather than the scores, Prediction Market Analysis ships the Kalshi and Polymarket trade history as a dataset. Metaculus and Good Judgment Open score their own forecasters directly, which is a different question from scoring a market's price.