# How to track and score your own forecasts

Five ways to keep a forecasting record that still means something a year later — two you own, two that supply the questions and the crowd, one for practice.

*https://predictionmarkets.tools/how-to/track-your-own-forecast-record · next to Forecasting Platforms*

**Answer:** Write the question down before you know the answer, with a probability and a resolution date, somewhere that timestamps it. Fatebook is the cheapest version of that; Confido is the one you host yourself. Metaculus and Good Judgment Open supply the questions and a score other people recognise. Then record which convention produced your number, because two of them differ by a factor of two.

## The approaches, in order

1. [Fatebook](https://predictionmarkets.tools/tools/fatebook.md) — Ten seconds a question, private by default, with a Slack command. Absolute and relative Brier day by day, a calibration chart, and a one-click CSV export.
2. [Confido](https://predictionmarkets.tools/tools/confido.md) — The same idea on a server you run, under AGPL. Its score is a rescaled Brier running −3 to 1 that points upwards, and it scores binary questions only.
3. [Metaculus](https://predictionmarkets.tools/tools/metaculus.md) — Somebody else's questions, resolved by staff against published criteria, scored on the log score. The record most likely to be read the same way by a stranger.
4. [Good Judgment Open](https://predictionmarkets.tools/tools/good-judgment-open.md) — Fixed Challenge question sets, scored by Relative Brier against the crowd's daily median. No forecast can be withdrawn or deleted, by policy.
5. [Quantified Intuitions](https://predictionmarkets.tools/tools/quantified-intuitions.md) — Pastcasting on questions that already resolved, scored against the crowd of the day — the feedback loop for while the real record takes a year to accrue.

*Ordered editorially. Paid placement does not affect this order.*

## The short way

The record is made at the moment you write the forecast down, not at the moment somebody scores
it. Three things have to be fixed before the answer is known: the probability, the resolution
criterion written out in enough words that a stranger could apply it, and the date by which it is
settled. Everything below is machinery for storing those three and doing arithmetic on them later.

[Fatebook](https://predictionmarkets.tools/tools/fatebook) is the shortest path to having them stored. A question is a title, a
probability and a resolve-by date; it is private until you share it; the platform reminds you when
the date arrives, and resolution is YES, NO or AMBIGUOUS. A Slack command opens a question to a
channel, and a browser extension opens one from any page. The API key is a query parameter rather
than a header, which is why the integrations people have built on it are Alfred workflows and iOS
shortcuts rather than services — writing a forecast down fits in a shell one-liner.

The scoring is where the detail is. The source walks the question day by day from creation to
resolution, computes a time-weighted average of the forecasts you made within each day, scores
that, and averages the days — so a forecast left standing counts for every day it stood, and
updating early is worth more than updating correctly at the end. Beside your absolute Brier score
it computes a relative one, subtracting the median of everyone else's score on that question that
day.

**Record which platform produced the number, in the same place you record the number.** There are
two Brier conventions in circulation and they differ by a factor of two on every figure:

```python
# The same forecast: 70% on something that then happened.
p, outcome = 0.7, 1

single = (p - outcome) ** 2                                   # 0.09, bounded at 1
both = (p - outcome) ** 2 + ((1 - p) - (1 - outcome)) ** 2     # 0.18, bounded at 2
```

Fatebook's source computes the second one. A score copied into a spreadsheet without its
provenance is a number you will not be able to interpret in a year, which is the failure this whole
exercise exists to avoid.

## What the options are

**A tracker you own.** [Fatebook](https://predictionmarkets.tools/tools/fatebook) above, hosted by somebody else and free, with a
one-click export of everything you have ever forecast. [Confido](https://predictionmarkets.tools/tools/confido) is the version you
run: rooms, members and questions on your own server under AGPL, which is the answer when the
questions themselves are the sensitive part. Two things about it are unlike everything else here.
Its score is a rescaled Brier — the source computes `(0.25 − (outcome − p)²) × 4`, which runs from
−3 to 1, sits at 0 for a 50% forecast and is **higher-is-better**, the opposite direction from every
other score on this page. And automatic scoring covers binary questions only: ask for a numeric
range and the scoring function returns nothing, leaving the CSV export and your own arithmetic.
Confido also has a `score time` on each question, and a prediction counts only if it was submitted
before that moment.

**Somebody else's questions, and a score with a public method.**
[Metaculus](https://predictionmarkets.tools/tools/metaculus) resolves against criteria published on the question, annuls a question
whose criterion turns out not to be evaluable, and scores on the log score under two rescalings.
The Baseline score compares your forecast to a fixed chance benchmark. The Peer score compares it
to the geometric mean of the other forecasters live at the same moment, scaled by the number of
them, which makes it the harsher and more informative figure. Both are weighted by coverage — the
share of the question's life your forecast was standing — and both have a spot variant that reads a
single moment instead. The API now needs an account token on every request, so plan for the record
to live on the site rather than in your own store.

[Good Judgment Open](https://predictionmarkets.tools/tools/good-judgment-open) is the Brier-based equivalent, organised into
Challenges with their own leaderboards and sponsors. Its Relative Brier Score is your average daily
Brier minus the crowd's average daily median, multiplied by your participation rate; lower is
better, negative beat the crowd, and a question you skipped scores zero rather than counting
against you. The property worth the most here is procedural rather than statistical: there is no
way to withdraw from a question or delete a forecast. A record you cannot tidy afterwards is the
only kind worth keeping.

**Practice, while the real record accrues.** [Quantified Intuitions](https://predictionmarkets.tools/tools/quantified-intuitions)
serves you questions that already resolved, presented as they stood on a vantage date with the
crowd forecast hidden and a search engine that filters out anything published after it. You learn
in ninety seconds instead of eighteen months, scored by a log score relative to the crowd of the
day. It is training, not a track record, and the site does not pretend otherwise.

## Where this breaks

**A Brier score without its convention is not a score.** Fatebook sums the squared error over both
outcomes and is bounded at 2; the single-outcome convention is bounded at 1; Confido rescales to
−3 to 1 and flips the direction of "good". Three numbers, three meanings, and none of them prints
the convention beside itself. [What a forecasting platform's score actually
measures](https://predictionmarkets.tools/guides/what-a-forecasting-score-measures) is the page for the arithmetic; the operational
version is to store the platform name in the same row as the figure, forever.

**A crowd-relative score describes the crowd, not you.** Fatebook subtracts the median of the other
forecasters on that question; Good Judgment Open subtracts the crowd's daily median and scales by
participation; Metaculus divides by the geometric mean of everyone live at that moment. Change the
crowd and your number moves while your judgement stands still, which is why none of these travels
to another site and why a good relative score on a thin question set is worth less than it looks.
It has a sharper edge on a private tracker: a question only you forecast has no crowd to subtract,
so the relative score is simply absent. [Which score you are
chasing](https://predictionmarkets.tools/compare/forecasting-platform-scoring) is the head to head.

**A record made of easy questions is not a better record.** Nothing in a Brier or log score knows
how hard the question was — that is precisely the work a crowd-relative score does, and it is the
thing you give up by asking your own questions. Twenty instances of "will I ship this by Friday"
resolving YES produce a calibration chart with a confident-and-correct bar and no information in
it. The cheap defences are to tag questions and read the chart per tag rather than in aggregate, to
write the question when it is genuinely uncertain rather than when it is nearly settled, and to keep
some fraction of your volume on a question set somebody else chose.

**On your own questions you are the forecaster and the judge.** Fatebook's author resolves, with no
moderator, no adjudication and no appeal; on Confido whoever runs the room enters the outcome.
Nothing stops a question being resolved generously and nobody will notice. The discipline that
replaces the missing referee is written before the fact: name the source that will decide, in the
question, and use the AMBIGUOUS button — the equivalent of a Metaculus annulment — rather than
forcing an unanswerable question to the answer that flatters the record. [Who decides how a
prediction market resolves](https://predictionmarkets.tools/guides/who-decides-the-outcome) is the same problem where money is
involved.

**Profit on a play-money platform is not accuracy.** [Manifold](https://predictionmarkets.tools/tools/manifold) is the place this
bites, because its profile number is mana profit and it reads like a score. Profit is position size
multiplied by price movement, summed; your stated probability appears nowhere in it, and it rewards
finding a mispriced question, being early in a thin one and sizing well. Those are real skills and
none of them is stating a well-calibrated probability. Manifold does publish a platform-wide
calibration curve, but that is a claim about the platform rather than about you — and on 19
September 2026 the figure on that page carried no unit, which is the first problem on this page all
over again.

## If you outgrow this

If the record has to be read by a research audience, the shape of the artefact changes. [Scientific
and government forecast hubs](https://predictionmarkets.tools/guides/scientific-forecast-hubs) are open, scored against
surveillance data and published as dated CSVs — but the unit of submission there is a team-and-model
pair, not a person, so there is no personal record to build. The [Social Science Prediction
Platform](https://predictionmarkets.tools/tools/social-science-prediction-platform) is the nearer thing: forecast what an
unpublished study will find, and be ranked by error against the estimate it eventually reports.

If the record is meant to lead somewhere, Good Judgment Open is the one with a published
destination — recruitment each autumn from forecasters who have answered at least 100 questions,
average accuracy per closed question weighted most heavily, then comment quality, then a
three-month probation.

And if the question you actually want answered is whether the *prices* were any good rather than
whether you were, that is a different measurement entirely: [Brier.fyi](https://predictionmarkets.tools/tools/brier-fyi) scores
four platforms against each other on hand-matched questions, with its pipeline paused and its method
published. [Everything here that computes a score](https://predictionmarkets.tools/collections/calibration-scoring) is the wider
listing, and it is worth reading before assuming that two of those numbers can be put side by side.

## FAQ

### How many questions does a record need before it says anything?

None of these platforms publishes a threshold, and inventing one would be worse than having none. The only published bar in this catalogue is a recruiting one — Good Judgment identifies potential Superforecasters from forecasters who have answered at least 100 GJ Open questions, weighing average accuracy per closed question most heavily. Below that order of magnitude, treat a calibration chart as a sketch.

### Can I move a record from one platform to another?

Rarely, and the direction matters. Fatebook imports from PredictionBook and from a spreadsheet, and exports every forecast you have made as CSV; Confido exports predictions as CSV, which its own guidance treats as the escape hatch for everything its scoring does not cover. Good Judgment Open publishes no API and its terms withhold permission for automated extraction, and Quantified Intuitions has no documented export at all.

### Does a good calibration chart mean I am a good forecaster?

It means one of the two halves. Calibration says that things you called 70 percent happen about 70 percent of the time, which a forecaster who says 50 percent to everything can also achieve on a balanced question set. The other half is resolution — saying 95 when 95 is right — and no calibration chart on its own distinguishes them.

### Should I keep the same record on several platforms at once?

Only if you want three short records under three incomparable formulas. The rules differ, the crowds differ, the sign conventions differ and two of the Brier numbers differ by a factor of two. Put the volume where the formula is the one you want to be measured by, and use the others for practice.

## Sources

1. [fatebook/lib/_scoring.ts](https://github.com/Sage-Future/fatebook/blob/main/lib/_scoring.ts) — Sage, read 2026-09-21
2. [confido1/src/commonMain/kotlin/Score.kt](https://github.com/confidoinstitute/confido1/blob/master/src/commonMain/kotlin/Score.kt) — Confido Institute, read 2026-09-21
3. [metaculus/scoring/score_math.py](https://github.com/Metaculus/metaculus/blob/main/scoring/score_math.py) — Metaculus, read 2026-09-21
4. [Frequently Asked Questions](https://www.gjopen.com/faq) — Good Judgment Open, read 2026-09-21

*Last updated 2026-09-21. A reference page, corrected in place — not a dated post.*