Scientific and government forecast hubs

There is a second forecasting world running alongside the one this catalogue lists, and almost nothing connects the two. Public-health agencies in Europe and the United States operate open forecast hubs: they publish a surveillance series, invite anybody in the world to forecast it, collect submissions weekly through a public code repository, build an ensemble, score every submission against what actually happened, and publish the scores as files you can download without an account.

By the standards this site usually applies, they are in remarkably good shape. The questions resolve against a named data series. The scoring rule is written down and is a proper one. The record goes back two years and is public. Nobody is selling anything.

And a reader cannot use any of it the way they would use a forecasting platform, because the hubs have no human dimension at all. There are no accounts. There are no personal scores. There is no way to submit one forecast about one question. The unit is a pair of names, a team and a model, and what gets ranked is the model. That is why these hubs are not cards in the forecasting platforms category, and it is also why they are worth an hour of anyone's attention: they are the best public evidence available on how hard short-horizon forecasting of a real quantity actually is.

The worked example throughout is the European Syndromic Indicators Forecasting Hub, part of the RespiCast consortium run by ECDC with the ISI Foundation and the London School of Hygiene and Tropical Medicine. It is the most inspectable of the three, because its evaluation output is committed to the repository rather than only rendered on a website. Every figure below was read out of that repository on 21 September 2026 and the file it came from is cited.

How it works

The chain is short, and every link in it is a file in a public repository.

1. An agency publishes the target data. ECDC communicates weekly ILI and ARI incidence for EU/EEA countries through the ERVISS report, generally on a Friday, covering up to the previous week. The hub copies that into a target-data folder, and its FAQ is explicit that the copy in that folder is the ground truth: forecasts are evaluated against it and not against whatever the agency later revises.
2. A submission window opens and closes. Submissions open on Saturday at 00:00 CET after the target data update and close the following Wednesday at 23:59 CET.
3. Submissions are validated automatically. A pull request against the repository triggers format checks. A file in the wrong shape fails and is not scored.
4. An ensemble is built. The hub generates its ensemble immediately after the Wednesday close, and forecasts plus the ensemble are published on the hub website on Thursday.
5. Scores are computed later, as the real data arrives, and written into model-evaluation/ as a rolling file and as dated weekly snapshots.

What a submission actually is

Not a probability. A submission is a table of quantiles. The hub's task configuration requires 23 quantile levels for each forecast: 0.01, 0.025, 0.05, then every 0.05 from 0.1 to 0.9, then 0.95, 0.975 and 0.99. Each one carries a predicted incidence value, and the set of them is the forecast. A median may be supplied as well; it is optional.

Each of those tables is tagged with a target (ILI incidence or ARI incidence), a country, an origin date, and a horizon from -1 to 4 weeks. Horizon -1 is the week that has already passed but has no consolidated data yet, which is why the hub calls the exercise nowcasting and forecasting rather than forecasting alone.

The unit is a model, and the hub says so in a schema

The clearest statement of what these hubs are is the metadata file every participant must supply. Its schema makes eight fields required: team_name, model_name, team_abbr, model_abbr, model_contributors, team_model_designation, methods and data_inputs. There is no field for a person's record, because the record is not a person's. model_contributors exists and can carry a name, an affiliation and an ORCID, but it is attribution on a model, the way an author list is attribution on a paper. team_model_designation takes one of primary, secondary, proposed, other, ensemble or hub_other, which is a team declaring which of its own models it stands behind.

The scores inherit that shape exactly. Every row of the evaluation file is keyed by team_id and model_id, and the identifiers you see in a ranking are things like ISI-GLEAM, QMUL-SEIR_agumented and respicast-quantileBaseline. Thirty-three model metadata files were in the repository on 21 September 2026 and 32 distinct team-and-model pairs appear anywhere in the score history.

Anyone can join, and at least one participant is one person

The hub's joining page invites "any group, individual, or institution", and the FAQ says individuals worldwide may submit forecasts for any EU/EEA country. This is not a formality. The metadata file for DHauser-FluChronos gives a team name of David Hauser, one contributor, and an affiliation of "Independent researcher"; its method is a fine-tune of a time-series foundation model on WHO FluID surveillance series. In the most recent scored round that model placed second of fourteen on one of the cells described below.

And it is still called DHauser-FluChronos. One person, working alone, producing a top-ranked forecast, and the hub records it as a model identifier, because that is the only kind of thing it records. This is the whole difference from a forecasting platform in a single row of a CSV.

The same shape in the United States

The US hubs are run by the CDC and differ in target, not in structure.

FluSight collects weekly nowcasts and forecasts of confirmed influenza hospital admissions for the nation, all 50 states, Washington DC and Puerto Rico, plus an optional target for the proportion of emergency department visits due to influenza. The 2026-2027 exercise is scheduled to begin on 7 October 2026 and run until 26 May 2027, with submissions due by 11PM Eastern each Wednesday. Predictions are compared against the NHSN Weekly Hospital Respiratory Dataset. Horizon -1 is accepted but is explicitly not scored in summary evaluations, and teams are told to submit it anyway because it helps detect calibration problems.

The COVID-19 Forecast Hub does the same for weekly confirmed COVID-19 hospital admissions, with an optional emergency-department-visit proportion target added in June 2025, for the same set of jurisdictions.

Both use the same quantile submission format, the same weekly cadence, the same public repository mechanics, and the same absence of any personal identity. FluSight adds target types a European reader will not have seen: probabilities over five rate-trend categories, and seasonal targets for peak week and peak incidence.

What a quantile submission is scored with, and why the metric is different

On a forecasting platform you state one number, the probability of an event, and you are graded with a rule built for that: a Brier score, or a log score, or some rescaling of one of them. Those rules take a probability and an outcome that either happened or did not.

Here the outcome is a count. ILI incidence in Belgium in the week ending 6 September 2026 was some number of notified cases per 100,000 people, and no probability attaches to it directly. So the hub grades the distribution you submitted against the number that arrived, with the Weighted Interval Score, and its evaluation wiki gives the formula.

In plain terms, WIS does two things at once and adds them up.

• It charges you for width. Take any central prediction interval you submitted, say the one running from the 10th to the 90th percentile. You pay its width, upper bound minus lower bound, whatever happens. A forecast that hedges by predicting everything pays for predicting everything.
• It charges you much more for being outside. If the reported value lands below your lower bound or above your upper bound, you additionally pay the distance by which you missed, multiplied by 2 divided by the interval's significance level. For a narrow, confident interval that multiplier is large. So narrowing an interval is cheap right up to the point where the truth falls outside it, and then it is very expensive.

Those two charges are computed for every interval implied by your 23 quantiles, averaged with weights, and added to a term for how far the median missed. The result is in the units of the thing being forecast, and lower is better.

The practical difference from a platform score is worth stating directly. A Brier score asks whether you were confident in the right direction. WIS asks whether the shape of your uncertainty was right: whether you were as sure as you claimed to be, no more and no less. You cannot do well on it by being vague, and you cannot do well on it by being bold. What a platform's score is rewarding instead, and why two of them rarely compare, is its own guide; the three formulas the platforms in this catalogue actually use are pulled apart in which score you are chasing.

The hub computes exactly two metrics: WIS, and Absolute Error, defined as the absolute difference between the predicted median and the reported value. AE looks at one number out of your 23 and ignores the rest, which is precisely why it is published next to WIS rather than instead of it. In the snapshot read for this page, WIS and AE rows were present in near-equal numbers: 133,101 and 132,845 respectively, out of 265,946 rows.

Baseline, and what a relative score means

A WIS of 4.04 means nothing on its own, because it is in cases per 100,000 and a hard week scores worse than an easy week for everybody. So the hub does what a forecasting platform does when it scores you against the crowd: it scores each model against a reference.

The reference is a model in the file like any other, respicast-quantileBaseline, a naive baseline the hub runs itself. Every row carries a value_absolute, which is the raw WIS or AE, and a value_relative, which the wiki defines as

log2( WIS of the baseline / WIS of the model )

Positive means better than the baseline, negative means worse, and the baseline scores exactly zero by construction. That last part is checkable rather than assumed: across all 28,377 baseline rows in the snapshot read for this page, value_relative was 0.0 in every one.

The base-2 logarithm is doing real work. It makes the scale symmetric, so that being twice as good as the baseline (+1) and twice as bad (-1) are the same distance from zero, which a plain ratio does not give you. It also converts back into something you can say out loud. Here is the cell this page has been quoting, from the snapshot dated 2026-09-09: target ILI incidence, location Belgium, horizon 1, metric WIS, 14 models scored.

Read the second column against the third and the transformation is obvious: +1.537 is a WIS 2.90 times lower than the baseline's, and -0.680 is a WIS 1.60 times higher. Eight of the fourteen models beat the naive baseline in that cell and five lost to it, which is the ordinary state of affairs and the reason the baseline is published as a competitor rather than as a footnote.

Two more columns matter and are easy to skip. n_models is the size of the field the rank was computed in, and it varies by cell: 14 here, but 7, 8, 10, 12 and 13 in other cells of the same round, because teams submit for the countries and horizons they choose. rank_score is a normalised position within that field. A rank without its n_models is not a fact.

What it costs

Nothing, in money. There is no fee, no account, no deposit and no prize. Submission is a pull request to a public repository. What participation and use actually cost is three other things, and all three are underestimated.

Compute, and the work behind it. The methods declared in the metadata range from ARIMA and SEIR models to gradient boosting, neural forecasting architectures and fine-tuned time-series foundation models. The independent participant quoted above declares a fine-tune of Chronos-2 across global WHO FluID series with five-year context windows, recency weighting and peak oversampling, rerun for each eligible target and location. That is not a number typed into a box. A hub asks for a pipeline that runs unattended, every week, for a season.

A weekly rhythm you have to hit. The window opens Saturday 00:00 CET and closes Wednesday 23:59 CET, and the ensemble is built the moment it closes. Miss it and the round is gone. The hub is gracious about this in its FAQ, where the answer to whether missing a week is acceptable is simply "Yes!" and participation in every round is not mandatory, but the cost lands on your own record: a model with few rounds has a thin history, which is the next point.

A wait of up to three weeks before you learn anything. This is the one that separates a hub from a platform most sharply, and it can be measured directly from the published snapshots.

In the snapshot dated 2026-09-09, the forecasts made on origin date 2026-09-09 have scores at horizon 1 only, for the week ending 2026-09-06. Forecasts made on 2026-09-02 have horizons 1 and 2. Forecasts made on 2026-08-26 have 1, 2 and 3. Only from 2026-08-19 backwards does the full set of horizons 1 to 4 appear. Compare it with the previous week's snapshot, dated 2026-09-02, and the boundary has moved by exactly one step: there, 2026-08-19 still had only horizons 1 to 3.

So the four-week-ahead forecast you submit today gets its score 21 days from now, one horizon at a time, as the surveillance data catches up. There is no instant feedback anywhere in this design, and there cannot be: the ground truth does not exist yet.

And a cost to the reader, which is smaller but real. The rolling latest-forecast_scores.csv was 30.8 MB and 265,946 data rows when read for this page, covering origin dates from 2024-10-23 to 2026-09-09 across 32 locations. It is not a leaderboard. It is a long file that you group yourself.

Where this stops

Be clear about how narrow this is before treating it as a general lesson.

The subject matter is respiratory disease incidence and nothing else. ILI and ARI incidence in EU/EEA countries; influenza and COVID-19 hospital admissions and emergency department visit proportions in the United States. Nothing about elections, macroeconomics, company events or any other subject a prediction market lists. A hub's demonstration that four-week-ahead forecasting is hard is a demonstration about a seasonal epidemiological series, which has structure that most questions do not.

There is no personal participation, and this is not an oversight to be worked around. You cannot open an account, you cannot answer one question, and nothing you do there accrues to you by name. The nearest thing in this catalogue to a scored, academically run exercise with a human on the other end is the Social Science Prediction Platform, where the unit is a person forecasting what a study will find.

A rank is not a track record. Two of the models in the score history appear in only one origin date, and DHauser-FluChronos, the second-placed model in the table above, appears in two. The baseline appears in 99. Reading a single-round rank as evidence about a model is the same error as reading a single resolved question as evidence about a forecaster, and the file gives you the participation count to avoid it.

Confirmed numbers are thinner than they look. Almost every specific figure on this page comes from one repository and its own wiki, read on one day. The evaluation folder's own readme file is empty, so the definition of the metrics lives only in the wiki. And the rolling file is not fresher than the snapshots: on 21 September 2026 latest-forecast_scores.csv had the identical SHA-256 digest to the snapshot dated 2026-09-09, twelve days earlier. If you need the state of play today, today is not what the file contains.

None of this tells you what will resolve. A hub scores models against surveillance data that is itself revised. The question of what a given contract resolves against, and who decides, is a separate mechanism entirely and is covered in who decides how a prediction market resolves.

What you can do about it

Download one snapshot and look at a single cell yourself. Go to model-evaluation/snapshots, take the newest dated file, and filter it to one origin_date, one target, one location, one horizon and metric == WIS. What comes back is a dozen or so rows, already ranked, with the baseline sitting somewhere in the middle. Eighty-six dated snapshots were in that folder on 21 September 2026, so you can also take last month's file and see what a forecast looked like before the answer was known.

Read n_models before you read rank. They are adjacent columns for a reason. Rank 3 of 14 and rank 3 of 7 are different claims, and in the round read for this page both field sizes occur.

Use value_relative as a difficulty gauge, not as a leaderboard. The useful question is not which model won. It is how many models beat a naive baseline at all, and by how much. In the cell above, eight of fourteen did, the best by a factor of 2.9 and the worst losing by a factor of 4. That spread is the honest answer to "how well is this problem solved", and it is worth carrying into any adjacent question you are about to take a position on. If the specialists with a season of tuning are clustered near a naive baseline four weeks out, a confident price on a related question four weeks out deserves more scepticism than its tightness suggests.

Check the horizon before you compare anything to anything. Horizon 1 in the European hub is the week that has already ended without consolidated data, and horizon 4 is three weeks past that. Scores at those two horizons are not comparable and were never meant to be. FluSight goes further and excludes its horizon -1 from summary evaluation entirely while still asking teams to submit it.

If what you want is a personal score, use a platform and know which formula it uses. Metaculus grades you with rescalings of the log score against the community, and Good Judgment Open with a Brier score relative to one crowd. Neither is comparable to the other, and neither is comparable to a WIS, for reasons set out in what a forecasting platform's score actually measures. If you want your own resolved questions scored across several platforms at once, brier.fyi is the card for that, and the whole forecasting platforms listing carries the scoring mechanism on every card.

If what you have is a model, submit it. The joining instructions are a fork, a metadata file and a pull request, and the hub's stated position is that it does not want the GitHub mechanics to deter anyone; the wiki gives an address to write to for help. A naive model is a legitimate submission. The hub already runs one and publishes its score at the top of the same table as everyone else's, which is the most useful convention in this entire corner of forecasting and the one no platform in this catalogue has copied.