# Scientific and government forecast hubs

Public-health agencies run open, scored forecast hubs anyone can join. The unit of submission is a model, not a person, and nothing there is a personal record.

*https://predictionmarkets.tools/guides/scientific-forecast-hubs · background to Forecasting Platforms*

**Answer:** Public-health agencies run open forecasting hubs: RespiCast under ECDC, FluSight and the COVID-19 hub under the US CDC. Anyone may join, every submission is scored against surveillance data, and the scores are published as dated CSV files. But the unit is a team-and-model pair, not a person. There are no accounts, no personal ratings, and no way to submit a single forecast rather than a model.

There is a second forecasting world running alongside the one this catalogue lists, and almost
nothing connects the two. Public-health agencies in Europe and the United States operate open
forecast hubs: they publish a surveillance series, invite anybody in the world to forecast it,
collect submissions weekly through a public code repository, build an ensemble, score every
submission against what actually happened, and publish the scores as files you can download
without an account.

By the standards this site usually applies, they are in remarkably good shape. The questions
resolve against a named data series. The scoring rule is written down and is a proper one. The
record goes back two years and is public. Nobody is selling anything.

And a reader cannot use any of it the way they would use a forecasting platform, because the
hubs have no human dimension at all. There are no accounts. There are no personal scores. There
is no way to submit one forecast about one question. The unit is a pair of names, a team and a
model, and what gets ranked is the model. That is why these hubs are not cards in the
[forecasting platforms](https://predictionmarkets.tools/categories/forecasting-platforms) category, and it is also why they are
worth an hour of anyone's attention: they are the best public evidence available on how hard
short-horizon forecasting of a real quantity actually is.

The worked example throughout is the European Syndromic Indicators Forecasting Hub, part of the
RespiCast consortium run by ECDC with the ISI Foundation and the London School of Hygiene and
Tropical Medicine. It is the most inspectable of the three, because its evaluation output is
committed to the repository rather than only rendered on a website. Every figure below was read
out of that repository on 21 September 2026 and the file it came from is cited.

## How it works

The chain is short, and every link in it is a file in a public repository.

1. **An agency publishes the target data.** ECDC communicates weekly ILI and ARI incidence for
   EU/EEA countries through the ERVISS report, generally on a Friday, covering up to the previous
   week. The hub copies that into a `target-data` folder, and its FAQ is explicit that the copy in
   that folder is the ground truth: forecasts are evaluated against it and not against whatever
   the agency later revises.
2. **A submission window opens and closes.** Submissions open on Saturday at 00:00 CET after the
   target data update and close the following Wednesday at 23:59 CET.
3. **Submissions are validated automatically.** A pull request against the repository triggers
   format checks. A file in the wrong shape fails and is not scored.
4. **An ensemble is built.** The hub generates its ensemble immediately after the Wednesday close,
   and forecasts plus the ensemble are published on the hub website on Thursday.
5. **Scores are computed later, as the real data arrives**, and written into `model-evaluation/`
   as a rolling file and as dated weekly snapshots.

### What a submission actually is

Not a probability. A submission is a table of quantiles. The hub's task configuration requires 23
quantile levels for each forecast: 0.01, 0.025, 0.05, then every 0.05 from 0.1 to 0.9, then 0.95,
0.975 and 0.99. Each one carries a predicted incidence value, and the set of them is the
forecast. A median may be supplied as well; it is optional.

Each of those tables is tagged with a target (ILI incidence or ARI incidence), a country, an
origin date, and a horizon from -1 to 4 weeks. Horizon -1 is the week that has already passed but
has no consolidated data yet, which is why the hub calls the exercise nowcasting and forecasting
rather than forecasting alone.

### The unit is a model, and the hub says so in a schema

The clearest statement of what these hubs are is the metadata file every participant must supply.
Its [schema](https://github.com/european-modelling-hubs/RespiCast-SyndromicIndicators/blob/main/hub-config/model-metadata-schema.json)
makes eight fields required: `team_name`, `model_name`, `team_abbr`, `model_abbr`,
`model_contributors`, `team_model_designation`, `methods` and `data_inputs`. There is no field
for a person's record, because the record is not a person's. `model_contributors` exists and can
carry a name, an affiliation and an ORCID, but it is attribution on a model, the way an author
list is attribution on a paper. `team_model_designation` takes one of `primary`, `secondary`,
`proposed`, `other`, `ensemble` or `hub_other`, which is a team declaring which of its own models
it stands behind.

The scores inherit that shape exactly. Every row of the evaluation file is keyed by `team_id` and
`model_id`, and the identifiers you see in a ranking are things like `ISI-GLEAM`,
`QMUL-SEIR_agumented` and `respicast-quantileBaseline`. Thirty-three model metadata files were in
the repository on 21 September 2026 and 32 distinct team-and-model pairs appear anywhere in the
score history.

### Anyone can join, and at least one participant is one person

The hub's joining page invites "any group, individual, or institution", and the FAQ says
individuals worldwide may submit forecasts for any EU/EEA country. This is not a formality. The
metadata file for `DHauser-FluChronos` gives a team name of David Hauser, one contributor, and an
affiliation of "Independent researcher"; its method is a fine-tune of a time-series foundation
model on WHO FluID surveillance series. In the most recent scored round that model placed second
of fourteen on one of the cells described below.

And it is still called `DHauser-FluChronos`. One person, working alone, producing a top-ranked
forecast, and the hub records it as a model identifier, because that is the only kind of thing it
records. This is the whole difference from a forecasting platform in a single row of a CSV.

### The same shape in the United States

The US hubs are run by the CDC and differ in target, not in structure.

**FluSight** collects weekly nowcasts and forecasts of confirmed influenza hospital admissions
for the nation, all 50 states, Washington DC and Puerto Rico, plus an optional target for the
proportion of emergency department visits due to influenza. The 2026-2027 exercise is scheduled
to begin on 7 October 2026 and run until 26 May 2027, with submissions due by 11PM Eastern each
Wednesday. Predictions are compared against the NHSN Weekly Hospital Respiratory Dataset. Horizon
-1 is accepted but is explicitly not scored in summary evaluations, and teams are told to submit
it anyway because it helps detect calibration problems.

**The COVID-19 Forecast Hub** does the same for weekly confirmed COVID-19 hospital admissions,
with an optional emergency-department-visit proportion target added in June 2025, for the same
set of jurisdictions.

Both use the same quantile submission format, the same weekly cadence, the same public repository
mechanics, and the same absence of any personal identity. FluSight adds target types a European
reader will not have seen: probabilities over five rate-trend categories, and seasonal targets for
peak week and peak incidence.

## What a quantile submission is scored with, and why the metric is different

On a forecasting platform you state one number, the probability of an event, and you are graded
with a rule built for that: a Brier score, or a log score, or some rescaling of one of them. Those
rules take a probability and an outcome that either happened or did not.

Here the outcome is a count. ILI incidence in Belgium in the week ending 6 September 2026 was some
number of notified cases per 100,000 people, and no probability attaches to it directly. So the
hub grades the distribution you submitted against the number that arrived, with the **Weighted
Interval Score**, and its
[evaluation wiki](https://github.com/european-modelling-hubs/RespiCast-SyndromicIndicators/wiki/Forecast-Evaluation)
gives the formula.

In plain terms, WIS does two things at once and adds them up.

- **It charges you for width.** Take any central prediction interval you submitted, say the one
  running from the 10th to the 90th percentile. You pay its width, upper bound minus lower bound,
  whatever happens. A forecast that hedges by predicting everything pays for predicting
  everything.
- **It charges you much more for being outside.** If the reported value lands below your lower
  bound or above your upper bound, you additionally pay the distance by which you missed,
  multiplied by 2 divided by the interval's significance level. For a narrow, confident interval
  that multiplier is large. So narrowing an interval is cheap right up to the point where the
  truth falls outside it, and then it is very expensive.

Those two charges are computed for every interval implied by your 23 quantiles, averaged with
weights, and added to a term for how far the median missed. The result is in the units of the
thing being forecast, and lower is better.

The practical difference from a platform score is worth stating directly. A Brier score asks
whether you were confident in the right direction. WIS asks whether the shape of your uncertainty
was right: whether you were as sure as you claimed to be, no more and no less. You cannot do well
on it by being vague, and you cannot do well on it by being bold. What a platform's score is
rewarding instead, and why two of them rarely compare, is
[its own guide](https://predictionmarkets.tools/guides/what-a-forecasting-score-measures); the three formulas the platforms in
this catalogue actually use are pulled apart in
[which score you are chasing](https://predictionmarkets.tools/compare/forecasting-platform-scoring).

The hub computes exactly two metrics: WIS, and **Absolute Error**, defined as the absolute
difference between the predicted median and the reported value. AE looks at one number out of your
23 and ignores the rest, which is precisely why it is published next to WIS rather than instead of
it. In the snapshot read for this page, WIS and AE rows were present in near-equal numbers:
133,101 and 132,845 respectively, out of 265,946 rows.

## Baseline, and what a relative score means

A WIS of 4.04 means nothing on its own, because it is in cases per 100,000 and a hard week scores
worse than an easy week for everybody. So the hub does what a forecasting platform does when it
scores you against the crowd: it scores each model against a reference.

The reference is a model in the file like any other, `respicast-quantileBaseline`, a naive
baseline the hub runs itself. Every row carries a `value_absolute`, which is the raw WIS or AE,
and a `value_relative`, which the wiki defines as

> log2( WIS of the baseline / WIS of the model )

Positive means better than the baseline, negative means worse, and the baseline scores exactly
zero by construction. That last part is checkable rather than assumed: across all 28,377 baseline
rows in the snapshot read for this page, `value_relative` was 0.0 in every one.

The base-2 logarithm is doing real work. It makes the scale symmetric, so that being twice as good
as the baseline (+1) and twice as bad (-1) are the same distance from zero, which a plain ratio
does not give you. It also converts back into something you can say out loud. Here is the cell
this page has been quoting, from the snapshot dated 2026-09-09: target ILI incidence, location
Belgium, horizon 1, metric WIS, 14 models scored.

| Model | WIS | log2 vs baseline | Rank |
| --- | --- | --- | --- |
| ItaLuxColab-EpiNetEKF | 3.973 | +1.561 | 1 of 14 |
| DHauser-FluChronos | 4.042 | +1.537 | 2 of 14 |
| ISI-LightGBM | 4.736 | +1.308 | 3 of 14 |
| respicast-hubEnsemble | 5.939 | +0.982 | 5 of 14 |
| respicast-quantileBaseline | 11.726 | 0.000 | 9 of 14 |
| ECDC-SARIMA | 18.786 | -0.680 | 12 of 14 |
| QMUL-SEIR_agumented | 46.729 | -1.995 | 14 of 14 |

Read the second column against the third and the transformation is obvious: +1.537 is a WIS 2.90
times lower than the baseline's, and -0.680 is a WIS 1.60 times higher. Eight of the fourteen
models beat the naive baseline in that cell and five lost to it, which is the ordinary state of
affairs and the reason the baseline is published as a competitor rather than as a footnote.

Two more columns matter and are easy to skip. `n_models` is the size of the field the rank was
computed in, and it varies by cell: 14 here, but 7, 8, 10, 12 and 13 in other cells of the same
round, because teams submit for the countries and horizons they choose. `rank_score` is a
normalised position within that field. A rank without its `n_models` is not a fact.

## What it costs

Nothing, in money. There is no fee, no account, no deposit and no prize. Submission is a pull
request to a public repository. What participation and use actually cost is three other things,
and all three are underestimated.

**Compute, and the work behind it.** The methods declared in the metadata range from ARIMA and
SEIR models to gradient boosting, neural forecasting architectures and fine-tuned time-series
foundation models. The independent participant quoted above declares a fine-tune of Chronos-2
across global WHO FluID series with five-year context windows, recency weighting and peak
oversampling, rerun for each eligible target and location. That is not a number typed into a box.
A hub asks for a pipeline that runs unattended, every week, for a season.

**A weekly rhythm you have to hit.** The window opens Saturday 00:00 CET and closes Wednesday
23:59 CET, and the ensemble is built the moment it closes. Miss it and the round is gone. The hub
is gracious about this in its FAQ, where the answer to whether missing a week is acceptable is
simply "Yes!" and participation in every round is not mandatory, but the cost lands on your own
record: a model with few rounds has a thin history, which is the next point.

**A wait of up to three weeks before you learn anything.** This is the one that separates a hub
from a platform most sharply, and it can be measured directly from the published snapshots.

In the snapshot dated 2026-09-09, the forecasts made on origin date 2026-09-09 have scores at
horizon 1 only, for the week ending 2026-09-06. Forecasts made on 2026-09-02 have horizons 1 and
2. Forecasts made on 2026-08-26 have 1, 2 and 3. Only from 2026-08-19 backwards does the full set
of horizons 1 to 4 appear. Compare it with the previous week's snapshot, dated 2026-09-02, and the
boundary has moved by exactly one step: there, 2026-08-19 still had only horizons 1 to 3.

So the four-week-ahead forecast you submit today gets its score 21 days from now, one horizon at a
time, as the surveillance data catches up. There is no instant feedback anywhere in this design,
and there cannot be: the ground truth does not exist yet.

**And a cost to the reader, which is smaller but real.** The rolling `latest-forecast_scores.csv`
was 30.8 MB and 265,946 data rows when read for this page, covering origin dates from 2024-10-23
to 2026-09-09 across 32 locations. It is not a leaderboard. It is a long file that you group
yourself.

## Where this stops

Be clear about how narrow this is before treating it as a general lesson.

**The subject matter is respiratory disease incidence and nothing else.** ILI and ARI incidence in
EU/EEA countries; influenza and COVID-19 hospital admissions and emergency department visit
proportions in the United States. Nothing about elections, macroeconomics, company events or any
other subject a prediction market lists. A hub's demonstration that four-week-ahead forecasting is
hard is a demonstration about a seasonal epidemiological series, which has structure that most
questions do not.

**There is no personal participation, and this is not an oversight to be worked around.** You
cannot open an account, you cannot answer one question, and nothing you do there accrues to you by
name. The nearest thing in this catalogue to a scored, academically run exercise with a human on
the other end is the
[Social Science Prediction Platform](https://predictionmarkets.tools/tools/social-science-prediction-platform), where the unit is
a person forecasting what a study will find.

**A rank is not a track record.** Two of the models in the score history appear in only one origin
date, and `DHauser-FluChronos`, the second-placed model in the table above, appears in two. The
baseline appears in 99. Reading a single-round rank as evidence about a model is the same error as
reading a single resolved question as evidence about a forecaster, and the file gives you the
participation count to avoid it.

**Confirmed numbers are thinner than they look.** Almost every specific figure on this page comes
from one repository and its own wiki, read on one day. The evaluation folder's own readme file is
empty, so the definition of the metrics lives only in the wiki. And the rolling file is not fresher
than the snapshots: on 21 September 2026 `latest-forecast_scores.csv` had the identical SHA-256
digest to the snapshot dated 2026-09-09, twelve days earlier. If you need the state of play today,
today is not what the file contains.

**None of this tells you what will resolve.** A hub scores models against surveillance data that
is itself revised. The question of what a given contract resolves against, and who decides, is a
separate mechanism entirely and is covered in
[who decides how a prediction market resolves](https://predictionmarkets.tools/guides/who-decides-the-outcome).

## What you can do about it

**Download one snapshot and look at a single cell yourself.** Go to
[model-evaluation/snapshots](https://github.com/european-modelling-hubs/RespiCast-SyndromicIndicators/tree/main/model-evaluation/snapshots),
take the newest dated file, and filter it to one `origin_date`, one `target`, one `location`, one
`horizon` and `metric == WIS`. What comes back is a dozen or so rows, already ranked, with the
baseline sitting somewhere in the middle. Eighty-six dated snapshots were in that folder on 21
September 2026, so you can also take last month's file and see what a forecast looked like before
the answer was known.

**Read `n_models` before you read `rank`.** They are adjacent columns for a reason. Rank 3 of 14
and rank 3 of 7 are different claims, and in the round read for this page both field sizes occur.

**Use `value_relative` as a difficulty gauge, not as a leaderboard.** The useful question is not
which model won. It is how many models beat a naive baseline at all, and by how much. In the cell
above, eight of fourteen did, the best by a factor of 2.9 and the worst losing by a factor of 4.
That spread is the honest answer to "how well is this problem solved", and it is worth carrying
into any adjacent question you are about to take a position on. If the specialists with a season
of tuning are clustered near a naive baseline four weeks out, a confident price on a related
question four weeks out deserves more scepticism than its tightness suggests.

**Check the horizon before you compare anything to anything.** Horizon 1 in the European hub is
the week that has already ended without consolidated data, and horizon 4 is three weeks past that.
Scores at those two horizons are not comparable and were never meant to be. FluSight goes further
and excludes its horizon -1 from summary evaluation entirely while still asking teams to submit
it.

**If what you want is a personal score, use a platform and know which formula it uses.**
[Metaculus](https://predictionmarkets.tools/tools/metaculus) grades you with rescalings of the log score against the community,
and [Good Judgment Open](https://predictionmarkets.tools/tools/good-judgment-open) with a Brier score relative to one crowd.
Neither is comparable to the other, and neither is comparable to a WIS, for reasons set out in
[what a forecasting platform's score actually measures](https://predictionmarkets.tools/guides/what-a-forecasting-score-measures).
If you want your own resolved questions scored across several platforms at once,
[brier.fyi](https://predictionmarkets.tools/tools/brier-fyi) is the card for that, and the whole
[forecasting platforms](https://predictionmarkets.tools/categories/forecasting-platforms) listing carries the scoring mechanism
on every card.

**If what you have is a model, submit it.** The joining instructions are a fork, a metadata file
and a pull request, and the hub's stated position is that it does not want the GitHub mechanics to
deter anyone; the wiki gives an address to write to for help. A naive model is a legitimate
submission. The hub already runs one and publishes its score at the top of the same table as
everyone else's, which is the most useful convention in this entire corner of forecasting and the
one no platform in this catalogue has copied.

## Tools this bears on

- [Metaculus](https://predictionmarkets.tools/tools/metaculus.md) — Proper scoring and public track records on questions nobody can take a position in.
- [Good Judgment Open](https://predictionmarkets.tools/tools/good-judgment-open.md) — Brier scores against the crowd, and the recruiting ground for Superforecasters.
- [Social Science Prediction Platform](https://predictionmarkets.tools/tools/social-science-prediction-platform.md) — Forecast what a study will find, before it finds it, and be scored against the estimate.
- [Brier.fyi](https://predictionmarkets.tools/tools/brier-fyi.md) — Brier scores and letter grades for matched questions across four platforms.

## FAQ

### Can I post my own forecast to one of these hubs?

Not as a forecast. The unit of submission is a model, and the metadata file the hub requires names a team, a model, the methods and the data inputs. A person who wants a personal score is looking for a forecasting platform, not a hub. What you can do is submit a model, including a trivial one, and it will be scored the same way as everyone else's.

### Why do these hubs use the Weighted Interval Score instead of a Brier score?

Because the thing being forecast is a count, not a yes or no. A submission to the European Syndromic Indicators hub is 23 quantiles of an incidence value, from the 1st percentile to the 99th. Brier and log scores grade a probability attached to an event that either happens or does not. The Weighted Interval Score grades a whole predicted distribution against the number that was eventually reported.

### Is participation free?

Yes, and there is no prize either. Submission is a pull request to a public GitHub repository, there is no fee and no account to fund. What it actually costs is compute to produce the forecast, a weekly submission window you have to hit, and a wait of up to three weeks before the longest horizon you submitted gets a score.

### How current are the published scores?

The European hub writes a dated snapshot roughly weekly and a rolling latest file alongside it. On 21 September 2026 the newest snapshot was dated 2026-09-09 and was byte-identical to the rolling file, which means the rolling file is simply the newest snapshot rather than something fresher.

### Does a good rank in one of these files mean the model is good?

Only in the cell it was computed in. A rank is per target, per country, per horizon, per metric and per round, and the number of models in that cell varies between about 7 and 14. The same file carries an n_models column precisely so a rank can be read against its field size.

## Sources

1. [European Syndromic Indicators Forecasting Hub, repository README](https://github.com/european-modelling-hubs/RespiCast-SyndromicIndicators) — European Centre for Disease Prevention and Control, ISI Foundation and London School of Hygiene and Tropical Medicine, read 2026-09-21
2. [Forecast Evaluation, hub wiki](https://github.com/european-modelling-hubs/RespiCast-SyndromicIndicators/wiki/Forecast-Evaluation) — European Respiratory Diseases Forecasting Hubs (RespiCast), read 2026-09-21
3. [Frequently Asked Questions, hub wiki](https://github.com/european-modelling-hubs/RespiCast-SyndromicIndicators/wiki/Frequently-Asked-Questions) — European Respiratory Diseases Forecasting Hubs (RespiCast), read 2026-09-21
4. [How to Join, hub wiki](https://github.com/european-modelling-hubs/RespiCast-SyndromicIndicators/wiki/How-to-Join) — European Respiratory Diseases Forecasting Hubs (RespiCast), read 2026-09-21
5. [hub-config/tasks.json, hubverse tasks schema v5.0.0](https://github.com/european-modelling-hubs/RespiCast-SyndromicIndicators/blob/main/hub-config/tasks.json) — European Syndromic Indicators Forecasting Hub, read 2026-09-21
6. [hub-config/model-metadata-schema.json](https://github.com/european-modelling-hubs/RespiCast-SyndromicIndicators/blob/main/hub-config/model-metadata-schema.json) — European Syndromic Indicators Forecasting Hub, read 2026-09-21
7. [Evaluation snapshot, 2026-09-09-forecast_scores.csv](https://github.com/european-modelling-hubs/RespiCast-SyndromicIndicators/blob/main/model-evaluation/snapshots/2026-09-09-forecast_scores.csv) — European Syndromic Indicators Forecasting Hub, 2026-09-09
8. [RespiCast forecast platform](https://respicast.ecdc.europa.eu/forecasts/) — European Centre for Disease Prevention and Control, read 2026-09-21
9. [FluSight 2026-2027, repository README](https://github.com/cdcepi/FluSight-forecast-hub) — US Centers for Disease Control and Prevention, Influenza Division, read 2026-09-21
10. [COVID-19 Forecast Hub, repository README](https://github.com/CDCgov/covid19-forecast-hub) — US Centers for Disease Control and Prevention, read 2026-09-21
11. [Evaluating epidemic forecasts in an interval format](https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1008618) — PLOS Computational Biology, 2021-02-12. The paper the hub's own evaluation wiki cites for the definition of the Weighted Interval Score it computes; the definition has not been restated.

*Last updated 2026-09-21. A reference page, corrected in place — not a dated post.*