Forecast Confidence Score tells you how much you can trust each forecast method — Statistical or ML — for a given choice. The score comes from a weekly test that compares each forecast against what actually sold. Use it to pick the method that fits your choice best, and to spot choices where the forecast is shaky before you plan around it.

Every week, Toolio re-runs both the Statistical and the ML forecast for every eligible choice — but instead of forecasting the future, it forecasts the recent past and checks how close each method came to actual sales. That comparison produces the scores you see in the popup.

The score compares the shape of the sales curve — peaks, dips, seasonality, trend — not the absolute unit numbers. For example, a forecast that nailed the holiday spike and the January dip scores well, even if the total unit numbers are a bit higher or lower than what actually sold.

This is the first version of the score. It will be refined in upcoming releases, so treat it as a strong directional input rather than an exact reading.

When you open the Forecast Confidence Metrics popup you see three metrics per side (Statistical and ML), plus an Overall Confidence Score at the top.

The headline number for the whole choice. It rolls up the per-cluster scores, weighted by recent sales, using the method that will actually run for each cluster.

The label tells you at a glance:

A HIGH score means the forecast tracked actual sales well in recent weeks. A LOW score is a heads-up to look at the choice more closely before relying on the forecast.

How much to trust that side's forecast for that cluster. It is a summary of the two metrics below — Model Accuracy and Data Quality — so a strong Confidence Score means both the model fit the sales pattern well and the underlying sales data was clean enough to learn from. Weakness on either side pulls the score down.

How well the forecast picked up changes in demand — the rises, drops, and trend shifts over time. This evaluates the model itself, assuming the underlying data is fine.

How clean and stable the actual sales data was during the test. When weekly sales swing wildly or jump around unpredictably, no model can fit them perfectly. A low Data Quality is a signal that the data, not the model, is the limit — even the best forecast cannot do much better with that history.

The shaded columns inside the popup mark the side that will actually run for each cluster — Statistical or ML. Those are the numbers that roll up into the Overall Confidence Score.

Each cluster of a choice has a Forecast Type setting in the popup. It tells Toolio which method to use next time you run Forecast for that choice. Changing it does not trigger a forecast on its own — it just records your preference for the next run.

When a new choice offering is created, Forecast Type is set to Organization Default. If your organization has not chosen a default, that means Statistical Forecast.

You have four options:

Statistical Forecast and ML Forecast are direct picks — Toolio honors them. Best Confidence and Organization Default let Toolio decide based on the scores or your organization's setting.

The popup re-shades the columns as soon as you save your change, so you can see right away which side will run for each cluster.

The Overall Confidence Score, the shaded columns, and the Forecast Confidence Score value on the main grid all update right away to reflect your new selection.

ML needs enough recent sales history at the channel to produce a reliable forecast. When a cluster does not have that history, the ML side of the popup is blank for that cluster — there is no score to show.

You can still pick ML Forecast for that cluster, but Toolio will quietly run Statistical instead so you still get a forecast. The blank cells in the popup tell you which clusters this applies to.

Gross Sales Lift Units shows the extra units a promotion is expected to drive — the difference between the promoted forecast and the underlying demand without the promo.

This metric is only meaningful for choices running the Statistical Forecast. The Statistical method produces a clean pre-promo baseline and then layers the promo lift on top, so the two are stored separately and the lift can be measured.

The ML Forecast works differently: it produces a single forecast that already has any promo effect built in. Because ML never separates "demand with promo" from "demand without promo," there is no lift to break out — so Gross Sales Lift Units shows 0 for ML choices. This is not the lift being hidden; for ML there simply is no separate lift figure to report.

If you plan around Gross Sales Lift Units, rely on it only for choices set to Statistical Forecast. For ML choices, treat the figure as not applicable rather than as "zero lift."

Forecast With AI in New Assortment Plan

Forecast in Assortment Plan

That cluster does not have enough recent sales history at the channel for ML. Toolio leaves the ML cells blank until the cluster has enough history. Selecting ML Forecast on that cluster will use Statistical until ML becomes available.

Those choices are running the ML Forecast. ML produces one combined forecast with the promo effect already included, so there's no separate baseline to measure the lift against. Gross Sales Lift Units is only available for choices using the Statistical Forecast.

HIGH is 70% or higher, MEDIUM is 40% to 69%, LOW is below 40%. They are quick visual cues — a green HIGH means the forecast tracked sales well recently; a red LOW is a flag to take a closer look.

The scores refresh once a week, when Toolio runs the weekly test against recent sales. The popup always shows the most recent week's result.

Best Confidence decides per cluster — for each cluster, it picks whichever side scored higher in the last weekly test. Organization Default follows your organization's preference, which can itself be Statistical, ML, or Best Confidence. Pick Best Confidence when you want each cluster optimized independently; pick Organization Default when you want a consistent rule across choices you have not explicitly set.

The new method runs the next time you trigger Forecast for that choice — changing Forecast Type records your preference, it does not start a forecast on its own.

A few things do update right away once you save: the Overall Confidence Score banner, the shaded columns in the popup, and the Forecast Confidence Score on the main grid all recalculate from the existing per-side scores to reflect your new selection.

The per-side metric values themselves — the Confidence Score, Model Accuracy, and Data Quality numbers under the Statistical Forecast and ML Forecast columns — do not change. They come from the last weekly test and refresh on Toolio's weekly cycle, regardless of which method you pick.

They should match. Both show the same number: an average of each cluster's score on the side that will run, weighted by recent sales. If they look different, refresh the page; the grid value updates automatically when you save in the popup.

Yes. Set Forecast Type to Best Confidence for the choice and Toolio picks the higher-scoring side for each cluster automatically. If you want this across all choices by default, ask your administrator to set Best Confidence as the organization default.