Forecast Confidence Score tells you how much you can trust each forecast engine — Statistical Forecast or AI Forecast — for a given choice. It is a test on past sales: did the forecast put the units at the right time?

The score is calculated for each individual choice, on its own sales. When a choice's own sales are too thin for a reliable score, Toolio falls back to the score of similar choices, and then to the partition level: the sales curve built for the choice's partition (for the AI Forecast, the product group of the channel). The popup marks which one you are looking at. Use the score to pick the engine that fits each choice best, and to spot choices where the forecast is shaky before you plan around it.

Note: AI Forecast (shown as ML Forecast in older versions of Toolio) is the same AI model that powers Forecast with AI. To use AI Forecast or see AI scores in the popup, Forecast with AI must be enabled for your tenant — see Forecast With AI in New Assortment Plan.

Toolio scores each forecast engine on its own. Both scores answer the same question — did the forecast put the units at the right time? — but each engine is checked against different data, and that one fact explains most of what you see in the popup.

Toolio does not compare single weeks. It adds the weeks checked together in blocks of 4, counting back from the most recent week, and compares the forecast with real sales block by block. Weeks left over at the start that do not fill a block are left out.

Weekly sales jump around by chance, especially for slow sellers (0, 2, 0, 1…), and no forecast can predict that. Blocks of 4 weeks smooth much of it out, so the score shows whether the forecast got the timing right rather than how noisy single weeks are. Both engines use the same blocks, so their scores can be compared.

Your organization can be set to compare single weeks instead — ask your Toolio team. The same forecast then scores lower.

Before comparing, Toolio turns both sides into shares of their total — this block had 10% of the units, the next 25%, and so on. A forecast with the right pattern over time but twice the units still scores well. A forecast with the right total but the units in the wrong weeks scores badly.

The score is a test on past sales, not a promise about future weeks.

Because the two engines are checked on different data, their Confidence Score, Model Accuracy and Data Quality are expected to differ, even for the same choice and cluster:

Compare the two sides to choose an engine, but do not expect their Model Accuracy or Data Quality to match. For a channel with a single cluster, such as Ecommerce, both engines see the same sales, so their Data Quality is usually close.

Imagine a choice sold 100 units over 16 weeks checked — four 4-week blocks — and the forecast spreads its own 100 units across the same weeks. Block by block, Toolio counts how many of the forecast's units match units that really sold in that block.

75 of the 100 units are in the right block. The score is 60%, not 75%, because each misplaced unit counts twice: 25 units the forecast put in the wrong block, and 25 real units it missed. So the score is 75 ÷ (75 + 25 + 25) = 60%.

4-week blocks smooth out much of the chance in weekly sales, but not all of it. With few sales, even the blocks jump around (2, 0, 3, 1…), and no forecast can predict that. So even a perfect forecast needs a minimum level of sales to reach a given score:

A choice that sells about 1 unit every 4 weeks can reach a little under 50% even with a perfect forecast, but not 60%. So a MEDIUM on a slow seller often means "too few sales to do better", not "bad forecast". The popup's Data Quality shows this ceiling for each choice.

The weekly sales are those of the choice in one cluster for the Statistical Forecast, and in the whole channel for the AI Forecast, when about a year of weeks is checked. If your organization compares single weeks instead of 4-week blocks, the same score needs about 4 times the weekly sales: about 2.6 units a week for 60% and about 5 for 70%.

Toolio first scores each choice on its own sales. Only when that score would not be reliable does it fall back, first to similar choices and then to the partition level (for the AI Forecast, the product group of the channel). Every score is the best one available, tried in this order — the closer it is to the choice, the more it says about this choice.

These rules count weeks with sales, not units: a week counts whether the choice sold 1 unit or 50. The number of units decides the Data Quality, not whether the choice gets its own score. So a choice can have its own score and still a low Data Quality.

Toolio does not average the similar choices. It lines up the similar choices that have their own score, sorted by Confidence Score, and takes the one in the middle. The Confidence Score, Model Accuracy and Data Quality are all copied from that one choice, so the three numbers on a row belong together.

A score from similar choices is usually close to what the choice itself would get. The choice switches to its own score once it has enough weeks with sales.

On the Statistical side, a wider-group score is the partition level. Toolio builds a sales curve for each partition (see Curve Partitioning), and the score is that partition's curve checked against its last-year version. On the AI side, it is a sample of 5 to 50 tested choices in the product group of the channel, scored together.

A wider-group score can be far from what the choice itself would get. Read it as "how good is this partition's sales curve" or "how good is the AI on this product group", not "how good is the forecast for this choice" — even when it reads HIGH.

When a score is borrowed — from similar choices or a wider group — the Model Accuracy and Data Quality next to it are borrowed too. They describe the similar choice or the partition, not this choice.

The Forecast Model Confidence Score column in New Assortment Plan shows a badge for the score of the forecast the choice is using (its Forecast Type). Hover over the badge to see the exact score, for example 76%.

The badge does not show whether a score is borrowed. Open the popup to see where it comes from.

The bands are the lines that turn a score into LOW, MEDIUM or HIGH. Your organization can use different bands from the defaults — ask your Toolio team to change them. Pick them by how close a match you want HIGH to mean, and whether your typical choice sells enough for a perfect forecast to reach that line:

Changing the bands only changes the label in the grid and the popup. The score, the forecast and Best Confidence's choice of engine stay the same.

The popup has one row per cluster and two sides, Statistical Forecast and AI Forecast. The side each cluster is using is shaded and in bold. The Total row shows, for each column, the average over the clusters that have a value, weighted by the units they sold. At the bottom, a legend lists the icons shown in the popup, and Learn More opens this article.

Hover over the ? next to each column header to see what that column means for that engine: what is compared, which weeks count, and when Model Accuracy is hidden.

The headline score for the whole choice, for example MEDIUM (47%). It averages each cluster's score for the engine that cluster uses, weighted by the units the cluster sold, so a cluster that sells more counts more. Clusters with no score for their engine are left out.

How closely the forecast followed real sales, block by block. This is the number behind the badge.

The best score a perfect forecast could have reached, given how many units were sold in the weeks checked. A low Data Quality means there were too few sales to forecast their timing — the data, not the model, is the limit.

How much of that best possible score the model reached: Confidence Score ÷ Data Quality. 90% or more means the model did about as well as the data allows. It is shown only when Data Quality is 50% or more; below that, the cell reads Too few sales to judge the model.

A score has at most two icons. Hover over a score to read what its icons mean. Grey icons tell you where the number comes from; amber icons ask you to take care.

From similar choices on the Statistical side, with its tooltip:

From a wider group on the Statistical side, where the score is the partition level:

AI forecast less reliable next to From similar choices on the AI side, for a choice with a short sales history:

In this example, the AI side reads No AI forecast for this choice for Ecommerce, and Model Accuracy reads Too few sales to judge the model where Data Quality is under 50%:

Each cluster of a choice has a Forecast Type setting in the popup. It tells Toolio which engine to use for that cluster.

When a new choice offering is created, Forecast Type is set to Organization Default. If your organization has not chosen a default, that means Statistical Forecast.

You have four options:

Statistical Forecast and AI Forecast are direct picks — Toolio honors them. Best Confidence and Organization Default let Toolio decide based on the scores or your organization's setting.

Saving starts a new forecast for every cluster of the choice. Until it finishes, the popup keeps showing the engine the previous forecast used, so the scores match the numbers in your plan.

The AI side of the popup reads No AI forecast for this choice when the AI cannot forecast the choice, for example because it has too little sales history.

You can still pick AI Forecast for that cluster, but Toolio runs the Statistical Forecast instead so you still get a forecast.

Gross Sales Lift Units shows the extra units a promotion is expected to drive — the difference between the promoted forecast and the underlying demand without the promo.

This metric is only meaningful for choices running the Statistical Forecast. The Statistical Forecast produces a clean pre-promo baseline and then layers the promo lift on top, so the two are stored separately and the lift can be measured.

The AI Forecast works differently: it produces a single forecast that already has any promo effect built in. Because the AI Forecast never separates "demand with promo" from "demand without promo," there is no lift to break out — so Gross Sales Lift Units shows 0 for AI Forecast choices. This is not the lift being hidden; for the AI Forecast there simply is no separate lift figure to report.

If you plan around Gross Sales Lift Units, rely on it only for choices set to Statistical Forecast. For AI Forecast choices, treat the figure as not applicable rather than as "zero lift." For more on how the Statistical Forecast is built, see Forecast in Assortment Plan.

The earlier score described the choice's product group. The per-choice score checks each choice against its own sales, which is stricter, so it is often lower — but it tells good forecasts from weak ones much better.

The default bands are now 30% / 60%; they used to be 40% / 70%. Scores are now calculated for each choice on its own sales, which is stricter, so they often run lower than the old group scores. 30% / 60% keeps HIGH within reach of a typical choice's forecast, while LOW still flags forecasts that missed (see Choosing the Bands above).

A score of 65%, for example, used to read MEDIUM and now reads HIGH. Only the label changed: the score, the forecast and Best Confidence's choice of engine stay the same. If your organization has its own bands, its badges did not change.

Each engine is checked on different data: one cluster and the sales curve's weeks for Statistical, the whole channel and the test weeks for AI. Different numbers are expected — see Why Statistical and AI Scores Differ above.

The AI side adds up the whole channel, so it usually has more weeks with sales than the Statistical side has in a single cluster. That is why small or quiet clusters often show From similar choices on the Statistical side only. The opposite also happens: a new choice can show From similar choices and AI forecast less reliable on the AI side only.

Yes, for a slow seller. With the default bands, a perfect forecast needs about 0.6 units a week to reach HIGH (60%). Below about 0.15 units a week (about 8 units a year), it can even read LOW. See Slow Sellers Have a Ceiling above.

Not quite, but it is usually close. A score from a wider group is much further from the choice; treat it with care.

Changing a choice (adding an offering, assigning another sales curve, linking, delinking or duplicating the choice, or changing its locations) refreshes its scores within minutes. A choice moves from From similar choices to its own score at the first update after it has enough weeks with sales.

Each score follows its forecast engine, so it updates once that engine has rebuilt on the new sales:

Until then, the scores rate the forecasts built on the earlier sales.

No. A forecast only changes when it runs again. Choices on Best Confidence can switch engine, but only when you re-forecast — the Scores changed since these forecasts ran banner tells you when that would happen.

No. The Confidence Score is a test on past sales before you use a forecast: how well would this engine have forecast this choice? Forecast Accuracy scores the forecast you actually used against what really sold afterwards. Use the Confidence Score to choose and trust a forecast, and Forecast Accuracy to check how it turned out.

Best Confidence decides per cluster — for each cluster, it picks whichever engine has the higher score. Organization Default follows your organization's preference, which can itself be Statistical Forecast, AI Forecast, or Best Confidence. Pick Best Confidence when you want each cluster optimized independently; pick Organization Default when you want a consistent rule across choices you have not explicitly set.

Yes. Set Forecast Type to Best Confidence for the choice and Toolio picks the higher-scoring engine for each cluster. If you want this across all choices by default, ask your administrator to set Best Confidence as the organization default.