Model evaluation component

The Evaluation component takes true target values and model predictions and computes performance metrics and visualizations. Its Actual port accepts as many wires as you like, so one node grades the whole grid: every model Inference scored, against every partition whose true labels reach it.

The grid

Wire Data Splitting’s Train y, Val y and Test y all into the Actual port. Each one is matched to the partition Inference used by reading the handle it came from, so nothing has to be configured. The report then reads:

Models down one axis — which algorithm is actually better.
Data sets across the other — whether that is true on data the model has never seen.

The gap

The scoreboard adds a Gap column: the training score minus the most held-out score. It is the single number that says whether a result is real.

Gap	Reading
≤ 0.03	Generalises well — the score is a result rather than a memory
0.03 – 0.10	Slight overfit — normal for most models, worth watching if you tune further
> 0.10	Overfitting — the model learned this particular data, not the pattern behind it
Negative	The held-out set beats training — usually a small or lucky partition; check the row counts

For error metrics (MAE, MSE, RMSE, MAPE) the sign is flipped before it is read, since a lower training error is the overfit direction.

Configuration

Option	Description	Default
Problem Type	`classification` or `regression`. Must match the model used.	`classification`
Metrics	Which metrics to compute. Leave empty to compute all defaults.	All

Which sets to grade is chosen on the node card rather than here, for the same reason Inference’s rosters are: it is a fact about the wiring, and the options only exist once the wires do.

Classification metrics

Metric	Description
Accuracy	Fraction of predictions that are correct
Precision	Of all positive predictions, how many are actually positive
Recall	Of all actual positives, how many were correctly predicted
F1	Harmonic mean of precision and recall
ROC AUC	Ranking quality. Needs the probability column Inference emits for binary classifiers
Confusion Matrix	Table showing true vs. predicted class counts

Regression metrics

Metric	Description
MAE	Mean Absolute Error — average absolute difference
MSE	Mean Squared Error — penalizes large errors more
RMSE	Root Mean Squared Error — in the same units as the target
R²	Proportion of variance explained by the model. 1.0 = perfect.
MAPE	Mean Absolute Percentage Error. Rows with a zero target are excluded

The report

Everything in the report is additive. A control bar across the top carries one chip per data set and one per model; ticking Validation adds a column, a series and a matrix — it never takes Test away. Every panel below renders the cross product of what is ticked, so “how does the forest do on validation versus test, next to the regression” is answered by looking rather than by clicking back and forth. Colour carries one meaning throughout, the same rule the canvas runs on: hue is the data set. Models are told apart by position, label and line dash.

Page	What it holds
Overview	The generalisation verdict for every model, headline tiles, and the scoreboard — models down, sets across, a bar behind every number and a gap column
Compare	The scoreboard as grouped bars; a slope chart tracing each model across the partitions, where a fall is a model that memorised; and every metric, every model, every set as one table
Diagnostics	Classification: all ROC curves on one axis with their AUCs, a grid of confusion matrices (models × sets, with a row-% toggle), class balance actual-against-predicted, and per-class precision/recall/F1. Regression: actual-vs-predicted for every partition on shared axes, and the error distributions overlaid
Rows	One row per record: what it really was, what each model said, and whether the models agreed — with an errors-only filter

The scoreboard’s row and column headers are themselves the selection controls, so narrowing the comparison happens where the comparison is. The last remaining chip in each row cannot be un-ticked; an empty report is never what un-ticking the last one means.

Headline numbers

The node card, the Overview page and the model version history each have room for one number. That number is the best-scoring model on the most held-out partition available — test where there is one, then validation, then the full frame, and only training as a last resort. A card reporting training accuracy would be a lie by omission.

Input / Output

	Type
Input	One or more target frames (Actual) + predictions from Inference (Predicted)
Output	Metrics, per-set and per-model, plus charts

Notes

A partition wired into Inference but not into Evaluation is called out in the report rather than silently dropped.
Actual and Predicted are matched by row index, so a partition arriving in a different order than it was scored in still lines up.

Get Started

Components

Data

Data Processing

Feature Engineering

Data Wrangling

Analytics

Models — Classification

Models — Regression

Evaluation & Inference

Model evaluation component

The grid

The gap

Configuration

Classification metrics

Regression metrics

The report

Headline numbers

Input / Output

Notes

​The grid

​The gap

​Configuration

​Classification metrics

​Regression metrics

​The report

​Headline numbers

​Input / Output

​Notes

The grid

The gap

Configuration

Classification metrics

Regression metrics

The report

Headline numbers

Input / Output

Notes