Ever wondered how close you duplicated a previous roast profile? A standard measure in statistics to express the average error between a curve and its target is know as Mean Squared Error (MSE) which computes the average of the squares of the single discrepancies between the data values and their targets.
Scott Rao remarked in one of his recent post on Instagram that "Picking one or two arbitrary landmark moments in a roast and calling two roasts replicated if those phase times match is a low bar for replication." With proper roast profiling systems in place we can express the quality of replication rather precisely as a numeric value based on the data of a roasts profile and its template.
Since v0.7.5 released mid 2014, Artisan implements such a measure that gives a quite good idea of the correlation between a profile and its template. The implementation of this measure was triggered by a small innovative roasting operation located in Budapest, Hungary, called Casino Mocca who focused on product consistency from the very beginning. This measure was therefore named CM, but it never got properly documented.
Note that the application of this measure makes most sense if one replicates a roast on the same machine, using the same coffee and the same batch size. A standard situation in production.
I also assume that the two profiles to be compared are about the same length and the temperatures at CHARGE are close. Those conditions can easily be quantified.
CM ET/BT
The CM values are computed by Artisan on the environment temperature (ET) and bean temperature (BT) curves as soon as a profile template is loaded in the background. The measures are displayed within the profile characteristics line, enabled via a corresponding flag in the Statistics dialog (menu Config >> Statistics), below the profile graph. Note that a right-click on the characteristics line might be need to toggle it.
Calculation
The CM measure is not calculated over all the recorded data, but just the most significant part of it. The begin of the second roasting phase, usually marked as yellow point (DRY), to the end of the roast (DROP) has been identified to be most significant. The readings reported by the bean temperature probe before DRY often differ too much from the real bean temperatures to make up a good base for this measure. In Artisan, the begin of the second roasting phase can be defined manually by modifying the minimum temperature of that second phase (menu Config >> Phases). Alternatively, the start temperature of that second roasting phase is taken from the DRY event of the current profile if the flag "Auto Adjusted" in the same dialog is ticked.
Before the measurement is computed, the two curves are internally aligned by the DROP event automatically. This has been done manually to generate the above roast graph by using the corresponding align setup in the Background dialog (menu Roast >> Background) for demonstration only.
The CM algorithm computes the residuals between the interpolated background ET and BT temperatures and the corresponding foreground temperatures per reading. It starts at DROP and iterates backwards until the being of the second roast phase, summing up all the error squares. The resulting sums of the errors are divided by the number of readings considered (cf. the formula shown in the cover picture of this post) and finally the square root is applied to get the calculated mean error expressed in our temperature unit. The CM algorithm implements essentially the standard mean squared error (MSE) as used in regression analysis.
Note that potentially the background reading at the sample time has to be interpolated between the two closest values in time, as Artisan supports profiles recorded at different sampling intervals.
On Instagram the idea popped up to calculate such a distance measure rather for the corresponding rate-of-rise (RoR) curves, most likely as the RoR is rather trendy nowadays. The following variant of the above roast graph, with the background profile shifted by 15C up, shows why this is clearly a bad idea. Of course the derived RoR curves do not change by such a shift of the temperature curves. However, the foreground profile would be for sure a bad replication of the shifted background one, as also found by the CM measure (now 14.7C / 15.4C).
To be copied...
