FAQ - Frequently Asked Questions
Issue:
-
Why do I get different cross-validation results from the command-line crossval routine compared to the Analysis window?
Possible Solutions:
-
There are three possible causes of this:
Preprocess Option Unassigned
The most common cause is that you have not set the same preprocessing in the crossval function (see the "preprocessing" option) as you are using in the Analysis window. You should use the preprocess function to define the same preprocessing you are using in Analysis and pass this into the crossval function via the preprocessing option.
Preprocessing Outside Cross-Validation
The next possible cause is that you are doing the preprocessing outside of cross-validation. Preprocessing should always be done inside of cross-validation to help avoid over-fitting models.
Autoscale Preprocess Shortcut Used (Version 3.5.2 and earlier)
The third possible cause is only present when using Version 3.5.2 or earlier of PLS_Toolbox and is related to using a preprocessing shortcut in the crossval preprocessing option. When using autoscaling in cross-validation, you may observe different results if you analyze data in the Analysis window compared to the same data analyzed at the command line using the crossval.m function. This discrepency (present through version 3.5.2 of PLS_Toolbox) is due to a bug in the preprocessing options of crossval.m experienced only if you use the shortcut "options.preprocessing=2" option.
If you use:
options.preprocessing = 2;the help claims you will get autoscaling. This, however, is not true. Any value other than zero (0) will give you mean centering. To do autoscaling in crossvalidation at the command line, use:
options.preprocessing = {preprocess('autoscale')};This will give you autoscaled results which will match the Analysis GUI's cross-validation results.
This bug will be fixed in Version 4.0 of PLS_Toolbox.
Still having problems? Check our documentation Wiki or try writing our helpdesk at helpdesk@eigenvector.com
