SKORCH: PyTorch Models Trained with a Scikit-Learn Wrapper
A guide to understand how easy and simple it is to train PyTorch models with SKORCH
As we can see, some values are fixed. For practical terms, I would like to highlight only lines 9 and 12. In line 9 we define 13 input features, this because the wine dataset contains 13 features. On the other hand, in line 12 we define an output of size 3, this because the classes that we are going to classify are 3 (that is, 3 types of wines).
Perfect, the PyTorch model is ready, it's time to see how we train this model with SKORCH, let's go to the next section!
We will analyze line by line. In line 2 we are importing the PyTorch model (which was defined in the previous section). In line 4 we are importing the class that will serve as a wrapper for our PyTorch model. This class receives a series of important parameters (line 7) which are: the PyTorch model, the number of epochs, learning rate, batch size and optimizer. Obviously, they are not the only parameters we can define in this class, however for practicality, we will only show those already mentioned in this example. Finally, on line 9 we execute the "fit" method, which will be in charge of performing the entire training phase.
You may be wondering, "what about the split into train and validation?", well the NeuralNetClassifier class takes care of this as well. By default, this class implements StratifiedKFold split in the data with a ratio of 80% for training and 20% for validation. Well, once the above mentioned, this would be the output:
In lines 5 and 6 we import the Pipeline and StandardScaler modules from scikit-learn. In line 12 we can see that we initialize the wrapper exactly the same as in the previous point (with fixed values), the interesting thing comes in lines 14 and 15 where the Pipeline is initialized, which contains the StandarScaler() module as well as the wrap of the PyTorch model. Running this we get:
So, as we can see in line 9 we import the EpochScoring callback. To make use of the callback, we only have to initialize it by passing the name of the metric we want to use as arguments, in this case, we initialize "_balancedaccuracy" and "accuracy" for the metrics. Also, we have to set the parameter "_lower_isbetter" as "False" because our problem seeks the maximization of the metrics, not the minimization.
So the result of executing the previous snippet would something like:
As we can see, the parameters have a particular aspect. We are adding the prefix "_nn_" and "_nnmodule_". These prefixes will help the wrapper to know if the parameter belongs to the definition of the PyTorch model or to the training phase. As we can see, we only use the prefix "_nn___" when we refer to parameters of the training phase and "_nnmodule___" when we refer to parameters of the PyTorch model. It is important that the name "nn" refers to the instantiation of the wrapper (line 19).
So if we want to know what the best parameters were, we can do it easily:
If you want to access the full implementation, take a look at: https://github.com/FernandoLpz/SKORCH-PyTorch-Wrapper
Conclusion
In this blog we have seen what SKORCH is and what its components are. We have also seen how to implement the NeuralNetClassifier wrapper to train a PyTorch model in a very simple way.
In my opinion, SKORCH is here to stay. Sometimes it is required to quickly and flexibly prototype PyTorch models, SKORCH does this wonderfully.
