202405021003
Status: #idea
Tags: Machine Learning
Validation Set
This is one of the most important components of Machine Learning if not the most important thing. As you need to be able to say how good your model is at fitting the data you were given.
It is the thing that will allow you to identify where roughly you are in the Bias-Variance Tradeoff spectrum, are you underfitting and introducing too much bias into the model (Bias can mean different things in different contexts, but in general it is related to the error caused by your model's rigidity) this will typically be due to not running enough epochs to train your model. You may be overfitting on the other hand which will generally be the result of running too many epochs, or modifying the hyperparameters too much.
Indeed a validation set is a part of the data that the model does not see during training, the issue is that while the model does not see it during training, you do see your validation set (or at least you can); and you will almost certainly adjust parameters to allow the model to perform better.
As a result, if you are not careful it is easy to overfit to the validation set simply because you kept tweaking the hyperparameters like the learning rate or whatever.