Htydjtk

In the "deep learning crash course" given by Leo Isikdogan in lecture 4 https://www.youtube.com/watch?v=ms-Ooh9mjiE&list=PLWKotBjTDoLj3rXBL-nEIPRN9V3a9Cx07&index=4
Overfitting, Underfitting, and Model Capacity
, He suggests that the data should be split in train, validation and test sets. The Train set is used to train the model, the validation set to optimize the hyperparameters and the test set to give an unbiased estimation of the generalization error.
When I look at how people implement the design, they usually use gridseachCV to evaluate the deep learning neural network to configure some of the hyperparameters such as the number of neurons, learning rate, optimizer, etc. After that, they used the validation set to perform the early stopping.

I think that it is possible to obtain a hyperparameter configuration from gridsearchCV that gave the best performance because it is already overfitted. Is it correct?

Is there another option to tune the hyperparameters with a validation set. Why they do not try different configurations in the validation set?

The validation set's purpose is to track your level of overfitting while you are experimenting with your network. When we try something we want to know how well it works on data it hasn't seen before.

By looking on the score on the training set we can't tell what is good from what is overfitting. If you instead use your test set to evaluate the results of all your experiments, then your test set stops being a reliable final evaluation since you have included it in your decisions. The test set is only for final evaluations.

To fill this need we set aside a part of our data as a validation set. A dataset that we do not directly train on, but instead the use to check results of training different layer/unit configurations, tuning hyperparameters, feature engineering and all the other things we possibly want to try. Early stopping is just one of the possible uses.

Regarding your comments about hyperparameters:

You don't use the validation set directly when you are tuning hyperparameters (with grid search or any other method). You still use the training set, but then you use the validation set to test what gave the best result.