How is validation of neural nets implemented? I know what are training, validation and testing stages. But, I want to know how validation is implemented. Let's consider the following data : data_train, data_val, data_test as respectively training, validation and test data. Let's suppose that the model is created with keras:
model = Sequential(...)

The validation stage occurs during the training stage:
for i in range(epochs):
    loss_train = my_function_computing_loss(model, data_train)
    # update weights and biases
    ...
    # Then try to validate
    my_validation_function(model, data_val)

In keras, when we let the model manage validation, we can see that
loss_train and loss_val are almost close (for good tuning).

What is the aspect of the code that makes loss_train and loss_val closer?

The validation data should not be influencing the training process. The validation data is used to a "sanity check" of the model, and for us to check for overfitting or undefitting.
So for example, if you have a overfitted model, the training accuracy will be really high, because the model "memorized" the data, and the validation accuracy will be significantly lower or it will stay constant regardless of the training steps.
If the training processes is going well, test and validation loss should be close and with a similar behavior.
