Hi.
Using evaluate() changes the loss.
I write sample code.
(network)

model = Sequential(name='sample_02')

model.add(Input(shape=(3,)))
model.add(BatchNormalization())
model.add(Dense(10))
model.add(Activation('relu'))
model.add(Dense(4))

model.compile(optimizer='adam', loss='mean_squared_error')

    |
    v

_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
batch_normalization (BatchNo (None, 3)                 12        
_________________________________________________________________
dense (Dense)                (None, 10)                40        
_________________________________________________________________
activation (Activation)      (None, 10)                0         
_________________________________________________________________
dense_1 (Dense)              (None, 4)                 44        
=================================================================
Total params: 96
Trainable params: 90
Non-trainable params: 6
_________________________________________________________________

Base training code is:
(source code)

# starts training
for i in range(max_loop):
    for j in range(max_in_loop):
        result_1 = model.train_on_batch(x_data_keras, y_data_keras)
#        result_2 = model.test_on_batch(x_data_keras, y_data_keras)
#        result_3 = model.evaluate(x_data_keras, y_data_keras, verbose = 0)
        print(str(result_1))
print()


(Dump result)

0.584841251373291
0.5780714750289917
0.5713894367218018
0.5647997856140137
0.5583046674728394
0.5519051551818848
0.5456019043922424
:
:

The loss is the same when using test_on_batch ().
(source code)

# starts training
for i in range(max_loop):
    for j in range(max_in_loop):
        result_1 = model.train_on_batch(x_data_keras, y_data_keras)
        result_2 = model.test_on_batch(x_data_keras, y_data_keras)
#        result_3 = model.evaluate(x_data_keras, y_data_keras, verbose = 0)
        print(str(result_1))
print()


(Dump result)

0.584841251373291
0.5780714750289917
0.5713894367218018
0.5647997856140137
0.5583046674728394
0.5519051551818848
0.5456019043922424
0.5394853949546814
:
:

The loss is different with and without evaluate ().
I would like to know the cause and solution of this.
(source code)

# starts training
for i in range(max_loop):
    for j in range(max_in_loop):
        result_1 = model.train_on_batch(x_data_keras, y_data_keras)
#        result_2 = model.test_on_batch(x_data_keras, y_data_keras)
        result_3 = model.evaluate(x_data_keras, y_data_keras, verbose = 0)
        print(str(result_1))
print()


(Dump result)

0.584841251373291        <- 1st same
0.5547470450401306      <- different! 
0.5488174557685852
0.5429757833480835
0.537223756313324
0.531562328338623
:
:

BRs
take hamster