Here is some code. The function below is the main function that builds the model.
> def build_model(self):
>         state_input = Input((self.NN_input_size,))
>         
>          
>         pre_dueling = Dense(512, activation='relu', name='Predueling')(state_input)
>         flatten = Flatten()(pre_dueling)
> 
>         # network separate state value and advantages
>         advantage_fc = Dense(256, activation='relu', name = 'AdvantageFC')(flatten)
>         advantage = Dense(self.action_dim, name = 'Adavantage')(advantage_fc)
>         
>         value_fc = Dense(256, activation='relu', name = 'ValueFC')(flatten)
>         value =  Dense(1, name = 'Value')(value_fc)
>         
>         # network merged and make Q Value
>         Q_values = (value + (advantage - tf.math.reduce_mean(advantage, axis = 1, keepdims = True)))
>         model = Model(inputs=state_input, outputs=Q_values)
>         
>         model.compile(loss=tf.keras.losses.Huber(), optimizer=Adam(self.lr))
>         
>         return model

In a Reinforcement learning settings I simply collect samples and pass them to the below function.
> losses = self.R.model.train_on_batch(self.b_obs, target_q, sample_weight = self.is_weight.flatten())
> 

This worked before on different problem. I am not able to understand when this error can occur