For the regularization loss penalizzation the sum is embedded in the code when you compile the loss:

  

      github.com
  

  
    keras-team/keras/blob/master/keras/engine/compile_utils.py#L231



      
      
            if (loss_obj.reduction == losses_utils.ReductionV2.SUM_OVER_BATCH_SIZE or
                loss_obj.reduction == losses_utils.ReductionV2.AUTO):
              loss_value = losses_utils.scale_loss_for_distribution(loss_value)
          
          
  loss_values.append(loss_value)
            loss_metric_values.append(loss_metric_value)
          
          
if regularization_losses:
            regularization_losses = losses_utils.cast_losses_to_common_dtype(
                regularization_losses)
            reg_loss = tf.add_n(regularization_losses)
            loss_metric_values.append(reg_loss)
            loss_values.append(losses_utils.scale_loss_for_distribution(reg_loss))
          
          
if loss_values:
            loss_metric_values = losses_utils.cast_losses_to_common_dtype(
                loss_metric_values)
            total_loss_metric_value = tf.add_n(loss_metric_values)
            self._loss_metric.update_state(
                total_loss_metric_value, sample_weight=batch_dim)
          
      

    
  


  

  
    
    
  

  


As a workaround probably you could create a custom regularizer that you can scale yourself (if you know the total number of regularizers) or you can control your loss more in detail with a custom trainning loop:

  
      

      TensorFlow
  

  
    

Writing a training loop from scratch  |  TensorFlow Core