Answers.Body: <p>You can save weights during training by passing checkpoint callback to model.fit() method.</p>
  <pre><code># Instantiate your model here
  model = create_model() 
  
  # Set model configurations here
  model.compile(loss=..., optimizer=..., metrics=...) 
  
  # Set checkpoint path
  checkpoint_path = &quot;model_weights.ckpt&quot;
  
  # Create a callback that saves the model's weights
      filepath=checkpoint_path,
      save_weights_only=True,
      monitor='val_loss',
      mode='min',
      save_best_only=True)
  
  # Train the model with the new callback
  model.fit(train_images_1, 
            train_labels_1,
            epochs=50, 
            batch_size=batch_size, 
            callbacks=[cp_callback],
            validation_data=(test_images_1, test_labels_1),
            verbose=0)
  </code></pre>
  <p>After finishing training 1st dataset, model weights will be saved in file called <em>model_weights.ckpt</em>. Before starting training next dataset, load the model weights as below</p>
  <pre><code># Create a new model instance
  model = create_model()
  
  # Set model configurations here
  model.compile(loss=..., optimizer=..., metrics=...) 
  
  # Set checkpoint path
  checkpoint_path = &quot;model_weights.ckpt&quot;
  
  # Load the previously saved weights
  model.load_weights(checkpoint_path)
  
  # Create a callback that saves the model's weights
  cp_callback = tf.keras.callbacks.ModelCheckpoint(
      filepath=checkpoint_path,
      save_weights_only=True,
      monitor='val_loss',
      mode='min',
      save_best_only=True)
  
  # Train the model with the new callback
  model.fit(train_images_2, 
            train_labels_2,
            epochs=50, 
            batch_size=batch_size, 
            callbacks=[cp_callback],
            validation_data=(test_images_2, test_labels_2),
            verbose=0)
  </code></pre>
  <p>Repeat this for all datasets.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-05-16T04:19:43.093Z
Answers.LastActivityDate: 2021-05-16T05:45:03.297Z
Answers.LastEditDate: 2021-05-16T05:45:03.297Z
Answers.UserDisplayName: spb
Answers.Body: <p>If you reinitialize your model weights before training it on a new subset, you erase everything it learned before; is it what you want to ?
  If not, saving your model after each subset and loading it before training on the next subset isn't a good practice neither because your model will see a lot of times a subset of samples and then move on the next ones without seing again these samples.</p>
  <p>A better solution is to load batches on-fly. This can be done using <a href="https://stanford.edu/%7Eshervine/blog/keras-how-to-generate-data-on-the-fly" rel="nofollow noreferrer">keras generators</a>. It allows you to load a batch and do a gradient descent step without changing the model.fit function.</p>
  <p>It will be a bit slower but it will work.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-05-18T12:43:22.000Z
Answers.LastActivityDate: 2021-05-18T12:43:22.000Z
Answers.UserDisplayName: bguetarni
Body: <p>I created a binary image classification model. The dataset contains about 500K images in each class, with ratio = Train : Validation : Test = 7 : 2 : 1. Total images = 1M</p>
  <p>I split my dataset into 5 parts (compute constraints)—5 training subsets, 5 validation subsets, and 1 test subset.</p>
  <p>I trained and evaluated my model stage by stage. In first stage (evaluation), my model's accuracy was 65%. I re-fitted it with 2nd dataset and the accuracy was 43%. I did same process with the rest, and my accuracies were: 65%, 43%, 57%, 21%, 30%.</p>
  <p>How can I train my model in staged training?</p>
  <p>I want to train models with different datasets without reinitialize the weight every training process.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-05-16T10:52:57.150Z
Comments.Text: Hello. I don't really understand what the question/doubt is here. Can you clarify that? I understand it has to do with `fit`, but apart from that I don't really understand the question.
Comments.UserDisplayName: nbro
Tags: <deep-learning><tensorflow><training>
Title: What is the process working on Tensorflow model.fit()?
UserDisplayName: Kue Satu
ThreadText: # What is the process working on Tensorflow model.fit()?
  
  (Asked by: username_0 on 2021-05-16)
  
  I created a binary image classification model. The dataset contains about 500K images in each class, with ratio = Train : Validation : Test = 7 : 2 : 1. Total images = 1M
  
  I split my dataset into 5 parts (compute constraints)—5 training subsets, 5 validation subsets, and 1 test subset.
  
  I trained and evaluated my model stage by stage. In first stage (evaluation), my model's accuracy was 65%. I re-fitted it with 2nd dataset and the accuracy was 43%. I did same process with the rest, and my accuracies were: 65%, 43%, 57%, 21%, 30%.
  
  How can I train my model in staged training?
  
  I want to train models with different datasets without reinitialize the weight every training process.
  
  **Tags:** deep-learning, tensorflow, training
  > **Comments:**
  > **username_1**: Hello. I don't really understand what the question/doubt is here. Can you clarify that? I understand it has to do with `fit`, but apart from that I don't really understand the question.  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_2 on 2021-05-16. Score: 1
  
  You can save weights during training by passing checkpoint callback to model.fit() method.
  # Instantiate your model here
  model = create_model() 
  
  # Set model configurations here
  model.compile(loss=..., optimizer=..., metrics=...) 
  
  # Set checkpoint path
  checkpoint_path = "model_weights.ckpt"
  
  # Create a callback that saves the model's weights
      filepath=checkpoint_path,
      save_weights_only=True,
      monitor='val_loss',
      mode='min',
      save_best_only=True)
  
  # Train the model with the new callback
  model.fit(train_images_1, 
            train_labels_1,
            epochs=50, 
            batch_size=batch_size, 
            callbacks=[cp_callback],
            validation_data=(test_images_1, test_labels_1),
            verbose=0)
  
  After finishing training 1st dataset, model weights will be saved in file called model_weights.ckpt. Before starting training next dataset, load the model weights as below
  # Create a new model instance
  model = create_model()
  
  # Set model configurations here
  model.compile(loss=..., optimizer=..., metrics=...) 
  
  # Set checkpoint path
  checkpoint_path = "model_weights.ckpt"
  
  # Load the previously saved weights
  model.load_weights(checkpoint_path)
  
  # Create a callback that saves the model's weights
  cp_callback = tf.keras.callbacks.ModelCheckpoint(
      filepath=checkpoint_path,
      save_weights_only=True,
      monitor='val_loss',
      mode='min',
      save_best_only=True)
  
  # Train the model with the new callback
  model.fit(train_images_2, 
            train_labels_2,
            epochs=50, 
            batch_size=batch_size, 
            callbacks=[cp_callback],
            validation_data=(test_images_2, test_labels_2),
            verbose=0)
  
  Repeat this for all datasets.
  
  
  ---
  
  ### Answer by username_3 on 2021-05-18. Score: 0
  
  If you reinitialize your model weights before training it on a new subset, you erase everything it learned before; is it what you want to ?
  If not, saving your model after each subset and loading it before training on the next subset isn't a good practice neither because your model will see a lot of times a subset of samples and then move on the next ones without seing again these samples.
  
  A better solution is to load batches on-fly. This can be done using [keras generators](https://stanford.edu/%7Eshervine/blog/keras-how-to-generate-data-on-the-fly). It allows you to load a batch and do a gradient descent step without changing the model.fit function.
  
  It will be a bit slower but it will work.
  
  
  ---
  
  