title: How Django bulks create check exist already in bulks objs and instance?
body: <p>I have a lot of data, that data is pretty dirty, example:</p>
  
  <p><strong>A table ORM :</strong></p>
  
  <pre><code>id = models.CharField(default='', max_length=50)
  time = models.DateTimeField(default=timezone.now)
  number = models.CharField(default='', max_length=20)
  value = models.CharField(default='', max_length=20)
  
  unique_together = ['id', 'time', 'number']
  </code></pre>
  
  <p><strong>A table DATA :</strong></p>
  
  <pre><code>id   time                   number   value
   1     2018-07-16 00:00:00   1         64
   1     2018-07-16 00:00:00   2         -99
   1     2018-07-16 00:00:00   3         655
   1     2018-07-16 00:00:00   4         3
   2     2018-07-16 00:00:00   0         12
  </code></pre>
  
  <p><strong>Import Datas (sample) :</strong></p>
  
  <pre><code>id   time                   number   value
   1     2018-07-16 00:00:00   1         64
   3     2018-07-16 00:00:00   0         -99
   3     2018-07-16 00:00:00   0         11
   4     2018-07-16 00:00:00   0         -99
   4     2018-07-16 00:00:00   1         -99
  </code></pre>
  
  <p>So, When I Do</p>
  
  <pre><code>for loop....
      objs = []
      objs.append(A(**kwargs))
  A.objects.bulk_create(objs, batch_size=50000) 
  </code></pre>
  
  <p>It will raise two kind duplicate.</p>
  
  <ol>
  <li>A Table already exist   <strong>" 1     2018-07-16 00:00:00   1"</strong></li>
  <li>Import Datas already exist  <strong>3     2018-07-16 00:00:00   0</strong> for two times in objs, so when I bulks create it will raise duplicate, then it will roll back all commit !!!</li>
  </ol>
  
  <p>the "1",  I can use get or create to solve it 
  but "2", I can't check now I append data exist in the objs or not
  I tried to use this to check exist or not, but when data row over 1000000, 
  the complexity  will be terrible.</p>
  
  <pre><code>def search(id, time, number, objs):
      for obj in objs:
          if obj['id'] == id and obj['time'] == time and obj['number'] == number:
              return True
      return False
  </code></pre>
  
  <p>Is there have any better way? thanks. </p>