title: Object Detection crash after 5428 steps, TypeError: 'numpy.float64' object cannot be interpreted as an integer
body: <p>My object detector has run multiple times but at this mark of 5428 it then crashes from TypeError's </p>
  
  <p>I'm running in in anaconda with:</p>
  
  <ul>
  <li>numpy 1.18.1</li>
  <li>numpy-base 1.18.1</li>
  <li>tensorflow-gpu 1.14</li>
  </ul>
  
  <p>This snippet below I think is the most important error?:</p>
  
  <pre><code>2020-02-19 13:56:06.901096: W tensorflow/core/framework/op_kernel.cc:1490] Invalid argument: TypeError: object of type &lt;class 'numpy.float64'&gt; cannot be safely interpreted as an integer.
  Traceback (most recent call last):
  
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\numpy\core\function_base.py", line 117, in linspace
      num = operator.index(num)
  
  TypeError: 'numpy.float64' object cannot be interpreted as an integer
  </code></pre>
  
  <p>The full traceback is below:</p>
  
  <pre><code>I0219 13:55:41.016854 15428 basic_session_run_hooks.py:260] loss = 0.0140173, step = 5400 (10.773 sec)
  INFO:tensorflow:Saving checkpoints for 5428 into training/model.ckpt.
  I0219 13:55:43.900022 15428 basic_session_run_hooks.py:606] Saving checkpoints for 5428 into training/model.ckpt.
  INFO:tensorflow:Calling model_fn.
  I0219 13:55:56.207441 15428 estimator.py:1145] Calling model_fn.
  INFO:tensorflow:Scale of 0 disables regularizer.
  I0219 13:55:58.009801 15428 regularizers.py:98] Scale of 0 disables regularizer.
  INFO:tensorflow:Scale of 0 disables regularizer.
  I0219 13:55:58.025418 15428 regularizers.py:98] Scale of 0 disables regularizer.
  INFO:tensorflow:depth of additional conv before box predictor: 0
  I0219 13:55:58.025418 15428 convolutional_box_predictor.py:151] depth of additional conv before box predictor: 0
  INFO:tensorflow:Scale of 0 disables regularizer.
  I0219 13:55:59.573186 15428 regularizers.py:98] Scale of 0 disables regularizer.
  INFO:tensorflow:Scale of 0 disables regularizer.
  I0219 13:55:59.588815 15428 regularizers.py:98] Scale of 0 disables regularizer.
  WARNING:tensorflow:From C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\object_detection-0.1-py3.7.egg\object_detection\eval_util.py:796: to_int64 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
  Instructions for updating:
  Use `tf.cast` instead.
  W0219 13:56:00.855241 15428 deprecation.py:323] From C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\object_detection-0.1-py3.7.egg\object_detection\eval_util.py:796: to_int64 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
  Instructions for updating:
  Use `tf.cast` instead.
  WARNING:tensorflow:From C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\object_detection-0.1-py3.7.egg\object_detection\utils\visualization_utils.py:498: py_func (from tensorflow.python.ops.script_ops) is deprecated and will be removed in a future version.
  Instructions for updating:
  tf.py_func is deprecated in TF V2. Instead, there are two
      options available in V2.
      - tf.py_function takes a python function which manipulates tf eager
      tensors instead of numpy arrays. It's easy to convert a tf eager tensor to
      an ndarray (just call tensor.numpy()) but having access to eager tensors
      means `tf.py_function`s can use accelerators such as GPUs as well as
      being differentiable using a gradient tape.
      - tf.numpy_function maintains the semantics of the deprecated tf.py_func
      (it is not differentiable, and manipulates numpy arrays). It drops the
      stateful argument making all functions stateful.
  
  W0219 13:56:01.105266 15428 deprecation.py:323] From C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\object_detection-0.1-py3.7.egg\object_detection\utils\visualization_utils.py:498: py_func (from tensorflow.python.ops.script_ops) is deprecated and will be removed in a future version.
  Instructions for updating:
  tf.py_func is deprecated in TF V2. Instead, there are two
      options available in V2.
      - tf.py_function takes a python function which manipulates tf eager
      tensors instead of numpy arrays. It's easy to convert a tf eager tensor to
      an ndarray (just call tensor.numpy()) but having access to eager tensors
      means `tf.py_function`s can use accelerators such as GPUs as well as
      being differentiable using a gradient tape.
      - tf.numpy_function maintains the semantics of the deprecated tf.py_func
      (it is not differentiable, and manipulates numpy arrays). It drops the
      stateful argument making all functions stateful.
  
  WARNING:tensorflow:From C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\object_detection-0.1-py3.7.egg\object_detection\utils\visualization_utils.py:1044: The name tf.summary.image is deprecated. Please use tf.compat.v1.summary.image instead.
  
  W0219 13:56:01.277014 15428 deprecation_wrapper.py:119] From C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\object_detection-0.1-py3.7.egg\object_detection\utils\visualization_utils.py:1044: The name tf.summary.image is deprecated. Please use tf.compat.v1.summary.image instead.
  
  WARNING:tensorflow:From C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\object_detection-0.1-py3.7.egg\object_detection\model_lib.py:484: The name tf.metrics.mean is deprecated. Please use tf.compat.v1.metrics.mean instead.
  
  W0219 13:56:01.386395 15428 deprecation_wrapper.py:119] From C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\object_detection-0.1-py3.7.egg\object_detection\model_lib.py:484: The name tf.metrics.mean is deprecated. Please use tf.compat.v1.metrics.mean instead.
  
  INFO:tensorflow:Done calling model_fn.
  I0219 13:56:01.749697 15428 estimator.py:1147] Done calling model_fn.
  INFO:tensorflow:Starting evaluation at 2020-02-19T13:56:01Z
  I0219 13:56:01.781106 15428 evaluation.py:255] Starting evaluation at 2020-02-19T13:56:01Z
  INFO:tensorflow:Graph was finalized.
  I0219 13:56:02.489665 15428 monitored_session.py:240] Graph was finalized.
  2020-02-19 13:56:02.508162: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1640] Found device 0 with properties:
  name: GeForce GTX 1080 Ti major: 6 minor: 1 memoryClockRate(GHz): 1.683
  pciBusID: 0000:06:00.0
  2020-02-19 13:56:02.512995: I tensorflow/stream_executor/platform/default/dlopen_checker_stub.cc:25] GPU libraries are statically linked, skip dlopen check.
  2020-02-19 13:56:02.516493: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1763] Adding visible gpu devices: 0
  2020-02-19 13:56:02.518703: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1181] Device interconnect StreamExecutor with strength 1 edge matrix:
  2020-02-19 13:56:02.523922: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1187]      0
  2020-02-19 13:56:02.526614: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1200] 0:   N
  2020-02-19 13:56:02.529223: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1326] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 8788 MB memory) -&gt; physical GPU (device: 0, name: GeForce GTX 1080 Ti, pci bus id: 0000:06:00.0, compute capability: 6.1)
  WARNING:tensorflow:From C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\tensorflow\python\training\saver.py:1276: checkpoint_exists (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.
  Instructions for updating:
  Use standard file APIs to check for files with this prefix.
  W0219 13:56:02.535778 15428 deprecation.py:323] From C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\tensorflow\python\training\saver.py:1276: checkpoint_exists (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.
  Instructions for updating:
  Use standard file APIs to check for files with this prefix.
  INFO:tensorflow:Restoring parameters from training/model.ckpt-5428
  I0219 13:56:02.538779 15428 saver.py:1280] Restoring parameters from training/model.ckpt-5428
  INFO:tensorflow:Running local_init_op.
  I0219 13:56:03.495252 15428 session_manager.py:500] Running local_init_op.
  INFO:tensorflow:Done running local_init_op.
  I0219 13:56:03.656017 15428 session_manager.py:502] Done running local_init_op.
  INFO:tensorflow:Performing evaluation on 5 images.
  I0219 13:56:06.852077 13368 coco_evaluation.py:205] Performing evaluation on 5 images.
  creating index...
  index created!
  INFO:tensorflow:Loading and preparing annotation results...
  I0219 13:56:06.867704 13368 coco_tools.py:115] Loading and preparing annotation results...
  INFO:tensorflow:DONE (t=0.00s)
  I0219 13:56:06.867704 13368 coco_tools.py:137] DONE (t=0.00s)
  creating index...
  index created!
  2020-02-19 13:56:06.901096: W tensorflow/core/framework/op_kernel.cc:1490] Invalid argument: TypeError: object of type &lt;class 'numpy.float64'&gt; cannot be safely interpreted as an integer.
  Traceback (most recent call last):
  
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\numpy\core\function_base.py", line 117, in linspace
      num = operator.index(num)
  
  TypeError: 'numpy.float64' object cannot be interpreted as an integer
  
  
  During handling of the above exception, another exception occurred:
  
  
  Traceback (most recent call last):
  
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\tensorflow\python\ops\script_ops.py", line 209, in __call__
      ret = func(*args)
  
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\object_detection-0.1-py3.7.egg\object_detection\metrics\coco_evaluation.py", line 384, in first_value_func
      self._metrics = self.evaluate()
  
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\object_detection-0.1-py3.7.egg\object_detection\metrics\coco_evaluation.py", line 215, in evaluate
      coco_wrapped_groundtruth, coco_wrapped_detections, agnostic_mode=False)
  
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\object_detection-0.1-py3.7.egg\object_detection\metrics\coco_tools.py", line 176, in __init__
      iouType=iou_type)
  
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\pycocotools\cocoeval.py", line 76, in __init__
      self.params = Params(iouType=iouType) # parameters
  
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\pycocotools\cocoeval.py", line 527, in __init__
      self.setDetParams()
  
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\pycocotools\cocoeval.py", line 507, in setDetParams
      self.iouThrs = np.linspace(.5, 0.95, np.round((0.95 - .5) / .05) + 1, endpoint=True)
  
    File "&lt;__array_function__ internals&gt;", line 6, in linspace
  
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\numpy\core\function_base.py", line 121, in linspace
      .format(type(num)))
  
  TypeError: object of type &lt;class 'numpy.float64'&gt; cannot be safely interpreted as an integer.
  
  
  Traceback (most recent call last):
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\tensorflow\python\client\session.py", line 1356, in _do_call
      return fn(*args)
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\tensorflow\python\client\session.py", line 1341, in _run_fn
      options, feed_dict, fetch_list, target_list, run_metadata)
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\tensorflow\python\client\session.py", line 1429, in _call_tf_sessionrun
      run_metadata)
  tensorflow.python.framework.errors_impl.OutOfRangeError: 2 root error(s) found.
    (0) Out of range: End of sequence
           [[{{node IteratorGetNext}}]]
    (1) Out of range: End of sequence
           [[{{node IteratorGetNext}}]]
           [[Loss/BoxClassifierLoss/assert_equal/Assert/Assert/data_4/_2449]]
  0 successful operations.
  0 derived errors ignored.
  
  During handling of the above exception, another exception occurred:
  
  Traceback (most recent call last):
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\tensorflow\python\training\evaluation.py", line 272, in _evaluate_once
      session.run(eval_ops, feed_dict)
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\tensorflow\python\training\monitored_session.py", line 754, in run
      run_metadata=run_metadata)
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\tensorflow\python\training\monitored_session.py", line 1252, in run
      run_metadata=run_metadata)
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\tensorflow\python\training\monitored_session.py", line 1353, in run
      raise six.reraise(*original_exc_info)
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\six.py", line 703, in reraise
      raise value
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\tensorflow\python\training\monitored_session.py", line 1338, in run
      return self._sess.run(*args, **kwargs)
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\tensorflow\python\training\monitored_session.py", line 1411, in run
      run_metadata=run_metadata)
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\tensorflow\python\training\monitored_session.py", line 1169, in run
      return self._sess.run(*args, **kwargs)
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\tensorflow\python\client\session.py", line 950, in run
      run_metadata_ptr)
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\tensorflow\python\client\session.py", line 1173, in _run
      feed_dict_tensor, options, run_metadata)
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\tensorflow\python\client\session.py", line 1350, in _do_run
      run_metadata)
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\tensorflow\python\client\session.py", line 1370, in _do_call
      raise type(e)(node_def, op, message)
  tensorflow.python.framework.errors_impl.OutOfRangeError: 2 root error(s) found.
    (0) Out of range: End of sequence
           [[node IteratorGetNext (defined at model_main.py:105) ]]
    (1) Out of range: End of sequence
           [[node IteratorGetNext (defined at model_main.py:105) ]]
           [[Loss/BoxClassifierLoss/assert_equal/Assert/Assert/data_4/_2449]]
  0 successful operations.
  0 derived errors ignored.
  
  Original stack trace for 'IteratorGetNext':
    File "model_main.py", line 109, in &lt;module&gt;
      tf.app.run()
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\tensorflow\python\platform\app.py", line 40, in run
      _run(main=main, argv=argv, flags_parser=_parse_flags_tolerate_undef)
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\absl\app.py", line 299, in run
      _run_main(main, args)
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\absl\app.py", line 250, in _run_main
      sys.exit(main(argv))
    File "model_main.py", line 105, in main
      tf.estimator.train_and_evaluate(estimator, train_spec, eval_specs[0])
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\tensorflow_estimator\python\estimator\training.py", line 473, in train_and_evaluate
      return executor.run()
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\tensorflow_estimator\python\estimator\training.py", line 613, in run
      return self.run_local()
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\tensorflow_estimator\python\estimator\training.py", line 714, in run_local
      saving_listeners=saving_listeners)
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\tensorflow_estimator\python\estimator\estimator.py", line 367, in train
      loss = self._train_model(input_fn, hooks, saving_listeners)
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\tensorflow_estimator\python\estimator\estimator.py", line 1158, in _train_model
      return self._train_model_default(input_fn, hooks, saving_listeners)
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\tensorflow_estimator\python\estimator\estimator.py", line 1192, in _train_model_default
      saving_listeners)
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\tensorflow_estimator\python\estimator\estimator.py", line 1484, in _train_with_estimator_spec
      _, loss = mon_sess.run([estimator_spec.train_op, estimator_spec.loss])
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\tensorflow\python\training\monitored_session.py", line 754, in run
      run_metadata=run_metadata)
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\tensorflow\python\training\monitored_session.py", line 1252, in run
      run_metadata=run_metadata)
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\tensorflow\python\training\monitored_session.py", line 1338, in run
      return self._sess.run(*args, **kwargs)
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\tensorflow\python\training\monitored_session.py", line 1419, in run
      run_metadata=run_metadata))
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\tensorflow\python\training\basic_session_run_hooks.py", line 594, in after_run
      if self._save(run_context.session, global_step):
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\tensorflow\python\training\basic_session_run_hooks.py", line 619, in _save
      if l.after_save(session, step):
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\tensorflow_estimator\python\estimator\training.py", line 519, in after_save
      self._evaluate(global_step_value)  # updates self.eval_result
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\tensorflow_estimator\python\estimator\training.py", line 539, in _evaluate
      self._evaluator.evaluate_and_export())
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\tensorflow_estimator\python\estimator\training.py", line 920, in evaluate_and_export
      hooks=self._eval_spec.hooks)
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\tensorflow_estimator\python\estimator\estimator.py", line 477, in evaluate
      name=name)
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\tensorflow_estimator\python\estimator\estimator.py", line 519, in _actual_eval
      return _evaluate()
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\tensorflow_estimator\python\estimator\estimator.py", line 501, in _evaluate
      self._evaluate_build_graph(input_fn, hooks, checkpoint_path))
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\tensorflow_estimator\python\estimator\estimator.py", line 1501, in _evaluate_build_graph
      self._call_model_fn_eval(input_fn, self.config))
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\tensorflow_estimator\python\estimator\estimator.py", line 1534, in _call_model_fn_eval
      input_fn, ModeKeys.EVAL)
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\tensorflow_estimator\python\estimator\estimator.py", line 1022, in _get_features_and_labels_from_input_fn
      self._call_input_fn(input_fn, mode))
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\tensorflow_estimator\python\estimator\util.py", line 65, in parse_input_fn_result
      result = iterator.get_next()
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\tensorflow\python\data\ops\iterator_ops.py", line 426, in get_next
      output_shapes=self._structure._flat_shapes, name=name)
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\tensorflow\python\ops\gen_dataset_ops.py", line 1947, in iterator_get_next
      output_shapes=output_shapes, name=name)
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\tensorflow\python\framework\op_def_library.py", line 788, in _apply_op_helper
      op_def=op_def)
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\tensorflow\python\util\deprecation.py", line 507, in new_func
      return func(*args, **kwargs)
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\tensorflow\python\framework\ops.py", line 3616, in create_op
      op_def=op_def)
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\tensorflow\python\framework\ops.py", line 2005, in __init__
      self._traceback = tf_stack.extract_stack()
  
  
  
  During handling of the above exception, another exception occurred:
  
  
  Traceback (most recent call last):
  
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\tensorflow\python\ops\script_ops.py", line 209, in __call__
      ret = func(*args)
  
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\object_detection-0.1-py3.7.egg\object_detection\metrics\coco_evaluation.py", line 384, in first_value_func
      self._metrics = self.evaluate()
  
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\object_detection-0.1-py3.7.egg\object_detection\metrics\coco_evaluation.py", line 215, in evaluate
      coco_wrapped_groundtruth, coco_wrapped_detections, agnostic_mode=False)
  
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\object_detection-0.1-py3.7.egg\object_detection\metrics\coco_tools.py", line 176, in __init__
      iouType=iou_type)
  
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\pycocotools\cocoeval.py", line 76, in __init__
      self.params = Params(iouType=iouType) # parameters
  
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\pycocotools\cocoeval.py", line 527, in __init__
      self.setDetParams()
  
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\pycocotools\cocoeval.py", line 507, in setDetParams
      self.iouThrs = np.linspace(.5, 0.95, np.round((0.95 - .5) / .05) + 1, endpoint=True)
  
    File "&lt;__array_function__ internals&gt;", line 6, in linspace
  
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\numpy\core\function_base.py", line 121, in linspace
      .format(type(num)))
  
  TypeError: object of type &lt;class 'numpy.float64'&gt; cannot be safely interpreted as an integer.
  
  
           [[node PyFunc_3 (defined at C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\object_detection-0.1-py3.7.egg\object_detection\metrics\coco_evaluation.py:394) ]]
  0 successful operations.
  0 derived errors ignored.
  
  Original stack trace for 'PyFunc_3':
    File "model_main.py", line 109, in &lt;module&gt;
      tf.app.run()
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\tensorflow\python\platform\app.py", line 40, in run
      _run(main=main, argv=argv, flags_parser=_parse_flags_tolerate_undef)
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\absl\app.py", line 299, in run
      _run_main(main, args)
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\absl\app.py", line 250, in _run_main
      sys.exit(main(argv))
    File "model_main.py", line 105, in main
      tf.estimator.train_and_evaluate(estimator, train_spec, eval_specs[0])
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\tensorflow_estimator\python\estimator\training.py", line 473, in train_and_evaluate
      return executor.run()
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\tensorflow_estimator\python\estimator\training.py", line 613, in run
      return self.run_local()
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\tensorflow_estimator\python\estimator\training.py", line 714, in run_local
      saving_listeners=saving_listeners)
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\tensorflow_estimator\python\estimator\estimator.py", line 367, in train
      loss = self._train_model(input_fn, hooks, saving_listeners)
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\tensorflow_estimator\python\estimator\estimator.py", line 1158, in _train_model
      return self._train_model_default(input_fn, hooks, saving_listeners)
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\tensorflow_estimator\python\estimator\estimator.py", line 1192, in _train_model_default
      saving_listeners)
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\tensorflow_estimator\python\estimator\estimator.py", line 1484, in _train_with_estimator_spec
      _, loss = mon_sess.run([estimator_spec.train_op, estimator_spec.loss])
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\tensorflow\python\training\monitored_session.py", line 754, in run
      run_metadata=run_metadata)
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\tensorflow\python\training\monitored_session.py", line 1252, in run
      run_metadata=run_metadata)
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\tensorflow\python\training\monitored_session.py", line 1338, in run
      return self._sess.run(*args, **kwargs)
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\tensorflow\python\training\monitored_session.py", line 1419, in run
      run_metadata=run_metadata))
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\tensorflow\python\training\basic_session_run_hooks.py", line 594, in after_run
      if self._save(run_context.session, global_step):
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\tensorflow\python\training\basic_session_run_hooks.py", line 619, in _save
      if l.after_save(session, step):
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\tensorflow_estimator\python\estimator\training.py", line 519, in after_save
      self._evaluate(global_step_value)  # updates self.eval_result
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\tensorflow_estimator\python\estimator\training.py", line 539, in _evaluate
      self._evaluator.evaluate_and_export())
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\tensorflow_estimator\python\estimator\training.py", line 920, in evaluate_and_export
      hooks=self._eval_spec.hooks)
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\tensorflow_estimator\python\estimator\estimator.py", line 477, in evaluate
      name=name)
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\tensorflow_estimator\python\estimator\estimator.py", line 519, in _actual_eval
      return _evaluate()
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\tensorflow_estimator\python\estimator\estimator.py", line 501, in _evaluate
      self._evaluate_build_graph(input_fn, hooks, checkpoint_path))
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\tensorflow_estimator\python\estimator\estimator.py", line 1501, in _evaluate_build_graph
      self._call_model_fn_eval(input_fn, self.config))
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\tensorflow_estimator\python\estimator\estimator.py", line 1537, in _call_model_fn_eval
      features, labels, ModeKeys.EVAL, config)
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\tensorflow_estimator\python\estimator\estimator.py", line 1146, in _call_model_fn
      model_fn_results = self._model_fn(features=features, **kwargs)
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\object_detection-0.1-py3.7.egg\object_detection\model_lib.py", line 482, in model_fn
      eval_config, list(category_index.values()), eval_dict)
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\object_detection-0.1-py3.7.egg\object_detection\eval_util.py", line 947, in get_eval_metric_ops_for_evaluators
      eval_dict))
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\object_detection-0.1-py3.7.egg\object_detection\metrics\coco_evaluation.py", line 394, in get_estimator_eval_metric_ops
      first_value_op = tf.py_func(first_value_func, [], tf.float32)
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\tensorflow\python\util\deprecation.py", line 324, in new_func
      return func(*args, **kwargs)
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\tensorflow\python\ops\script_ops.py", line 480, in py_func
      return py_func_common(func, inp, Tout, stateful, name=name)
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\tensorflow\python\ops\script_ops.py", line 462, in py_func_common
      func=func, inp=inp, Tout=Tout, stateful=stateful, eager=False, name=name)
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\tensorflow\python\ops\script_ops.py", line 285, in _internal_py_func
      input=inp, token=token, Tout=Tout, name=name)
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\tensorflow\python\ops\gen_script_ops.py", line 159, in py_func
      "PyFunc", input=input, token=token, Tout=Tout, name=name)
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\tensorflow\python\framework\op_def_library.py", line 788, in _apply_op_helper
      op_def=op_def)
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\tensorflow\python\util\deprecation.py", line 507, in new_func
      return func(*args, **kwargs)
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\tensorflow\python\framework\ops.py", line 3616, in create_op
      op_def=op_def)
    File "C:\Users\luke9\Anaconda3\envs\tf_gpu\lib\site-packages\tensorflow\python\framework\ops.py", line 2005, in __init__
      self._traceback = tf_stack.extract_stack()
  </code></pre>
  
  <p>This is missing some of the traceback because of the character limit, but it is all numpy TypeError's.</p>