text: Title: Numba list of sets causes malfunctioning when filled by function not decorated with @jit
  Body: <!--
  
  Thanks for opening an issue! To help the Numba team handle your information
  efficiently, please first ensure that there is no other issue present that
  already describes the issue you have
  (search at https://github.com/numba/numba/issues?&q=is%3Aissue).
  
  -->
  
  ## Reporting a bug
  
  <!--
  
  Before submitting a bug report please ensure that you can check off these boxes:
  
  -->
  
  - [x] I have tried using the latest released version of Numba (most recent is
    visible in the release notes
    (https://numba.readthedocs.io/en/stable/release-notes-overview.html).
  - [x] I have included a self contained code sample to reproduce the problem.
    i.e. it's possible to run as 'python bug.py'.
  
  <!--
  
  Please include details of the bug here, including, if applicable, what you
  expected to happen!
  
  -->
  
  - Summary
  The goal of the code is to generate a numba typed list of sets with values based on the values of a numba typed list of numpy arrays. When this function is defined without the `@jit` decorator, values in the sets are not those expected and segmentation fault rises. When it is called with the `@jit` decorator, everything works as expected.
  
  - Problematic code
  ```
  import numba
  from numba.core.types import ListType
  from numpy import array, asarray, ceil, floor, isnan
  
  def calculate_coarse_grained(
          grid_longitude_length: float,
          grid_latitude_length: float,
          longitude_length: float,
          longitude_min_value: float,
          latitude_min_value: float,
          interpolated_trajectories: ListType,
          initialized_set: ListType
  ):
      lengthwise_division_number = ceil(longitude_length / grid_longitude_length)
      corrected_lengthwise_division_number = lengthwise_division_number + 1
      for i in range(len(interpolated_trajectories)):
          trajectory = interpolated_trajectories[i]
          if not trajectory.shape[0]: continue
          coarse_grained_positions = \
              floor((trajectory[:, 2] - longitude_min_value) / grid_longitude_length) + \
              corrected_lengthwise_division_number * floor((trajectory[:, 1] - latitude_min_value) / grid_latitude_length)
  
          filtered_coarse_grained_position = coarse_grained_positions[~isnan(coarse_grained_positions)]
          full_coarse_grained_position = \
              set(filtered_coarse_grained_position) | set(filtered_coarse_grained_position + 1) | \
              set(filtered_coarse_grained_position - 1) | \
              set(filtered_coarse_grained_position - corrected_lengthwise_division_number) | \
              set(filtered_coarse_grained_position + corrected_lengthwise_division_number)
  
          initialized_set[i] = full_coarse_grained_position
  
      return initialized_set
  
  
  
  if __name__ == "__main__":
      grid_longitude_length = 0.09
      grid_latitude_length = 0.09
      min_longitude = -15.
      max_longitude = 30.
      min_latitude = 35.
      max_latitude = 70.
      min_altitude = 100.
      longitude_length = max_longitude - min_longitude
  
      trial_array = array([array([[1., 40., 0., 150.], [2., 45., 1., 200.]]), array([[1., 50., 10., 150.], [2., 55., 11., 200.]])])
      trial_typed_list = numba.typed.List(trial_array)
  
      set_array = asarray([{0.0}] * len(trial_typed_list))
      set_list = numba.typed.List(set_array)
  
  
      coarse_grained_list = calculate_coarse_grained(
          grid_longitude_length,
          grid_latitude_length,
          longitude_length,
          min_longitude,
          min_latitude,
          trial_typed_list,
          set_list
      )
  
      print(coarse_grained_list[0], coarse_grained_list[-1])
  
      for i in [1]:
          coarse_grained_list.pop(i)
  ```
  - Output from problematic code
  ```
  {'', '.4', 'retptr', '.6', '.1', 'arg.l.1', '.2', 'excinfo', 'arg.index', 'arg.l.0', '.5', '.3'} {0}
  
  Process finished with exit code -1073740940 (0xC0000374)
  ```
  Where the first line is the ouput of  `print(coarse_grained_list[0], coarse_grained_list[-1])`, and the second one of the action `coarse_grained_list.pop(i)`.
  
  - Functioning code
  ```
  import numba
  from numba import jit
  from numba.core.types import ListType
  from numpy import array, asarray, ceil, floor, isnan
  
  @jit(
      'ListType(Set(float64, reflected=True))(float64, float64, float64, float64, float64, ListType(float64[:, ::1]), ListType(Set(float64, reflected=True)))',
      nopython=True,
      cache=True,
      nogil=True
  )
  def calculate_coarse_grained(
          grid_longitude_length: float,
          grid_latitude_length: float,
          longitude_length: float,
          longitude_min_value: float,
          latitude_min_value: float,
          interpolated_trajectories: ListType,
          initialized_set: ListType
  ):
      lengthwise_division_number = ceil(longitude_length / grid_longitude_length)
      corrected_lengthwise_division_number = lengthwise_division_number + 1
      for i in range(len(interpolated_trajectories)):
          trajectory = interpolated_trajectories[i]
          if not trajectory.shape[0]: continue
          coarse_grained_positions = \
              floor((trajectory[:, 2] - longitude_min_value) / grid_longitude_length) + \
              corrected_lengthwise_division_number * floor((trajectory[:, 1] - latitude_min_value) / grid_latitude_length)
  
          filtered_coarse_grained_position = coarse_grained_positions[~isnan(coarse_grained_positions)]
          full_coarse_grained_position = \
              set(filtered_coarse_grained_position) | set(filtered_coarse_grained_position + 1) | \
              set(filtered_coarse_grained_position - 1) | \
              set(filtered_coarse_grained_position - corrected_lengthwise_division_number) | \
              set(filtered_coarse_grained_position + corrected_lengthwise_division_number)
  
          initialized_set[i] = full_coarse_grained_position
  
      return initialized_set
  
  
  
  if __name__ == "__main__":
      grid_longitude_length = 0.09
      grid_latitude_length = 0.09
      min_longitude = -15.
      max_longitude = 30.
      min_latitude = 35.
      max_latitude = 70.
      min_altitude = 100.
      longitude_length = max_longitude - min_longitude
  
      trial_array = array([array([[1., 40., 0., 150.], [2., 45., 1., 200.]]), array([[1., 50., 10., 150.], [2., 55., 11., 200.]])])
      trial_typed_list = numba.typed.List(trial_array)
  
      set_array = asarray([{0.0}] * len(trial_typed_list))
      set_list = numba.typed.List(set_array)
  
  
      coarse_grained_list = calculate_coarse_grained(
          grid_longitude_length,
          grid_latitude_length,
          longitude_length,
          min_longitude,
          min_latitude,
          trial_typed_list,
          set_list
      )
  
      print(coarse_grained_list[0], coarse_grained_list[-1])
  
      for i in [1]:
          coarse_grained_list.pop(i)
  ```
  - Output from the functioning code
  ```
  C:\Users\Raúl\Desktop\Fisica\Doctorado\interaction-networks-git\git_hub_real_preprocess_and_tracker_main.py:37: NumbaTypeSafetyWarning: unsafe cast from set(float64) to reflected set(float64). Precision may be lost.
    initialized_set[i] = full_coarse_grained_position
  {56289.0, 27720.0, 27721.0, 27722.0, 55787.0, 55788.0, 55789.0, 27220.0, 55287.0, 28222.0} {111009.0, 83944.0, 112011.0, 83442.0, 83443.0, 83444.0, 111509.0, 111510.0, 111511.0, 82942.0}
  
  Process finished with exit code 0
  ```
  Where the line with the printed sets is the result of  `print(coarse_grained_list[0], coarse_grained_list[-1])`
  
  - Versions and packages:
  I am running this code on a Windows 10 machine. Python version: 3.9.18 and a conda virtual environment with the following packages:
   
  ![image](https://github.com/numba/numba/assets/83119155/462cf01d-0682-4ff7-9e5a-a146cc01f7ef)
  
  - Further comments
  Considering that when `@jit` is added the code works correctly, it seems like python working without the jit functionality is unable to work correctly with numba typed lists of numba in this very specific scenario.
  