title: Sagemaker directory opt/ml/models does not store models to load them for inference
body: <h1>Problem</h1>
  <p>Sagemaker does not load my models from the s3 bucket.</p>
  <p><strong>Note: I am passing in sagemaker the S3 URI, where my models should be at. All the models are archived as .tar.gz and they are .onnx models.</strong></p>
  <h1>Dockerfile</h1>
  <p>My initial MODEL_BASE_PATH was /opt/ml/model(this is where sagemaker is supposed to store the loaded models from the s3 bucket where you specify the path for).</p>
  <p>I modified this path upon recomendation(as /opt/ml/model throws the same exception) to:</p>
  <pre><code>    ENV MODEL_BASE_PATH=/opt/ml/models
  </code></pre>
  <h1>Problem analysis and code#</h1>
  <p>I added a few loggers to check if the path in sagemaker of /opt/ml/models actually contains something. Here are the logs, that describe the error (I deleted date and time for them to be easier to read).</p>
  <blockquote>
  <p>/opt/ml: ['models', '.sagemaker_infra']</p>
  </blockquote>
  <blockquote>
  <p>/opt/ml/models: []</p>
  </blockquote>
  <blockquote>
  <p>Exception on /ping [GET]</p>
  </blockquote>
  <blockquote>
  <p>FileNotFoundError: [Errno 2] No such file or directory: '/opt/ml/models/&lt;my_model_name&gt;.tar.gz'</p>
  </blockquote>
  <blockquote>
  <p>&quot;GET /ping HTTP/1.1&quot; 500 265 &quot;-&quot; &quot;AHC/2.0&quot;</p>
  </blockquote>
  <blockquote>
  <p>&quot;GET /models HTTP/1.1&quot; 404 2 &quot;-&quot; &quot;AHC/2.0&quot;</p>
  </blockquote>
  <p>As you can see from the above logs, /opt/ml/models is empty, so the models from the s3 bucket are not loaded there. So I get an exception from the code below, that points out the model could not be loaded as there is no such file or directory(Because nothing exists at that location).</p>
  <pre><code>    @app.route(&quot;/ping&quot;, methods=[&quot;GET&quot;])
      def ping():
      &quot;&quot;&quot;Determine if the container is working and healthy. In this sample container, we declare
      it healthy if we can load the model successfully.&quot;&quot;&quot;
          health = ScoringService.get_model() is not None  # You can insert a health check here
  
          status = 200 if health else 404
          return flask.Response(response=&quot;\n&quot;, status=status, mimetype=&quot;application/json&quot;)
  </code></pre>
  <p>&quot;ScoringService.get_model()&quot; Is supposed to return a string with the path of the model I am using. This works on a simultaion of the sagemaker structure on my local computer with a request from Postman.</p>
  <p>More on the project structure: <a href="https://sagemaker-workshop.com/custom/containers.html" rel="nofollow noreferrer">https://sagemaker-workshop.com/custom/containers.html</a></p>
  <p>My project structure:</p>
  <pre><code>    project_name/
          Dockerfile
          build_and_push.sh
          inference/
              nginx.conf
              predictor.py
              serve
              wsgi.py
  </code></pre>
  <p><strong>Note: I have left out the file 'train', as I will not be adding further models to my application.</strong></p>
  <p><strong>Endnote: Any insight on what I am doing wrong is really helpful.</strong></p>