Answers: <p>After diving into the rabbit hole of netfilter and flannel routing / firewall rules it turns out to be something really simple.</p>
  <p>There were multiple deployments with the same label!</p>
  <p>When different pods have the same selector labels k8s selects the wrong one sometimes and then gives connection refused. You have to have unique names for the selectors. Weird that this is not logged anywhere which pod is selected or why it is refused. So the following is wrong because 'app' exists over multiple deployments.</p>
  <pre><code>apiVersion: v1
  kind: Service
  metadata:
   name: revproxy-svc
  spec:
   selector:
     role: app
   type: NodePort
  
  ---
  
  apiVersion: v1
  kind: Service
  metadata:
   name: loadbalance-svc
  spec:
   selector:
     role: app
   type: NodePort
  
  ---
  
  apiVersion: apps/v1
  kind: Deployment
  metadata:
   name: loadbalancer-python
  spec:
   replicas: 1
   selector:
     matchLabels:
       role: app
   template:
     metadata:
       labels:
         role: app
  
  ---
  
  apiVersion: apps/v1
  kind: Deployment
  metadata:
   name: revproxy
  spec:
   replicas: 2
   selector:
     matchLabels:
       role: app
   template:
     metadata:
       labels:
         role: app
  </code></pre>
  
Answers: CC BY-SA 4.0
Answers: 2021-09-08T09:07:37.897Z
Answers: 2021-09-08T09:07:37.897Z
Answers: Serve Laurijssen
Body: <p>I've got a simple python http server running on port 8080.</p>
  <pre><code>class Handler(BaseHTTPRequestHandler):
      def do_GET(self):
          print(self.path)
          self.wfile.write(b'\n')
          return
  
  class ThreadedHTTPServer(ThreadingMixIn, HTTPServer):
      &quot;&quot;&quot;Handle requests in a separate thread.&quot;&quot;&quot;
  
  if __name__ == '__main__':
      server = ThreadedHTTPServer(('0.0.0.0', 8080), Handler)
      server.serve_forever()
  </code></pre>
  <p>fire up 'python3 balancer.py' and it works fine.</p>
  <blockquote>
  <p>curl -X GET <a href="http://10.203.32.90:8080/?path=/project-2" rel="nofollow noreferrer">http://10.203.32.90:8080/?path=/project-2</a></p>
  </blockquote>
  <p>works every time. from the same machine and others.</p>
  <p>Now after creating a docker image from this script and deploy it in k8s cluster on port 32003 it does fire up.
  But when calling the service with</p>
  <blockquote>
  <p>curl -X GET <a href="http://10.203.32.90:32003/?path=/project-2" rel="nofollow noreferrer">http://10.203.32.90:32003/?path=/project-2</a></p>
  </blockquote>
  <p>a connection refused error <strong>sometimes</strong> is returned if it's called from another machine.
  and when the curl is done from the same machine more often than not a connection refused is returned. sometimes (1 out of 10) it works though.</p>
  <pre><code>laurijssen@ub:~/pablo$ curl -X GET http://ub:32003/
  curl: (7) Failed to connect to ub port 32003: Connection refused
  laurijssen@ub:~/pablo$ curl -X GET http://ub:32003/
  {&quot;success&quot;: &quot;true&quot;, &quot;code&quot;: 0}
  
  laurijssen@ub:~/pablo$ curl -X GET http://ub:32003/
  curl: (7) Failed to connect to ub port 32003: Connection refused
  </code></pre>
  <p>The docker file of the webserver:</p>
  <pre><code>FROM python:3.8-slim-buster
  RUN pip install requests
  ADD balancer.py /
  CMD [&quot;python3&quot;, &quot;balancer.py&quot;]
  </code></pre>
  <p>The yaml of the k8s service/deployment:</p>
  <pre><code>apiVersion: v1
  kind: Service
  metadata:
   name: loadbalance-svc
  spec:
   selector:
     role: app
   ports:
     - protocol: TCP
       port: 8080
       targetPort: 8080
       nodePort: 32003
   type: NodePort
   externalName: lb.nl
  
  ---
  
  apiVersion: apps/v1
  kind: Deployment
  metadata:
   name: loadbalancer-deployment 
   labels:
     role: app
  spec:
   replicas: 1
   selector:
     matchLabels:
       role: app
   template:
     metadata:
       labels:
         role: app
     spec: 
       containers:
       - name: web
         image: &quot;10.203.32.90:5000/loadbalancer&quot;
         env:
         imagePullPolicy: Always
  </code></pre>
  <p>Dont know how to fix this. What logs to check? What could cause this sometimes it works, sometimes it doesnt behaviour?</p>
  
Tags: <docker><kubernetes>
Title: connection refused from webserver in container
UserDisplayName: Serve Laurijssen
ThreadText: # connection refused from webserver in container
  
  (Asked by: username_0 on 2021-09-06)
  
  I've got a simple python http server running on port 8080.
  
  ```
  class Handler(BaseHTTPRequestHandler):
      def do_GET(self):
          print(self.path)
          self.wfile.write(b'\n')
          return
  
  class ThreadedHTTPServer(ThreadingMixIn, HTTPServer):
      """Handle requests in a separate thread."""
  
  if __name__ == '__main__':
      server = ThreadedHTTPServer(('0.0.0.0', 8080), Handler)
      server.serve_forever()
  
  ```
  
  fire up 'python3 balancer.py' and it works fine.
  
  > curl -X GET <http://10.203.32.90:8080/?path=/project-2>
  
  works every time. from the same machine and others.
  
  Now after creating a docker image from this script and deploy it in k8s cluster on port 32003 it does fire up.
  But when calling the service with
  
  > curl -X GET <http://10.203.32.90:32003/?path=/project-2>
  
  a connection refused error **sometimes** is returned if it's called from another machine.
  and when the curl is done from the same machine more often than not a connection refused is returned. sometimes (1 out of 10) it works though.
  
  ```
  username_0@ub:~/pablo$ curl -X GET http://ub:32003/
  curl: (7) Failed to connect to ub port 32003: Connection refused
  username_0@ub:~/pablo$ curl -X GET http://ub:32003/
  {"success": "true", "code": 0}
  
  username_0@ub:~/pablo$ curl -X GET http://ub:32003/
  curl: (7) Failed to connect to ub port 32003: Connection refused
  
  ```
  
  The docker file of the webserver:
  
  ```
  FROM python:3.8-slim-buster
  RUN pip install requests
  ADD balancer.py /
  CMD ["python3", "balancer.py"]
  
  ```
  
  The yaml of the k8s service/deployment:
  
  ```
  apiVersion: v1
  kind: Service
  metadata:
   name: loadbalance-svc
  spec:
   selector:
     role: app
   ports:
     - protocol: TCP
       port: 8080
       targetPort: 8080
       nodePort: 32003
   type: NodePort
   externalName: lb.nl
  
  ---
  
  apiVersion: apps/v1
  kind: Deployment
  metadata:
   name: loadbalancer-deployment 
   labels:
     role: app
  spec:
   replicas: 1
   selector:
     matchLabels:
       role: app
   template:
     metadata:
       labels:
         role: app
     spec: 
       containers:
       - name: web
         image: "10.203.32.90:5000/loadbalancer"
         env:
         imagePullPolicy: Always
  
  ```
  
  Dont know how to fix this. What logs to check? What could cause this sometimes it works, sometimes it doesnt behaviour?
  
  **Tags:** docker, kubernetes
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_0 on 2021-09-08. Score: 0 | Accepted answer
  
  After diving into the rabbit hole of netfilter and flannel routing / firewall rules it turns out to be something really simple.
  
  There were multiple deployments with the same label!
  
  When different pods have the same selector labels k8s selects the wrong one sometimes and then gives connection refused. You have to have unique names for the selectors. Weird that this is not logged anywhere which pod is selected or why it is refused. So the following is wrong because 'app' exists over multiple deployments.
  
  ```
  apiVersion: v1
  kind: Service
  metadata:
   name: revproxy-svc
  spec:
   selector:
     role: app
   type: NodePort
  
  ---
  
  apiVersion: v1
  kind: Service
  metadata:
   name: loadbalance-svc
  spec:
   selector:
     role: app
   type: NodePort
  
  ---
  
  apiVersion: apps/v1
  kind: Deployment
  metadata:
   name: loadbalancer-python
  spec:
   replicas: 1
   selector:
     matchLabels:
       role: app
   template:
     metadata:
       labels:
         role: app
  
  ---
  
  apiVersion: apps/v1
  kind: Deployment
  metadata:
   name: revproxy
  spec:
   replicas: 2
   selector:
     matchLabels:
       role: app
   template:
     metadata:
       labels:
         role: app
  
  ```
  
  
  ---
  
  