Answers: <p>I think your ingress rule is fine but I think your Traefik service needs to be changed. Check out this article <a href="https://kubernetes.io/docs/tutorials/services/source-ip/" rel="nofollow noreferrer">https://kubernetes.io/docs/tutorials/services/source-ip/</a></p>
  
  <p>I think what you are looking for is <code>nodePort</code> with <code>externalTrafficPolicy</code> set to <code>local</code>.</p>
  
  <p>This should allow Traefik to enforce <code>whitelist-source-range</code> annotation on the ingress. </p>
  
  <p>EDIT:</p>
  
  <p>I was going to go into more detail but it was so old I thought you might have already found a solution. Ingress controllers are not allowed to bind to 80 and 443 but most organizations are using the load balancer service to load balance traffic to the Ingress controller and then out to the kubernetes services and then to the pods. </p>
  
  <pre><code>DNS    HostName Routing       Port routing
  LB --&gt; Ingress nodePort --&gt; Service ClusterIP --&gt; Pod
  </code></pre>
  
  <p>I don't usually use the cloud providers LB. So usually I will wildcard my dns <code>*.mycluster.com</code> and point that to nginx. In nginx I will then have a couple of configs to redirect that traffic to either the ingress controller or some of my services that are running nodePort because they are fragile. </p>
  
  <p>Hope this helps</p>
  
Answers: CC BY-SA 4.0
Answers: 2019-01-21T21:39:15.093Z
Answers: Ok, haven't seen that option the last time I checked the documentation. Thanks for the hint :) It does not yet fulfill eveything though. Without changing the whole setup of the cluster, you cannot bind NodePort services to ports 80 and 443. But maybe a combination of the DaemonSet/NET_BIND approach and your suggestion will do it. I will check woon and then report back here. Thanks so far! :)
Answers: Max N.
Answers: CC BY-SA 4.0
Answers: 2019-01-22T22:20:36.953Z
Answers: @razr I have updated my answer with a reply to your comment.
Answers: Levi
Answers: CC BY-SA 4.0
Answers: 2019-01-22T22:46:13.443Z
Answers: Hm. The LB approach does not apply, since I do not run in the cloud. I'm not sure if I understand the nginx setup correctly, but if I need to setup another service to do this, I can also do the _second ingress controller_ approach, I explained above...
Answers: Max N.
Answers: CC BY-SA 4.0
Answers: 2019-01-22T22:47:40.287Z
Answers: On the 80 and 443 are not allowed: Traefik docu says something else there: DaemonSets can be run with the NET_BIND_SERVICE capability, which will allow it to bind to port 80/443/etc on each host. This will allow bypassing the kube-proxy, and reduce traffic hops. Note that this is against the Kubernetes Best Practices Guidelines, and raises the potential for scheduling/scaling issues. Despite potential issues, this remains the choice for most ingress controllers. (see link in original post)
Answers: Max N.
Answers: CC BY-SA 4.0
Answers: 2019-01-23T14:59:25.777Z
Answers: @razr The LB approach just allows you to wild card your DNS to that LB (nginx) and then reverse proxy on hostname to your cluster's IP's and the port's that are running the nodePorts service for your ingress controller. 
  
  Thats a nice little tid bit about dameonsets, I use the helm chart and the default is the deployment method.
Answers: Levi
Answers: CC BY-SA 4.0
Answers: 2019-01-21T15:47:29.297Z
Answers: 2019-01-22T22:19:21.470Z
Answers: 2019-01-22T22:19:21.470Z
Answers: Levi
Body: <p>I have a k8s cluster (three vms on my own hardware; no aws, google cloud, ...) that uses <em>traefik</em> (<a href="https://traefik.io/" rel="noreferrer">https://traefik.io/</a>) as a reverse proxy to address services/deployments in the background.</p>
  
  <p>For this I use the deployment-variant from this part of the documentation: <a href="https://docs.traefik.io/user-guide/kubernetes/#deploy-trfik-using-a-deployment-or-daemonset" rel="noreferrer">https://docs.traefik.io/user-guide/kubernetes/#deploy-trfik-using-a-deployment-or-daemonset</a></p>
  
  <p>Now I have multiple applications deployed in the cluster, which all have some ingresses assigned, which are read by the <em>traefik-ingress-controller</em>. Some of those applications are internal ones, like <em>kibana</em> or the <em>traefik-web-ui</em>, and some others are external, like the applications themselves. I distinguish those two by having different dns entries (like <code>https://dashboard.internal.mycoolapp.com</code> and <code>https://app1.external.mycoolapp.com</code>) and the internal dns is not resolved from the outside world (=the internet) whereas the external is (like from google dns).</p>
  
  <p>That's for the setup. Now, let's come to the problem:</p>
  
  <p>A couple of days ago, I thought: What happens, if I create a wildcard dns entry for <code>*.internal.mycoolapp.com</code> on a machine, that is outside my network, and just resolve it to the same ip(s) the external dns entry resolves to. Et voila, my internal services are accessible from the outside!</p>
  
  <p>So this is, of course, a state which is not acceptable. So I'm searching for solutions on this.</p>
  
  <p>What first came to mind was to block all incoming requests on rules for internal services, if the host ip of the requester is outside of our network:</p>
  
  <pre><code>...
  kind: Ingress
  metadata:
    name: app1
    namespace: default
    annotations:
      traefik.ingress.kubernetes.io/whitelist-source-range: "10.0.0.0/8"
      ingress.kubernetes.io/whitelist-x-forwarded-for: "true"
  ...
  </code></pre>
  
  <p>Theoretically this should work. But as I found out later on, before reaching the <em>traefik-ingress-controller</em>, all requests are handled by <em>kube-proxy</em> and their host addresses are translated to local addresses (<em>(S)NAT</em>), so every request has an internal host address set.</p>
  
  <p>So this is the point I am currently looking for a solution.</p>
  
  <p>One solution supposedly is to deploy the <em>traefik-ingress-controller</em> not as a deployment, but as a daemon set and bind it to the ports on the host directly (as said here <a href="https://docs.traefik.io/user-guide/kubernetes/#deploy-trfik-using-a-deployment-or-daemonset" rel="noreferrer">https://docs.traefik.io/user-guide/kubernetes/#deploy-trfik-using-a-deployment-or-daemonset</a>). I tried that yesterday by just changing my traefik configuration to a daemon set and add the <code>NET_BIND_SERVICE</code> capability to it, but it didn't really change anything there. So has someone any idea what I might have done wrong there? Or has someone a good how-to/tutorial/... on how to pass the actual requester host through to the ingress controller?</p>
  
  <p>Here's my current configuration file for traefik:</p>
  
  <pre><code>---
  kind: Deployment
  apiVersion: extensions/v1beta1
  metadata:
    name: traefik-ingress-controller
    namespace: kube-system
    labels:
      k8s-app: traefik-ingress-lb
  spec:
    replicas: 1
    selector:
      matchLabels:
        k8s-app: traefik-ingress-lb
    template:
      metadata:
        labels:
          k8s-app: traefik-ingress-lb
          name: traefik-ingress-lb
      spec:
        serviceAccountName: traefik-ingress-controller
        terminationGracePeriodSeconds: 60
        containers:
        - image: traefik:v1.6.4
          name: traefik-ingress-lb
          volumeMounts:
          - mountPath: /ssl/external
            name: ssl-external
          - mountPath: /ssl/internal
            name: ssl-internal
          - name: traefik-toml
            subPath: traefik.toml
            mountPath: /config/traefik.toml
          ports:
          - name: http
            containerPort: 80
          - name: https
            containerPort: 443
          - name: admin
            containerPort: 8080
          args:
          - --configfile=/config/traefik.toml
          - --api
          - --kubernetes
          - --logLevel=INFO
        volumes:
        - name: ssl-external
          secret:
            secretName: external.mycoolapp.com.cert
        - name: ssl-internal
          secret:
            secretName: internal.mycoolapp.com.cert
        - name: traefik-toml
          configMap:
            name: traefik-toml
  ---
  kind: Service
  apiVersion: v1
  metadata:
    name: traefik-ingress-service
    namespace: kube-system
  spec:
    selector:
      k8s-app: traefik-ingress-lb
    ports:
    - protocol: TCP
      port: 80
      name: web
    - protocol: TCP
      port: 443
      name: sweb
    externalIPs:
    - 10.2.3.1
    - 10.2.3.2
    - 10.2.3.3
  </code></pre>
  
  <p>The toml-file just contains http-to-https redirect and the paths to the certificate files.</p>
  
  <p>I have some other solutions in mind, but they do all have the downside that they need more <em>things</em> deployed and maintained, like a second ingress controller on another port, that handles only specific ingress objects, or a new entrypoint for external requests in front of the whole cluster, that deletes host headers from requests, when it is not my external address (does this even work with ssl?).</p>
  
  <p>So I really need some help on this. Thanks in advance!</p>
  
Tags: <docker><kubernetes><rproxy>
Title: Handling request based on source ip with traefik in k8s
UserDisplayName: Max N.
ThreadText: # Handling request based on source ip with traefik in k8s
  
  (Asked by: username_0 on 2018-07-04)
  
  I have a k8s cluster (three vms on my own hardware; no aws, google cloud, ...) that uses *traefik* (<https://traefik.io/>) as a reverse proxy to address services/deployments in the background.
  
  For this I use the deployment-variant from this part of the documentation: <https://docs.traefik.io/user-guide/kubernetes/#deploy-trfik-using-a-deployment-or-daemonset>
  
  Now I have multiple applications deployed in the cluster, which all have some ingresses assigned, which are read by the *traefik-ingress-controller*. Some of those applications are internal ones, like *kibana* or the *traefik-web-ui*, and some others are external, like the applications themselves. I distinguish those two by having different dns entries (like `https://dashboard.internal.mycoolapp.com` and `https://app1.external.mycoolapp.com`) and the internal dns is not resolved from the outside world (=the internet) whereas the external is (like from google dns).
  
  That's for the setup. Now, let's come to the problem:
  
  A couple of days ago, I thought: What happens, if I create a wildcard dns entry for `*.internal.mycoolapp.com` on a machine, that is outside my network, and just resolve it to the same ip(s) the external dns entry resolves to. Et voila, my internal services are accessible from the outside!
  
  So this is, of course, a state which is not acceptable. So I'm searching for solutions on this.
  
  What first came to mind was to block all incoming requests on rules for internal services, if the host ip of the requester is outside of our network:
  
  ```
  ...
  kind: Ingress
  metadata:
    name: app1
    namespace: default
    annotations:
      traefik.ingress.kubernetes.io/whitelist-source-range: "10.0.0.0/8"
      ingress.kubernetes.io/whitelist-x-forwarded-for: "true"
  ...
  
  ```
  
  Theoretically this should work. But as I found out later on, before reaching the *traefik-ingress-controller*, all requests are handled by *kube-proxy* and their host addresses are translated to local addresses (*(S)NAT*), so every request has an internal host address set.
  
  So this is the point I am currently looking for a solution.
  
  One solution supposedly is to deploy the *traefik-ingress-controller* not as a deployment, but as a daemon set and bind it to the ports on the host directly (as said here <https://docs.traefik.io/user-guide/kubernetes/#deploy-trfik-using-a-deployment-or-daemonset>). I tried that yesterday by just changing my traefik configuration to a daemon set and add the `NET_BIND_SERVICE` capability to it, but it didn't really change anything there. So has someone any idea what I might have done wrong there? Or has someone a good how-to/tutorial/... on how to pass the actual requester host through to the ingress controller?
  
  Here's my current configuration file for traefik:
  
  ```
  ---
  kind: Deployment
  apiVersion: extensions/v1beta1
  metadata:
    name: traefik-ingress-controller
    namespace: kube-system
    labels:
      k8s-app: traefik-ingress-lb
  spec:
    replicas: 1
    selector:
      matchLabels:
        k8s-app: traefik-ingress-lb
    template:
      metadata:
        labels:
          k8s-app: traefik-ingress-lb
          name: traefik-ingress-lb
      spec:
        serviceAccountName: traefik-ingress-controller
        terminationGracePeriodSeconds: 60
        containers:
        - image: traefik:v1.6.4
          name: traefik-ingress-lb
          volumeMounts:
          - mountPath: /ssl/external
            name: ssl-external
          - mountPath: /ssl/internal
            name: ssl-internal
          - name: traefik-toml
            subPath: traefik.toml
            mountPath: /config/traefik.toml
          ports:
          - name: http
            containerPort: 80
          - name: https
            containerPort: 443
          - name: admin
            containerPort: 8080
          args:
          - --configfile=/config/traefik.toml
          - --api
          - --kubernetes
          - --logLevel=INFO
        volumes:
        - name: ssl-external
          secret:
            secretName: external.mycoolapp.com.cert
        - name: ssl-internal
          secret:
            secretName: internal.mycoolapp.com.cert
        - name: traefik-toml
          configMap:
            name: traefik-toml
  ---
  kind: Service
  apiVersion: v1
  metadata:
    name: traefik-ingress-service
    namespace: kube-system
  spec:
    selector:
      k8s-app: traefik-ingress-lb
    ports:
    - protocol: TCP
      port: 80
      name: web
    - protocol: TCP
      port: 443
      name: sweb
    externalIPs:
    - 10.2.3.1
    - 10.2.3.2
    - 10.2.3.3
  
  ```
  
  The toml-file just contains http-to-https redirect and the paths to the certificate files.
  
  I have some other solutions in mind, but they do all have the downside that they need more *things* deployed and maintained, like a second ingress controller on another port, that handles only specific ingress objects, or a new entrypoint for external requests in front of the whole cluster, that deletes host headers from requests, when it is not my external address (does this even work with ssl?).
  
  So I really need some help on this. Thanks in advance!
  
  **Tags:** docker, kubernetes, rproxy
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2019-01-21. Score: 1
  
  I think your ingress rule is fine but I think your Traefik service needs to be changed. Check out this article <https://kubernetes.io/docs/tutorials/services/source-ip/>
  
  I think what you are looking for is `nodePort` with `externalTrafficPolicy` set to `local`.
  
  This should allow Traefik to enforce `whitelist-source-range` annotation on the ingress.
  
  EDIT:
  
  I was going to go into more detail but it was so old I thought you might have already found a solution. Ingress controllers are not allowed to bind to 80 and 443 but most organizations are using the load balancer service to load balance traffic to the Ingress controller and then out to the kubernetes services and then to the pods.
  
  ```
  DNS    HostName Routing       Port routing
  LB --> Ingress nodePort --> Service ClusterIP --> Pod
  
  ```
  
  I don't usually use the cloud providers LB. So usually I will wildcard my dns `*.mycluster.com` and point that to nginx. In nginx I will then have a couple of configs to redirect that traffic to either the ingress controller or some of my services that are running nodePort because they are fragile.
  
  Hope this helps
  
  > **Comments:**
  > **username_0**: Ok, haven't seen that option the last time I checked the documentation. Thanks for the hint :) It does not yet fulfill eveything though. Without changing the whole setup of the cluster, you cannot bind NodePort services to ports 80 and 443. But maybe a combination of the DaemonSet/NET_BIND approach and your suggestion will do it. I will check woon and then report back here. Thanks so far! :)  
  > **username_1**: @razr I have updated my answer with a reply to your comment.  
  > **username_0**: Hm. The LB approach does not apply, since I do not run in the cloud. I'm not sure if I understand the nginx setup correctly, but if I need to setup another service to do this, I can also do the _second ingress controller_ approach, I explained above...  
  > **username_0**: On the 80 and 443 are not allowed: Traefik docu says something else there: DaemonSets can be run with the NET_BIND_SERVICE capability, which will allow it to bind to port 80/443/etc on each host. This will allow bypassing the kube-proxy, and reduce traffic hops. Note that this is against the Kubernetes Best Practices Guidelines, and raises the potential for scheduling/scaling issues. Despite potential issues, this remains the choice for most ingress controllers. (see link in original post)  
  > **username_1**: @razr The LB approach just allows you to wild card your DNS to that LB (nginx) and then reverse proxy on hostname to your cluster's IP's and the port's that are running the nodePorts service for your ingress controller.   Thats a nice little tid bit about dameonsets, I use the helm chart and the default is the deployment method.  
  
  ---
  
  