<p dir="auto">This issue to enumerate and track for 1.5 release the items needed to move the following @kubernetes/sig-scheduling features out of alpha/beta and into product:</p>
<p dir="auto">Taints and Tolerations<br>
Inter-pod Affinity and Anti-Affinity</p>
<p dir="auto">/cc <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/ConnorDoyle/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/ConnorDoyle">@ConnorDoyle</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/ingvagabund/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/ingvagabund">@ingvagabund</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/jayunit100/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/jayunit100">@jayunit100</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/davidopp/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/davidopp">@davidopp</a></p>
<p dir="auto">We should socialize the checklist of items here.</p>
<p dir="auto">Hey all, I’m looking to extend the default scheduler with a capability that I don’t think exists yet. I wanted to ask here just in case I'm duplicating work.</p>
<p dir="auto">My workload:<br>
A few services/forever pods, but a large number of terminating pods (not yet using the jobs object, but that’s the general idea). These jobs can run anywhere from 5 min to 3 days, and we pretty much always have more “requests” of the cluster than capacity (this is by design). We currently have a fairly hacky way of setting up a priority queue, where certain jobs are submitted to the cluster before others. The details of this hacky system are not that relevant for this discussion.</p>
<p dir="auto">The need:<br>
It is my task to rewrite this system to be more robust. While my use case in prod will be a more complicated and dynamic, a more or less FIFO queue is sufficient to discuss the scheduler code in question (as long as the priority of a job can be changed later, such as through an annotation on the pod).</p>
<p dir="auto">Wait, doesn't the scheduler already do this?<br>
Currently the default scheduler effectively does a for loop over all pods, which somewhat randomly selects pods to schedule in order. If it can't schedule a pod, it will move onto the next one. I don't really want to touch the logic of determining which <em>node</em> to schedule the pod on; I just want to modify which <em>pods</em> are selected to be scheduled. For my needs, the scheduler doesn't have to be lightning fast as long as it's reasonable.</p>
<p dir="auto">Gotchas<br>
Large request pods: It's possible that small pods can get blocked behind large pods in the queue. I have a straightforward way to overcome this, but that adds more complexity to this issue and I'd rather wait until later.</p>
<p dir="auto">relevant issues:<br>
Multiple schedulers (wip)<br>
<a href="https://github.com/kubernetes/kubernetes/blob/master/docs/proposals/multiple-schedulers.md">https://github.com/kubernetes/kubernetes/blob/master/docs/proposals/multiple-schedulers.md</a><br>
Dispatcher feature request<br>
<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="117595164" data-permission-text="Title is private" data-url="https://github.com/kubernetes/kubernetes/issues/17430" data-hovercard-type="pull_request" data-hovercard-url="/kubernetes/kubernetes/pull/17430/hovercard" href="https://github.com/kubernetes/kubernetes/pull/17430">#17430</a></p>
<p dir="auto">None of the above has been implemented, and frankly I'm ok with just extending my existing scheduler (and running all pods through this mechanism) until multiple schedulers are supported.</p>
<p dir="auto">relevant issue labels: area/scheduling</p>
<p dir="auto">relevant code:<br>
I believe all I have to do is implement this function differently:<br>
<a href="https://github.com/kubernetes/kubernetes/blob/master/plugin/pkg/scheduler/scheduler.go#L66">https://github.com/kubernetes/kubernetes/blob/master/plugin/pkg/scheduler/scheduler.go#L66</a><br>
Is that it or am I missing something? I am also unclear of where that actually is calculated...</p>
<p dir="auto">Can anyone give me a few pointers/suggestions and I'll code it up?<br>
Once I code it up, where should I contribute this back to? I'm sure I'm not the only one trying to schedule a ton of jobs on a cluster.</p>