title: Limiting the number of concurrent jobs on Azure Functions queue
body: <p>I have a Function app in Azure that is triggered when an item is put on a queue.  It looks something like this (greatly simplified):</p>
  
  <pre class="lang-cs prettyprint-override"><code>public static async Task Run(string myQueueItem, TraceWriter log)
  {
      using (var client = new HttpClient())
      {
          client.BaseAddress = new Uri(Config.APIUri);
          client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
  
          StringContent httpContent = new StringContent(myQueueItem, Encoding.UTF8, "application/json");
          HttpResponseMessage response = await client.PostAsync("/api/devices/data", httpContent);
          response.EnsureSuccessStatusCode();
  
          string json = await response.Content.ReadAsStringAsync();
          ApiResponse apiResponse = JsonConvert.DeserializeObject&lt;ApiResponse&gt;(json);
  
          log.Info($"Activity data successfully sent to platform in {apiResponse.elapsed}ms.  Tracking number: {apiResponse.tracking}");
      }
  }
  </code></pre>
  
  <p>This all works great and runs pretty well.  Every time an item is put on the queue, we send the data to some API on our side and log the response.  Cool.</p>
  
  <p>The problem happens when there's a big spike in "the thing that generates queue messages" and a lot of items are put on the queue at once.  This tends to happen around 1,000 - 1,500 items in a minute.  The error log will have something like this:</p>
  
  <blockquote>
    <p>2017-02-14T01:45:31.692 mscorlib: Exception while executing function:
    Functions.SendToLimeade. f-SendToLimeade__-1078179529: An error
    occurred while sending the request. System: Unable to connect to the
    remote server. System: Only one usage of each socket address
    (protocol/network address/port) is normally permitted
    123.123.123.123:443.</p>
  </blockquote>
  
  <p>At first, I thought this was an issue with the Azure Function app running out of local sockets, as <a href="https://blogs.msdn.microsoft.com/dgorti/2005/09/18/only-one-usage-of-each-socket-address-protocolnetwork-addressport-is-normally-permitted/" rel="nofollow noreferrer">illustrated here</a>.  However, then I noticed the IP address.  The IP address 123.123.123.123 (of course changed for this example) is our IP address, the one that the HttpClient is posting to.  So, now I'm wondering if it is <em>our</em> servers running out of sockets to handle these requests.</p>
  
  <p>Either way, we have a scaling issue going on here.  I'm trying to figure out the best way to solve it.</p>
  
  <p><strong>Some ideas:</strong></p>
  
  <ol>
  <li>If it's a local socket limitation, the <a href="https://blogs.msdn.microsoft.com/dgorti/2005/09/18/only-one-usage-of-each-socket-address-protocolnetwork-addressport-is-normally-permitted/" rel="nofollow noreferrer">article above</a> has an example of increasing the local port range using <code>Req.ServicePoint.BindIPEndPointDelegate</code>.  This seems promising, but what do you do when you <em>truly</em> need to scale?  I don't want this problem coming back in 2 years.</li>
  <li>If it's a remote limitation, it looks like I can control how many messages the Functions runtime will process at once.  There's an interesting article here that says you can set <code>serviceBus.maxConcurrentCalls</code> to 1 and only a single message will be processed at once.  Maybe I could set this to a relatively low number.  Now, at some point our queue will be filling up faster than we can process them, but at that point the answer is adding more servers on our end.</li>
  <li>Multiple Azure Functions apps?  What happens if I have more than one Azure Functions app and they all trigger on the same queue?  Is Azure smart enough to divvy up the work among the Function apps and I could have an army of machines processing my queue, which could be scaled up or down as needed?</li>
  <li>I've also come across keep-alives.  It seems to me if I could somehow keep my socket open as queue messages were flooding in, it could perhaps help greatly.  Is this possible, and any tips on how I'd go about doing this?</li>
  </ol>
  
  <p>Any insight on a recommended (scalable!) design for this sort of system would be greatly appreciated!</p>