Question: We are experiencing an issue with our SonicWall NSA 2400 Firewall. We have a secondary gateway over IPSec setup in the event of a failure of our main ISP (which is unfortunately common). The secondary gateway is a 4G connection through Verizon and the cost grows as data usage increases.

The firewall switches over to the secondary gateway properly, but then *sometimes* will not renegotiate and switch back to the primary when it comes back up. Hitting 'Renegotiate' on both sides seems to fix the issue, but I am wondering if there is something I am missing that may be causing it to stay on the secondary.

I don't think this has anything to do with the settings as it seems to work 50% of the time, but here they are anyway in case someone has some tips on how to ensure switching back to primary when the connection is restored.

Policy Type: Site to Site

Auth Method: IKE using Preshared Secret

IKE Phase 1 proposal:

Exchange: Main Mode

DH Group: Group 1

Encrypt: AES-256

Auth: SHA1

Lifetime: 3600 (seconds)

Phase 2 proposal:

Protocol: ESP

Encrypt: AES-256

Auth: SHA1

Lifetime: 900 (seconds)

Keep Alive is enabled, and Preempt Secondary Gateway is enabled at 120 second interval.

Answer: 
You will get a "Back-end server is at capacity" when the ELB load balancer performs its health checks and receives a "page not found" (or other simple error) due to a mis-configuration (typically with the NameVirtual host).

Try grepping the log files folder using the "ELB-HealthChecker" user agent. e.g. 

```
grep ELB-HealthChecker  /var/log/httpd/*

```

This will typically give you a 4x or 5x error which is easily fixed. e.g. Flooding, MaxClients etc is giving the problem way too much credit.

FYI Amazon: Why not show the returned response from request? Even a status code would help.
[EDIT after understanding the question better]
Not having any experience of the ELB, I still think this sounds suspiciously like the 503 error which may be thrown when Apache fronts a Tomcat and floods the connection.

The effect is that if Apache delivers more connection requests than can be processed by the backend, the backend input queues fill up until no more connections can be accepted. When that happens, the corresponding output queues of Apache start filling up. When the queues are full Apache throws a 503. It would follow that the same could happen when Apache is the backend, and the frontend delivers at such a rate as to make the queues fill up.

The (hypothetical) solution is to size the input connectors of the backend and output connectors of the frontend. This turns into a balancing act between the anticipated flooding level and the available RAM of the computers involved.

So as this happens, check your maxclients settings and monitor your busy workers in Apache (mod\_status.). Do the same if possible with whatever ELB has that corresponds to Tomcats connector backlog, maxthreads etc. In short, look at everything concerning the input queues of Apache and the output queues of ELB.

Although I fully understand it is not directly applicable, this link contains a sizing guide for the Apache connector. You would need to research the corresponding ELB queue technicalities, then do the math:
<http://www.cubrid.org/blog/dev-platform/maxclients-in-apache-and-its-effect-on-tomcat-during-full-gc/>

As observed in the commentary below, to overwhelm the Apache connector a spike in traffic is not the only possibility. If some requests are slower served than others, a higher ratio of those can also lead to the connector queues filling up. This was true in my case.

Also, when this happened to me I was baffled that I had to restart the Apache service in order to not get served 503:s again. Simply waiting out the connector flooding was not enough. I never got that figured out, but one can speculate in Apache serving from its cache perhaps?

After increasing the number of workers and the corresponding pre-fork maxclients settings (this was multithreaded Apache on Windows which has a couple of other directives for the queues if I remember correctly), the 503-problem disappeared. I actually didn't do the math, but just tweaked the values up until I could observe a wide margin to the peak consumption of the queue resources. I let it go at that.

Hope this was of some help.