Answers.Body: <p>There are a couple of points to this.</p>
  
  <p>1 - SMTP is chatty, it is a fairly slow protocol. To combat this, run several mailing jobs in parallel. This can be achieved by calling the process_mailing cron job multiple more frequently. Ensure that the 'cron job limit' is equal to the maximum number mailing jobs you would want to be active at any one time. This is useful if you have a sending limit on your SMTP server of x per second.</p>
  
  <p>2 - The issue with 10 emails sent 'on the same second' and then a delay: MailingJob.php processes 10 emails at once, and then writes the transactions to the database all at once. So it looks like the emails were all sent on the same second. The '10' is defined in BULK_MAIL_INSERT_COUNT in CRM_Mailing_Config. Ideally the actual time of sending would be recorded in the database, rather than the time of writing.</p>
  
  <p>To speed up mailings we now call the process_mailing job independently every 5 minutes, and limit the number of emails that are sent on a cron run. This means we have several cron jobs running sending emails at once for large mailings, which greatly improves our send rate.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2016-01-20T08:48:12.143Z
Answers.LastActivityDate: 2016-04-13T08:21:29.187Z
Answers.LastEditDate: 2016-04-13T08:21:29.187Z
Answers.UserDisplayName: John
Answers.Body: <p>when civimail uses smtp to deliver, it doesn't create a new connection to the smtp server for every contact, but pipeline them. this might explain the bursts pattern?</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2016-01-19T13:39:41.950Z
Answers.Text: Ah I need to update this! The 'bursts' are actually a side effect of CiviCRM's coding... We've found out quite a lot since asking this question.
Answers.UserDisplayName: John
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2016-01-19T13:38:25.837Z
Answers.LastActivityDate: 2016-01-19T13:38:25.837Z
Answers.UserDisplayName: Xavier
Body: <p>We're currently investigating why our mailings are sending slowly from CiviCRM. We use an external SMTP server.</p>
  
  <p>Our mailing settings:</p>
  
  <ul>
  <li>Mailer batch limit: 30000</li>
  <li>Mailer throttle time: 2100</li>
  <li>Mailer job size: 0</li>
  <li>Cron job limit: 1</li>
  </ul>
  
  <p>However, when we look at the delivery table, the times individual emails were delivered are a bit strange:</p>
  
  <ul>
  <li>10 emails delivered all on the same second</li>
  <li>A delay of 2-3 seconds</li>
  <li>Another 10 emails delivered on the same second</li>
  <li>A delay of 2-3 seconds</li>
  <li>Rinse, repeat</li>
  </ul>
  
  <p>Has anyone run into a similar issue?</p>
  
  <p>Update: After some testing, this only happens for SMTP (ie, not mail()), and it is happening with two separate SMTP providers. Investigations continue...</p>
  
  <p>Update: Emails also send slowly from a standalone script which uses the PHPMailer library on the same server. That actually sends slower, at 2/3 emails per second but without a delay inbetween. The next step would be for us to try concurrent sending jobs (once this is fixed in 4.6.10).</p>
  
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2015-10-26T18:09:41.410Z
Comments.Text: Any luck with your investigation? Was it a CiviCRM issue, problem at your server, or with your SMTP provider?
Comments.UserDisplayName: Sanjay Jain
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2015-10-27T10:44:48.777Z
Comments.Text: Still no luck I'm afraid Sanjay. Our server team says there are no limits on SMTP connections from the server, so that was a dead end. We'll be focusing back on this issue soon. Any thoughts appreciated in the mean time!
Comments.UserDisplayName: John
Tags: <civimail><performance><smtp>
Title: What could be causing slow mailing from CiviCRM (via SMTP)
ThreadText: # What could be causing slow mailing from CiviCRM (via SMTP)
  
  (Asked by: username_0 on 2015-10-22)
  
  We're currently investigating why our mailings are sending slowly from CiviCRM. We use an external SMTP server.
  
  Our mailing settings:
  
  * Mailer batch limit: 30000
  * Mailer throttle time: 2100
  * Mailer job size: 0
  * Cron job limit: 1
  
  However, when we look at the delivery table, the times individual emails were delivered are a bit strange:
  
  * 10 emails delivered all on the same second
  * A delay of 2-3 seconds
  * Another 10 emails delivered on the same second
  * A delay of 2-3 seconds
  * Rinse, repeat
  
  Has anyone run into a similar issue?
  
  Update: After some testing, this only happens for SMTP (ie, not mail()), and it is happening with two separate SMTP providers. Investigations continue...
  
  Update: Emails also send slowly from a standalone script which uses the PHPMailer library on the same server. That actually sends slower, at 2/3 emails per second but without a delay inbetween. The next step would be for us to try concurrent sending jobs (once this is fixed in 4.6.10).
  
  **Tags:** civimail, performance, smtp
  
  > **Comments:**
  > **username_1**: Any luck with your investigation? Was it a CiviCRM issue, problem at your server, or with your SMTP provider?  
  > **username_0**: Still no luck I'm afraid Sanjay. Our server team says there are no limits on SMTP connections from the server, so that was a dead end. We'll be focusing back on this issue soon. Any thoughts appreciated in the mean time!  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_0 on 2016-01-20. Score: 2
  
  There are a couple of points to this.
  
  1 - SMTP is chatty, it is a fairly slow protocol. To combat this, run several mailing jobs in parallel. This can be achieved by calling the process\_mailing cron job multiple more frequently. Ensure that the 'cron job limit' is equal to the maximum number mailing jobs you would want to be active at any one time. This is useful if you have a sending limit on your SMTP server of x per second.
  
  2 - The issue with 10 emails sent 'on the same second' and then a delay: MailingJob.php processes 10 emails at once, and then writes the transactions to the database all at once. So it looks like the emails were all sent on the same second. The '10' is defined in BULK\_MAIL\_INSERT\_COUNT in CRM\_Mailing\_Config. Ideally the actual time of sending would be recorded in the database, rather than the time of writing.
  
  To speed up mailings we now call the process\_mailing job independently every 5 minutes, and limit the number of emails that are sent on a cron run. This means we have several cron jobs running sending emails at once for large mailings, which greatly improves our send rate.
  
  
  ---
  
  ### Answer by username_2 on 2016-01-19. Score: 1
  
  when civimail uses smtp to deliver, it doesn't create a new connection to the smtp server for every contact, but pipeline them. this might explain the bursts pattern?
  
  > **Comments:**
  > **username_0**: Ah I need to update this! The 'bursts' are actually a side effect of CiviCRM's coding... We've found out quite a lot since asking this question.  
  
  ---
  
  