Answers.Body: <p>I found the culprit. It was CiviRules. In the system, a rule configured on activity with a complicated test. For every e-mail sent, a mass mail activity is created and the activity rule fires. Disabling the rule did speed up the email job.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-09-21T14:37:42.900Z
Answers.Text: Well, we don't have that extension installed and have problems. But I do have civiextensions/nz.co.fuzion.cssinline installed, and I know that uses a lot of resources per email. I still don't know why the problem cropped up only after upgrade to 5.48.0.
Answers.UserDisplayName: TimETown
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-09-21T13:41:57.497Z
Answers.LastActivityDate: 2022-09-21T13:41:57.497Z
Answers.UserDisplayName: Kainuk - CiviCooP
Answers.Body: <p>Our problems started in April of 2022. when we upgraded to 5.48.0 from 5.46.3. That's when we had to start batching or the entire process would hang.</p>
  <p>(Embarrassed to say I have been using the Drupal Cron and so just runs 1x/hour, which I need to fix.)</p>
  <p><a href="https://i.sstatic.net/e6A7q.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/e6A7q.png" alt="our settings" /></a></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-09-16T21:25:58.077Z
Answers.LastActivityDate: 2022-09-16T21:25:58.077Z
Answers.UserDisplayName: TimETown
Answers.Body: <p>We usually get around 6000 per hour through AWS, with the following settings and a cron job running every ~5 minutes:</p>
  <p><a href="https://i.sstatic.net/G1miR.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/G1miR.png" alt="enter image description here" /></a></p>
  <p>Perhaps try a lower throttle time? I don't think the batch size should be much different between 400 and 500, but I do recall some performance improvement from reducing it in the past.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-09-16T21:23:00.930Z
Answers.Text: Our problems started in April of 2022. when we upgraded to 5.48.0 from 5.46.3. That's when we had to start batching or the entire process would hang.
Answers.UserDisplayName: TimETown
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-09-15T20:11:00.967Z
Answers.LastActivityDate: 2022-09-15T20:21:24.787Z
Answers.LastEditDate: 2022-09-15T20:21:24.787Z
Answers.UserDisplayName: Lars SG
Body: <p>Tips for troubleshooting email performance.</p>
  <p>I have a CiviCRM instance that currently sends 800 emails an hour and I have the feeling that it can do better. Some details.</p>
  <ul>
  <li>CiviCRM 5.51.3</li>
  <li>The CMS is Drupal 9.</li>
  <li>The mail is sent by SMTP to AmazonSES (The Paris data-center is used, and port 587)</li>
  </ul>
  <p>The CiviMail Settings are as follows:<a href="https://i.sstatic.net/9SXXo.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/9SXXo.png" alt="enter image description here" /></a></p>
  <p>The cron starts 4 times per hour. So, it is ready for 2000 emails.</p>
  <p>The mailing process keeps the server busy. There is a long-running PHP process that asks 25% processor time, and MySQL needs 15%.</p>
  <p>The SMTP server sometimes emits the following error.</p>
  <pre><code>(code: 451, response: 4.4.2 Timeout waiting for data from client.)]
  </code></pre>
  <p>This suggests that Amazon is fast enough and waits for CiviCRM. It also bothers me a bit, and that is why the database update rate is 1 (to prevent double emails).</p>
  <p>The sending limit for Amazon is 50 mails a second (with a maximum of, 100000 a day).</p>
  <p>Has someone tips, how to troubleshoot this? I already disabled the custom tokens, unfortunately without impact.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-09-13T22:43:38.197Z
Comments.Text: We started to have this problem several versions ago (trying to find which one) and I limited the batches to 250 (with no throttle time). Without batches, it would crash and not send at all and had to dupe email and start over. We also use AWS SES, and it used to work like a dream. (We have database Update Frequency at 10). email-smtp.us-east-1.amazonaws.com and port 587. Drupal 7.
Comments.UserDisplayName: TimETown
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-09-14T13:36:23.713Z
Comments.Text: What do you have as values for the other settings? Do you have specific settings for the scheduled mail cronjob?
Comments.UserDisplayName: Sven Schelfaut
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-09-20T09:08:52.443Z
Comments.Text: We had throttling problems with a client using SES as well. after roughly 1000 Emails we had SMTP errors that weren't parsed as errors in CiviCRM. We ended up using a relay Postfix that was able to queue mails and could adapt to SES sending rate. 
  
  I don't exactly know what the problem was. We have other clients using SES with default civi settings that send without problems.
Comments.UserDisplayName: Phil_B
Tags: <civimail><performance><amazon-ses>
Title: Tips for troubleshooting email performance
ThreadText: # Tips for troubleshooting email performance
  
  (Asked by: username_0 on 2022-09-07)
  
  Tips for troubleshooting email performance.
  I have a CiviCRM instance that currently sends 800 emails an hour and I have the feeling that it can do better. Some details.
  
  CiviCRM 5.51.3
  The CMS is Drupal 9.
  The mail is sent by SMTP to AmazonSES (The Paris data-center is used, and port 587)
  
  The CiviMail Settings are as follows:
  The cron starts 4 times per hour. So, it is ready for 2000 emails.
  The mailing process keeps the server busy. There is a long-running PHP process that asks 25% processor time, and MySQL needs 15%.
  The SMTP server sometimes emits the following error.
  (code: 451, response: 4.4.2 Timeout waiting for data from client.)]
  
  This suggests that Amazon is fast enough and waits for CiviCRM. It also bothers me a bit, and that is why the database update rate is 1 (to prevent double emails).
  The sending limit for Amazon is 50 mails a second (with a maximum of, 100000 a day).
  Has someone tips, how to troubleshoot this? I already disabled the custom tokens, unfortunately without impact.
  
  **Tags:** civimail, performance, amazon-ses
  
  > **Comments:**
  > **username_1**: We started to have this problem several versions ago (trying to find which one) and I limited the batches to 250 (with no throttle time). Without batches, it would crash and not send at all and had to dupe email and start over. We also use AWS SES, and it used to work like a dream. (We have database Update Frequency at 10). email-smtp.us-east-1.amazonaws.com and port 587. Drupal 7.  
  > **username_2**: What do you have as values for the other settings? Do you have specific settings for the scheduled mail cronjob?  
  > **username_3**: We had throttling problems with a client using SES as well. after roughly 1000 Emails we had SMTP errors that weren't parsed as errors in CiviCRM. We ended up using a relay Postfix that was able to queue mails and could adapt to SES sending rate.   I don't exactly know what the problem was. We have other clients using SES with default civi settings that send without problems.  
  
  ---
  
  ## 3 answers
  
  ### Answer by username_0 on 2022-09-21. Score: 2
  
  I found the culprit. It was CiviRules. In the system, a rule configured on activity with a complicated test. For every e-mail sent, a mass mail activity is created and the activity rule fires. Disabling the rule did speed up the email job.
  
  > **Comments:**
  > **username_1**: Well, we don't have that extension installed and have problems. But I do have civiextensions/nz.co.fuzion.cssinline installed, and I know that uses a lot of resources per email. I still don't know why the problem cropped up only after upgrade to 5.48.0.  
  
  ---
  
  ### Answer by username_1 on 2022-09-16. Score: 0
  
  Our problems started in April of 2022. when we upgraded to 5.48.0 from 5.46.3. That's when we had to start batching or the entire process would hang.
  
  (Embarrassed to say I have been using the Drupal Cron and so just runs 1x/hour, which I need to fix.)
  
  [![our settings](https://i.sstatic.net/e6A7q.png)](https://i.sstatic.net/e6A7q.png)
  
  
  ---
  
  ### Answer by username_4 on 2022-09-15. Score: 0
  
  We usually get around 6000 per hour through AWS, with the following settings and a cron job running every ~5 minutes:
  
  [![enter image description here](https://i.sstatic.net/G1miR.png)](https://i.sstatic.net/G1miR.png)
  
  Perhaps try a lower throttle time? I don't think the batch size should be much different between 400 and 500, but I do recall some performance improvement from reducing it in the past.
  
  > **Comments:**
  > **username_1**: Our problems started in April of 2022. when we upgraded to 5.48.0 from 5.46.3. That's when we had to start batching or the entire process would hang.  
  
  ---
  
  