Body: <p>I'm seeing persistent issues when running larger mailings using Mosaico/Flexmailer for one site. Small mailings will process without any issues. But larger mailings (e.g. 25k+) will terminate in the process_mailings job. The specific point of failure is in /src/FlexMailer.php - fireWalkBatches(). The $event object appears to be constructed properly, but it dies on the dispatch call.</p>
  
  <p>It's been challenging to troubleshoot as there are no errors in the Civi log or apache logs. It simply dies at that point.</p>
  
  <p>Anyone else see similar issues or have suggestions for troubleshooting?</p>
  
  <hr>
  
  <p>Additional info --</p>
  
  <p>The issue only occurs when triggering jobs via the cli (this site is running Drupal and triggering cli.php). If I trigger process_mailings via the URL/cron.php, including triggering it with wget from a cron, it processes correctly.</p>
  
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2018-04-17T19:20:48.620Z
Comments.Text: I think we have had recent problems that we have not been able to reproduce/debug. Let me ask @edsel to provide details on whether your symptoms are consistent with ours.
Comments.UserDisplayName: Joe Murray
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2018-04-18T20:28:13.423Z
Comments.Text: Who is the cron user? Often there are issues with triggering mail not as www-data.
Comments.UserDisplayName: Tommy B
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2018-04-20T23:01:49.387Z
Comments.Text: In addition to the point about checking users/permissions, I'd also look for differences in the `php.ini` used by the web and CLI processes. For example, if the call-stack got really tall, and if xdebug was only enabled on CLI, you could see a weird crash in CLI.
Comments.UserDisplayName: Tim Otten
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2018-04-21T06:54:06.973Z
Comments.Text: Did a trial-run trying to replicate. Report: https://github.com/civicrm/org.civicrm.flexmailer/issues/19
Comments.UserDisplayName: Tim Otten
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2018-04-22T20:48:26.300Z
Comments.Text: We have fairly big mailings (>200k) and it works fine. We are using drush to run the cronjob, that might, or might not be the difference?
Comments.UserDisplayName: Xavier
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2018-04-30T20:54:29.390Z
Comments.Text: We're seeing the same issue as this i.e. works in wget but not from cli. Are you using the mandril plugin? We're leaning towards issues with fleximailer and the mandrill extension not playing nicely.
Comments.UserDisplayName: Parvez Saleh
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2018-04-30T22:06:36.227Z
Comments.Text: (1) Another possible difference: by default, Flexmailer abdicates on traditional mailings but takes control of Mosaico (& any non-traditional) mailings. (2) Batch-limits/memory-limits may be more important as discussed in https://github.com/veda-consulting/uk.co.vedaconsulting.mosaico/issues/167#issuecomment-385416790
Comments.UserDisplayName: Tim Otten
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2018-05-01T02:19:50.150Z
Comments.Text: It's not a cron user/perms issue -- I run the cron under the web user. xdebug is not enabled and php.ini are comparable except in the areas where you'd expect the CLI to run better (i.e. max_execution_time = 0). We're not using the Mandril plugin. The issue is exclusively Mosaico/Flexmailer mailings. The last time this happened I converted the mailing content into a standard Civi email, disabled Mosaico/Flexmailer and ran it using the standard setup without issue. I've not had a chance to debug much more.
Comments.UserDisplayName: lcdservices
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2018-05-01T02:22:38.240Z
Comments.Text: It's working with wget/cron.php and is only replicated with large mailings, so it's difficult to troubleshoot without impacting/holding up mailings that need to get out. As indicated above, the point of failure is fireWalkBatches() when dispatch is called. I'm finding it difficult to troubleshoot beyond that point -- if you have suggestions, let me know.
Comments.UserDisplayName: lcdservices
Tags: <flexmailer>
Title: flexmailer terminating without processing mail
ThreadText: # flexmailer terminating without processing mail
  
  (Asked by: username_0 on 2018-04-17)
  
  I'm seeing persistent issues when running larger mailings using Mosaico/Flexmailer for one site. Small mailings will process without any issues. But larger mailings (e.g. 25k+) will terminate in the process\_mailings job. The specific point of failure is in /src/FlexMailer.php - fireWalkBatches(). The $event object appears to be constructed properly, but it dies on the dispatch call.
  
  It's been challenging to troubleshoot as there are no errors in the Civi log or apache logs. It simply dies at that point.
  
  Anyone else see similar issues or have suggestions for troubleshooting?
  
  ---
  
  Additional info --
  
  The issue only occurs when triggering jobs via the cli (this site is running Drupal and triggering cli.php). If I trigger process\_mailings via the URL/cron.php, including triggering it with wget from a cron, it processes correctly.
  
  **Tags:** flexmailer
  
  > **Comments:**
  > **username_1**: I think we have had recent problems that we have not been able to reproduce/debug. Let me ask @edsel to provide details on whether your symptoms are consistent with ours.  
  > **username_2**: Who is the cron user? Often there are issues with triggering mail not as www-data.  
  > **username_3**: In addition to the point about checking users/permissions, I'd also look for differences in the `php.ini` used by the web and CLI processes. For example, if the call-stack got really tall, and if xdebug was only enabled on CLI, you could see a weird crash in CLI.  
  > **username_3**: Did a trial-run trying to replicate. Report: https://github.com/civicrm/org.civicrm.flexmailer/issues/19  
  > **username_4**: We have fairly big mailings (>200k) and it works fine. We are using drush to run the cronjob, that might, or might not be the difference?  
  > **username_5**: We're seeing the same issue as this i.e. works in wget but not from cli. Are you using the mandril plugin? We're leaning towards issues with fleximailer and the mandrill extension not playing nicely.  
  > **username_3**: (1) Another possible difference: by default, Flexmailer abdicates on traditional mailings but takes control of Mosaico (& any non-traditional) mailings. (2) Batch-limits/memory-limits may be more important as discussed in https://github.com/veda-consulting/uk.co.vedaconsulting.mosaico/issues/167#issuecomment-385416790  
  > **username_0**: It's not a cron user/perms issue -- I run the cron under the web user. xdebug is not enabled and php.ini are comparable except in the areas where you'd expect the CLI to run better (i.e. max_execution_time = 0). We're not using the Mandril plugin. The issue is exclusively Mosaico/Flexmailer mailings. The last time this happened I converted the mailing content into a standard Civi email, disabled Mosaico/Flexmailer and ran it using the standard setup without issue. I've not had a chance to debug much more.  
  > **username_0**: It's working with wget/cron.php and is only replicated with large mailings, so it's difficult to troubleshoot without impacting/holding up mailings that need to get out. As indicated above, the point of failure is fireWalkBatches() when dispatch is called. I'm finding it difficult to troubleshoot beyond that point -- if you have suggestions, let me know.  
  
  ---
  
  ## 0 answers
  
  