Answers.Body: <p>The error message comes from refreshing the browser rather than the upgrade process.
  The last entry before the error is:</p>
  
  <pre><code>Dec 21 12:20:56  [info] Running task: Upgrade mailing foreign key constraints
  </code></pre>
  
  <p>You can find the detail in <code>CRM/Upgrade/Incremental/php/FourSeven.php</code>
  Adding foreign key constraints creates an index which can take a while on a big table.  How many rows (count(*)) do your <code>civicrm_mailing_event_queue</code> and <code>civicrm_mailing_recipients</code> tables have?</p>
  
  <p>Once the upgrade has started, I don't know how dependent it is on the browser, so it would be worth checking the log later to see if you have any further messages from the upgrade.</p>
  
  <p>If you need to start again, then I'd try this:</p>
  
  <ol>
  <li>Restore your database and files back to your starting point at 4.3.8 (you did take full backups didn't you??)</li>
  <li>Upgrade to 4.7.7</li>
  <li>Check all is working as expected</li>
  <li>Take new backups</li>
  <li>Try upgrading to 4.7.8</li>
  </ol>
  
  <p>If your mailing tables are large you'll need to be patient. If it fails, you can easily go back to 4.7.7 and then work out what to do next.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2016-12-22T18:51:07.157Z
Answers.Text: civicrm_mailing_event_queue: 2230662; civicrm_mailing_recipients: 3025260;
Answers.UserDisplayName: Christia
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2016-12-22T18:55:13.217Z
Answers.Text: Then you'll need a very large cup of coffee.... If you don't need all that history, might be worth posting a separate question about ways to trim those tables.
Answers.UserDisplayName: Aidan
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-11-22T22:01:33.443Z
Answers.Text: Just to second this approach. We had over a million records in both the civicrm_mailing_event_queue and civicrm_mailing_recipients tables.
  
  I backed them both up, truncated them, carried out the upgrades, then re-imported the data. 
  
  Seemed to do the job!
Answers.UserDisplayName: Jon-man
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2016-12-22T11:50:39.830Z
Answers.LastActivityDate: 2016-12-22T11:50:39.830Z
Answers.UserDisplayName: Aidan
Body: <p>My upgrade has been running for about 8 hours, but it stalled after completing 4.7.7, from 4.3.8 at. I checked <code>top</code> to see if Drupal or CiviCRM were running, and there was no indication of the process. I decided to refresh my browser and my cookie must have expired, because Drupal asked me to log in again. Then I pulled up the queue runner in the browser (<code>https://dev.example.com/civicrm/upgrade/queue/runner</code>) it spit out the error:</p>
  
  <blockquote>
    <p>Sorry but we are not able to provide this at the moment.
    Could not find valid value for qrid</p>
  </blockquote>
  
  <p>The log shows that it could not find valid value for qrid. Below is the log from <code>/sites/files/civicrm/ConfigAndLog</code>, starting after completing upgrade to 4.7.6:</p>
  
  <pre><code>Dec 21 12:18:17  [info] Running task: Begin Upgrade to 4.7.7
  Dec 21 12:18:19  [info] Running task: Upgrade DB to 4.7.7
  Dec 21 12:18:21  [info] Running task: Upgrade DB to 4.7.7: SQL
  Dec 21 12:18:29  [info] Running task: Finish Upgrade DB to 4.7.7
  Dec 21 12:20:15  [info] Running task: Begin Upgrade to 4.7.8
  Dec 21 12:20:17  [info] Running task: Upgrade DB to 4.7.8
  Dec 21 12:20:23  [info] Running task: Upgrade DB to 4.7.8: SQL
  Dec 21 12:20:56  [info] Running task: Upgrade mailing foreign key constraints
  Dec 21 13:26:27  [info] $Fatal Error Details = Array
  
  (
      [message] =&gt; Could not find valid value for qrid
      [code] =&gt; 
  )
  
  Dec 21 13:26:27  [info] $backTrace = #0 /var/www/html/drupal/docroot/sites/all/modules/civicrm/CRM/Core/Error.php(373): CRM_Core_Error::backtrace("backTrace", TRUE)
  #1 /var/www/html/drupal/docroot/sites/all/modules/civicrm/CRM/Utils/Request.php(120): CRM_Core_Error::fatal("Could not find valid value for qrid")
  #2 /var/www/html/drupal/docroot/sites/all/modules/civicrm/CRM/Queue/Page/Runner.php(52): CRM_Utils_Request::retrieve("qrid", "String", Object(CRM_Queue_Page_Runner), TRUE)
  #3 /var/www/html/drupal/docroot/sites/all/modules/civicrm/CRM/Core/Invoke.php(312): CRM_Queue_Page_Runner-&gt;run((Array:4), NULL)
  #4 /var/www/html/drupal/docroot/sites/all/modules/civicrm/CRM/Core/Invoke.php(86): CRM_Core_Invoke::runItem((Array:13))
  #5 /var/www/html/drupal/docroot/sites/all/modules/civicrm/CRM/Core/Invoke.php(54): CRM_Core_Invoke::_invoke((Array:4))
  #6 /var/www/html/drupal/docroot/sites/all/modules/civicrm/drupal/civicrm.module(448): CRM_Core_Invoke::invoke((Array:4))
  #7 [internal function](): civicrm_invoke("upgrade", "queue", "runner")
  #8 /var/www/html/drupal/docroot/includes/menu.inc(527): call_user_func_array("civicrm_invoke", (Array:3))
  #9 /var/www/html/drupal/docroot/index.php(21): menu_execute_active_handler()
  #10 {main}
  </code></pre>
  
  <p>I suspect that the qrid is the queue runner ID and that the queue was lost when Drupal asked me to log in again. The log doesn't show that there was any other issues before the qrid problem. I'm not certain if something was wrong before I was forced to log in again or if that is what did it it.</p>
  
  <p>Furthermore, the file permissions for the civicrm directory in sites/all/modules was somehow changed to an owner I am not familiar with on our server (test2?):</p>
  
  <pre><code>    drwxr-xr-x 19 test2 csf      4096 Nov  2 19:32 civicrm
  drwxr-sr-- 80 fclub www-data 4096 Oct 16  2015 contrib
  drwxr-sr-- 15 fclub www-data 4096 Oct 16  2015 custom
  -rw-r--r--  1 fclub www-data  952 Aug  5  2015 README.txt
  drwxr-sr-x  2 fclub www-data 4096 Nov  3 22:40 timeago
  </code></pre>
  
  <p>I tried loading this URL to see if that would help:</p>
  
  <pre><code>civicrm/upgrade/queue/runner?reset=1&amp;qrid=‌​CRM_Upgrade
  </code></pre>
  
  <p>but then received the error:</p>
  
  <blockquote>
    <p>Queue runner must be configured before execution.</p>
  </blockquote>
  
  <p>I should have used Drush to run the upgrade so I wouldn't have to deal with Browser issues.</p>
  
  <p><strong>Question 1:</strong> Is there anyway to salvage the progress and continue from 4.7.7, repair the database or reinstate the queue runner?</p>
  
  <p><strong>Question 2:</strong> How can I figure out what went wrong so that I can try running the upgrade again without this happening?</p>
  
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2016-12-21T23:00:06.753Z
Comments.Text: It looks like it is possible to go through the upgrade steps manually until I can identify where the upgrade process left off until the DB is stable enough to let the upgrade script continue. There are .sql and .php files in `civicrm/CRM/Upgrade/` that I can check out.
  https://forum.civicrm.org/index.php?topic=28606.0
Comments.UserDisplayName: Christia
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2016-12-22T00:47:21.970Z
Comments.Text: I'm getting a MySQL error at CRM-16189 in the 4.7.8.mysql.tpl from sites/all/modules/civicrm/CRM/Upgrade/Incremental/sql when entering the commands via command line. Boo!
Comments.UserDisplayName: Christia
Tags: <drupal><upgrade><error>
Title: Upgrade froze at 4.7.8, Can't find valid value for qrid
ThreadText: # Upgrade froze at 4.7.8, Can't find valid value for qrid
  
  (Asked by: username_0 on 2016-12-21)
  
  My upgrade has been running for about 8 hours, but it stalled after completing 4.7.7, from 4.3.8 at. I checked top to see if Drupal or CiviCRM were running, and there was no indication of the process. I decided to refresh my browser and my cookie must have expired, because Drupal asked me to log in again. Then I pulled up the queue runner in the browser (https://dev.example.com/civicrm/upgrade/queue/runner) it spit out the error:
  
  Sorry but we are not able to provide this at the moment.
    Could not find valid value for qrid
  
  The log shows that it could not find valid value for qrid. Below is the log from /sites/files/civicrm/ConfigAndLog, starting after completing upgrade to 4.7.6:
  Dec 21 12:18:17  [info] Running task: Begin Upgrade to 4.7.7
  Dec 21 12:18:19  [info] Running task: Upgrade DB to 4.7.7
  Dec 21 12:18:21  [info] Running task: Upgrade DB to 4.7.7: SQL
  Dec 21 12:18:29  [info] Running task: Finish Upgrade DB to 4.7.7
  Dec 21 12:20:15  [info] Running task: Begin Upgrade to 4.7.8
  Dec 21 12:20:17  [info] Running task: Upgrade DB to 4.7.8
  Dec 21 12:20:23  [info] Running task: Upgrade DB to 4.7.8: SQL
  Dec 21 12:20:56  [info] Running task: Upgrade mailing foreign key constraints
  Dec 21 13:26:27  [info] $Fatal Error Details = Array
  
  (
      [message] => Could not find valid value for qrid
      [code] => 
  )
  
  Dec 21 13:26:27  [info] $backTrace = #0 /var/www/html/drupal/docroot/sites/all/modules/civicrm/CRM/Core/Error.php(373): CRM_Core_Error::backtrace("backTrace", TRUE)
  #1 /var/www/html/drupal/docroot/sites/all/modules/civicrm/CRM/Utils/Request.php(120): CRM_Core_Error::fatal("Could not find valid value for qrid")
  #2 /var/www/html/drupal/docroot/sites/all/modules/civicrm/CRM/Queue/Page/Runner.php(52): CRM_Utils_Request::retrieve("qrid", "String", Object(CRM_Queue_Page_Runner), TRUE)
  #3 /var/www/html/drupal/docroot/sites/all/modules/civicrm/CRM/Core/Invoke.php(312): CRM_Queue_Page_Runner->run((Array:4), NULL)
  #4 /var/www/html/drupal/docroot/sites/all/modules/civicrm/CRM/Core/Invoke.php(86): CRM_Core_Invoke::runItem((Array:13))
  #5 /var/www/html/drupal/docroot/sites/all/modules/civicrm/CRM/Core/Invoke.php(54): CRM_Core_Invoke::_invoke((Array:4))
  #6 /var/www/html/drupal/docroot/sites/all/modules/civicrm/drupal/civicrm.module(448): CRM_Core_Invoke::invoke((Array:4))
  #7 [internal function](): civicrm_invoke("upgrade", "queue", "runner")
  #8 /var/www/html/drupal/docroot/includes/menu.inc(527): call_user_func_array("civicrm_invoke", (Array:3))
  #9 /var/www/html/drupal/docroot/index.php(21): menu_execute_active_handler()
  #10 {main}
  
  I suspect that the qrid is the queue runner ID and that the queue was lost when Drupal asked me to log in again. The log doesn't show that there was any other issues before the qrid problem. I'm not certain if something was wrong before I was forced to log in again or if that is what did it it.
  Furthermore, the file permissions for the civicrm directory in sites/all/modules was somehow changed to an owner I am not familiar with on our server (test2?):
      drwxr-xr-x 19 test2 csf      4096 Nov  2 19:32 civicrm
  drwxr-sr-- 80 fclub www-data 4096 Oct 16  2015 contrib
  drwxr-sr-- 15 fclub www-data 4096 Oct 16  2015 custom
  -rw-r--r--  1 fclub www-data  952 Aug  5  2015 README.txt
  drwxr-sr-x  2 fclub www-data 4096 Nov  3 22:40 timeago
  
  I tried loading this URL to see if that would help:
  civicrm/upgrade/queue/runner?reset=1&qrid=‌​CRM_Upgrade
  
  but then received the error:
  
  Queue runner must be configured before execution.
  
  I should have used Drush to run the upgrade so I wouldn't have to deal with Browser issues.
  Question 1: Is there anyway to salvage the progress and continue from 4.7.7, repair the database or reinstate the queue runner?
  Question 2: How can I figure out what went wrong so that I can try running the upgrade again without this happening?
  
  **Tags:** drupal, upgrade, error
  
  > **Comments:**
  > **username_0**: It looks like it is possible to go through the upgrade steps manually until I can identify where the upgrade process left off until the DB is stable enough to let the upgrade script continue. There are .sql and .php files in `civicrm/CRM/Upgrade/` that I can check out. https://forum.civicrm.org/index.php?topic=28606.0  
  > **username_0**: I'm getting a MySQL error at CRM-16189 in the 4.7.8.mysql.tpl from sites/all/modules/civicrm/CRM/Upgrade/Incremental/sql when entering the commands via command line. Boo!  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2016-12-22. Score: 3 | Accepted answer
  
  The error message comes from refreshing the browser rather than the upgrade process.
  The last entry before the error is:
  Dec 21 12:20:56  [info] Running task: Upgrade mailing foreign key constraints
  
  You can find the detail in CRM/Upgrade/Incremental/php/FourSeven.php
  Adding foreign key constraints creates an index which can take a while on a big table.  How many rows (count(*)) do your civicrm_mailing_event_queue and civicrm_mailing_recipients tables have?
  Once the upgrade has started, I don't know how dependent it is on the browser, so it would be worth checking the log later to see if you have any further messages from the upgrade.
  If you need to start again, then I'd try this:
  
  Restore your database and files back to your starting point at 4.3.8 (you did take full backups didn't you??)
  Upgrade to 4.7.7
  Check all is working as expected
  Take new backups
  Try upgrading to 4.7.8
  
  If your mailing tables are large you'll need to be patient. If it fails, you can easily go back to 4.7.7 and then work out what to do next.
  
  > **Comments:**
  > **username_0**: civicrm_mailing_event_queue: 2230662; civicrm_mailing_recipients: 3025260;  
  > **username_1**: Then you'll need a very large cup of coffee.... If you don't need all that history, might be worth posting a separate question about ways to trim those tables.  
  > **username_2**: Just to second this approach. We had over a million records in both the civicrm_mailing_event_queue and civicrm_mailing_recipients tables.  I backed them both up, truncated them, carried out the upgrades, then re-imported the data.   Seemed to do the job!  
  
  ---
  
  