Answers.Body: <p>There's no single perfect solution to this, but there are several adequate solutions. To highlight some trade-offs among the approaches raised so far:</p>
  
  <ul>
  <li><em>For each security principal, use separate data-directories</em> (like bgm's answer).
  <ul>
  <li><strong>Pro</strong>: Portable -- works in many different environments. Doesn't require any special tools or root permissions.</li>
  <li><strong>Con</strong>: Only works when data can be split apart by user. For example, it could potentially work for logs/caches (because they don't really need to be shared) but is more problematic for uploads/attachments (because both users should have read/write on the same objects).</li>
  </ul></li>
  <li><em>Create a cronjob or daemon to adjust permissions</em> (like Chris's answer)
  <ul>
  <li><strong>Pro</strong>: Portable -- you can design the daemon or cronjob to build on the traditional POSIX UGO/RWX triplets. Works with all data (logs/caches/attachments).</li>
  <li><strong>Con</strong>: Requires installing/running a new root-permissioned service. Watch out for symlinks/hardlinks.</li>
  </ul></li>
  <li><em>Use the operating system's ACL features</em> (like <a href="https://github.com/totten/amp/blob/master/doc/perm.md" rel="nofollow">amp</a>'s linuxAcl and osxAcl policies)
  <ul>
  <li><strong>Pro</strong>: Works with all data (logs/caches/attachments). Doesn't require daemon or regularly running as root.</li>
  <li><strong>Con</strong>: Not standardized -- technical steps depend on the environment (OS/distro/filesystem). Sometimes applications try to be clever and override the permissions -- which <a href="https://issues.civicrm.org/jira/browse/CRM-12169" rel="nofollow">breaks ACLs</a> for specific files/use-cases.</li>
  <li>(Note: <code>amp</code> tries to abstract this, but <a href="https://xkcd.com/927/" rel="nofollow">as with any abstraction</a>, it's probably not worth the overhead unless you buy into the <a href="https://github.com/totten/amp" rel="nofollow">general vision</a>.)</li>
  </ul></li>
  </ul>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2015-06-02T07:27:51.120Z
Answers.Text: Do you see any case where drush could need to create/modify a file that needs to be shared with the www-data user?
Answers.UserDisplayName: Xavier
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2015-06-02T10:24:11.710Z
Answers.Text: (1) Destroying+recreating demo sites. (For *.demo.civicrm.org, we get bitten by CRM-12169 and need to use work-arounds to let the console user delete attachments created by the web user.) (2) Importing or exporting data via the Attachment API. (3) Flushing templates_c as part of upgrading/migration/development. (Strictly speaking, one could work-around #3 by putting a nonce in the path for templates_c and changing the nonce whenever you flush the cache; then it doesn't matter if you delete the stale cache files. But that's not an obvious change.)
Answers.UserDisplayName: Tim Otten
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2015-06-02T12:10:23.203Z
Answers.Text: duh, having different paths on my patch means NOT deleting the templates_c if drush CC. Not so good at all. @bmg workaround idea not good enough imo
Answers.UserDisplayName: Xavier
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2015-06-02T05:59:47.623Z
Answers.LastActivityDate: 2015-06-02T11:08:42.107Z
Answers.LastEditDate: 2015-06-02T11:08:42.107Z
Answers.UserDisplayName: Tim Otten
Answers.Body: <p>Setting directories group-setuid might get you part of the way (with chmod g+ws templates_c), but maybe we should consider something like this in the CiviCRM drush module?:</p>
  
  <pre><code>$config = CRM_Core_Config::singleton( );
  $config-&gt;configAndLogDir = $config-&gt;configAndLogDir . 'drush/';
  $config-&gt;templateCompileDir = $config-&gt;templateCompileDir . 'drush/';
  </code></pre>
  
  <p>i.e. create templates in a sub-directory of templates_c/drush/, this way the web server (www-data) will not have trouble writing in the templates/logs later on.</p>
  
  <p>This is what Aegir did for a long time, but due to a regression it has not been working for a while and since then I run into that issue as well.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2015-06-01T23:48:10.193Z
Answers.Text: I like this (could we just set the FS cache to /dev/null in Drush?), it seems like it would work around issues like [CRM-15632: Don't exit if templates_c unwriteable](https://issues.civicrm.org/jira/browse/CRM-15632). Caching to an unpermissioned environment (eg memcache) might also eliminate the same. Or perhaps include the current UID in the cache path or hash?
Answers.UserDisplayName: Chris Burgess
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2015-06-02T08:35:39.570Z
Answers.Text: Hi, tried, it chokes on the templates_c in the singleton(). Workaround I found, please test and comment: https://github.com/civicrm/civicrm-drupal/pull/286
Answers.UserDisplayName: Xavier
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2015-06-03T14:21:05.657Z
Answers.Text: Furthermore, as Tim pointed out, this will break things such as "drush cvapi System.flush", since CiviCRM will not flush the correct template_c directory.
Answers.UserDisplayName: bgm
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2015-06-01T18:38:49.483Z
Answers.LastActivityDate: 2015-06-01T18:38:49.483Z
Answers.UserDisplayName: bgm
Answers.Body: <p>Tim describes a solution based on linux ACL:</p>
  
  <p><a href="https://github.com/totten/amp/blob/master/doc/perm.md" rel="nofollow">https://github.com/totten/amp/blob/master/doc/perm.md</a></p>
  
  <p>But it requires amp, that I (might be wrongly) see a dev only tool. Does someone use something like that in production?</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2015-06-02T02:00:25.137Z
Answers.Text: We make use of POSIX ACL with XFS at the Apache website directory /srv/www. I am not quite totally satisfied with the operational results... there still are times I must sudo as the ACL did not override the file system operation. Yes, to clean up temp files CiviCRM creates, even though due to the ACL new objects get created as group www-data, and my ID is a member of that group, for now I must sudo to be able to purge.
Answers.UserDisplayName: Michael Lueck
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2015-06-02T05:49:01.403Z
Answers.Text: FWIW, you don't need to use ```amp``` to use ACLs. The first few sections of ```perm.md``` are general/architectural. The ```amp datadir``` commands are thin wrappers for equivalent commands using [```setfacl``` (Linux)](https://help.ubuntu.com/community/FilePermissionsACLs) or [```chmod +a``` (OSX)](http://www.techrepublic.com/blog/apple-in-the-enterprise/introduction-to-os-x-access-control-lists-acls/). If you want to see exactly what ```amp datadir``` does in each environment, read https://github.com/totten/amp/blob/07aceace79/app/defaults/services.yml#L132
Answers.UserDisplayName: Tim Otten
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2015-06-01T19:45:52.120Z
Answers.LastActivityDate: 2015-06-01T19:45:52.120Z
Answers.UserDisplayName: Xavier
Answers.Body: <p>Our solution to mixed access to CiviCRM's cache files (and other situations where permissions issues arose) has been a daemon which monitors filesystem entries and "corrects" permissions immediately on file creation/modification.</p>
  
  <ul>
  <li><a href="https://github.com/torrance/enforced" rel="nofollow">Enforced on Github</a></li>
  </ul>
  
  <p>There are still some issues with this approach (eg issue linked in comment on bgm's answer), but it seems to work out better than just relying on setgid (which I'd hope <em>could</em> be a complete solution).</p>
  
  <p>I'm still a bit unsure of our taking this route - I'd prefer for UNIX permissions to work for people! They are simple and well-tested after all ... - but at least it's no longer a support headache :)</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2015-06-01T23:46:06.680Z
Answers.LastActivityDate: 2015-06-02T01:14:55.037Z
Answers.LastEditDate: 2015-06-02T01:14:55.037Z
Answers.UserDisplayName: Chris Burgess
Body: <p>Two users are trying to access the temp files (where, for instance, the compiled templates are stored): www-data (when it's accessed from the website) <strong>and</strong> the sysadmin that run drush commands</p>
  
  <p>By default, all these temporary files can only modified by the user that created it, so sometimes it's the sysadmin, sometimes it's the webserver user</p>
  
  <p>Is there a way so all the temp files can be created/modified/deleted by only the admin user(s) and www-data?</p>
  
  <p>One possible fix is a patch on drush, so it uses a separate templates_c folder than the one used by www-data. Could you test if it works in your environment/workflow?</p>
  
  <p><a href="https://github.com/civicrm/civicrm-drupal/pull/286" rel="nofollow">https://github.com/civicrm/civicrm-drupal/pull/286</a></p>
  
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2015-06-01T23:51:56.453Z
Comments.Text: [CRM-15632: Do not exit if templates_c unwriteable](https://issues.civicrm.org/jira/browse/CRM-15632) is related.
Comments.UserDisplayName: Chris Burgess
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-05-18T20:12:46.380Z
Comments.Text: One day, I hope for there to be an answer here which says "this is no longer an issue, because CivICRM uses a caching mechanism which avoids system user permission issues - eg by avoiding filesystem storage entirely". See eg [dev/cloud-native#1](https://lab.civicrm.org/dev/cloud-native/issues/1).
Comments.UserDisplayName: Chris Burgess
Tags: <configuration><sysadmin><drush>
Title: CiviCRM does not have permission to write temp files
ThreadText: # CiviCRM does not have permission to write temp files
  
  (Asked by: username_0 on 2015-06-01)
  
  Two users are trying to access the temp files (where, for instance, the compiled templates are stored): www-data (when it's accessed from the website) **and** the sysadmin that run drush commands
  
  By default, all these temporary files can only modified by the user that created it, so sometimes it's the sysadmin, sometimes it's the webserver user
  
  Is there a way so all the temp files can be created/modified/deleted by only the admin user(s) and www-data?
  
  One possible fix is a patch on drush, so it uses a separate templates\_c folder than the one used by www-data. Could you test if it works in your environment/workflow?
  
  <https://github.com/civicrm/civicrm-drupal/pull/286>
  
  **Tags:** configuration, sysadmin, drush
  
  > **Comments:**
  > **username_1**: [CRM-15632: Do not exit if templates_c unwriteable](https://issues.civicrm.org/jira/browse/CRM-15632) is related.  
  > **username_1**: One day, I hope for there to be an answer here which says "this is no longer an issue, because CivICRM uses a caching mechanism which avoids system user permission issues - eg by avoiding filesystem storage entirely". See eg [dev/cloud-native#1](https://lab.civicrm.org/dev/cloud-native/issues/1).  
  
  ---
  
  ## 4 answers
  
  ### Answer by username_2 on 2015-06-02. Score: 5
  
  There's no single perfect solution to this, but there are several adequate solutions. To highlight some trade-offs among the approaches raised so far:
  
  * *For each security principal, use separate data-directories* (like username_3's answer).
    + **Pro**: Portable -- works in many different environments. Doesn't require any special tools or root permissions.
    + **Con**: Only works when data can be split apart by user. For example, it could potentially work for logs/caches (because they don't really need to be shared) but is more problematic for uploads/attachments (because both users should have read/write on the same objects).
  * *Create a cronjob or daemon to adjust permissions* (like Chris's answer)
    + **Pro**: Portable -- you can design the daemon or cronjob to build on the traditional POSIX UGO/RWX triplets. Works with all data (logs/caches/attachments).
    + **Con**: Requires installing/running a new root-permissioned service. Watch out for symlinks/hardlinks.
  * *Use the operating system's ACL features* (like [amp](https://github.com/totten/amp/blob/master/doc/perm.md)'s linuxAcl and osxAcl policies)
    + **Pro**: Works with all data (logs/caches/attachments). Doesn't require daemon or regularly running as root.
    + **Con**: Not standardized -- technical steps depend on the environment (OS/distro/filesystem). Sometimes applications try to be clever and override the permissions -- which [breaks ACLs](https://issues.civicrm.org/jira/browse/CRM-12169) for specific files/use-cases.
    + (Note: `amp` tries to abstract this, but [as with any abstraction](https://xkcd.com/927/), it's probably not worth the overhead unless you buy into the [general vision](https://github.com/totten/amp).)
  
  > **Comments:**
  > **username_0**: Do you see any case where drush could need to create/modify a file that needs to be shared with the www-data user?  
  > **username_2**: (1) Destroying+recreating demo sites. (For *.demo.civicrm.org, we get bitten by CRM-12169 and need to use work-arounds to let the console user delete attachments created by the web user.) (2) Importing or exporting data via the Attachment API. (3) Flushing templates_c as part of upgrading/migration/development. (Strictly speaking, one could work-around #3 by putting a nonce in the path for templates_c and changing the nonce whenever you flush the cache; then it doesn't matter if you delete the stale cache files. But that's not an obvious change.)  
  > **username_0**: duh, having different paths on my patch means NOT deleting the templates_c if drush CC. Not so good at all. @bmg workaround idea not good enough imo  
  
  ---
  
  ### Answer by username_3 on 2015-06-01. Score: 3
  
  Setting directories group-setuid might get you part of the way (with chmod g+ws templates_c), but maybe we should consider something like this in the CiviCRM drush module?:
  $config = CRM_Core_Config::singleton( );
  $config->configAndLogDir = $config->configAndLogDir . 'drush/';
  $config->templateCompileDir = $config->templateCompileDir . 'drush/';
  
  i.e. create templates in a sub-directory of templates_c/drush/, this way the web server (www-data) will not have trouble writing in the templates/logs later on.
  This is what Aegir did for a long time, but due to a regression it has not been working for a while and since then I run into that issue as well.
  
  > **Comments:**
  > **username_1**: I like this (could we just set the FS cache to /dev/null in Drush?), it seems like it would work around issues like [CRM-15632: Don't exit if templates_c unwriteable](https://issues.civicrm.org/jira/browse/CRM-15632). Caching to an unpermissioned environment (eg memcache) might also eliminate the same. Or perhaps include the current UID in the cache path or hash?  
  > **username_0**: Hi, tried, it chokes on the templates_c in the singleton(). Workaround I found, please test and comment: https://github.com/civicrm/civicrm-drupal/pull/286  
  > **username_3**: Furthermore, as Tim pointed out, this will break things such as "drush cvapi System.flush", since CiviCRM will not flush the correct template_c directory.  
  
  ---
  
  ### Answer by username_0 on 2015-06-01. Score: 2
  
  Tim describes a solution based on linux ACL:
  
  <https://github.com/totten/amp/blob/master/doc/perm.md>
  
  But it requires amp, that I (might be wrongly) see a dev only tool. Does someone use something like that in production?
  
  > **Comments:**
  > **username_4**: We make use of POSIX ACL with XFS at the Apache website directory /srv/www. I am not quite totally satisfied with the operational results... there still are times I must sudo as the ACL did not override the file system operation. Yes, to clean up temp files CiviCRM creates, even though due to the ACL new objects get created as group www-data, and my ID is a member of that group, for now I must sudo to be able to purge.  
  > **username_2**: FWIW, you don't need to use ```amp``` to use ACLs. The first few sections of ```perm.md``` are general/architectural. The ```amp datadir``` commands are thin wrappers for equivalent commands using [```setfacl``` (Linux)](https://help.ubuntu.com/community/FilePermissionsACLs) or [```chmod +a``` (OSX)](http://www.techrepublic.com/blog/apple-in-the-enterprise/introduction-to-os-x-access-control-lists-acls/). If you want to see exactly what ```amp datadir``` does in each environment, read https://github.com/totten/amp/blob/07aceace79/app/defaults/services.yml#L132  
  
  ---
  
  ### Answer by username_1 on 2015-06-01. Score: 2
  
  Our solution to mixed access to CiviCRM's cache files (and other situations where permissions issues arose) has been a daemon which monitors filesystem entries and "corrects" permissions immediately on file creation/modification.
  
  * [Enforced on Github](https://github.com/torrance/enforced)
  
  There are still some issues with this approach (eg issue linked in comment on username_3's answer), but it seems to work out better than just relying on setgid (which I'd hope *could* be a complete solution).
  
  I'm still a bit unsure of our taking this route - I'd prefer for UNIX permissions to work for people! They are simple and well-tested after all ... - but at least it's no longer a support headache :)
  
  
  ---
  
  