Answers.Body: <blockquote>
  <p>... a JavaScript file ... When I upload a new version of the file it takes quite some time until it is picked up by CiviCRM, sometimes hours. I have cleared caches and disabled asset caching, didn't help. ... <code>composer civicrm:publish</code> doesn't seem to help with this... I am a total newbie in CiviCRM development and don't understand the caching going on here.</p>
  </blockquote>
  <p>Yeah, there are many layers of caching in real-world systems. Even experienced folks will lose track of them.</p>
  <p><strong>Short thought</strong>: Check the references to your JS file. There should be a &quot;resource caching code&quot; (<em>random suffix</em>).</p>
  <p><strong>Long explanation</strong>:  These symptoms sound like an issue in the browser-based caching and URL-construction. I don't have a reference to your codebase, so I'll make up an example.</p>
  <p>Many HTTP servers enable caching for static files, such as <code>jquery.min.js</code>. It works like so:</p>
  <pre><code>&lt;!-- The HTML page requires `jquery.min.js` --&gt;
  &lt;script
    type=&quot;text/javascript&quot; 
    src=&quot;/sites/all/modules/civicrm/bower_components/jquery/dist/jquery.min.js&quot;&gt;
  &lt;/script&gt;
  </code></pre>
  <pre><code>&lt;!-- So the browser requests the file --&gt;
  GET /sites/all/modules/civicrm/bower_components/jquery/dist/jquery.min.js
  </code></pre>
  <pre><code>&lt;!-- Then the server sends the file along with caching instructions --&gt;
  Content-Type: application/javascript
  Content-Length: 96830
  Cache-Control: max-age=7200
  
  !function(a,b){&quot;object&quot;==typeof...
  </code></pre>
  <p>The server tells the browser to cache <code>jquery.min.js</code> for 7200 seconds (2 hours) -- which is great for performance but frustrating for deployment. For a clean deployment, one needs a work-around.</p>
  <p>In CiviCRM, the <em>conventional work-around</em> is to append a random suffix to the URL. For example, I went to <a href="https://dmaster.demo.civicrm.org/civicrm/event/info?id=3&amp;reset=1" rel="nofollow noreferrer">this demo page</a> and used &quot;View Source&quot; to find <code>jquery.min.js</code>. The actual URL has a suffix <code>?s3tkpz</code>.</p>
  <pre><code>&lt;!-- Here's how the HTML actually references the file --&gt;
  &lt;script
    type=&quot;text/javascript&quot; 
    src=&quot;https://dmaster.demo.civicrm.org/sites/all/modules/civicrm
                   /bower_components/jquery/dist/jquery.min.js?s3tkpz&quot;&gt;
  &lt;/script&gt;
  </code></pre>
  <pre><code>&lt;!-- So the browser requests the file --&gt;
  GET /sites/all/modules/civicrm/bower_components/jquery/dist/jquery.min.js?s3tkpz
  </code></pre>
  <p>When CiviCRM has a major change on the server-side, it will switch out the suffix.</p>
  <p><strong><code>jquery.min.js?s3tkpz</code> =&gt; <code>jquery.min.js?neWc0d3</code></strong></p>
  <p>This doesn't actually <em>clear</em> the browser caches -- but as far as the browser is concerned, it looks like a different URL for a different file. The old caches become irrelevant. The browser will re-read the file.</p>
  <p>This convention is <em>extremely common</em> for CiviCRM code. Several methods - like <code>addScriptFile()</code> and <code>addStyleFile()</code> from <a href="https://docs.civicrm.org/dev/en/latest/framework/resources/" rel="nofollow noreferrer">Resources API</a> - will add the code by default. (It's so common that we take for granted...) But it's not absolute or universal. You might need to do one of these:</p>
  <ul>
  <li>Switch to <code>Civi::resources()-&gt;addScriptFile()</code> or <code>Civi::resources()-&gt;addStyleFile()</code></li>
  <li>Use <code>Civi::resources()-&gt;getUrl()</code> and set the third option (<code>$addCacheCode=TRUE</code>).</li>
  <li>Lookup the cache-code (<code>Civi::settings()-&gt;get('resCacheCode')</code>) and manually append it to a URL.</li>
  <li>Tell your browser to force-reload (usu <code>Ctrl-Shift-R</code> or <code>Cmd-Shift-R</code>).</li>
  </ul>
  <p>(<em>I don't know if the above is actually your problem - but it does fit the symptoms: you're working with static JS files and clearing server-side caches and not seeing the expected content. Also, the behavior of HTTP caches can vary between different web-servers - e.g. your local dev server and production web server might have different caching policies.</em>)</p>
  <p>(<em>Also note that the random suffix may look a bit different depending on the environment and code-flow. It might look like <code>?xxxxxx</code> or <code>?r=xxxxxx</code>, but it's the same concept either way.</em>)</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-11-19T12:53:22.090Z
Answers.Text: Thank you for this comprehensive answer! My solution was easier, but you pointed me in the right direction. I basically had to purge and configure our proxy cache. Your explanation gave me some extra learnings though. :)
Answers.UserDisplayName: Jan
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-11-08T22:02:54.643Z
Answers.LastActivityDate: 2023-11-08T22:18:15.620Z
Answers.LastEditDate: 2023-11-08T22:18:15.620Z
Answers.UserDisplayName: Tim Otten
Answers.Body: <p>Following-up on Coleman's comment about <code>composer civicrm:publish</code>.... I don't know if that was confirmed to be part of the issue here. But here's a little braindump about when that should or shouldn't be necessary.</p>
  <p><strong>Assumptions</strong>: You have a typical Civi-D9/D10 layout (as per <code>civicrm/civicrm-asset-plugin</code>). You are deploying custom/contrib extensions.</p>
  <p><strong>Considerations</strong>: The location and mechanism of the download can vary, depending on your local-configuration/preferences/etc.</p>
  <ul>
  <li><strong>Location</strong>: Where is the extension code stored? Typically, it's either the public folder (<code>sites/default/files/civicrm/ext/</code>) or the private folder (<code>vendor/</code>).</li>
  <li><strong>Mechanism</strong>: Which mechanism performs the download? Typically it's either the built-in downloader (<code>CRM_Extension_Downloader</code>) or a manual downloader (<code>git clone</code> or <code>wget &amp;&amp; unzip</code>) or a <code>composer</code>-based downloader</li>
  </ul>
  <p><strong>Expectations</strong>: There are two typical approaches.</p>
  <ol>
  <li>For extensions in the public folder <code>sites/default/files/civicrm/ext/</code>...
  <ul>
  <li>It makes sense to me to use the built-in downloader and/or manual downloader.</li>
  <li>The assets (JS/CSS/PNG/etc) are already public. These downloaders do not integrate with <code>civicrm-asset-plugin</code>. (It would be complex+unnecessary.)</li>
  <li>You shouldn't have to call <code>composer civicrm:publish</code>... because it's fundamentally not needed.</li>
  </ul>
  </li>
  <li>For extensions in the private folder (<code>vendor/XXX/YYY/</code>)...
  <ul>
  <li>It makes sense to me to use <code>composer</code> as the downloader.</li>
  <li>The assets (JS/CSS/PNG/etc) are private and need to be copied to a public location. This is why <code>civicrm-asset-plugin</code> integrates into <code>composer</code> - so that it can automatically publish these assets.</li>
  <li>You shouldn't have to call <code>composer civicrm:publish</code> in typical usage... because <code>civicrm-asset-plugin</code> hooks into other <code>composer</code> commands. It should run automatically (during <code>composer install</code> or <code>composer require</code> or <code>composer update</code>).</li>
  </ul>
  </li>
  <li>For other arrangements... that's getting into oddball territory. I might see cases where you need to manually call <code>composer civicrm:publish</code>. But we really need to clarify the scenario. Here are some examples of oddball scenarios:
  <ul>
  <li>Manually editing files in <code>vendor/</code></li>
  <li>Configuring <code>extensionsDir</code> to point at a private folder</li>
  <li>Using <code>composer</code> to download extensions into a public folder</li>
  </ul>
  </li>
  </ol>
  <p>(There may be mitigations to make the oddball scenarios better. For example, if you frequently do direct edits in the <code>vendor/</code> tree, then you should configure <a href="https://lab.civicrm.org/dev/civicrm-asset-plugin#options" rel="nofollow noreferrer"><code>CIVICRM_COMPOSER_ASSET=symdir</code> or <code>file-mode: symdir</code></a>.)</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-11-07T22:42:39.330Z
Answers.Text: I am uploading the javascript file with scp into the `sites/default/files/civicrm/ext/<myext>` directory and as you have said `composer civicrm:publish` doesn't seem to help with this. I am a total newbie in CiviCRM development and don't understand the caching going on here.
Answers.UserDisplayName: Jan
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-11-07T05:31:49.427Z
Answers.LastActivityDate: 2023-11-07T05:31:49.427Z
Answers.UserDisplayName: Tim Otten
Answers.Body: <p>The solution is to run the command <code>composer civicrm:publish</code>. See <a href="https://docs.civicrm.org/sysadmin/en/latest/upgrade/drupal8/#clear-cached-files" rel="nofollow noreferrer">note in the upgrade guide</a> (which really ought to be more prominent)!</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-11-07T22:45:58.753Z
Answers.Text: This command doesn't seem to help with an extension in `sites/default/files/civicrm/ext/` does it?
Answers.UserDisplayName: Jan
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-11-06T17:26:04.533Z
Answers.LastActivityDate: 2023-11-06T17:26:04.533Z
Answers.UserDisplayName: Coleman
Body: <p>I embedded a JavaScript file inside a custom extension via the buildForm hook. The file gets loaded and does its job, however it is not reloaded reliably after an update. When I upload a new version of the file it takes quite some time until it is picked up by CiviCRM, sometimes hours. I have cleared caches and disabled asset caching, didn't help.
  I am on Drupal 9 and CiviCRM 5.67.0. Any help is appreciated.</p>
  
Tags: <javascript><cache>
Title: Custom javascript file is not refreshed
ThreadText: # Custom javascript file is not refreshed
  
  (Asked by: username_0 on 2023-11-06)
  
  I embedded a JavaScript file inside a custom extension via the buildForm hook. The file gets loaded and does its job, however it is not reloaded reliably after an update. When I upload a new version of the file it takes quite some time until it is picked up by CiviCRM, sometimes hours. I have cleared caches and disabled asset caching, didn't help.
  I am on Drupal 9 and CiviCRM 5.67.0. Any help is appreciated.
  
  **Tags:** javascript, cache
  
  
  ---
  
  ## 3 answers
  
  ### Answer by username_1 on 2023-11-08. Score: 1 | Accepted answer
  
  ... a JavaScript file ... When I upload a new version of the file it takes quite some time until it is picked up by CiviCRM, sometimes hours. I have cleared caches and disabled asset caching, didn't help. ... composer civicrm:publish doesn't seem to help with this... I am a total newbie in CiviCRM development and don't understand the caching going on here.
  
  Yeah, there are many layers of caching in real-world systems. Even experienced folks will lose track of them.
  Short thought: Check the references to your JS file. There should be a "resource caching code" (random suffix).
  Long explanation:  These symptoms sound like an issue in the browser-based caching and URL-construction. I don't have a reference to your codebase, so I'll make up an example.
  Many HTTP servers enable caching for static files, such as jquery.min.js. It works like so:
  
  
  
  
  GET /sites/all/modules/civicrm/bower_components/jquery/dist/jquery.min.js
  
  
  Content-Type: application/javascript
  Content-Length: 96830
  Cache-Control: max-age=7200
  
  !function(a,b){"object"==typeof...
  
  The server tells the browser to cache jquery.min.js for 7200 seconds (2 hours) -- which is great for performance but frustrating for deployment. For a clean deployment, one needs a work-around.
  In CiviCRM, the conventional work-around is to append a random suffix to the URL. For example, I went to this demo page and used "View Source" to find jquery.min.js. The actual URL has a suffix ?s3tkpz.
  
  
  
  
  GET /sites/all/modules/civicrm/bower_components/jquery/dist/jquery.min.js?s3tkpz
  
  When CiviCRM has a major change on the server-side, it will switch out the suffix.
  jquery.min.js?s3tkpz => jquery.min.js?neWc0d3
  This doesn't actually clear the browser caches -- but as far as the browser is concerned, it looks like a different URL for a different file. The old caches become irrelevant. The browser will re-read the file.
  This convention is extremely common for CiviCRM code. Several methods - like addScriptFile() and addStyleFile() from Resources API - will add the code by default. (It's so common that we take for granted...) But it's not absolute or universal. You might need to do one of these:
  
  Switch to Civi::resources()->addScriptFile() or Civi::resources()->addStyleFile()
  Use Civi::resources()->getUrl() and set the third option ($addCacheCode=TRUE).
  Lookup the cache-code (Civi::settings()->get('resCacheCode')) and manually append it to a URL.
  Tell your browser to force-reload (usu Ctrl-Shift-R or Cmd-Shift-R).
  
  (I don't know if the above is actually your problem - but it does fit the symptoms: you're working with static JS files and clearing server-side caches and not seeing the expected content. Also, the behavior of HTTP caches can vary between different web-servers - e.g. your local dev server and production web server might have different caching policies.)
  (Also note that the random suffix may look a bit different depending on the environment and code-flow. It might look like ?xxxxxx or ?r=xxxxxx, but it's the same concept either way.)
  
  > **Comments:**
  > **username_0**: Thank you for this comprehensive answer! My solution was easier, but you pointed me in the right direction. I basically had to purge and configure our proxy cache. Your explanation gave me some extra learnings though. :)  
  
  ---
  
  ### Answer by username_1 on 2023-11-07. Score: 1
  
  Following-up on username_2's comment about `composer civicrm:publish`.... I don't know if that was confirmed to be part of the issue here. But here's a little braindump about when that should or shouldn't be necessary.
  
  **Assumptions**: You have a typical Civi-D9/D10 layout (as per `civicrm/civicrm-asset-plugin`). You are deploying custom/contrib extensions.
  
  **Considerations**: The location and mechanism of the download can vary, depending on your local-configuration/preferences/etc.
  
  * **Location**: Where is the extension code stored? Typically, it's either the public folder (`sites/default/files/civicrm/ext/`) or the private folder (`vendor/`).
  * **Mechanism**: Which mechanism performs the download? Typically it's either the built-in downloader (`CRM_Extension_Downloader`) or a manual downloader (`git clone` or `wget && unzip`) or a `composer`-based downloader
  
  **Expectations**: There are two typical approaches.
  
  1. For extensions in the public folder `sites/default/files/civicrm/ext/`...
     * It makes sense to me to use the built-in downloader and/or manual downloader.
     * The assets (JS/CSS/PNG/etc) are already public. These downloaders do not integrate with `civicrm-asset-plugin`. (It would be complex+unnecessary.)
     * You shouldn't have to call `composer civicrm:publish`... because it's fundamentally not needed.
  2. For extensions in the private folder (`vendor/XXX/YYY/`)...
     * It makes sense to me to use `composer` as the downloader.
     * The assets (JS/CSS/PNG/etc) are private and need to be copied to a public location. This is why `civicrm-asset-plugin` integrates into `composer` - so that it can automatically publish these assets.
     * You shouldn't have to call `composer civicrm:publish` in typical usage... because `civicrm-asset-plugin` hooks into other `composer` commands. It should run automatically (during `composer install` or `composer require` or `composer update`).
  3. For other arrangements... that's getting into oddball territory. I might see cases where you need to manually call `composer civicrm:publish`. But we really need to clarify the scenario. Here are some examples of oddball scenarios:
     * Manually editing files in `vendor/`
     * Configuring `extensionsDir` to point at a private folder
     * Using `composer` to download extensions into a public folder
  
  (There may be mitigations to make the oddball scenarios better. For example, if you frequently do direct edits in the `vendor/` tree, then you should configure [`CIVICRM_COMPOSER_ASSET=symdir` or `file-mode: symdir`](https://lab.civicrm.org/dev/civicrm-asset-plugin#options).)
  
  > **Comments:**
  > **username_0**: I am uploading the javascript file with scp into the `sites/default/files/civicrm/ext/<myext>` directory and as you have said `composer civicrm:publish` doesn't seem to help with this. I am a total newbie in CiviCRM development and don't understand the caching going on here.  
  
  ---
  
  ### Answer by username_2 on 2023-11-06. Score: 0
  
  The solution is to run the command `composer civicrm:publish`. See [note in the upgrade guide](https://docs.civicrm.org/sysadmin/en/latest/upgrade/drupal8/#clear-cached-files) (which really ought to be more prominent)!
  
  > **Comments:**
  > **username_0**: This command doesn't seem to help with an extension in `sites/default/files/civicrm/ext/` does it?  
  
  ---
  
  