<h3 dir="auto">System info</h3>
<ul dir="auto">
<li>Playwright Version: [v1.XX]</li>
<li>Operating System: [macOS 13.2, etc.]</li>
<li>Browser:  Chromium]</li>
<li>Other info:</li>
</ul>
<h3 dir="auto">Source code</h3>
<ul class="contains-task-list">
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I provided exact source code that allows reproducing the issue locally.</li>
</ul>




<p dir="auto"><strong>Link to the GitHub repository with the repro</strong></p>
<p dir="auto">[https://github.com/your_profile/playwright_issue_title]</p>
<p dir="auto">or</p>
<p dir="auto"><strong>Config file</strong></p>
<div class="highlight highlight-source-java notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="// playwright.java
            Playwright playwright = Playwright.create();
            Browser browser = playwright.chromium().launch(
                    new BrowserType.LaunchOptions().setHeadless(false)
                            .setChannel(&quot;chrome&quot;)
                            .setDevtools(true)
                            .setTimeout(300000)
            );
            BrowserContext browserContext = browser.newContext();
            Page page = browserContext.newPage();
            page.setDefaultTimeout(300000);
            page.setDefaultNavigationTimeout(300000);
            page.navigate(url, new Page.NavigateOptions().setTimeout(300000));
            page.close();
            browserContext.close();
            browser.close();
            playwright.close();"><pre class="notranslate"><span class="pl-c">// playwright.java</span>
            <span class="pl-smi">Playwright</span> <span class="pl-s1">playwright</span> = <span class="pl-smi">Playwright</span>.<span class="pl-en">create</span>();
            <span class="pl-smi">Browser</span> <span class="pl-s1">browser</span> = <span class="pl-s1">playwright</span>.<span class="pl-en">chromium</span>().<span class="pl-en">launch</span>(
                    <span class="pl-k">new</span> <span class="pl-smi">BrowserType</span>.<span class="pl-smi">LaunchOptions</span>().<span class="pl-en">setHeadless</span>(<span class="pl-c1">false</span>)
                            .<span class="pl-en">setChannel</span>(<span class="pl-s">"chrome"</span>)
                            .<span class="pl-en">setDevtools</span>(<span class="pl-c1">true</span>)
                            .<span class="pl-en">setTimeout</span>(<span class="pl-c1">300000</span>)
            );
            <span class="pl-smi">BrowserContext</span> <span class="pl-s1">browserContext</span> = <span class="pl-s1">browser</span>.<span class="pl-en">newContext</span>();
            <span class="pl-smi">Page</span> <span class="pl-s1">page</span> = <span class="pl-s1">browserContext</span>.<span class="pl-en">newPage</span>();
            <span class="pl-s1">page</span>.<span class="pl-en">setDefaultTimeout</span>(<span class="pl-c1">300000</span>);
            <span class="pl-s1">page</span>.<span class="pl-en">setDefaultNavigationTimeout</span>(<span class="pl-c1">300000</span>);
            <span class="pl-s1">page</span>.<span class="pl-en">navigate</span>(<span class="pl-s1">url</span>, <span class="pl-k">new</span> <span class="pl-smi">Page</span>.<span class="pl-smi">NavigateOptions</span>().<span class="pl-en">setTimeout</span>(<span class="pl-c1">300000</span>));
            <span class="pl-s1">page</span>.<span class="pl-en">close</span>();
            <span class="pl-s1">browserContext</span>.<span class="pl-en">close</span>();
            <span class="pl-s1">browser</span>.<span class="pl-en">close</span>();
            <span class="pl-s1">playwright</span>.<span class="pl-en">close</span>();</pre></div>
<p dir="auto"><strong>Steps</strong></p>
<ul dir="auto">
<li>[Run the test]</li>
<li>[...]Run the above code in parallel using executor service (10 threads)</li>
</ul>
<p dir="auto"><strong>Expected</strong></p>
<p dir="auto">[Successful navigation]</p>
<p dir="auto"><strong>Actual</strong></p>
<div class="highlight highlight-source-java notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="[WARNING: Timed out while waiting for driver process to exit
com.microsoft.playwright.PlaywrightException: Error {
  message='net::ERR_ABORTED at url
=========================== logs ===========================
navigating to &quot;url&quot;, waiting until &quot;load&quot;
============================================================
  name='Error
  stack='Error: net::ERR_ABORTED at url
=========================== logs ===========================
navigating to &quot;url&quot;, waiting until &quot;load&quot;
============================================================
    at FrameSession._navigate (/private/var/folders/mg/1pyjgqbs4sj89gyr8wy3zkkc0000gq/T/playwright-java-1669077734166046551/package/lib/server/chromium/crPage.js:509:35)
]"><pre class="notranslate">[<span class="pl-c1">WARNING</span>: <span class="pl-smi">Timed</span> <span class="pl-s1">out</span> <span class="pl-s1">while</span> <span class="pl-s1">waiting</span> <span class="pl-s1">for</span> <span class="pl-s1">driver</span> <span class="pl-smi">process</span> <span class="pl-s1">to</span> <span class="pl-smi">exit</span>
<span class="pl-s1">com</span>.<span class="pl-s1">microsoft</span>.<span class="pl-s1">playwright</span>.<span class="pl-s1">PlaywrightException</span>: <span class="pl-s1">Error</span> {
  <span class="pl-s1">message</span>='net::ERR_ABORTED at url
=========================== <span class="pl-s1">logs</span> ===========================
<span class="pl-smi">navigating</span> <span class="pl-s1">to</span> <span class="pl-s">"url"</span>, <span class="pl-s1">waiting</span> <span class="pl-s1">until</span> "<span class="pl-s1">load</span><span class="pl-s">"</span>
<span class="pl-s">============================================================</span>
<span class="pl-s">  name='Error</span>
<span class="pl-s">  stack='Error: net::ERR_ABORTED at url</span>
<span class="pl-s">=========================== logs ===========================</span>
<span class="pl-s">navigating to "</span><span class="pl-s1">url</span><span class="pl-s">", waiting until "</span><span class="pl-s1">load</span>"
============================================================
    at FrameSession._navigate (/private/var/folders/mg/1pyjgqbs4sj89gyr8wy3zkkc0000gq/T/playwright-java-1669077734166046551/package/lib/server/chromium/crPage.js:509:35)
]</pre></div>
<h3 dir="auto">System info</h3>
<ul dir="auto">
<li>Playwright Version: [v1.34.3]</li>
<li>Operating System: [Windows 10, Ubuntu 22.04]</li>
<li>Browser: [Chromium]</li>
</ul>
<h3 dir="auto">Source code</h3>
<ul class="contains-task-list">
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I provided exact source code that allows reproducing the issue locally.</li>
</ul>




<p dir="auto"><strong>Link to the GitHub repository with the repro</strong></p>
<p dir="auto"><a href="https://github.com/edumserrano/playwright-bug-vrt-diff">edumserrano/playwright-bug-vrt-diff</a></p>
<h3 dir="auto">Bug report</h3>
<p dir="auto">The issue I have is that some scenarios with screenshot comparison don't seem to work as expected, or at least as I'd expect. Perhaps some of these issues aren't qualified to be bugs.</p>
<p dir="auto">I've found issues with unexpected differences in screenshot tests:</p>
<ul dir="auto">
<li>running in headless vs not headless mode.</li>
<li>running on Chromium Windows vs Chromium Ubuntu.</li>
<li>running outside debug vs in debug mode.</li>
</ul>
<p dir="auto">Please see the details below.</p>
<h3 dir="auto">Issues trying to get consistent VRT</h3>
<div class="markdown-alert markdown-alert-note" dir="auto"><p dir="auto"><span class="color-fg-accent text-semibold d-inline-flex flex-items-center mb-1"><svg class="octicon octicon-info mr-2" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path d="M0 8a8 8 0 1 1 16 0A8 8 0 0 1 0 8Zm8-6.5a6.5 6.5 0 1 0 0 13 6.5 6.5 0 0 0 0-13ZM6.5 7.75A.75.75 0 0 1 7.25 7h1a.75.75 0 0 1 .75.75v2.75h.25a.75.75 0 0 1 0 1.5h-2a.75.75 0 0 1 0-1.5h.25v-2h-.25a.75.75 0 0 1-.75-.75ZM8 6a1 1 0 1 1 0-2 1 1 0 0 1 0 2Z"></path></svg>Note</span></p><p dir="auto">All these scenarios were executed from a Windows OS as the base OS.</p></div>
<h4 dir="auto">Scenario 1: Hardware acceleration and headless mode</h4>
<ul dir="auto">
<li>Go to the root of the repo.</li>
<li>Run the tests via <code class="notranslate">npm run test</code>.</li>
<li>Run the app via <code class="notranslate">npm run start</code>.</li>
<li>Compare the screenshot generated via <code class="notranslate">npm run test</code> at <code class="notranslate">tests/example.spec.ts-snapshots/basic-test-1-chromium-win32.png</code> with how the app running via <code class="notranslate">npm run start</code> looks like in regards to the blur effect.</li>
</ul>
<details>
  <summary>Screenshot from npm run test</summary>
<p dir="auto"><a target="_blank" rel="noopener noreferrer" href="https://github.com/edumserrano/playwright-bug-vrt-diff/tree/main/docs/images/hardware-acceleration-npm-run-test.png"><img src="https://github.com/edumserrano/playwright-bug-vrt-diff/tree/main/docs/images/hardware-acceleration-npm-run-test.png" alt="screenshot from npm run test" style="max-width: 100%;"></a></p>
</details>
<details>
  <summary>Screenshot from npm run start</summary>
<div class="markdown-alert markdown-alert-note" dir="auto"><p dir="auto"><span class="color-fg-accent text-semibold d-inline-flex flex-items-center mb-1"><svg class="octicon octicon-info mr-2" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path d="M0 8a8 8 0 1 1 16 0A8 8 0 0 1 0 8Zm8-6.5a6.5 6.5 0 1 0 0 13 6.5 6.5 0 0 0 0-13ZM6.5 7.75A.75.75 0 0 1 7.25 7h1a.75.75 0 0 1 .75.75v2.75h.25a.75.75 0 0 1 0 1.5h-2a.75.75 0 0 1 0-1.5h.25v-2h-.25a.75.75 0 0 1-.75-.75ZM8 6a1 1 0 1 1 0-2 1 1 0 0 1 0 2Z"></path></svg>Note</span></p><p dir="auto">My PC has a higher resolution than 1920x1080, I've used the browser tools to try and get close for the manual screenshot I took. Anyways, what's important in this screenshot is looking at how the blur looks correct and how different it is from when running the test via <code class="notranslate">npm run test</code>.</p></div>
<p dir="auto"><a target="_blank" rel="noopener noreferrer" href="https://github.com/edumserrano/playwright-bug-vrt-diff/tree/main/docs/images/hardware-acceleration-npm-run-start.png"><img src="https://github.com/edumserrano/playwright-bug-vrt-diff/tree/main/docs/images/hardware-acceleration-npm-run-start.png" alt="screenshot from npm run test" style="max-width: 100%;"></a></p>
</details>
<p dir="auto">When running the tests with <code class="notranslate">npm run test</code> it seems the chromium browser running in <code class="notranslate">headless</code> mode does not have hardware acceleration enabled, therefore the image produced is not as expected.</p>
<p dir="auto">You can compare the image produced from the test with how the app looks like when running via <code class="notranslate">npm run start</code>. I believe this is related with hardware acceleration because when running the app via <code class="notranslate">npm run start</code> if you enable/disable hardware accelearation in the browser settings, the blur applied looks different.</p>
<p dir="auto">Another way to view this is related with hardware acceleration is that if you add:</p>
<div class="highlight highlight-source-ts notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="launchOptions: {
  args: [&quot;--disable-gpu&quot;],
},"><pre class="notranslate"><span class="pl-s1">launchOptions</span>: <span class="pl-kos">{</span>
  <span class="pl-c1">args</span>: <span class="pl-kos">[</span><span class="pl-s">"--disable-gpu"</span><span class="pl-kos">]</span><span class="pl-kos">,</span>
<span class="pl-kos">}</span><span class="pl-kos">,</span></pre></div>
<p dir="auto">to the chromium project settings in the <a href="/playwright.config.ts">playwright.config.ts</a> file, then the blur of screenshot from debug will look equal to when running without debug.</p>
<div class="markdown-alert markdown-alert-note" dir="auto"><p dir="auto"><span class="color-fg-accent text-semibold d-inline-flex flex-items-center mb-1"><svg class="octicon octicon-info mr-2" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path d="M0 8a8 8 0 1 1 16 0A8 8 0 0 1 0 8Zm8-6.5a6.5 6.5 0 1 0 0 13 6.5 6.5 0 0 0 0-13ZM6.5 7.75A.75.75 0 0 1 7.25 7h1a.75.75 0 0 1 .75.75v2.75h.25a.75.75 0 0 1 0 1.5h-2a.75.75 0 0 1 0-1.5h.25v-2h-.25a.75.75 0 0 1-.75-.75ZM8 6a1 1 0 1 1 0-2 1 1 0 0 1 0 2Z"></path></svg>Note</span></p><p dir="auto">The way the blur is intended to look like is how it looks when hardware acceleration is enabled</p>
<p dir="auto">The blur effect is being created with:</p>
<pre lang="css" class="notranslate"><code class="notranslate">background-color: transparent;
-webkit-backdrop-filter: blur(50px);
backdrop-filter: blur(50px);
</code></pre></div>
<h4 dir="auto">Scenario 2: Chromium screenshot in Windows vs Chromium screenshot in Ubuntu</h4>
<ul dir="auto">
<li>Go to the root of the repo.</li>
<li>Uncomment the line <code class="notranslate">// ignoreDefaultArgs: ["--hide-scrollbars"]</code> in the <a href="/playwright.config.ts">playwright.config.ts</a> file.</li>
<li>Run the tests via <code class="notranslate">npm run test</code> and generate the screenshot in Windows.</li>
<li>Duplicate the screenshot <code class="notranslate">basic-test-1-chromium-win32.png</code> at <code class="notranslate">/tests/example.spec.ts-snapshots</code> and rename it to <code class="notranslate">basic-test-1-chromium-linux.png</code>.</li>
<li>Run the tests in Ubuntu via docker by executing <code class="notranslate">docker run -it --rm --ipc=host --workdir=/app -v ${PWD}:/app mcr.microsoft.com/playwright:v1.34.3-jammy npx playwright test</code>.</li>
<li>Open the html report via <code class="notranslate">npx playwright show-report tests/test-results/reporters/html</code> and look at the differences in the screenshots.</li>
</ul>
<p dir="auto">The screenshot generated from Ubuntu will have some slight differences regarding the font (looks bolder in windows?). Furthermore, the page scrollbar on Windows is a bit wider than the one in Ubuntu.</p>
<details>
  <summary>Diff</summary>
<p dir="auto"><a target="_blank" rel="noopener noreferrer" href="https://github.com/edumserrano/playwright-bug-vrt-diff/tree/main/docs/images/windows-ubuntu-diff.png"><img src="https://github.com/edumserrano/playwright-bug-vrt-diff/tree/main/docs/images/windows-ubuntu-diff.png" alt="diff" style="max-width: 100%;"></a></p>
</details>
<details>
  <summary>Screenshot from Windows</summary>
<p dir="auto"><a target="_blank" rel="noopener noreferrer" href="https://github.com/edumserrano/playwright-bug-vrt-diff/tree/main/docs/images/windows.png"><img src="https://github.com/edumserrano/playwright-bug-vrt-diff/tree/main/docs/images/windows.png" alt="Windows" style="max-width: 100%;"></a></p>
</details>
<details>
  <summary>Screenshot from Ubuntu</summary>
<p dir="auto"><a target="_blank" rel="noopener noreferrer" href="https://github.com/edumserrano/playwright-bug-vrt-diff/tree/main/docs/images/ubuntu.png"><img src="https://github.com/edumserrano/playwright-bug-vrt-diff/tree/main/docs/images/ubuntu.png" alt="Ubuntu" style="max-width: 100%;"></a></p>
</details>
<div class="markdown-alert markdown-alert-note" dir="auto"><p dir="auto"><span class="color-fg-accent text-semibold d-inline-flex flex-items-center mb-1"><svg class="octicon octicon-info mr-2" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path d="M0 8a8 8 0 1 1 16 0A8 8 0 0 1 0 8Zm8-6.5a6.5 6.5 0 1 0 0 13 6.5 6.5 0 0 0 0-13ZM6.5 7.75A.75.75 0 0 1 7.25 7h1a.75.75 0 0 1 .75.75v2.75h.25a.75.75 0 0 1 0 1.5h-2a.75.75 0 0 1 0-1.5h.25v-2h-.25a.75.75 0 0 1-.75-.75ZM8 6a1 1 0 1 1 0-2 1 1 0 0 1 0 2Z"></path></svg>Note</span></p><p dir="auto">The font I'm using on the app is <a href="https://fonts.google.com/specimen/Lato?preview.text=Whereas%20recognition%20of%20the%20inherent%20dignity&amp;preview.text_type=custom" rel="nofollow">Lato Regular 400 from google fonts</a>.</p>
<p dir="auto">See the below at <a href="/src/index.html">index.html</a>:</p>
<pre lang="html" class="notranslate"><code class="notranslate">&lt;link rel="preconnect" href="https://fonts.googleapis.com"&gt;
&lt;link rel="preconnect" href="https://fonts.gstatic.com" crossorigin&gt;
&lt;link href="https://fonts.googleapis.com/css2?family=Lato&amp;display=swap" rel="stylesheet"&gt;
</code></pre>
<p dir="auto">And CSS at <a href="/src/styles/styles.less">styles.less</a>:</p>
<pre lang="css" class="notranslate"><code class="notranslate">@font-family-lato: 'Lato', sans-serif;

body {
  font-family: @font-family-lato;
}
</code></pre></div>
<h4 dir="auto">Scenario 3: Differences when running the test in/out of debug mode</h4>
<p dir="auto">This one might not be a bug but it still feels like a bit of an issue.</p>
<ul dir="auto">
<li>Install the <a href="https://marketplace.visualstudio.com/items?itemName=ms-playwright.playwright" rel="nofollow">playwright extension for VSCode</a>.</li>
<li>Go to the root of the repo.</li>
<li>Uncomment the line <code class="notranslate">// ignoreDefaultArgs: ["--hide-scrollbars"]</code> in the <a href="/playwright.config.ts">playwright.config.ts</a> file.</li>
<li>Generate the test screenshot by running the tests with <code class="notranslate">npm run test</code>.</li>
<li>Go to VS Code and debug the test using the playwright extension for VS Code.</li>
<li>Open the html report via <code class="notranslate">npx playwright show-report tests/test-results/reporters/html</code> and look at the differences in the screenshots.</li>
</ul>
<p dir="auto">The debug will fail because there are differences in the screenshots:</p>
<ul dir="auto">
<li>the screenshot from debug mode will have will have the appropriate blur. Perhaps it runs with hardware acceleration enabled and therefore shows as expected (equal to what is visible when running the app with <code class="notranslate">npm run start</code>).</li>
<li>the screenshot from debug mode will have slight differences in the font. Or perhaps this is incorrect, perhaps the issue is not with the font but with the fact that the scrollbar width is slightly different and the effect that the difference in blur also has on the fonts.</li>
<li>the screenshot from debug mode seems to have "weaker" coloured red line (the red line below the label <code class="notranslate">Where do you study or work?</code>). This again might just be related with the blur difference.</li>
<li>the screenshot from debug mode has a scrollbar with smaller width.</li>
</ul>
<details>
  <summary>Diff</summary>
<p dir="auto"><a target="_blank" rel="noopener noreferrer" href="https://github.com/edumserrano/playwright-bug-vrt-diff/tree/main/docs/images/debug-diff.png"><img src="https://github.com/edumserrano/playwright-bug-vrt-diff/tree/main/docs/images/debug-diff.png" alt="diff" style="max-width: 100%;"></a></p>
</details>
<details>
  <summary>Screenshot from running via "npm run test"</summary>
<p dir="auto"><a target="_blank" rel="noopener noreferrer" href="https://github.com/edumserrano/playwright-bug-vrt-diff/tree/main/docs/images/not-debug.png"><img src="https://github.com/edumserrano/playwright-bug-vrt-diff/tree/main/docs/images/not-debug.png" alt="Windows" style="max-width: 100%;"></a></p>
</details>
<details>
  <summary>Screenshot from running in debug via VS Code playwright extension</summary>
<p dir="auto"><a target="_blank" rel="noopener noreferrer" href="https://github.com/edumserrano/playwright-bug-vrt-diff/tree/main/docs/images/debug.png"><img src="https://github.com/edumserrano/playwright-bug-vrt-diff/tree/main/docs/images/debug.png" alt="Ubuntu" style="max-width: 100%;"></a></p>
</details>
<div class="markdown-alert markdown-alert-note" dir="auto"><p dir="auto"><span class="color-fg-accent text-semibold d-inline-flex flex-items-center mb-1"><svg class="octicon octicon-info mr-2" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path d="M0 8a8 8 0 1 1 16 0A8 8 0 0 1 0 8Zm8-6.5a6.5 6.5 0 1 0 0 13 6.5 6.5 0 0 0 0-13ZM6.5 7.75A.75.75 0 0 1 7.25 7h1a.75.75 0 0 1 .75.75v2.75h.25a.75.75 0 0 1 0 1.5h-2a.75.75 0 0 1 0-1.5h.25v-2h-.25a.75.75 0 0 1-.75-.75ZM8 6a1 1 0 1 1 0-2 1 1 0 0 1 0 2Z"></path></svg>Note</span></p><p dir="auto">To make the comparison easier in debug vs not debug I've added <code class="notranslate">ignoreDefaultArgs: ["--hide-scrollbars"]</code> in the <a href="/playwright.config.ts">playwright.config.ts</a> file. This makes sure scrollbar is always visible whether running the test in debug mode or not.</p>
<p dir="auto">However I'd say that this was also unexpected to me even if it's default behavior. Couldn't find any mention to the fact that the scrollbar is disabled in headless mode when searching the <a href="https://playwright.dev/docs" rel="nofollow">playwright docs</a>. Perhaps a chance for improving the docs?</p></div>
<h3 dir="auto">Related</h3>
<p dir="auto">Perhaps related with the effort on <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="968519938" data-permission-text="Title is private" data-url="https://github.com/microsoft/playwright/issues/8161" data-hovercard-type="issue" data-hovercard-url="/microsoft/playwright/issues/8161/hovercard" href="https://github.com/microsoft/playwright/issues/8161">#8161</a></p>