<h3 dir="auto">System info</h3>
<ul dir="auto">
<li>Playwright Version: [v1.31.0]</li>
<li>Operating System: [All]</li>
<li>Browser: [All, Chromium, Firefox, WebKit]</li>
<li>Other info:</li>
</ul>
<p dir="auto">I am doing visual regression test with @web/test-runner and playwright library, after the playwright upgrade to 1.31.0 from 1.22.0 then screenshots started breaking due to fonts rendering change and screenshot width and height change, most of the issues are in firefox and webkit</p>
<p dir="auto"><strong>Context:</strong></p>
<ul dir="auto">
<li>Playwright Version: @playwright/test, @playwright/experimental-ct-react, playwright 1.27.1</li>
<li>Operating System: PopOS 20.04 LTS</li>
<li>Node.js version: 19.0.0</li>
<li>Browser: All</li>
<li>Extra:</li>
</ul>
<p dir="auto"><strong>Describe the bug</strong></p>
<p dir="auto">My tests were just working fine until a few days ago. I have no idea what I changed to make it behave this way. I have tried removing node_modules and <code class="notranslate">./playwright/.cache/</code> and re-installing, but the same errors occur. I have no idea why the tests are running twice and there is no info online about the error I'm seeing.</p>
<p dir="auto">Failed tests:</p>
<p dir="auto"><a target="_blank" rel="noopener noreferrer nofollow" href="https://camo.githubusercontent.com/505c0828506baf289224e654314737d93b3f9698d71dee915e09e38ff5f56c72/68747470733a2f2f692e696d6775722e636f6d2f4250577168324d2e706e67"><img src="https://camo.githubusercontent.com/505c0828506baf289224e654314737d93b3f9698d71dee915e09e38ff5f56c72/68747470733a2f2f692e696d6775722e636f6d2f4250577168324d2e706e67" alt="test failure" data-canonical-src="https://i.imgur.com/BPWqh2M.png" style="max-width: 100%;"></a></p>
<p dir="auto"><strong>Code Snippet</strong></p>
<p dir="auto">One of the tests that both fails and succeeds:</p>
<div class="highlight highlight-source-ts notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="import { test, expect } from '@playwright/experimental-ct-react'

import FolderSelector from './FolderSelector'

test.describe('has correct styles', async () =&gt; {
  test('has full width', async ({ mount }) =&gt; {
    const component = await mount(&lt;FolderSelector /&gt;)

    const button = component.locator('button')

    await expect(component).toHaveClass(/w-full/)
    await expect(button).toHaveClass(/w-full/)
  })

  test('has correct icon', async ({ mount }) =&gt; {
    const component = await mount(&lt;FolderSelector /&gt;)

    const button = component.locator('button&gt;svg')

    await expect(button).toHaveScreenshot()
  })
})

test('has correct text', async ({ mount }) =&gt; {
  const component = await mount(&lt;FolderSelector /&gt;)

  const button = component.locator('button')

  await expect(button).toContainText('Select Data Folder')
})

test('has file input', async ({ mount }) =&gt; {
  const component = await mount(&lt;FolderSelector /&gt;)

  const input = component.locator('input')

  await expect(input).toHaveAttribute('type', 'file')
})"><pre class="notranslate"><span class="pl-k">import</span> <span class="pl-kos">{</span> <span class="pl-s1">test</span><span class="pl-kos">,</span> <span class="pl-s1">expect</span> <span class="pl-kos">}</span> <span class="pl-k">from</span> <span class="pl-s">'@playwright/experimental-ct-react'</span>

<span class="pl-k">import</span> <span class="pl-smi">FolderSelector</span> <span class="pl-k">from</span> <span class="pl-s">'./FolderSelector'</span>

<span class="pl-s1">test</span><span class="pl-kos">.</span><span class="pl-c1">describe</span><span class="pl-kos">(</span><span class="pl-s">'has correct styles'</span><span class="pl-kos">,</span> <span class="pl-k">async</span> <span class="pl-kos">(</span><span class="pl-kos">)</span> <span class="pl-c1">=&gt;</span> <span class="pl-kos">{</span>
  <span class="pl-s1">test</span><span class="pl-kos">(</span><span class="pl-s">'has full width'</span><span class="pl-kos">,</span> <span class="pl-k">async</span> <span class="pl-kos">(</span><span class="pl-kos">{</span> mount <span class="pl-kos">}</span><span class="pl-kos">)</span> <span class="pl-c1">=&gt;</span> <span class="pl-kos">{</span>
    <span class="pl-k">const</span> <span class="pl-s1">component</span> <span class="pl-c1">=</span> <span class="pl-k">await</span> <span class="pl-en">mount</span><span class="pl-kos">(</span><span class="pl-kos">&lt;</span><span class="pl-smi">FolderSelector</span><span class="pl-kos"></span> <span class="pl-c1">/</span>&gt;<span class="pl-kos">)</span>

    <span class="pl-s1">const</span> <span class="pl-s1">button</span> <span class="pl-c1">=</span> <span class="pl-s1">component</span><span class="pl-kos">.</span><span class="pl-en">locator</span><span class="pl-kos">(</span><span class="pl-s">'button'</span><span class="pl-kos">)</span>

    <span class="pl-k">await</span> <span class="pl-en">expect</span><span class="pl-kos">(</span><span class="pl-s1">component</span><span class="pl-kos">)</span><span class="pl-kos">.</span><span class="pl-en">toHaveClass</span><span class="pl-kos">(</span><span class="pl-pds"><span class="pl-c1">/</span>w-full<span class="pl-c1">/</span></span><span class="pl-kos">)</span>
    <span class="pl-k">await</span> <span class="pl-en">expect</span><span class="pl-kos">(</span><span class="pl-s1">button</span><span class="pl-kos">)</span><span class="pl-kos">.</span><span class="pl-en">toHaveClass</span><span class="pl-kos">(</span><span class="pl-pds"><span class="pl-c1">/</span>w-full<span class="pl-c1">/</span></span><span class="pl-kos">)</span>
  <span class="pl-kos">}</span><span class="pl-kos">)</span>

  <span class="pl-en">test</span><span class="pl-kos">(</span><span class="pl-s">'has correct icon'</span><span class="pl-kos">,</span> <span class="pl-k">async</span> <span class="pl-kos">(</span><span class="pl-kos">{</span> mount <span class="pl-kos">}</span><span class="pl-kos">)</span> <span class="pl-c1">=&gt;</span> <span class="pl-kos">{</span>
    <span class="pl-k">const</span> <span class="pl-s1">component</span> <span class="pl-c1">=</span> <span class="pl-k">await</span> <span class="pl-en">mount</span><span class="pl-kos">(</span><span class="pl-kos">&lt;</span><span class="pl-smi">FolderSelector</span><span class="pl-kos"></span> <span class="pl-c1">/</span>&gt;<span class="pl-kos">)</span>

    <span class="pl-s1">const</span> <span class="pl-s1">button</span> <span class="pl-c1">=</span> <span class="pl-s1">component</span><span class="pl-kos">.</span><span class="pl-en">locator</span><span class="pl-kos">(</span><span class="pl-s">'button&gt;svg'</span><span class="pl-kos">)</span>

    <span class="pl-k">await</span> <span class="pl-en">expect</span><span class="pl-kos">(</span><span class="pl-s1">button</span><span class="pl-kos">)</span><span class="pl-kos">.</span><span class="pl-en">toHaveScreenshot</span><span class="pl-kos">(</span><span class="pl-kos">)</span>
  <span class="pl-kos">}</span><span class="pl-kos">)</span>
<span class="pl-kos">}</span><span class="pl-kos">)</span>

<span class="pl-s1">test</span><span class="pl-kos">(</span><span class="pl-s">'has correct text'</span><span class="pl-kos">,</span> <span class="pl-k">async</span> <span class="pl-kos">(</span><span class="pl-kos">{</span> mount <span class="pl-kos">}</span><span class="pl-kos">)</span> <span class="pl-c1">=&gt;</span> <span class="pl-kos">{</span>
  <span class="pl-k">const</span> <span class="pl-s1">component</span> <span class="pl-c1">=</span> <span class="pl-k">await</span> <span class="pl-en">mount</span><span class="pl-kos">(</span><span class="pl-kos">&lt;</span><span class="pl-smi">FolderSelector</span><span class="pl-kos"></span> <span class="pl-c1">/</span>&gt;<span class="pl-kos">)</span>

  <span class="pl-s1">const</span> <span class="pl-s1">button</span> <span class="pl-c1">=</span> <span class="pl-s1">component</span><span class="pl-kos">.</span><span class="pl-en">locator</span><span class="pl-kos">(</span><span class="pl-s">'button'</span><span class="pl-kos">)</span>

  <span class="pl-k">await</span> <span class="pl-en">expect</span><span class="pl-kos">(</span><span class="pl-s1">button</span><span class="pl-kos">)</span><span class="pl-kos">.</span><span class="pl-en">toContainText</span><span class="pl-kos">(</span><span class="pl-s">'Select Data Folder'</span><span class="pl-kos">)</span>
<span class="pl-kos">}</span><span class="pl-kos">)</span>

<span class="pl-s1">test</span><span class="pl-kos">(</span><span class="pl-s">'has file input'</span><span class="pl-kos">,</span> <span class="pl-k">async</span> <span class="pl-kos">(</span><span class="pl-kos">{</span> mount <span class="pl-kos">}</span><span class="pl-kos">)</span> <span class="pl-c1">=&gt;</span> <span class="pl-kos">{</span>
  <span class="pl-k">const</span> <span class="pl-s1">component</span> <span class="pl-c1">=</span> <span class="pl-k">await</span> <span class="pl-en">mount</span><span class="pl-kos">(</span><span class="pl-kos">&lt;</span><span class="pl-smi">FolderSelector</span><span class="pl-kos"></span> <span class="pl-c1">/</span>&gt;<span class="pl-kos">)</span>

  <span class="pl-s1">const</span> <span class="pl-s1">input</span> <span class="pl-c1">=</span> <span class="pl-s1">component</span><span class="pl-kos">.</span><span class="pl-en">locator</span><span class="pl-kos">(</span><span class="pl-s">'input'</span><span class="pl-kos">)</span>

  <span class="pl-k">await</span> <span class="pl-en">expect</span><span class="pl-kos">(</span><span class="pl-s1">input</span><span class="pl-kos">)</span><span class="pl-kos">.</span><span class="pl-en">toHaveAttribute</span><span class="pl-kos">(</span><span class="pl-s">'type'</span><span class="pl-kos">,</span> <span class="pl-s">'file'</span><span class="pl-kos">)</span>
<span class="pl-kos">}</span><span class="pl-kos">)</span></pre></div>
<p dir="auto">The passing tests look completely normal:</p>
<p dir="auto"><a target="_blank" rel="noopener noreferrer nofollow" href="https://camo.githubusercontent.com/4e611435a7a8e58314bf97fec9da9e3c24c40e40674ee021717a0f045f7f3561/68747470733a2f2f692e696d6775722e636f6d2f3050476a394f4d2e706e67"><img src="https://camo.githubusercontent.com/4e611435a7a8e58314bf97fec9da9e3c24c40e40674ee021717a0f045f7f3561/68747470733a2f2f692e696d6775722e636f6d2f3050476a394f4d2e706e67" alt="passing-test" data-canonical-src="https://i.imgur.com/0PGj9OM.png" style="max-width: 100%;"></a></p>
<p dir="auto">Here's the error from the respective failing test:</p>
<p dir="auto"><a target="_blank" rel="noopener noreferrer nofollow" href="https://camo.githubusercontent.com/b68aeb68fbd65bb1fff2965be947b14c446a3ce92e540f2d5d81dbeff4dce9f6/68747470733a2f2f692e696d6775722e636f6d2f743655794f71702e706e67"><img src="https://camo.githubusercontent.com/b68aeb68fbd65bb1fff2965be947b14c446a3ce92e540f2d5d81dbeff4dce9f6/68747470733a2f2f692e696d6775722e636f6d2f743655794f71702e706e67" alt="failing-test" data-canonical-src="https://i.imgur.com/t6UyOqp.png" style="max-width: 100%;"></a></p>
<p dir="auto"><code class="notranslate">playwright-ct.config.js</code> - almost completely unchanged from default:</p>
<div class="highlight highlight-source-js notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="import type { PlaywrightTestConfig } from '@playwright/experimental-ct-react'
import { devices } from '@playwright/experimental-ct-react'

/**
 * See https://playwright.dev/docs/test-configuration.
 */
const config: PlaywrightTestConfig = {
  testDir: './',
  /* The base directory, relative to the config file, for snapshot files created with toMatchSnapshot and toHaveScreenshot. */
  snapshotDir: './__snapshots__',
  /* Folders and files to ignore. */
  testIgnore: ['e2e/**'],
  /* Maximum time one test can run for. */
  timeout: (process.env.CI ? 30 : 10) * 1000,
  /* Run tests in files in parallel */
  fullyParallel: true,
  /* Fail the build on CI if you accidentally left test.only in the source code. */
  forbidOnly: !!process.env.CI,
  /* Retry on CI only */
  retries: process.env.CI ? 2 : 0,
  /* Opt out of parallel tests on CI. */
  workers: process.env.CI ? 1 : undefined,
  /* Reporter to use. See https://playwright.dev/docs/test-reporters */
  reporter: 'html',
  /* Shared settings for all the projects below. See https://playwright.dev/docs/api/class-testoptions. */
  use: {
    /* Collect trace when retrying the failed test. See https://playwright.dev/docs/trace-viewer */
    trace: 'on-first-retry',

    /* Port to use for Playwright component endpoint. */
    ctPort: 3100,

    ctViteConfig: {
      css: {
        preprocessorOptions: {
          less: {
            javascriptEnabled: true,
          },
        },
      },
      resolve: {
        alias: [{ find: /^~/, replacement: '' }],
      },
    },
  },

  /* Configure projects for major browsers */
  projects: [
    {
      name: 'chromium',
      use: {
        ...devices['Desktop Chrome'],
      },
    },
    // {
    //   name: 'firefox',
    //   use: {
    //     ...devices['Desktop Firefox'],
    //   },
    // },
    // {
    //   name: 'webkit',
    //   use: {
    //     ...devices['Desktop Safari'],
    //   },
    // },
  ],
}

export default config"><pre class="notranslate"><span class="pl-k">import</span> <span class="pl-s1">type</span> <span class="pl-kos">{</span> <span class="pl-v">PlaywrightTestConfig</span> <span class="pl-kos">}</span> <span class="pl-k">from</span> <span class="pl-s">'@playwright/experimental-ct-react'</span>
<span class="pl-k">import</span> <span class="pl-kos">{</span> <span class="pl-s1">devices</span> <span class="pl-kos">}</span> <span class="pl-k">from</span> <span class="pl-s">'@playwright/experimental-ct-react'</span>

<span class="pl-c">/**</span>
<span class="pl-c"> * See https://playwright.dev/docs/test-configuration.</span>
<span class="pl-c"> */</span>
<span class="pl-k">const</span> <span class="pl-s1">config</span>: <span class="pl-v">PlaywrightTestConfig</span> <span class="pl-c1">=</span> <span class="pl-kos">{</span>
  <span class="pl-c1">testDir</span>: <span class="pl-s">'./'</span><span class="pl-kos">,</span>
  <span class="pl-c">/* The base directory, relative to the config file, for snapshot files created with toMatchSnapshot and toHaveScreenshot. */</span>
  <span class="pl-c1">snapshotDir</span>: <span class="pl-s">'./__snapshots__'</span><span class="pl-kos">,</span>
  <span class="pl-c">/* Folders and files to ignore. */</span>
  <span class="pl-c1">testIgnore</span>: <span class="pl-kos">[</span><span class="pl-s">'e2e/**'</span><span class="pl-kos">]</span><span class="pl-kos">,</span>
  <span class="pl-c">/* Maximum time one test can run for. */</span>
  <span class="pl-c1">timeout</span>: <span class="pl-kos">(</span><span class="pl-s1">process</span><span class="pl-kos">.</span><span class="pl-c1">env</span><span class="pl-kos">.</span><span class="pl-c1">CI</span> ? <span class="pl-c1">30</span> : <span class="pl-c1">10</span><span class="pl-kos">)</span> <span class="pl-c1">*</span> <span class="pl-c1">1000</span><span class="pl-kos">,</span>
  <span class="pl-c">/* Run tests in files in parallel */</span>
  <span class="pl-c1">fullyParallel</span>: <span class="pl-c1">true</span><span class="pl-kos">,</span>
  <span class="pl-c">/* Fail the build on CI if you accidentally left test.only in the source code. */</span>
  <span class="pl-c1">forbidOnly</span>: <span class="pl-c1">!</span><span class="pl-c1">!</span><span class="pl-s1">process</span><span class="pl-kos">.</span><span class="pl-c1">env</span><span class="pl-kos">.</span><span class="pl-c1">CI</span><span class="pl-kos">,</span>
  <span class="pl-c">/* Retry on CI only */</span>
  <span class="pl-c1">retries</span>: <span class="pl-s1">process</span><span class="pl-kos">.</span><span class="pl-c1">env</span><span class="pl-kos">.</span><span class="pl-c1">CI</span> ? <span class="pl-c1">2</span> : <span class="pl-c1">0</span><span class="pl-kos">,</span>
  <span class="pl-c">/* Opt out of parallel tests on CI. */</span>
  <span class="pl-c1">workers</span>: <span class="pl-s1">process</span><span class="pl-kos">.</span><span class="pl-c1">env</span><span class="pl-kos">.</span><span class="pl-c1">CI</span> ? <span class="pl-c1">1</span> : <span class="pl-c1">undefined</span><span class="pl-kos">,</span>
  <span class="pl-c">/* Reporter to use. See https://playwright.dev/docs/test-reporters */</span>
  <span class="pl-c1">reporter</span>: <span class="pl-s">'html'</span><span class="pl-kos">,</span>
  <span class="pl-c">/* Shared settings for all the projects below. See https://playwright.dev/docs/api/class-testoptions. */</span>
  <span class="pl-c1">use</span>: <span class="pl-kos">{</span>
    <span class="pl-c">/* Collect trace when retrying the failed test. See https://playwright.dev/docs/trace-viewer */</span>
    <span class="pl-c1">trace</span>: <span class="pl-s">'on-first-retry'</span><span class="pl-kos">,</span>

    <span class="pl-c">/* Port to use for Playwright component endpoint. */</span>
    <span class="pl-c1">ctPort</span>: <span class="pl-c1">3100</span><span class="pl-kos">,</span>

    <span class="pl-c1">ctViteConfig</span>: <span class="pl-kos">{</span>
      <span class="pl-c1">css</span>: <span class="pl-kos">{</span>
        <span class="pl-c1">preprocessorOptions</span>: <span class="pl-kos">{</span>
          <span class="pl-c1">less</span>: <span class="pl-kos">{</span>
            <span class="pl-c1">javascriptEnabled</span>: <span class="pl-c1">true</span><span class="pl-kos">,</span>
          <span class="pl-kos">}</span><span class="pl-kos">,</span>
        <span class="pl-kos">}</span><span class="pl-kos">,</span>
      <span class="pl-kos">}</span><span class="pl-kos">,</span>
      <span class="pl-c1">resolve</span>: <span class="pl-kos">{</span>
        <span class="pl-c1">alias</span>: <span class="pl-kos">[</span><span class="pl-kos">{</span> <span class="pl-c1">find</span>: <span class="pl-pds"><span class="pl-c1">/</span><span class="pl-cce">^</span>~<span class="pl-c1">/</span></span><span class="pl-kos">,</span> <span class="pl-c1">replacement</span>: <span class="pl-s">''</span> <span class="pl-kos">}</span><span class="pl-kos">]</span><span class="pl-kos">,</span>
      <span class="pl-kos">}</span><span class="pl-kos">,</span>
    <span class="pl-kos">}</span><span class="pl-kos">,</span>
  <span class="pl-kos">}</span><span class="pl-kos">,</span>

  <span class="pl-c">/* Configure projects for major browsers */</span>
  <span class="pl-c1">projects</span>: <span class="pl-kos">[</span>
    <span class="pl-kos">{</span>
      <span class="pl-c1">name</span>: <span class="pl-s">'chromium'</span><span class="pl-kos">,</span>
      <span class="pl-c1">use</span>: <span class="pl-kos">{</span>
        ...<span class="pl-s1">devices</span><span class="pl-kos">[</span><span class="pl-s">'Desktop Chrome'</span><span class="pl-kos">]</span><span class="pl-kos">,</span>
      <span class="pl-kos">}</span><span class="pl-kos">,</span>
    <span class="pl-kos">}</span><span class="pl-kos">,</span>
    <span class="pl-c">// {</span>
    <span class="pl-c">//   name: 'firefox',</span>
    <span class="pl-c">//   use: {</span>
    <span class="pl-c">//     ...devices['Desktop Firefox'],</span>
    <span class="pl-c">//   },</span>
    <span class="pl-c">// },</span>
    <span class="pl-c">// {</span>
    <span class="pl-c">//   name: 'webkit',</span>
    <span class="pl-c">//   use: {</span>
    <span class="pl-c">//     ...devices['Desktop Safari'],</span>
    <span class="pl-c">//   },</span>
    <span class="pl-c">// },</span>
  <span class="pl-kos">]</span><span class="pl-kos">,</span>
<span class="pl-kos">}</span>

<span class="pl-k">export</span> <span class="pl-k">default</span> <span class="pl-s1">config</span></pre></div>
<p dir="auto">The e2e tests on my main Electron process are working just fine and only run once like normal. I have absolutely no idea why the component tests started running twice and failing.</p>
<p dir="auto">Thanks for any help.</p>