Stack Exchange
How to click and select element within async function page.evalue in pupeteer
Trying to iterate over invalid fields and entering/selecting values on the page.
const enterTextFields = async (page) => {
  const txtQuestText = await page.evaluate(() =>
    Array.from(
      document.querySelectorAll('div[aria-invalid="true"]'),
      async (element) => {
        const span = element.querySelector(
          'label span[data-test-form-element-label-title="true"]'
        );
        const ques = span.textContent.trim();
        const ans = await window.getAns(ques);
        if (ans) {
          const txtField = element.querySelector('div input[type="text"]');
          if (txtField) {
            txtField.type(ans);
          }
          const dropdownField = element.querySelector(
            "div select[data-test-fb-dropdown-select]"
          );
          if (dropdownField) {
            dropdownField.select(ans);
          }
        }
        return ques + " : " + ans;
      }
    )
  );
  console.log("txtQuestText : ", txtQuestText);
}

On the execution of the above page.evaluate, neither text nor dropdown field is getting filled even though there are ans available for the ques.
In txtQuestText getting:
(2) [{…}, {…}]
0: {}
1: {}
length: 2
[[Prototype]]: Array(0)
[[Prototype]]: Object

If I remove async from async (element) => {, await window.getAns(ques) call and if block then I can see the list of correct Ques strings in txtQuestText.
Cannot remove async due to await window.getAns(ques).
Not sure what is going wrong and or how to use it. Please Help!
HTML Page sample:
<div ...>
    <div ... aria-invalid="true">
        <label ...>
            <span ... data-test-form-element-label-title="true">
                Are you currently based in USA? <!-- This is the ques -->
            </span>
            <span ...>
              Required
            </span>
        </label>
        <div ...>
            <select ... data-test-fb-dropdown-select="">
                <option value="">Select an option</option>
                <option value="Yes">Yes</option>
                <option value="No">No</option>
                <!-- Need to select the answer here -->
            </select>
        </div>
        <p ... data-test-form-element-error-message="true">
            Please enter a valid answer
        </p>
    </div>
</div>

<div>
    <div ... aria-invalid="true">
        <label ...>
            <span ... data-test-form-element-label-title="true">
                What is phone number? <!-- This is the ques -->
            </span>
            <span ...>
              Required
            </span>
        </label>
        <div ...>
            <input ... type="text" >
            <!-- Need to enter the answer here -->
            </input>
        </div>
        <p ... data-test-form-element-error-message="true">
            Please enter a valid answer
        </p>
    </div>
</div>


Can you share the page and/or a [mcve]? Thanks.

Thank you for the reply @ggorlen, in the above example, you can ignore the console.log and one of the nested if blocks. The rest must be there because I must show that I am doing querySelector on each element.

The problem is, I can't run it, so I'm basically guessing as to what the problem/solution is. That makes it really difficult to answer your question and usually results in misunderstandings, back-and-forth and wasted time. I suspect most of the code isn't really relevant to the problem other than the fact that there's an async mapper that isn't being awaited, and that's easy enough to minimize. Does adding Promise.all() around the array resolve the issue on your use case? If not, please share more details (preferably, share a link to the site) so I can fix my answer.

Well, it worked only if I want to return the list of ans. Because within (element) => { it is a promise I cannot do txtField.type([object Promise]) is has to be txtField.type('50000').
The use case is pretty simple, click on the next button then the mandatory fields will give the error. Then check for div[aria-invalid="true"] which are mandatory but empty fields. Iterate through it, read its label (ques) by label span tag and get the value to be entered from getAns function, and set the value in its text field by div input.

Please suggest if there's better way to do this.

What is txtField.type()? element.type() is generally a Puppeteer function, not a native DOM function (that I know of) and you can't run Node/Puppeteer code in the browser. Same thing for dropdownField.select(ans);--I think you're confusing Puppeteer and browser code. So if you run this, I'd expect it to throw an error.

Let us continue this discussion in chat.

I can't run your code, but the problem appears to be returning an array from evaluate rather than a single promise object. Puppeteer isn't smart enough to realize this is an array of promises, so it doesn't resolve them. The promises degrade into empty objects once they pass through the serialization/deserialization process.
Here's a reproducible example:
const puppeteer = require("puppeteer"); // ^18.0.4

let browser;
(async () => {
  browser = await puppeteer.launch();
  const [page] = await browser.pages();
  const html = "<html><body><p>test</p><p>test2</p></body></html>";
  await page.setContent(html);

  const result = await page.evaluate(() =>
    Array.from(
      document.querySelectorAll("p"),
      async el => el.textContent // pretend we need to await in this func
    )
  );

  console.log(result); // => [ {}, {} ]
})()
  .catch(err => console.error(err))
  .finally(() => browser?.close())
;

The fix is to return a single promise rather than an array of promises:
const puppeteer = require("puppeteer"); // ^18.0.4

let browser;
(async () => {
  browser = await puppeteer.launch();
  const [page] = await browser.pages();
  const html = "<html><body><p>test</p><p>test2</p></body></html>";
  await page.setContent(html);

  const result = await page.evaluate(() =>
    Promise.all(Array.from(
      document.querySelectorAll("p"),
      async el => el.textContent
    ))
  );

  console.log(result); // => [ 'test', 'test2' ]
})()
  .catch(err => console.error(err))
  .finally(() => browser?.close())
;


Thank you @ggorlen. Yes, it was returning promises and now I am getting actual ans text instead of promises just by enclosing Array.from( call into Promise.all( in the question example.
Now it seems I need to look for the correct function instead of element.select for select[data-test-fb-dropdown-select] types of elements. Could please help me with the same?

Glad it's working, but I'm totally flying blind here and I can't see the page or its source so I'm not sure what you're trying to do. Are you trying to call a Puppeteer function in the browser? That won't work--either change the select value with native DOM functions or return an array of elementHandles with page.$$ and manipulate it from Node/Puppeteer. If you share the page and explain what you're trying to do, I can help and give a complete solution, otherwise, I can only guess and make general, handwavey statements and it'll probably be a waste of your time as well as mine.

Hi @ggorlen, I have added the Sample HTML input form page in the question.
The basic goal is to fill in the mandatory fields and keep on clicking the next button until reaches the Finish button or any question is missing in the answer. If a question is missing in the file answers.json then add the question in the file, skip the current working form and move to the next one.

Thanks, but like I said there's too much back-and-forth here and not enough clarity, so we're just burning cycles and wasting time. Feel free to ping me when you have a runnable [mcve], complete, valid HTML that shows the button and the JSON and a full specification and I can very likely provide you a complete, verified working solution. Thanks.
