Can you undo close all Finder windows (alt + click the red close button)? I just inadvertently closed around 20 Finder windows in one click, each related to ongoing projects in them, by clicking the red window button and having "alt" key pressed. 

Is there something I can do to reopen all the windows that have just been closed?

It might be too late for me now but if I make this mistake again it will save me a lot of time to reopen these windows.
This functionality is not available natively in Finder, but it can be gained by the use of third party software such as XtraFinder and perhaps Default Folder X. Please note though that you would have to check compatibility as you don't specify what version of macOS you're using.

Another option that may interest you is to access your recent folders. To do this:


Ensure you have Finder active
Click on the Go menu
Hover over the Recent Folders sub-menu
You'll now see your most recently opened folders
Select the folders one by one to reopen them


Notes: 


This is not an undo option, but at least you can easily reopen most of your Finder windows (although not likely as many as 20 of them).
I have no affiliation whatsoever with either XtraFinder or Default Folder X.