You may install tensorflow first by following instructions GitHub - apple/tensorflow_macos: TensorFlow for macOS 11.0+ accelerated using Apple's ML Compute framework. 264. This step should solve the NumPy issue.
I followed instructions from How to install TensorFlow and NumPy on Apple Silicon (M1) - YouTube 243 and could get tensorflow installed.
To install Pytorch with pip3, I enabled the terminal to run with Rosetta 2. For that, you need to right-click on the terminal app in Applications, then select ‘Get Info’ and select the option ‘Open using Rosetta’.
After enabling the option,  launching terminal would automatically run all subsequent commands in Rosetta 2  and therefore commands that work for Intel-based Macs would work for M1 Mac. So you can simply install pytorch with pip3 install torch .
I believe running it in Rosetta would not get you the optimum performance for the platform but it may help you to get started with at least using your M1 before PyTorch is compiled for apple chip.