Use Google speech to text instead of Apple's Google currently has much higher quality speech transcription. You can try it easily by going to doc.new and pressing Cmd-Shift-S and talking. Compare to using the built in system from Apple (pressing Fn twice).

Ideally there would be a good way to do this, but if that doesn't work, hacky is the next best thing. An AppleScript or Hammerspoon script that instantly pops up a Google Docs window, and then inserts the result is better than nothing.

Given that things like this https://dictation.io/speech exist, or this jsfiddle it seems like it should be possible to at least have an extension that can interface with other applications to enable dictation input, but I haven't found any yet. 

Is there any way to somehow use this instead of the built in transcription, e.g. with a keyboard shortcut?
Check out the Voice In Speech to Text Chrome Extension. It builds on Google's speech to text to allow you to dictate directly on many websites like Gmail, Wordpress, Zendesk, Facebook, etc. Note: it works only in the Chrome browser.
You can download from the Chrome Webstore