I found two limitations here, both of which @vercel/og also seems to be battling with from a quick glance at their code:
1. You need to use the WASM version.
2. All resources need to be embedded
3. As an implication of 2, you cannot render <text> elements either