Should I use an ORM like SQLAlchemy for a lightweight Flask web service?
Your question is too open to anyone guarantee SQLAlchemy is not a good fit, but SQLAlchemy probably will never be your problem to handle scalability. You'll have to handle almost the same problems with or without SQLAlchemy.
Of course SQLAlchemy has some performance impact, it is a layer above the database driver, but it also will help you a lot.
That said, if you want to use SQLAlchemy to help with your security (SQL escaping), you can use the SQLAlchemy just to execute your raw SQL queries, but I recommend it to fix specific bottlenecks, never to avoid the ORM.
I'd rather just use raw MySQL, but I'm wondering if I'm missing something besides security concerns.  Does SQLAlchemy or another ORM handle scaling any better than just using pymysql or MySQLdb?