I upvoted because I think you are absolutely correct.
Better security could be had with a two-factor system -- plug the phone into a cryptobox to decrypt. Having everything in one place is vulnerable.