Question: I know that in a transformer the power is conserved on both sides, and hence there's no power amplification as such. That's fine!

But if I instead want only a voltage amplification, then choosing proper turns ratio and connecting the load across the secondary would serve my purpose. So can you consider it as voltage amplifier? If yes, then why is it not used as one?

Answer: 
It's just a subtlety of the terminology. If there's no (capability of) power amplification, then we don't call it an "amplifier" even though the output voltage might be higher than the input voltage.

As others have pointed out, there are other functional differences between transformers and amplifiers, one being that power can flow both ways through a transformer but generally only one way through an amplifier.

Finally, we already have a word for a device that can change voltages without providing power gain, so we don't need to call these things *amplifiers*. We can just call them *transformers*.
Amplifiers typically provide input-output isolation. Changes in output energy need not require more input energy (tho the VDD rail must support the output demands)

A transformer of course does not provide isolation.