Question: I know that in a transformer the power is conserved on both sides, and hence there's no power amplification as such. That's fine!

But if I instead want only a voltage amplification, then choosing proper turns ratio and connecting the load across the secondary would serve my purpose. So can you consider it as voltage amplifier? If yes, then why is it not used as one?

Answer: 
It's just a subtlety of the terminology. If there's no (capability of) power amplification, then we don't call it an "amplifier" even though the output voltage might be higher than the input voltage.

As others have pointed out, there are other functional differences between transformers and amplifiers, one being that power can flow both ways through a transformer but generally only one way through an amplifier.

Finally, we already have a word for a device that can change voltages without providing power gain, so we don't need to call these things *amplifiers*. We can just call them *transformers*.
Only for ideal transformers, power is conserved.  

Real transformers have losses, which may be load dependent. So, in that case, the 'output' voltage is not the gain times the 'input' voltage. 

For voltage amplifiers, you also want them to have unlimited (or at least large) current supply. This unlimited current is needed, so the output is not dependent to the load, i.e. the voltage should not collapse because of the load.  

For transformers, this 'output' current is dependent and limited.