Question: I am a Trinity grade 5 pianist and have been playing piano for 2-3 years(earlier played keyboard).

I find very difficult to sight read. I usually play the passage once very carefully and memorize it well so I don't have look at the music while playing. I play while continuously looking down at the keys and find it difficult to lift my head up and register the notes and then play. I know all the notes on the staff, but I can't register them in time to play them on the spot.

What should I do to get better at sight reading?

Answer: 
Sight read more! The best way to get better at doing something is to practice it. Force yourself to keep your eyes glued to the page. Play slow, deliberately, and correct your mistakes.
I'm going to approach this answer from a different perspective, my own. For years I had trouble sight reading because it took more time to focus on the notes and be certain of their place on the stave. I have problems with my sight where I see double images when I look at things from certain angles so I can see double notes, double staves etc. when trying to sight read. Very confusing when sight reading. I had my eyes checked by several doctors with no solution, but I kept trying and finally found a doctor who knew the solution. He tested me, gave me a new prescription for glasses and now I don't see double when I'm wearing them. If your problem stems from eye problems, you'll need to get that fixed before you do anything else.