Question: In the final exam of a course, a student has solved some problems in a really ingenious way.
He gets perfect score anyway.
What is the appropriate way to inform him about my appreciation of his solutions?
(Also what if I don't have the occasion to meet him again?)

Answer: 
Well, as a student, I will be glad if my work gets appreciated. Recognition like these act as confidence booster. But, don't overdo it, as it may make student think too high of himself. 

Assuming that the final exam sheets are returned to the students, you can attach a (very) short note appreciating his ingenuity. Trust me, I get excited even on a simple "Good Job!" on my exam sheet (my professor does it rarely though). Other option could be to write him a short email. You may invite him to have a discussion on what further he can do with the ideas he demonstrated in the answer sheet. Depending on how high you think of his subject knowledge, you can offer him a TA job, as pointed by Stephen Kolassa. 

But, most importantly, you should make sure that the student does something big with his talent, by referring him a book, an online course, or any other course in your or other department.
You already did: you gave him a perfect score.