I do not really understand what you exactly mean with similarity especially if you want to talk about meaning. You would need a dataset to denote meaning unto words. A popular example of this would be sentiment analysis. If you got a lot of textual data, say tweets from twitter, you might want to know if the data is mostly positive or negative. To do this you would find a dataset of similar nature who has labelled the data already into categories. Then you can use this data to classify the texts into categories (e.g with a Naive Bayes classifier). In this way you can denote meaning on texts computationally.
This would allow general evaluations but also evaluations on an input to input basis on how well they scored across different categories of meaning.
I'm not sure if that's what you're looking for in an answer though.
Say you have two random words ('yellow' and 'ambient' or 'goose' and 'kettle'). What tech could be used to rate how similar or different they are in meaning as informed by popular usage? For example, from 0 to 1 where antonyms are 0 and synonyms are 1, 'yellow' and 'ambient' might be 0.65 similar.
Note: I'm not talking about how close the two strings are to each other, but rather an approximation of how similar their meanings are.
How to Measure Similarity or Difference of Meaning Between Words?