Information measure of rank changes?
I'm trying to compute the information content of a function that reranks a list. Perhaps more precisely, I'm trying to compare the information content of different arguments to that function.
Suppose that I have a scoring function, S(i,n) that takes and item, i, and gives is a score of, say, 0-1, where n is some sort of parameter that we'll get back to in a moment. I can run this on a set of items, i, and thereby (ignoring ties) rank these items S(i,n1) -> {s1, s2, s3 ...} where these scores correspond to the items, and so can be ordered by simple sorting to rank the items. (It's possible that the actual scores are enough, that we don't need the rank ordering, in what is to follow.)
The scoring function depends on n, so S(i,n1) usually != S(i,n2) (Again, ignoring the unusual case of a tie.) What I'm interested in is measuring the "information utility" of the ns. 143
So, for example, in the (neglected by definition) case where so S(i,n1) == S(i,n2), n2 provide no additional information over n1 (and v.v.), but in the more common case where S(i,n1) != S(i,n2) the scores, and perhaps the rankings will differ, and so n2 provides some more information over n1 (and v.v.)
(Note that the reason I keep harping on the ranking and not the scores is that there is an infinitude of cases where the scores will change by the rankings NOT change! I really care a more about the rankings than the scores. Say, for example, I'm trying to choose the best drug, or the best stock, or something like that. There which comes in first matters more than the specific scores.)
There are, of course, tests, like the Wilcoxon's and Spearman's that tell me on a Z scale how much the rankings differ under different ns. And perhaps the simplest thing to do is just to pick the largest n and call the most informative n the one that gives the largest Z on one of those tests.
But is there a well understood way to directly measure the information change between two lists, that would seem to be a more theoretically well motivated way of doing this.
Thanks in advance for your suggestions. 946