This invention relates to technology for statistical prediction and, in particular, to technology for prediction based on Bayes procedure.
Conventionally, a wide variety of methods have been proposed to statistically predict a data on the basis of a sequence of data generated from the unknown source. Among the methods, Bayes prediction procedure has been widely known and has been described or explained in various textbooks concerned with statistics and so forth.
As a problem to be solved by such statistical prediction, there is a problem for sequentially predicting, by use of an estimation result, next data which appear after the data sequence. As regards this problem, proof has been made about the fact that a specific Bayes procedure exhibits a very good minimax property by using a particular prior distribution which may be referred to as Jeffreys prior distribution. Such a specific Bayes procedure will be called Jeffreys procedure hereinafter. This proof is done by B. Clarke and A. R. Barron in an article which is published in Journal of Statistical Planning and Inference, 41:37-60, 1994, and which is entitled xe2x80x9cJeffreys prior is asymptotically least favorable under entropy riskxe2x80x9d. This procedure is guaranteed to be always optimum whenever a probability distribution hypothesis class is assumed to be a general smooth model class, although some mathematical restrictions are required in strict sense.
Herein, let logarithmic regret be used as another index. In this event also, it is again proved that the Jeffery procedure has a minimax property on the assumption that a probability distribution hypothesis class belongs to an exponential family. This proof is made by J. Takeuchi and A. R. Barron in a paper entitled xe2x80x9cAsymptotically minimax regret for exponential familiesxe2x80x9d, in Proceedings of 20th Symposium on Information Theory and Its Applications, pp. 665-668, 1997.
Furthermore, the problem of the sequential prediction can be replaced by a problem which provides a joint (or simultaneous) probability distribution of a data sequence obtained by cumulatively multiplying prediction probability distributions.
These proofs suggest that the Jeffreys procedure can have excellent performance except that the prediction problem is sequential, when the performance measure is the logarithmic loss.
Thus, it has been proved by Clarke and Barron and by Takeuchi and Barron that the Bayes procedure is effective when the Jeffreys prior distribution is used. However, the Bayes procedure is effective only when the model class of the probability distribution is restricted to the exponential family which is very unique, in the case where the performance measure is the logarithmic regret instead of redundancy.
Under the circumstances, it is assumed that the probability distribution model class belongs to a general smooth model class which is different from the exponential family. In this case, the Jeffreys procedure described in above B. Clarke and A. R. Barron""s document does not guarantee the minimax property. To the contrary, it is confirmed by the instant inventors in this case that the Jeffreys procedure does not have the minimax property.
Furthermore, it often happens that a similar reduction of performance takes place in a general Bayes procedure different from the Jeffreys procedure when estimation is made by using the logarithmic regret in lieu of the redundancy.
It is an object of this invention to provide a method which is capable of preventing a reduction of performance.
It is a specific object of this invention to provide improved Jeffreys procedure which can accomplish a minimax property even when logarithmic regret is used a performance measure instead of redundancy.
According to a first embodiment of the invention, a Bayes mixture density calculator operable in response to a sequence of vectors "khgr"n=("khgr"1, "khgr"2, . . . , "khgr"n) selected from a vector value set "khgr" to produce a Bayes mixture density on occurrence of the "khgr"n, comprising a probability density calculator, supplied with a sequence of data "khgr"t and a vector value parameter u, for calculating a probability density for the "khgr"t, p("khgr"t|u), a Bayes mixture calculator for calculating a first approximation value of a Bayes mixture density pw("khgr"n) on the basis of a prior distribution w(u) predetermined by the probability density calculator to produce the first approximation value, an enlarged mixture calculator for calculating a second approximation value of a Bayes mixture m("khgr"n) on exponential fiber bundle in cooperation with the probability density calculator to produce the second approximation value, and a whole mixture calculator for calculating (1xe2x88x92xcex5)pw("khgr"n)+xcex5xc2x7m("khgr"n) to produce a calculation result by mixing the first approximation value of the Bayes mixture density pw("khgr"n) with a part of the second approximation value of the Bayes mixture m("khgr"n) at a rate of 1xe2x88x92xcex5:xcex5 to produce the calculation result where xcex5 is a value smaller than unity.
According to a second embodiment of the invention which can be modified based on the first embodiment of the invention, a Jeffreys mixture density calculator operable in response to a sequence of vector "khgr"n=(x1, x2, . . . , xn) selected from a vector value set "khgr" to produce a Bayes mixture density on occurrence of the xn, comprising a probability density calculator responsive to a sequence of data "khgr"t and a vector value parameter u for calculating a probability density p("khgr"t|u) for the "khgr"t, a Jeffreys mixture calculator for calculating a first approximation value of a Bayes mixture density pJ("khgr"n) based on a Jeffreys prior distribution wJ(u) in cooperation with the probability density calculator to produce the first approximation value, an enlarged mixture calculator for calculating a second approximation value of a Bayes mixture m("khgr"n) on exponential fiber bundle in cooperation with the probability density calculator to produce the second approximation value, and a whole mixture calculator for calculating (1xe2x88x92xcex5)pJ("khgr"n)+xcex5xc2x7m("khgr"n) to produce a calculation result by mixing the first approximation value of the Bayes mixture density pJ("khgr"n) with a part of the second approximation value of the Bayes mixture m("khgr"n) at a rate of 1xe2x88x92xcex5:xcex5 to produce the calculation result where xcex5 is a value smaller than unity.
Also, when hypothesis class is curved exponential family, it is possible to provide with a third embodiment of the invention by modifying the first embodiment of the invention. According to the third embodiment of the invention, a Bayes mixture density calculator operable in response to a sequence of vector "khgr"n=("khgr"1, "khgr"2, . . . , "khgr"n selected from a vector value set "khgr" to produce a Bayes mixture density on occurrence of the "khgr"n, comprising a probability density calculator responsive to a sequence of data "khgr"t and a vector value parameter u for outputting probability density p("khgr"t|u) for the "khgr"t on curved exponential family, a Bayes mixture calculator for calculating a first approximation value of a Bayes mixture density pw("khgr"n) on the basis of a prior distribution w(u) predetermined by the probability density calculator to produce the first approximation value, an enlarged mixture calculator for calculating a second approximation value of a Bayes mixture m("khgr"n) on exponential family including curved exponential family in cooperation with the probability density calculator to produce the second approximation value, and a whole mixture calculator for calculating (1xe2x88x92xcex5)pw("khgr"n)+xcex5xc2x7m("khgr"n) to produce a calculation result by mixing the first approximation value of the Bayes mixture density pw("khgr"n) with a part of the second approximation value of the Bayes mixture m("khgr"n) at a rate of 1xe2x88x92xcex5:xcex5 to produce the calculation result where xcex5 is a value smaller than unity.
According to a forth embodiment of the invention which can be modified based on the third embodiment of the invention, a Jeffreys mixture density calculator operable in response to a sequence of vector xn=(x1, x2, . . . , xn) selected from a vector value set "khgr" to produce a Bayes mixture density on occurrence of the "khgr"n, comprising a probability density calculator responsive to a sequence of data "khgr"t and a vector value parameter u for calculating probability density p("khgr"t|u) for the "khgr"t on curved exponential family, a Jeffreys mixture calculator for calculating a first approximation value of a Bayes mixture density pJ("khgr"n) based on a Jeffreys prior distribution wJ(u) in cooperation with the probability density calculator to produce the first approximation value, an enlarged mixture calculator for calculating a second approximation value of a Bayes mixture m("khgr"n) on exponential family including curved exponential family in cooperation with the probability density calculator to produce the second approximation value, and a whole mixture calculator for calculating (1xe2x88x92xcex5)pJ("khgr"n)+xcex5xc2x7m("khgr"n) to produce a calculation result by mixing the first approximation value of the Bayes mixture density pJ("khgr"n) with a part of the second approximation value of the Bayes mixture m("khgr"n) at a ratio of 1xe2x88x92xcex5:xcex5 to produce the calculation result where xcex5 is a value smaller than unity.
According to a fifth embodiment of the invention, a prediction probability density calculator operable in response to a sequence of vector xn=(x1, X2, . . . , xn) selected from a vector value set "khgr" and xn+1 to produce a prediction probability density on occurrence of the xn+1, comprising a joint probability calculator structured by the Bayes mixture density calculator of the first embodiment of the invention for calculating a modified Bayes mixture density q(xcex5)(xn) and q(xcex5)(xn+1) based on predetermined prior distribution to produce first calculation results and a divider responsive to the calculation results for calculating probability density q(xcex5)(xn+1)/q(xcex5)(xn) to produce a second calculation result with the first calculation results kept intact.
According to a sixth embodiment of the invention which can be modified based on the fifth embodiment of the invention, a prediction probability density calculator operable in response to a sequence of xn=(x1, x2, . . . , xn) selected from a vector value set "khgr" and xn+1 to produce a prediction probability density on occurrence of the xn+1, comprising a joint probability calculator structured by the Jeffreys mixture density calculator of the second embodiment of the invention for calculating a modified Jeffreys mixture density q(xcex5)(xn) and q(xcex5)(xn+1) to produce first calculation results and a divider response to the calculation results for calculating a probability density q(xcex5)(xn+l)/q(xcex5)( xn) to produce a second calculation result with the first calculation results kept intact.
Also, when hypothesis class is curved exponential family, it is possible to provide with a seventh embodiment of the invention by modifying the fifth embodiment of the invention. According the seventh embodiment of the invention, a prediction probability density calculator operable in response to a sequence of vector xn=(x1, x2, . . . , xn) selected from a vector value set "khgr" and xn+1 to produce a prediction probability density on occurrence of the xn+1, comprising a joint probability calculator structured by the Bayes mixture density calculator of the third embodiment of the invention for calculating a modified Bayes mixture density q(xcex5)(xn) and q(xcex5)(xn+1) based on a predetermined prior distribution to produce first calculation results and a divider responsive to the calculation results for calculating a probability density q(xcex5)(xn+1)/q(xcex5)(xn) to produce a second calculation result with the first calculation results kept intact.
According to an eighth embodiment of the invention which can be modified based on the seventh embodiment of the invention, a prediction probability density calculator operable in response to a sequence of vector xn=(x1, x2, . . . , xn) selected from a vector value set "khgr"and xn+1 to produce a prediction probability density on occurrence of the xn+1, comprising a joint probability calculator structured by the Jeffreys mixture density probability calculator of the fourth embodiment of the invention for calculating a modified Jeffreys mixture density q(xcex5)(xn+1) and q(xcex5)(xn) to produce first calculation results and a divider responsive to the calculation results for calculating a probability density q(xcex5)(xn+1)/q(xcex5)(xn) to produce a second calculation result with the first calculation results kept intact.