1. Field of the Invention
The present invention relates generally to remote security protocols in computer systems and more specifically to a challenge-response protocol for ensuring the authenticity of a process operating in a different address space.
2. Description of Related Art
Current methods of performing challenge-response protocols in a cryptographic system require the parties or processes performing the protocol to share a high-value persistent secret or both parties must process appropriate pieces of two asymmetric key pairs. In both of these scenarios, prior communication of the secrets between the parties or knowledge of the public component of the other party's key pair is required. Most versions of such challenge-response protocols are vulnerable to "man-in-the-middle" attacks, which are particularly problematic if the two parties are communicating over a network. Hence, general challenge-response protocols prove only that a verified channel between two endpoints sharing a secret has been set up, but one of the endpoints could be insecure. It would be better if one party could authenticate the other party to ensure that the other party has not be tampered with or "hacked", as opposed to just validating that the other party shares the secret. This can be done when the parties share the same address space by checking the contents of memory of the other party, computing its digital signature, and verifying its integrity. However, this cannot be accomplished across different process address spaces unless the memory is shared. Various challenge-response protocols and their deficiencies are described in "Applied Cryptography", by Bruce Schneier, second edition, 1996. What is needed is a protocol that overcomes the above problems and deficiencies of the prior art by securely communicating a required secret as part of an authentication process such that the secret can be a nonce of no persistent worth and wherein the secret need not be previously communicated.