I am not sure you are missing the point. We can produce a Hidden variables model to reproduce the results of quantum mechanics for a two dimensional Hilbert space, but the Kochen-Specker theorem shows that this cannot be done for a Hilbert space of dimension more than 2 (<PERSON> considered infinite dimensional Hilbert space). <PERSON>'s theorem can also be interpreted as meaning that the only probability measure on a Hilbert space of dimension at least 3 is that given by quantum mechanics, thereby excluding hidden variables).
I regard contextuality as a red herring in these arguments. It is also a red herring to consider Hilbert space of dimension 2, since that does not describe quantum mechanics. 682
Whenever contextuality is introduced, it seems to be to confuse the issue by changing the subject (the current Wikipedia article on <PERSON>'s theorem is a case in point; compare with the statement of <PERSON>'s theorem which was quoted in this SE answer)
Contextuality means that the hidden variable determines just a particular measurement or class of measurements. It does not satisfy classical determinism, which means the determination of all possible measurement results, not just the result of the measurement performed. It is not sufficient for contextuality to say that the hidden variable contains parts that determine either position or momentum and that since only one of these measurements can be performed the other exists but is not used; conjugacy between position and momentum states that when one measurement is performed the hidden variable for the other measurement cannot exist.
It makes no difference that we can in practice perform only one measurement, because quantum logic describes not only the measurement performed but also what would happen in measurements which are not performed. Contextuality would require that different hidden variables pop in and out of existence whenever an experimenter changes his mind about which measurement he is to perform. This would imply different underlying physics depending on what a physicist will choose to do.
In short, whenever I see arguments about contextuality, I find physicists clinging to determinism and seeking a loophole which does not exist. 682