pKa differences and reactivity I know a lot of chemists like to quote $\mathrm{p}K_\mathrm{a}$ values and subtract them to reinforce some notion about some acid-base reaction being highly favorable or highly unfavorable or whatnot. 

For example, the $\mathrm{p}K_\mathrm{a}$ of acetic acid is about 5 and the $\mathrm{p}K_\mathrm{a}$ of LDA's conjugate acid is about 36. 36 minus 5 gives us 31 "zeros" of favorability. 

However, $\mathrm{p}K_\mathrm{a}$ values themselves don't tell you what solvent you are using … can using this method blindly lead to problems when working across solvents? And as far as I know, a lot of $\mathrm{p}K_\mathrm{a}$ values are measured in water … but in organic chemistry we don't just use water as a solvent. So does this method even have general validity?
An important case of acidity constants changing with solvent, with consequences in organic chemistry, can be found for compounds with hydroxyls, especially water. It is possible to deduce that hydroxyl compounds are more acidic in protic solvents compared to aprotic solvents, which is attributed to an abnormally high stabilization of the resulting conjugate base via strong $\ce{O^{-}\ ^...\  H-O}$ hydrogen bonds.

The difference is most clear for water. Water molecules solvated by water itself have a $\mathrm{p}K_\mathrm{a}$ of 15.7 in ambient conditions, but if the unusual stabilization from hydrogen bonding were removed, they would have a $\mathrm{p}K_\mathrm{a}$ of around 28. In other words, $\ce{H2O}$ is a weaker acid by some 10 orders of magnitude in non-protic solvents compared to water, and correspondingly, its conjugate base $\ce{OH^{-}}$ is a stronger base by the same factor. This means hydroxide ions can effect quantitative deprotonation of even rather weak acids, including the hydrocarbons cyclopentadiene, indene and fluorene, in solvents such as 1,2-dimethoxyethane.

Unfortunately, compounds containing hydroxide ions have somewhat limited use as superbases in non-protic solvents because the solubility of common hydroxide bases such as $\ce{NaOH}$ or $\ce{KOH}$ is rather small. Other hydroxides such as quaternary ammonium hydroxides have appreciable solubility in non-protic solvents, but they are always found in the form of hydrates. Attempts to prepare anhydrous $\ce{R_4N^+OH^{-}}$ inevitably result in decomposition, because the crystallized water solvates the hydroxide ion and tames its reactivity; once the water is removed, the bare hydroxide ion immediately attacks the quaternary ammonium cation creating an alcohol and a tertiary amine.

As mentioned previously, other compounds containing hydroxyls also show changes in $\mathrm{p}K_\mathrm{a}$ comparing water and non-protic solvents, though the difference tends to be smaller. For ethanol, the difference is at most about 4 units of $\mathrm{p}K$.