Order of reactivity of hydrogen halides Solomon and Fryhle $11^{th}$ edition says :

Order of reactivity of HX in the acid catalyzed conversion of alcohols to alkyl halides follows the order : HI>HBr>HCl

We can account for this by saying that the acidity order is also HI>HBr>HCl
and hence the $-OH$ is easily protonated by the more acidic halide but the reaction is already acid catalyzed by a strong acid and the conversion of $-OH$ to  $-H_2O^+$ is fast so the acidity of $HX$ should make a negligible difference. 

The slow step of the reaction is the formation of the carbocation and that does not seem to involve the halide so how can we explain this order ?
The halide ion is involved, indirectly.  The hydrogen halide has to protonate the alcohol first, and this requires dissociation of the proton from  the halide ion.  Chloride ion is more strongly basic than iodide ion, an effect we do not see in water (leveling effect) but can see it in the reaction with the alcohol.  So we can protonate more of the alcohol at any time, with hydrogen iodide, making more opportunities to then form the carbocation, than with hydrogen chloride.

Hydrogen fluoride would be way off the scale here, in the direction of less reactivity.  It is only a weak acid in water, let alone in this reaction mechanism.  It's one manifestation of fluorination being more difficult than other halogenations.