In what sense is the Inductive effect a permanent effect? Almost everywhere I checked, the Inductive effect is described as a permanent effect with almost no mention of what that actually means. 


What exactly is permanent about the effect?
If the effect is permanent, does that mean that even if the attached electron
withdrawing/donating group is removed, the induced effect still continues to
exist?
The effect is permanent because the group that does the 'induction' is always there in the molecule. For example, if the induction is caused by a t-butyl group, that group is held by a strong carbon to carbon ($\ce{C-C}$) bond that will not easily be broken. One could also consider the inductive effect of the Fluorine held by a very strong $\ce{C-F}$ bond. 

A temporary effect might be caused by protontation of an oxygen atom where the protonation can catalyse a reaction. The protonation is reversible. The bond between the proton and oxygen would be much weaker, as $\ce{C=O^+-H}$ will be in equilibrium with $\ce{C=O  +  H^+}$.

Removal of the group that does the induction will remove the inductive effect.