Can we use Sigmoid directly in front of Mean Square Error in the linear regression model or MLP? I know that a sigmoid function in front of the MSE in the logistic regression lets gradient be 0, if the prediction is 1 (no matter what the ground truth is). According to this math, this problem will happen in the linear regression model or multilayer perceptron model as well, right?
For example, a linear regression model or MLP (with sigmoid function directly in front of MSE) may output 1, but the ground truth may be 0.5. In this situation, the gradient will be 0 so that the weights of the model will not be updated even though the prediction is wrong.
I know that there are other reasons that we would not use sigmoid at the output layer for linear regression. Again, does this mean that we should not put sigmoid function directly in front of the MSE (no matter what kind of model we are using)?

I believe you mean output activation by in front of the MSE. In logistic regression, cross entropy is used for the loss function, not MSE (mean squared error). But, independent from the loss function, the gradient portion produced  by the sigmoid will contain $\sigma (1-\sigma)$ multiplier, and if $\sigma$ was $1$, the gradient would be $0$ irrespective of the output. However, sigmoid cannot output $0$ or $1$ for real inputs, because there is no real $x$ for the following function to be $0$ or $1$:
$$\sigma(x)=\frac{1}{1+e^{-x}}$$
So, it's theoretically not possible, but, it can happen in finite precision machines (e.g. practically all digital computers we use) when $x$ is too small or big. It's not going to happen for all the dataset you have unless your parameter initialization is too off.
Sigmoid is not evil, but of course it has limitations, like it can be the cause of vanishing gradients problem in deep networks due to the accumulating sigmoid derivative multipliers during the back-propagation.
Also note that, for the output layer, softmax's derivative is quite similar but it's being used extensively.
