Question: For example, we always assumed that the data or signal error is a Gaussian distribution? why?

I have asked this question on stackoverflow, the link: <https://stackoverflow.com/questions/12616406/anyone-can-tell-me-why-we-always-use-the-gaussian-distribution-in-machine-learni>

Answer: 
Machine learning (and statistics as well) treats data as the mix of deterministic (causal) and random parts. The random part of data usually has normal distribution. (Really, the causal relation is reverse: the distribution of random part of variable is called normal). Central limit theorem says that the sum of large number of varibles each having a small influence on the result approximate normal distribution. 
1. Why data is treated as normally distributed? In machine learning we want to express dependent variable as some function of a number of independent variables. If this function is sum (or expressed as a sum of some other funstions) and we are suggesting that the number of independent variables is really high, then the dependent variable should have normal distribution (due to central limit theorem).
2. Why errors are looked to be normally distributed? The dependent variable ($Y$) consists of deterministic and random parts. In machine learning we are trying to express deterministic part as a sum of deterministic independent variables: $$deterministic + random = func(deterministic(1))+...+func(deterministic(n))+model\\_error$$ If the whole deterministic part of $Y$ is explained by $X$ then the $model\\_error$ depicts only $random$ part, and thus should have normal distribution. So if error distribution is normal, then we may suggest that the model is successful. Else there are some other features that are absent in model, but have large enough influence on $Y$ (the model is incomplete) or the model is incorrect.
One reason that normal distributions are often (but not always!) assumed: the nature of the distribution often leads to extremely efficient computation. For example, in generalized linear regression, the solution is technically in closed form when your distribution is Gaussian:

$\hat \beta = (X^T X)^{-1} X^T Y$

where as for other distributions, iterative algorithms must be used. Technical note: using this direct computation to find $\hat \beta$ is both inefficient and unstable.

Quite often, both the theoretical math and numerical methods required are *substaintially* easier if the distribution is a linear transformation of normal variables. Because of this, methods are frequently first developed under the assumption that the data is normal, as the problem is considerably more tractable. Later, the more difficult problem of addressing non-normality is addressed by statistical/machine learning researchers.