Question: For example, we always assumed that the data or signal error is a Gaussian distribution? why?

I have asked this question on stackoverflow, the link: <https://stackoverflow.com/questions/12616406/anyone-can-tell-me-why-we-always-use-the-gaussian-distribution-in-machine-learni>

Answer: 
I had the same question "what the is advantage of doing a Gaussian transformation on predictors or target?" Infact, caret package has a pre-processing step that enables this transformation. 

I tried reasoning this out and am summarizing my understanding - 

1. Usually the data distribution in Nature follows a Normal distribution ( few examples like - age, income, height, weight etc., ) . So its the best approximation when we are not aware of the underlying distribution pattern.
2. Most often the goal in ML/ AI is to strive to make the data linearly separable even if it means projecting the data into higher dimensional space so as to find a fitting "hyperplane" (for example - SVM kernels, Neural net layers, Softmax etc.,). The reason for this being "Linear boundaries always help in reducing variance and is the most simplistic, natural and interpret-able" besides reducing mathematical / computational complexities. And, when we aim for linear separability, its always good to reduce the effect of outliers, influencing points and leverage points. Why? Because the hyperplane is very sensitive to the influencing points and leverage points (aka outliers) - To undertstand this - Lets shift to a 2D space where we have one predictor (X) and one target(y) and assume there exists a good positive correlation between X and y. Given this, if our X is normally distributed and y is also normally distributed, you are most likely to fit a straight line that has many points centered in the middle of the line rather than the end-points (aka outliers, leverage / influencing points). So the predicted regression line will most likely suffer little variance when predicting on unseen data.

Extrapolating the above understanding to a n-dimensional space and fitting a hyperplane to make things linearly separable does infact really makes sense because it helps in reducing the variance.
i'm currently studying machine learning and the same question popped to my mind. What I think the reason should be is that in every machine learning problem we assume we have abundant observational data available and whenever data tends to infinity it gets normally distributed around its mean and thats what Normal distribution(Gaussian Distribution) says. Although its not necessary that Gaussian DIstribution will always be a perfect fit to any data that tends to infinity like take a case when your data is always positive then you can easily see if you try to fit gaussian dist to it it will give some weight to negative values of x also(although negligible but still some weight is given) so in such case distribution like Zeta is more suited.