What is an acceptable value of square loss in machine learning (using mxnet gluon's square loss function)? I have checked the gluon's square loss function implementation in mxnet, where it is calculated as follows:
$$L=\frac{1}{2}\sum_i\left|label_i−pred_i\right|^2$$
https://mxnet.apache.org/versions/1.8.0/api/python/docs/api/gluon/loss/index.html#mxnet.gluon.loss.L2Loss
The key here is that the loss is squared and then multiplied by $\frac{1}{2}$.
Now, let's suppose I want to predict a label ∈ {0,1} where P(1)=P(0)=0.5
And instead of doing any calculation I simply always predict that my value is equal to 0.5
If the label is 1, I get a loss $L=\frac{1}{2}\sum_i\left|1−0.5\right|^2$=0.125
And if my label is 0, I also get $L=\frac{1}{2}\sum_i\left|0−0.5\right|^2$=0.125
Does this mean I should aim for my loss to be less than 0.125 for my predictions to be of any use?
Or in this case ( if labels are ∈ {0,1} ), would it be more sensible to switch to softmax model?

In addition to the other answers, I'll add that usually with machine learning we fit the model by finding the weights that minimise the loss, but we then usually evaluate the model using a different metric that has more "intuitive" meaning (and to be robust this should be on a different dataset that wasn't used to train the model, i.e. a train-test split).
For example with classification we often use Cross Entropy as the loss, however when we evaluate the model we use either accuracy, F1 score or some other metric to actually decide if our model is "good enough". For regression problems such as the one you're working on, we more commonly use squared error loss, and it's fairly common to use something like r-squared as a metric (although it's certainly not a perfect one).
