Mean logarithmic square error I have a fallowing problems: I'm training a neural network against some set of output values (regression problem). Those values are between -inf to inf and I can't normalize them, because they come continuously from a stream of data.
Now I'm using MSE, but sometimes it results in very high losses and disturbs training. To avoid it I wanted to somehow limit the value of loss. Thus I came up with the fallowing loss:
$$ L(true, false) = \frac{1}{N}\sum_{i=1}^N{log((true - false)^2+1)}$$
The questions is, if there's a loss like this in the literature, because I can't find anything like that. If not, would it have properties like square loss or something different ?

I am not aware of a loss function like this, but I'm looking forward to other answers giving pointers to literature.
Your other question is about the properties of your loss function. The inner part is a squared loss, which is minimized in expectation by the conditional mean of your observations. You then apply a monotonic transformation $x\mapsto \log(x+1)$ to this inner expression, so the minimizer stays the same. Overall, your loss function incentivizes your model to yield the conditional mean (unlike, say, the mean absolute error, which draws your predictions towards the conditional median). If this is the functional you want to elicit, go for it.
You may find a paper of mine (Kolassa, 2020, IJF) helpful.
