I think that you have a point as far as staying true to what the status codes mean, but it comes with a price.  To make the resource invisible to the user by 404ing instead of 403ing, you protect yourself from people searching for endpoints that return personal info... and all urls are guessable.  If you get a hit on some resource in a sea of 404s, you are that much closer to gaining private info.