With little to no government interference, the border of my property ends where the border of your property begins. There is no allotment for roads or utilities.  It's every property owner for themself.  If people cross my property to get from one place to the next, there is nothing stopping me from building a giant wall.  Or building a path/road with a toll booth.  Or cutting your electric/cable/water lines.  It doesn't become easier to get around with no government interference; it becomes much much harder.  You get twisted mazes of narrow paths that are dangerous to travel down.  
For example, look to the various slums of the world:  these are places where many people are forced to live with essentially no government interference.  You don't end up with nice, easily traversable, clean streets with readily available amenities for all.