Is there a way to order all the files and folders automatically? I am new to OS X, but there's one thing that annoys me a little bit, which is that every time I have to clean up all the files (by name, for example) because they are always pasted in an unordered way.

I would not like to lose time to order them every time I change the content of the folders.
Instead of using Clean Up By, use Arrange By. This both cleans up the layout and sets the default sort order for the folder for the future so that any modifications to the folder will update the sort order.