Question: I understand and very much appreciate the practice of separating header (.h, .hpp) and source files (.cpp). 

However, many simple classes can be expressed entirely in the header file itself. I could define some methods within the class and others (e.g. template methods, inline methods) below the class. 

What advantage is there to creating a .cpp file when I could just declare *and* define the entire class in the header file? (Do I even need to?)

Answer: 
As already mentioned compilation time is one reason for having separate translation units. Especially if you have many cores parallelization can dramatically decrease compile time. Another reason is that non-template classes and functions as well as variables in header files would lead to duplicate symbols when included in different translation units. As soon as you use virtual functions inlining is no viable option.

You could use templates to avoid the duplicate symbol issue but in addition to higher compilation times this also leads to higher linking time because the linker has to collapse the template-instances (see <https://gcc.gnu.org/onlinedocs/gcc/Template-Instantiation.html>). Nevertheless STL and many other header-only libraries do exactly this.
Nothing holds you back from having everything in header files. But doing otherwise can have various effects on the project itself, in certain cases improving readability and positively improving compilation speed due to not having compile everything that has a dependency on files you just made changes to. 

Usually, for small classes or classes which change very rarely, it has almost no impact whatsoever. 

I would note that with > c++20 and inclusion of modules things will slowly change and we will start seeing more code that is purely written in header files (if they will still be called like that).

Nonetheless, it is too opinion based question to have a good answer.