Can I access an OS X Yosemite Time Machine backup if I revert to OS X Mavericks? I've had problems with Yosemite since I've installed it but I'm also wondering if it is even worth it to revert since I've done everything on Yosemite?
Generally speaking, no, you can't simply 'restore' from a newer version of TM to an older OS.

That said, you can access the data.  And, if you understand how TimeMachine works at the UNIX level, you can use the hardlinks created to manually restore what you'd like.