Question: We would like to get rid of the Tridion Broker as part of a project and publish content directly to a repository as part of a Storage Extension. 

My question is, is there any deep dependency that Tridion has on the Broker that might mean things work in a sub-optimal way, or gotchas that I should be aware of?

Update - We are going to be using Tridion purely as a content store which will push out JSON. There will be no functionality or any form of business logic/UI.

Answer: 
I will assume that what you mean with the Broker is the Tridion Broker database.

It is possible to publish everything to the file system, it includes, content, linking info, metadata and references, however this functionality is deprecated and will be removed in future releases, having said that, in future releases, linking, metadata and references will always be stored in the Tridion broker database.

If what you want is to override the storage layer and create a custom one, it will for sure affect key features like the Tridion CD API, Dynamic Linking and of course it won't be supported by either Customer Support or R&D.

Other key feature that depends on the Tridion Broker database is the Session Preview functionality.
This sounds like an unusual implementation.

It might be worth blueprinting your website to allow you to localize the templates to render the json. The new child publication could be used with a new deployer for you to test if this gives you the output you need without any missing functionality.