Question: We would like to get rid of the Tridion Broker as part of a project and publish content directly to a repository as part of a Storage Extension. 

My question is, is there any deep dependency that Tridion has on the Broker that might mean things work in a sub-optimal way, or gotchas that I should be aware of?

Update - We are going to be using Tridion purely as a content store which will push out JSON. There will be no functionality or any form of business logic/UI.

Answer: 
1. Dynamic Linking is the big one as Rob mentions.
2. any DCPs embedded on Pages will render
 UCs/Tags.
3. Any code that queries CPs using Criteria/Query API will need to be gutted/replaced.
4. If you use P&P, then any personalized content will be rendered and filtered via User Controls/Tags, and this will need replacing
5. If you use UGC, same as #4.

When starting a fresh implementation Broker/no-Broker is a decision to be made, but once it's done and the site is live, you have a lot of work on your hands to get rid of the Broker.
This sounds like an unusual implementation.

It might be worth blueprinting your website to allow you to localize the templates to render the json. The new child publication could be used with a new deployer for you to test if this gives you the output you need without any missing functionality.