As a retired small business owner I can say that something needs doing but tinkering is not it. Broad strokes are required. Eliminate entire departments.

Some actual examples: a business owner with ONE employee has to do CPP, EI, Income Tax deductions, WSIB and actually pay them. Roll all these into one and let me pay say 15% to be divided up by the government.

I saw a small deck being replaced. It was worth about $1200. Because it was too close to the water (the new one was going where the old one was) a variance was needed: $800. Then an environmental assessment was needed: $500. The Conservation Authority took $300. A building permit cost $80. Really? $1680 in fees for a $1200 deck? And, Oh Yeah! Then we had to charge HST and send it along. 

Go through the regulations with a wire brush. Eliminate most of them.