Well I am sure you can 'try', but he is the boss and if anyone of you are dumb enough to go against his rules then you certainly don't value your job much!  Document your reasoning and proposed guidelines and assurances to not interfere with work and present them but just ask yourself,is your paycheck worth playing a game?