Though you may be right, doesn't live and let live work the other way as well?  The baker could have baked the cake, made some profit, and said "have a nice day" and gotten on with his life.  Instead he refuses and here we are.  (The word "fascist" is bandied about way too often any more - it seems nobody really knows the proper time to use it - on both sides).