If you want to raise your kids bilingually one way is for you to only speak to them say in Norwegian and your wife to speak to them only in English. It is a very interesting subject especially fro an evolutionary point of view - what's happening to you when you speak to your friends is normal it happens to all my friends an myself. You can imagine if your friends and you and your families were isolated say in Island for 200  years your descendants would have been speaking a new language composed of Norwegian, French and English.