I drink 2-3 liters of milk a day too, but I workout a lot too. I keep track of calories in a day and my current caloric intake keeps me the same weight. I also eat normal amounts, I just don't drink soda or eat candy or snacks too much. Milk is a great source of calories and protein if you workout a lot, but if you stop that and drink the same amount, you will gain fat, obviously.