Who said only men were hunters? Don't you think every capable individual of the tribe did something to provide food? Also it is assumed that the hunter and gatherer societies of the Paleolithic only needed 2-3 hours a day to get the food they needed. So in your logic, the male brains would only be used for such a short time of the day, the other 21 hours the females have to use it for everything else that needs to be done.