People like sex (citation needed). Sex produces babies. The reason population growth slowed down is obviously an easy access to contraception (and to a lesser degree - abortion). There are two ways of causing baby boom - a) contraception stops working, b) people choose not to use it.
1. No contraception available
For some reason contraception became less available. It can be religious, political, technological or any other. Religious fundamentalist politics is done to death in novels and tv these days, techno is more interesting.
1a Technoloy
When the oil runs out many technologies will become very expensive, almost everything we make is based on oil products, like plastics. That includes condoms, but also a lot of medicines. The problem with this story is that you have bigger problem than lack of condoms when you run out of oil.
1b Biology
Evolution decided to save us from going extinct and people developed severe allergic reaction to condoms or pills or both. 1006
Or even better, pills stop working. This will take longer than 50 years though, you would have to move your story 200 years or more in the future. If a gene of resistance to contraception appears it will obviously spread quite quickly, in couple of generations it will dominate the population because people without it have 0-1 child and people with it have 6-10.
2. Contraception is dangerous
More interesting option is some disease that affects women on pill only. If taking a pill makes you likely to die you will have no option but to abstain from sex unless you are ready for children, most likely in a stable marriage.
3. No need for contraception
People try to prevent pregnancies, even in stable marriage, because children are very costly. They cost the mother a year or more of her career. They need a spare room, new clothes every few months, toys, school fees, etc. If your civilization somehow becomes post-scarcity many of these problems go away.
Conclusion
There will obviously always be women who just don't want children and do everything to prevent or terminate pregnancy, and if all else fails even resort to infanticide ("fourth trimester abortion"). But even if only 25% women have 5 children each, and another 50% have 3 each, that's already population explosion. 998