Remove the door knockers. Pull down the shutters. Pretend no one’s home. Your adult children are coming back — for good.

One-in-nine baby boomer parents said their adult children returned home within the last year, according to a new report from financial services firm Fidelity Investments and Stanford Center on Longevity, which surveyed 9,000 employees.The adult children save money on rent and household goods, but their parents are the ones who appear to be suffering: 68% said they were more stressed, 53% said they were less happy and another 53% said they had less leisure time after the return of their “boomerang kids.” More than three-quarters (76%) said they took on higher expenses, too.

Even people who are now in their 40s and 50s are considering mom and dad an option. Older millennials are 2.7 times more likely to live in their parents’ home than people under 55 years old than in 1999, while Generation-Xers, who are now in their mid-30s to early 50s, were 2.2 times as likely to live with their parents, according to separate data released last week by real estate site Trulia. “No parent is going to want to say no to a child who needs help, but certainly being realistic about the financial situation is important,” said Katie Taylor, vice president of thought leadership at Fidelity.

See: The real reason college grads move home with their parents may surprise you

More American adults are living with their parents and grandparents than ever before — 19% of the U.S. population (or nearly 61 million people) lived in a multigenerational household, up from 17% (42 million) in 2009 and 12% (27.5 million) in 1980, according to the Pew Research Center, nonprofit think tank based in Washington, D.C.

But not all millennials are as “lazy” or “entitled,” as they are often accused of being. About one in four 25- to 34-year-olds who live at home and are not working or going to school do so because of a health-related reason or because they are acting as caregivers to their family members. And more than a third of Americans, including millennials, expect to financially help their parents within the next few years, another survey found. Some are even making efforts to help their parents save for retirement.

Get a daily roundup of the top reads in personal finance delivered to your inbox. Subscribe to MarketWatch's free Personal Finance Daily newsletter. Sign up here.