While it makes no sense to worry that you are living well at your grandchildren's expense, you might legitimately worry that someone else is living well at your grandchildren's expense.
They are childless without an heir. 