While the unemployment rate has declined steadily for nearly four years, that’s due in no small part to people simply leaving the labor force — taking early retirement, going back to graduate school, or simply giving up looking for work.