RATIONALE: Panobinostat may stop the growth of tumor cells by blocking some of the enzymes needed for cell growth. Estrogen can cause the growth of breast cancer cells. Hormone therapy using letrozole may fight breast cancer by lowering the amount of estrogen the body makes. Giving panobinostat together with letrozole may be an effective treatment for breast cancer.