This phase I/II trial is studying the side effects and best dose of vorinostat when given together with trastuzumab and to see how well they work in treating patients with metastatic breast canceror breast cancer that has recurred in the chest wall. Vorinostat may stop the growth of tumor cells by blocking some of the enzymes needed for cell growth. Monoclonal antibodies, such as trastuzumab, can block tumor growth in different ways. Some find tumor cells and kill them or carry tumor-killing substances to them. Others interfere with the ability of tumor cells to grow and spread. Vorinostat and trastuzumab also may stop the growth of tumor cells by blocking blood flow to the tumor. Giving vorinostat together with trastuzumab may be a better way to block tumor growth.