PRIMARY OBJECTIVES: I. Determine the overall response rate (ORR), including complete responses (CR) and partial responses (PR), with combined lenalidomide and panobinostat in patients with relapsed or refractory Hodgkin's lymphoma. SECONDARY OBJECTIVES: I. Assess the safety and tolerability of combined lenalidomide and panobinostat in patients with previously treated Hodgkin's lymphoma. II. Determine progression-free survival (PFS) in patients with previously treated Hodgkin's lymphoma receiving combined lenalidomide and panobinostat. III. Evaluate changes in natural killer (NK) cell number and function, plasma cytokines, gene expression profile of peripheral blood, and plasma proteins via proteomics, before, during, and after lenalidomide and panobinostat therapy. IV. Determine levels of histone acetylation before and after treatment with panobinostat. V. Determine the pharmacokinetics of lenalidomide in the presence of panobinostat. VI. To collect deoxyribonucleic acid (DNA) samples for potential later analysis of associations between outcomes and polymorphisms in genes coding for drug metabolizing enzymes, transporters and molecular targets of lenalidomide or panobinostat. OUTLINE: Patients receive panobinostat orally (PO) on days 1, 3, and 5 of weeks 1-4 and lenalidomide PO on days 1-7 of weeks 1-3. Courses repeat every 28 days in the absence of disease progression or unacceptable toxicity. After completion of study treatment, patients are followed up at least every 3 months for 2 years, and then every 6 months for 3-5 years