This phase I trial studies the side effects and best dose of lenalidomide when given together with brentuximab vedotin in treating patients with T-cell lymphomas that have come back or do not respond to treatment. Monoclonal antibodies, such as brentuximab vedotin, may interfere with the ability of cancer cells to grow and spread. Drugs used in chemotherapy, such as lenalidomide, work in different ways to stop the growth of cancer cells, either by killing the cells, by stopping them from dividing, or by stopping them from spreading. Giving brentuximab vedotin and lenalidomide may work better in treating patients with T-cell lymphomas.