In this research study, the investigators are trying to determine whether the combination of alectinib and cobimetinib is safe and well tolerated. This study will help determine the doses of the drugs that should be used in the Phase II portion of this study. Another purpose of this study is to determine whether adding cobimetinib to alectinib is effective for treating lung cancer that has already stopped responding to alectinib alone. In order to help understand why the cancer may have stopped responding to alectinib and whether treatment with the two drugs effectively blocks growth signals, all participants in this study will have periodic blood collections and undergo multiple biopsies.