This phase II trial studies how well olaparib works in treating patients with glioma, cholangiocarcinoma, or solid tumors with IDH1 or IDH2 mutations that have spread to other places in the body and usually cannot be cured or controlled with treatment. Olaparib may stop the growth of tumor cells by blocking some of the enzymes needed for cell growth.