PURPOSE: This phase I trial is studying the side effects and best dose of erlotinib hydrochloride when given together with gemcitabine hydrochloride and oxaliplatin in treating patients with advanced biliary tract cancer, pancreatic cancer, duodenal cancer, or ampullary cancer.