This phase I trial is studying the side effects and best dose of erlotinib hydrochloride when given together with oxaliplatin, fluorouracil, and radiation before surgery and alone after surgery in treating patients with locally advanced cancer of the esophagus and gastroesophageal junction. Drugs used in chemotherapy, such as oxaliplatin and fluorouracil, work in different ways to stop the growth of tumor cells, either by killing the cells or by stopping them from dividing. Erlotinib hydrochloride may stop the growth of tumor cells by blocking some of the enzymes needed for cell growth. Radiation therapy uses high-energy x-rays to kill tumor cells. Giving combination chemotherapy together with erlotinib hydrochloride and radiation therapy before surgery may make the tumor smaller and reduce the amount of normal tissue that needs to be removed. Giving erlotinib hydrochloride after surgery may kill any tumor cells that remain after surgery