title: Exploring the MIT Mathematics and EECS Curriculum Using Large Language
    Models
summary: We curate a comprehensive dataset of 4,550 questions and solutions from
  problem sets, midterm exams, and final exams across all MIT Mathematics and
  Electrical Engineering and Computer Science (EECS) courses required for
  obtaining a degree. We evaluate the ability of large language models to fulfill
  the graduation requirements for any MIT major in Mathematics and EECS. Our
  results demonstrate that GPT-3.5 successfully solves a third of the entire MIT
  curriculum, while GPT-4, with prompt engineering, achieves a perfect solve rate
  on a test set excluding questions based on images. We fine-tune an open-source
  large language model on this dataset. We employ GPT-4 to automatically grade
  model responses, providing a detailed performance breakdown by course,
  question, and answer type. By embedding questions in a low-dimensional space,
  we explore the relationships between questions, topics, and classes and
  discover which questions and classes are required for solving other questions
  and classes through few-shot learning. Our analysis offers valuable insights
  into course prerequisites and curriculum design, highlighting language models'
  potential for learning and improving Mathematics and EECS education.
paper_authors._id: 648bc37095d85bfac020d484
paper_authors.name: Sarah J. Zhang
paper_authors.status: extracted_infirmed
paper_authors.statusLastChangedAt: 2024-05-09T15:50:08.998Z
paper_authors._id: 648bc37095d85bfac020d485
paper_authors.name: Samuel Florin
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2023-06-16T18:06:50.672Z
paper_authors._id: 6406a026c3ab325efa9bb411
paper_authors.avatarUrl: /avatars/971c17600395b1037f676710b32b3909.svg
paper_authors.fullname: Sam Florin
paper_authors.type: user
paper_authors.user: sflorin
paper_authors._id: 648bc37095d85bfac020d486
paper_authors.name: Ariel N. Lee
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2023-06-16T02:36:08.413Z
paper_authors._id: 638bcfa91987d67b340e6c1c
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/638bcfa91987d67b340e6c1c/3tHCB_J6c4-lsEZ_zJSlp.jpeg
paper_authors.fullname: Ariel N. Lee
paper_authors.type: user
paper_authors.user: arielnlee
paper_authors._id: 648bc37095d85bfac020d487
paper_authors.name: Eamon Niknafs
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-06-20T08:50:15.096Z
paper_authors._id: 63ecfa8ec5b3c734085daebe
paper_authors.avatarUrl: /avatars/cdad9af0749a52945116b385e2c2591b.svg
paper_authors.fullname: Eamon Niknafs
paper_authors.type: user
paper_authors.user: eamonniknafs
paper_authors._id: 648bc37095d85bfac020d488
paper_authors.name: Andrei Marginean
paper_authors._id: 648bc37095d85bfac020d489
paper_authors.name: Annie Wang
paper_authors._id: 648bc37095d85bfac020d48a
paper_authors.name: Keith Tyser
paper_authors._id: 648bc37095d85bfac020d48b
paper_authors.name: Zad Chin
paper_authors._id: 648bc37095d85bfac020d48c
paper_authors.name: Yann Hicke
paper_authors._id: 648bc37095d85bfac020d48d
paper_authors.name: Nikhil Singh
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2023-06-16T07:56:07.095Z
paper_authors._id: 6374384c250cf1379bc278f8
paper_authors.avatarUrl: /avatars/e436988005ad4f9d99fcafb8bdfade9c.svg
paper_authors.fullname: Nikhil Singh
paper_authors.type: user
paper_authors.user: nsingh1
paper_authors._id: 648bc37095d85bfac020d48e
paper_authors.name: Madeleine Udell
paper_authors._id: 648bc37095d85bfac020d48f
paper_authors.name: Yoon Kim
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2023-06-16T02:05:38.166Z
paper_authors._id: 630ee22c05b5c5280e3cdcdf
paper_authors.avatarUrl: /avatars/b282162d6d9ae8512873eba0275f603e.svg
paper_authors.fullname: Yoon Kim
paper_authors.type: user
paper_authors.user: yoon-kim
paper_authors._id: 648bc37095d85bfac020d490
paper_authors.name: Tonio Buonassisi
paper_authors._id: 648bc37095d85bfac020d491
paper_authors.name: Armando Solar-Lezama
paper_authors._id: 648bc37095d85bfac020d492
paper_authors.name: Iddo Drori
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-06-16T13:57:32.533Z
paper_authors._id: 62c9f32a334a6ee11c35d601
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1657402192472-62c9f32a334a6ee11c35d601.png
paper_authors.fullname: Iddo Drori
paper_authors.type: user
paper_authors.user: idrori
paper_title: Exploring the MIT Mathematics and EECS Curriculum Using Large Language
    Models
paper_summary: We curate a comprehensive dataset of 4,550 questions and solutions from
  problem sets, midterm exams, and final exams across all MIT Mathematics and
  Electrical Engineering and Computer Science (EECS) courses required for
  obtaining a degree. We evaluate the ability of large language models to fulfill
  the graduation requirements for any MIT major in Mathematics and EECS. Our
  results demonstrate that GPT-3.5 successfully solves a third of the entire MIT
  curriculum, while GPT-4, with prompt engineering, achieves a perfect solve rate
  on a test set excluding questions based on images. We fine-tune an open-source
  large language model on this dataset. We employ GPT-4 to automatically grade
  model responses, providing a detailed performance breakdown by course,
  question, and answer type. By embedding questions in a low-dimensional space,
  we explore the relationships between questions, topics, and classes and
  discover which questions and classes are required for solving other questions
  and classes through few-shot learning. Our analysis offers valuable insights
  into course prerequisites and curriculum design, highlighting language models'
  potential for learning and improving Mathematics and EECS education.