title: Feasible Learning
summary: We introduce Feasible Learning (FL), a sample-centric learning paradigm where
  models are trained by solving a feasibility problem that bounds the loss for
  each training sample. In contrast to the ubiquitous Empirical Risk Minimization
  (ERM) framework, which optimizes for average performance, FL demands
  satisfactory performance on every individual data point. Since any model that
  meets the prescribed performance threshold is a valid FL solution, the choice
  of optimization algorithm and its dynamics play a crucial role in shaping the
  properties of the resulting solutions. In particular, we study a primal-dual
  approach which dynamically re-weights the importance of each sample during
  training. To address the challenge of setting a meaningful threshold in
  practice, we introduce a relaxation of FL that incorporates slack variables of
  minimal norm. Our empirical analysis, spanning image classification, age
  regression, and preference optimization in large language models, demonstrates
  that models trained via FL can learn from data while displaying improved tail
  behavior compared to ERM, with only a marginal impact on average performance.
paper_authors._id: 67986d764fccd4b95149db0b
paper_authors.name: Juan Ramirez
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-28T16:53:24.742Z
paper_authors._id: 65555c6c6947208b77271f1e
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/CQPL5xB-Y-F89BxJrB7tg.png
paper_authors.fullname: Juan Ramírez
paper_authors.type: user
paper_authors.user: juanramirezneilson
paper_authors._id: 67986d764fccd4b95149db0c
paper_authors.name: Ignacio Hounie
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-28T19:40:24.939Z
paper_authors._id: 6508b3f15b34509e16c78fea
paper_authors.avatarUrl: /avatars/f663439c405354504b27f5ffab5c401a.svg
paper_authors.fullname: Ignacio Hounie
paper_authors.type: user
paper_authors.user: ihounie
paper_authors._id: 67986d764fccd4b95149db0d
paper_authors.name: Juan Elenter
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-28T19:40:32.041Z
paper_authors._id: 65a04adddb5d37ad5e6b9d29
paper_authors.avatarUrl: /avatars/071a137c2c1aad0699ee1b8b001e4a58.svg
paper_authors.fullname: Elenter
paper_authors.type: user
paper_authors.user: juanelenter
paper_authors._id: 67986d764fccd4b95149db0e
paper_authors.name: Jose Gallego-Posada
paper_authors._id: 67986d764fccd4b95149db0f
paper_authors.name: Meraj Hashemizadeh
paper_authors._id: 67986d764fccd4b95149db10
paper_authors.name: Alejandro Ribeiro
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-28T19:24:31.422Z
paper_authors._id: 6724c42c055ce33014feaec2
paper_authors.avatarUrl: /avatars/8854289d890b555ca9562f95edeab784.svg
paper_authors.fullname: Alejandro Ribeiro Prieto
paper_authors.type: user
paper_authors.user: Prieto
paper_authors._id: 67986d764fccd4b95149db11
paper_authors.name: Simon Lacoste-Julien
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-28T16:53:32.537Z
paper_authors._id: 6406bcaca577649430c6bff4
paper_authors.avatarUrl: /avatars/77b386172b0d86d40cddd7a7a8744491.svg
paper_authors.fullname: Simon Lacoste-Julien
paper_authors.type: user
paper_authors.user: slacoste
paper_title: Feasible Learning
paper_summary: We introduce Feasible Learning (FL), a sample-centric learning paradigm where
  models are trained by solving a feasibility problem that bounds the loss for
  each training sample. In contrast to the ubiquitous Empirical Risk Minimization
  (ERM) framework, which optimizes for average performance, FL demands
  satisfactory performance on every individual data point. Since any model that
  meets the prescribed performance threshold is a valid FL solution, the choice
  of optimization algorithm and its dynamics play a crucial role in shaping the
  properties of the resulting solutions. In particular, we study a primal-dual
  approach which dynamically re-weights the importance of each sample during
  training. To address the challenge of setting a meaningful threshold in
  practice, we introduce a relaxation of FL that incorporates slack variables of
  minimal norm. Our empirical analysis, spanning image classification, age
  regression, and preference optimization in large language models, demonstrates
  that models trained via FL can learn from data while displaying improved tail
  behavior compared to ERM, with only a marginal impact on average performance.
paper_ai_summary: Feasible Learning trains models by ensuring satisfactory performance on each sample, using a primal-dual approach and minimal norm slack variables, improving tail behavior with slight impact on average performance.
paper_ai_keywords: Feasible Learning sample-centric learning feasibility problem Empirical Risk Minimization primal-dual approach slack variables