title: LLM-FE: Automated Feature Engineering for Tabular Data with LLMs as
    Evolutionary Optimizers
summary: Automated feature engineering plays a critical role in improving predictive
  model performance for tabular learning tasks. Traditional automated feature
  engineering methods are limited by their reliance on pre-defined
  transformations within fixed, manually designed search spaces, often neglecting
  domain knowledge. Recent advances using Large Language Models (LLMs) have
  enabled the integration of domain knowledge into the feature engineering
  process. However, existing LLM-based approaches use direct prompting or rely
  solely on validation scores for feature selection, failing to leverage insights
  from prior feature discovery experiments or establish meaningful reasoning
  between feature generation and data-driven performance. To address these
  challenges, we propose LLM-FE, a novel framework that combines evolutionary
  search with the domain knowledge and reasoning capabilities of LLMs to
  automatically discover effective features for tabular learning tasks. LLM-FE
  formulates feature engineering as a program search problem, where LLMs propose
  new feature transformation programs iteratively, and data-driven feedback
  guides the search process. Our results demonstrate that LLM-FE consistently
  outperforms state-of-the-art baselines, significantly enhancing the performance
  of tabular prediction models across diverse classification and regression
  benchmarks.
paper_authors._id: 67dc4650129a710a8042aa99
paper_authors.name: Nikhil Abhyankar
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2025-03-20T16:46:09.424Z
paper_authors._id: 6490a82b231a197da764ace8
paper_authors.avatarUrl: /avatars/5d8b7cafa2658a01e706037a3994ebb9.svg
paper_authors.fullname: Nikhil Abhyankar
paper_authors.type: user
paper_authors.user: nikhilsa
paper_authors._id: 67dc4650129a710a8042aa9a
paper_authors.name: Parshin Shojaee
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2025-03-20T17:22:12.321Z
paper_authors._id: 6520621836008ecc88699622
paper_authors.avatarUrl: /avatars/b08c00af00f1736a4f4938443e575b0e.svg
paper_authors.fullname: Parshin Shojaee
paper_authors.type: user
paper_authors.user: parshinsh
paper_authors._id: 67dc4650129a710a8042aa9b
paper_authors.name: Chandan K. Reddy
paper_title: LLM-FE: Automated Feature Engineering for Tabular Data with LLMs as
    Evolutionary Optimizers
paper_summary: Automated feature engineering plays a critical role in improving predictive
  model performance for tabular learning tasks. Traditional automated feature
  engineering methods are limited by their reliance on pre-defined
  transformations within fixed, manually designed search spaces, often neglecting
  domain knowledge. Recent advances using Large Language Models (LLMs) have
  enabled the integration of domain knowledge into the feature engineering
  process. However, existing LLM-based approaches use direct prompting or rely
  solely on validation scores for feature selection, failing to leverage insights
  from prior feature discovery experiments or establish meaningful reasoning
  between feature generation and data-driven performance. To address these
  challenges, we propose LLM-FE, a novel framework that combines evolutionary
  search with the domain knowledge and reasoning capabilities of LLMs to
  automatically discover effective features for tabular learning tasks. LLM-FE
  formulates feature engineering as a program search problem, where LLMs propose
  new feature transformation programs iteratively, and data-driven feedback
  guides the search process. Our results demonstrate that LLM-FE consistently
  outperforms state-of-the-art baselines, significantly enhancing the performance
  of tabular prediction models across diverse classification and regression
  benchmarks.
paper_ai_summary: LLM-FE integrates domain knowledge and evolutionary search to automate feature engineering, enhancing tabular prediction model performance.
paper_ai_keywords: LLM-FE evolutionary search large language models feature engineering feature transformation programs data-driven feedback