title: Understanding Visual Feature Reliance through the Lens of Complexity
summary: Recent studies suggest that deep learning models inductive bias towards
  favoring simpler features may be one of the sources of shortcut learning. Yet,
  there has been limited focus on understanding the complexity of the myriad
  features that models learn. In this work, we introduce a new metric for
  quantifying feature complexity, based on V-information and
  capturing whether a feature requires complex computational transformations to
  be extracted. Using this V-information metric, we analyze the
  complexities of 10,000 features, represented as directions in the penultimate
  layer, that were extracted from a standard ImageNet-trained vision model. Our
  study addresses four key questions: First, we ask what features look like as a
  function of complexity and find a spectrum of simple to complex features
  present within the model. Second, we ask when features are learned during
  training. We find that simpler features dominate early in training, and more
  complex features emerge gradually. Third, we investigate where within the
  network simple and complex features flow, and find that simpler features tend
  to bypass the visual hierarchy via residual connections. Fourth, we explore the
  connection between features complexity and their importance in driving the
  networks decision. We find that complex features tend to be less important.
  Surprisingly, important features become accessible at earlier layers during
  training, like a sedimentation process, allowing the model to build upon these
  foundational elements.
paper_authors._id: 668db3b3e990292e5fbb6553
paper_authors.name: Thomas Fel
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-10T11:56:40.525Z
paper_authors._id: 620dbbf8611163230f7d0fae
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/620dbbf8611163230f7d0fae/xDqa6GMiSBhsb9G_9jdne.jpeg
paper_authors.fullname: Thomas FEL
paper_authors.type: user
paper_authors.user: ThomasFEL
paper_authors._id: 668db3b3e990292e5fbb6554
paper_authors.name: Louis Bethune
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-10T11:56:47.617Z
paper_authors._id: 662a8d7faa7781f8ea521f16
paper_authors.avatarUrl: /avatars/c05f9a7885b89c8818d52b98b52ecc7b.svg
paper_authors.fullname: Louis Bethune
paper_authors.type: user
paper_authors.user: AlgueRythme
paper_authors._id: 668db3b3e990292e5fbb6555
paper_authors.name: Andrew Kyle Lampinen
paper_authors._id: 668db3b3e990292e5fbb6556
paper_authors.name: Thomas Serre
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-10T11:57:00.057Z
paper_authors._id: 64c134cb7fd776c64945f066
paper_authors.avatarUrl: /avatars/dea3c78673dd0d430e267acd310fb79c.svg
paper_authors.fullname: Thomas Serre
paper_authors.type: user
paper_authors.user: tserre
paper_authors._id: 668db3b3e990292e5fbb6557
paper_authors.name: Katherine Hermann
paper_title: Understanding Visual Feature Reliance through the Lens of Complexity
paper_summary: Recent studies suggest that deep learning models inductive bias towards
  favoring simpler features may be one of the sources of shortcut learning. Yet,
  there has been limited focus on understanding the complexity of the myriad
  features that models learn. In this work, we introduce a new metric for
  quantifying feature complexity, based on V-information and
  capturing whether a feature requires complex computational transformations to
  be extracted. Using this V-information metric, we analyze the
  complexities of 10,000 features, represented as directions in the penultimate
  layer, that were extracted from a standard ImageNet-trained vision model. Our
  study addresses four key questions: First, we ask what features look like as a
  function of complexity and find a spectrum of simple to complex features
  present within the model. Second, we ask when features are learned during
  training. We find that simpler features dominate early in training, and more
  complex features emerge gradually. Third, we investigate where within the
  network simple and complex features flow, and find that simpler features tend
  to bypass the visual hierarchy via residual connections. Fourth, we explore the
  connection between features complexity and their importance in driving the
  networks decision. We find that complex features tend to be less important.
  Surprisingly, important features become accessible at earlier layers during
  training, like a sedimentation process, allowing the model to build upon these
  foundational elements.
paper_ai_keywords: $\mathscr{V}$-information inductive bias shortcut learning feature complexity penultimate layer visual hierarchy residual connections