Answers.Body: <p>You can find a brief explanation of <em>hierarchical feature selection</em> in the following from &quot;<a href="https://core.ac.uk/download/pdf/154748089.pdf" rel="nofollow noreferrer">An Empirical Evaluation of Hierarchical
  Feature Selection Methods for Classification in
  Bioinformatics Datasets with Gene Ontology-based
  Features</a>&quot; paper:</p>
  <blockquote>
  <p><strong>Hierarchical feature selection</strong> is a new research area in machine
  learning/data mining, which <em>consists of performing feature selection by exploiting dependency relationships among hierarchically structured features</em>.</p>
  </blockquote>
  <p>Therefore, hierarchical features correspond to the dependency structure between features.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-10-09T19:11:44.217Z
Answers.LastActivityDate: 2021-10-09T19:11:44.217Z
Answers.UserDisplayName: OmG
Body: <p>Features in machine learning are the attributes of the elements of a data set. They are considered as random variables in probability.</p>
  <p>Consider the following excerpt from <a href="https://pytorch.org/assets/deep-learning/Deep-Learning-with-PyTorch.pdf#page=35" rel="nofollow noreferrer">1.1: The deep learning revolution</a> of the textbook named <strong>Deep learning with PyTorch</strong> by <em>Eli Stevens, Luca Antiga, Thomas Viehmann</em>,</p>
  <blockquote>
  <p>On the right, with deep learning, the raw data is fed to an algorithm
  that <strong>extracts hierarchical features automatically</strong>, guided by the
  optimization of its own performance on the task; the results will be
  as good as the ability of the practitioner to drive the algorithm
  toward its goal.</p>
  </blockquote>
  <p>When can we call a feature hierarchical? Does it refer to a random variable that is a (function on) derived from some other random variables?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-10-08T12:35:09.863Z
Comments.Text: It may be a good idea to quote/cite something that uses the term/expression "hierarchical feature". I suppose that you're talking about the hierarchical nature of neural networks, which are interpreted as (hierarchical) feature extractors. However, I am not sure if "hierarchical feature" is a common term.
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-10-08T22:07:53.427Z
Comments.Text: Yeah, @nbro. It is true. So, we can call a feature hierarchial only if it is obtained from hidden or output layers of a neural network?
Comments.UserDisplayName: hanugm
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-01-15T00:44:28.287Z
Comments.Text: I'm not sure if this term is well-defined. It's common to say that neural networks learn "hierarchical representations of the data" (because they are essentially hierarchies of layers), and representations can be viewed as "features" of the data, but I've never seen a mathematical definition of "hierarchical feature". Intuitively, hierarchical features are associated with the hierarchy that the neural network represents, but I don't know the exact answer to your question because, as I said, I am not aware of a mathematical definition of "hierarchical feature".
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-01-15T00:46:31.543Z
Comments.Text: I guess they are just referring to the hierarchical nature of the neural network, which "stores" the "representation" of the data. For example, [here](https://people.idsia.ch/~ciresan/data/icann2011.pdf), after a quick search, they use the term "Hierarchical Feature Extraction". This makes more sense to me, because they are just saying that the "feature extraction" is hierarchical, i.e. we extract the features as a hierarchy, they are not saying that features are hierarchical.
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-01-15T00:46:33.747Z
Comments.Text: But, saying that features are hierarchical would also be ok, if you're just referring to the neural network as the representation/features of the data. Note: this is just my interpretation of that term. So, take these words with a grain of salt.
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-01-15T00:51:33.103Z
Comments.Text: Ha, @nbro. It seems that they used the word hierarchical in a loose sense.
Comments.UserDisplayName: hanugm
Tags: <deep-learning><terminology><features><random-variable>
Title: When can we call a feature "hierarchical"?
UserDisplayName: hanugm
ThreadText: # When can we call a feature "hierarchical"?
  
  (Asked by: username_0 on 2021-10-08)
  
  Features in machine learning are the attributes of the elements of a data set. They are considered as random variables in probability.
  
  Consider the following excerpt from [1.1: The deep learning revolution](https://pytorch.org/assets/deep-learning/Deep-Learning-with-PyTorch.pdf#page=35) of the textbook named **Deep learning with PyTorch** by *Eli Stevens, Luca Antiga, Thomas Viehmann*,
  
  > On the right, with deep learning, the raw data is fed to an algorithm
  > that **extracts hierarchical features automatically**, guided by the
  > optimization of its own performance on the task; the results will be
  > as good as the ability of the practitioner to drive the algorithm
  > toward its goal.
  
  When can we call a feature hierarchical? Does it refer to a random variable that is a (function on) derived from some other random variables?
  
  **Tags:** deep-learning, terminology, features, random-variable
  > **Comments:**
  > **username_1**: It may be a good idea to quote/cite something that uses the term/expression "hierarchical feature". I suppose that you're talking about the hierarchical nature of neural networks, which are interpreted as (hierarchical) feature extractors. However, I am not sure if "hierarchical feature" is a common term.  
  > **username_0**: Yeah, @nbro. It is true. So, we can call a feature hierarchial only if it is obtained from hidden or output layers of a neural network?  
  > **username_1**: I'm not sure if this term is well-defined. It's common to say that neural networks learn "hierarchical representations of the data" (because they are essentially hierarchies of layers), and representations can be viewed as "features" of the data, but I've never seen a mathematical definition of "hierarchical feature". Intuitively, hierarchical features are associated with the hierarchy that the neural network represents, but I don't know the exact answer to your question because, as I said, I am not aware of a mathematical definition of "hierarchical feature".  
  > **username_1**: I guess they are just referring to the hierarchical nature of the neural network, which "stores" the "representation" of the data. For example, [here](https://people.idsia.ch/~ciresan/data/icann2011.pdf), after a quick search, they use the term "Hierarchical Feature Extraction". This makes more sense to me, because they are just saying that the "feature extraction" is hierarchical, i.e. we extract the features as a hierarchy, they are not saying that features are hierarchical.  
  > **username_1**: But, saying that features are hierarchical would also be ok, if you're just referring to the neural network as the representation/features of the data. Note: this is just my interpretation of that term. So, take these words with a grain of salt.  
  > **username_0**: Ha, @nbro. It seems that they used the word hierarchical in a loose sense.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2021-10-09. Score: 1
  
  You can find a brief explanation of *hierarchical feature selection* in the following from "[An Empirical Evaluation of Hierarchical
  Feature Selection Methods for Classification in
  Bioinformatics Datasets with Gene Ontology-based
  Features](https://core.ac.uk/download/pdf/154748089.pdf)" paper:
  
  > **Hierarchical feature selection** is a new research area in machine
  > learning/data mining, which *consists of performing feature selection by exploiting dependency relationships among hierarchically structured features*.
  
  Therefore, hierarchical features correspond to the dependency structure between features.
  
  
  ---
  
  