Answers.Body: <p>In language theory, there are generally several admitted levels that can be studied in relation with one another or independently. The semantic level is the one dealing with the meaning of the text ("semantic" comes from the greek and means "to signify"). The semantic level is therefore generally independent from the syntax and even the language used to convey the message. 
  Here is an interesting picture I found on the internet to illustrate my point.
  <a href="https://i.sstatic.net/zdWU1.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/zdWU1.png" alt="enter image description here"></a></p>
  
  <p>EDIT: I took some time reading the paper. I think "semantic levels" refers here to the different neural networks layers used for the exogenous features. </p>
  
  <p>Here is a modified version of their figure I've drawned to make it clearer:
  <a href="https://i.sstatic.net/kB6xD.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/kB6xD.png" alt="enter image description here"></a></p>
  
  <p>In particular, from what I have understood so far, the attention coefficients apply to the whole semantic level (which I find not really clearly indicated on their figure)</p>
  
  <p>The LHS of their figure would then be better described by this new one:</p>
  
  <p><a href="https://i.sstatic.net/tf6tP.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/tf6tP.png" alt="enter image description here"></a></p>
  
  <p>Hope this helps!</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-11-18T13:39:41.833Z
Answers.Text: I think it would be nice if you could also explain the meaning of the expression "semantic level" in the specific context of the paper that the OP is quoting.
Answers.UserDisplayName: nbro
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-11-18T15:01:57.967Z
Answers.Text: @nbro You're right, I read the question too fast. I took some time reading the paper. I think "semantic levels" refers to the possible interactions between the different layers of the neural network. Each layer of the neural network representing possibly a semantic level. The architecture of the RHN seems (does it?) to enable such direct interactions (cf. RHS of fig. 2 page 3 of the paper). Not too sure about this, I'm thrilled to see other's replies as I find the subject really interesting! :-)
Answers.UserDisplayName: Jeanba
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-11-18T17:05:16.210Z
Answers.Text: You should edit your answer to include any new details you think are relevant to the question. Comments are only temporary and can be deleted later.
Answers.UserDisplayName: nbro
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-11-19T08:54:24.587Z
Answers.Text: In the paper the authors mention that the first time they use RHN is to summarize the exogenous data into different **semantics** at different **levels**. So now, whit what @Jenba have said, I understand that the term "semantic levels" refers to the layers RHN shown below in fig. 2 at the bottom in green. Each layer is a different semantic tha is in a different level.
Answers.UserDisplayName: MikelBa
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-11-19T09:57:31.177Z
Answers.Text: Yes I think we finally figured this out. I modified my original response with two new drawings to make it clearer. :-)
Answers.UserDisplayName: Jeanba
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-11-18T11:18:32.330Z
Answers.LastActivityDate: 2019-11-19T10:24:29.257Z
Answers.LastEditDate: 2019-11-19T10:24:29.257Z
Answers.UserDisplayName: Jeanba
Body: <p>I am reading the paper <a href="https://arxiv.org/pdf/1806.00685v1.pdf" rel="nofollow noreferrer">Hierarchical Attention-Based Recurrent Highway Networks for Time Series Prediction</a> (2018) by Yunzhe Tao et al.</p>
  
  <p>In this paper, they use several times the expression "semantic levels". Some examples:</p>
  
  <ul>
  <li><em>HRHN can adaptively select the relevant exogenous features in different <strong>semantic levels</em></strong></li>
  <li><em>the temporal information is usually complicated and may occur at different <strong>semantic levels</em></strong></li>
  <li><em>The encoder RHN reads the convolved features <span class="math-container">$(w_1,w_2,···,w_{T−1})$</span> and models their temporal dependencies at different <strong>semantic levels</em></strong></li>
  <li><em>Then an RHN is used to model the temporal dependencies among convolved input features at different <strong>semantic levels</em></strong></li>
  </ul>
  
  <p>What is the <em>semantic level</em>?</p>
  
Tags: <neural-networks><deep-learning><terminology><papers><time-series>
Title: What is the "semantic level"?
UserDisplayName: MikelBa
ThreadText: # What is the "semantic level"?
  
  (Asked by: username_0 on 2019-11-18)
  
  I am reading the paper [Hierarchical Attention-Based Recurrent Highway Networks for Time Series Prediction](https://arxiv.org/pdf/1806.00685v1.pdf) (2018) by Yunzhe Tao et al.
  
  In this paper, they use several times the expression "semantic levels". Some examples:
  
  * *HRHN can adaptively select the relevant exogenous features in different **semantic levels***
  * *the temporal information is usually complicated and may occur at different **semantic levels***
  * *The encoder RHN reads the convolved features $(w\_1,w\_2,···,w\_{T−1})$ and models their temporal dependencies at different **semantic levels***
  * *Then an RHN is used to model the temporal dependencies among convolved input features at different **semantic levels***
  
  What is the *semantic level*?
  
  **Tags:** neural-networks, deep-learning, terminology, papers, time-series
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2019-11-18. Score: 4 | Accepted answer
  
  In language theory, there are generally several admitted levels that can be studied in relation with one another or independently. The semantic level is the one dealing with the meaning of the text ("semantic" comes from the greek and means "to signify"). The semantic level is therefore generally independent from the syntax and even the language used to convey the message.
  Here is an interesting picture I found on the internet to illustrate my point.
  [![enter image description here](https://i.sstatic.net/zdWU1.png)](https://i.sstatic.net/zdWU1.png)
  
  EDIT: I took some time reading the paper. I think "semantic levels" refers here to the different neural networks layers used for the exogenous features.
  
  Here is a modified version of their figure I've drawned to make it clearer:
  [![enter image description here](https://i.sstatic.net/kB6xD.png)](https://i.sstatic.net/kB6xD.png)
  
  In particular, from what I have understood so far, the attention coefficients apply to the whole semantic level (which I find not really clearly indicated on their figure)
  
  The LHS of their figure would then be better described by this new one:
  
  [![enter image description here](https://i.sstatic.net/tf6tP.png)](https://i.sstatic.net/tf6tP.png)
  
  Hope this helps!
  
  > **Comments:**
  > **username_2**: I think it would be nice if you could also explain the meaning of the expression "semantic level" in the specific context of the paper that the OP is quoting.  
  > **username_1**: @username_2 You're right, I read the question too fast. I took some time reading the paper. I think "semantic levels" refers to the possible interactions between the different layers of the neural network. Each layer of the neural network representing possibly a semantic level. The architecture of the RHN seems (does it?) to enable such direct interactions (cf. RHS of fig. 2 page 3 of the paper). Not too sure about this, I'm thrilled to see other's replies as I find the subject really interesting! :-)  
  > **username_2**: You should edit your answer to include any new details you think are relevant to the question. Comments are only temporary and can be deleted later.  
  > **username_0**: In the paper the authors mention that the first time they use RHN is to summarize the exogenous data into different **semantics** at different **levels**. So now, whit what @Jenba have said, I understand that the term "semantic levels" refers to the layers RHN shown below in fig. 2 at the bottom in green. Each layer is a different semantic tha is in a different level.  
  > **username_1**: Yes I think we finally figured this out. I modified my original response with two new drawings to make it clearer. :-)  
  
  ---
  
  