Body: <p>Bidirectional RNNs are used for generating the semantic vectors of the text at the sentence level and word level.</p>
  <p>In order to train a CNN for the classification tasks, images, and labels/outputs are required in general.</p>
  <p>If the data required is dependent on the algorithm used then please consider the algorithm that uses <em>less possible types of data</em>. Suppose if we need to perform a classification task on images: then in general, images with corresponding labels are expected. But, there may be algorithms that also work better with fewer labels available. But, the types of data in both the cases are the same: Images, labels.</p>
  <p>I want to know such bare minimum types of data requirements for a bidirectional RNN. Afaik, a text file is enough for it to get semantic vectors since learning in text processing generally happens based on the <a href="https://web.stanford.edu/%7Ejurafsky/slp3/6.pdf" rel="nofollow noreferrer">distributional hypothesis</a>.</p>
  <p>Am I correct? If not, then what should be the other <strong>necessary</strong> data requirements for a bidirectional RNN to generate semantic embeddings?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-12-16T17:55:12.327Z
Comments.Text: What do you mean by "distributional hypothesis"?
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-12-16T22:40:02.603Z
Comments.Text: @nbro It means the words with similar meanings occur in similar contexts.
Comments.UserDisplayName: hanugm
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-12-17T23:03:16.467Z
Comments.Text: Ok. I had heard of it before, but I didn't recall it. It may be a good idea to provide some reference that defines this term. Anyway, this question seems to be a bit vague because there are specific techniques to learn embeddings, like word2vec. So, asking what is necessary to learn the embeddings could potentially have many answers, I think. Do you have in mind a specific example (link to code or paper) where embeddings are learned in the context of bidirectional RNNs?
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-12-17T23:08:37.377Z
Comments.Text: I am asking what is the necessary *data/information* required by a bidirectional RNN. I am not asking about other techniques for word embeddings.
Comments.UserDisplayName: hanugm
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-12-17T23:09:27.373Z
Comments.Text: Suppose images and labels are necessary (bare minimum) for a CNN to perform classification task @nbro
Comments.UserDisplayName: hanugm
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-12-17T23:10:59.667Z
Comments.Text: Yes, but to learn the embeddings you need some technique, right? It's not just a model. Of course, the data is needed, like in every machine learning technique, but I don't understand what you're asking here then, if not something like how to process the data with the bidirectional RNN to learn the embeddings. Typically, in ML, you need data + algorithms + models. You have the data, you have the model (bidirectional RNN), then the only thing that is missing are the algorithms from your question.
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-12-17T23:23:05.373Z
Comments.Text: @nbro So, is it true that the algorithm influences the data requirements in the case of RNN? In CNN,  I can see that this is not the case as the task alone influences data requirement.
Comments.UserDisplayName: hanugm
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-12-17T23:27:48.803Z
Comments.Text: @nbro I think the question is clear now. Previously I did not use the word `data`. So, it might be confusing.
Comments.UserDisplayName: hanugm
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-12-18T08:57:00.187Z
Comments.Text: Well, now, I am not sure why you're asking for the "data requirements" or "what is the necessary data?". Are you asking if we need more than one file to learn the embeddings? It's still not clear to me what do you mean by "data requirements" here. Regarding the algorithm affecting the data. Some algorithms may converge faster than others. So, in this sense, they may require more or less data than others.
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-12-18T09:09:08.407Z
Comments.Text: @nbro I am asking for bare requirements only for the algorithms that require minimum data.
Comments.UserDisplayName: hanugm
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-12-18T09:10:00.773Z
Comments.Text: I still don't understand. Minimum requirements in terms of data, but in terms of the size of the data or what? I don't understand.
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-12-18T09:12:19.917Z
Comments.Text: Consider an analogy with CNN. For the classification task, what data do we require? **dataset of images along with labels right**? Similarly for the word embedding task using RNN, what data do we need? is the question. Size or any other aspect is immaterial for the question. @nbro
Comments.UserDisplayName: hanugm
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-12-18T09:14:20.393Z
Comments.Text: That's why the algorithm is important. Not all CNNs are trained in a supervised way, i.e. with labels. This question needs to be more specific. You cannot just say "what are the data requirements for learning word embeddings" as there are different approaches for word embeddings that probably use different ways to learn from data, some labelled and maybe some unlabelled. Do you see now the problem with your question?
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-12-18T09:24:59.417Z
Comments.Text: @nbro I got it now. Changed the question accordingly, please check now whether it is clear or not.
Comments.UserDisplayName: hanugm
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-12-18T09:47:19.123Z
Comments.Text: Ok, let's put it in that way. What would be an example of an answer you're looking for? Are you looking for an answer like "What you need is just unlabelled text. It doesn't matter the size, but is probably better because of reasong X"? By the way, I wouldn't call "labels" a type of data. Labels are part of the data that you have or not. So, you can have labelled data or unlabelled data.
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-12-18T09:50:48.190Z
Comments.Text: @nbro The primary intention is to know what is all data needed for classical algorithms for generating word embeddings using bidirectional RNN. Yes, I am waiting for an answer like that. If an answer says the algorithm name along with data expected, then it would be great.
Comments.UserDisplayName: hanugm
Tags: <recurrent-neural-networks><datasets><semantics><vector-semantics><bidirectional-rnn>
Title: What is all necessary types of data for a bidirectional RNN to learn embeddings?
UserDisplayName: hanugm
ThreadText: # What is all necessary types of data for a bidirectional RNN to learn embeddings?
  
  (Asked by: username_0 on 2021-12-16)
  
  Bidirectional RNNs are used for generating the semantic vectors of the text at the sentence level and word level.
  
  In order to train a CNN for the classification tasks, images, and labels/outputs are required in general.
  
  If the data required is dependent on the algorithm used then please consider the algorithm that uses *less possible types of data*. Suppose if we need to perform a classification task on images: then in general, images with corresponding labels are expected. But, there may be algorithms that also work better with fewer labels available. But, the types of data in both the cases are the same: Images, labels.
  
  I want to know such bare minimum types of data requirements for a bidirectional RNN. Afaik, a text file is enough for it to get semantic vectors since learning in text processing generally happens based on the [distributional hypothesis](https://web.stanford.edu/%7Ejurafsky/slp3/6.pdf).
  
  Am I correct? If not, then what should be the other **necessary** data requirements for a bidirectional RNN to generate semantic embeddings?
  
  **Tags:** recurrent-neural-networks, datasets, semantics, vector-semantics, bidirectional-rnn
  > **Comments:**
  > **username_1**: What do you mean by "distributional hypothesis"?  
  > **username_0**: @nbro It means the words with similar meanings occur in similar contexts.  
  > **username_1**: Ok. I had heard of it before, but I didn't recall it. It may be a good idea to provide some reference that defines this term. Anyway, this question seems to be a bit vague because there are specific techniques to learn embeddings, like word2vec. So, asking what is necessary to learn the embeddings could potentially have many answers, I think. Do you have in mind a specific example (link to code or paper) where embeddings are learned in the context of bidirectional RNNs?  
  > **username_0**: I am asking what is the necessary *data/information* required by a bidirectional RNN. I am not asking about other techniques for word embeddings.  
  > **username_0**: Suppose images and labels are necessary (bare minimum) for a CNN to perform classification task @nbro  
  > **username_1**: Yes, but to learn the embeddings you need some technique, right? It's not just a model. Of course, the data is needed, like in every machine learning technique, but I don't understand what you're asking here then, if not something like how to process the data with the bidirectional RNN to learn the embeddings. Typically, in ML, you need data + algorithms + models. You have the data, you have the model (bidirectional RNN), then the only thing that is missing are the algorithms from your question.  
  > **username_0**: @nbro So, is it true that the algorithm influences the data requirements in the case of RNN? In CNN,  I can see that this is not the case as the task alone influences data requirement.  
  > **username_0**: @nbro I think the question is clear now. Previously I did not use the word `data`. So, it might be confusing.  
  > **username_1**: Well, now, I am not sure why you're asking for the "data requirements" or "what is the necessary data?". Are you asking if we need more than one file to learn the embeddings? It's still not clear to me what do you mean by "data requirements" here. Regarding the algorithm affecting the data. Some algorithms may converge faster than others. So, in this sense, they may require more or less data than others.  
  > **username_0**: @nbro I am asking for bare requirements only for the algorithms that require minimum data.  
  > **username_1**: I still don't understand. Minimum requirements in terms of data, but in terms of the size of the data or what? I don't understand.  
  > **username_0**: Consider an analogy with CNN. For the classification task, what data do we require? **dataset of images along with labels right**? Similarly for the word embedding task using RNN, what data do we need? is the question. Size or any other aspect is immaterial for the question. @nbro  
  > **username_1**: That's why the algorithm is important. Not all CNNs are trained in a supervised way, i.e. with labels. This question needs to be more specific. You cannot just say "what are the data requirements for learning word embeddings" as there are different approaches for word embeddings that probably use different ways to learn from data, some labelled and maybe some unlabelled. Do you see now the problem with your question?  
  > **username_0**: @nbro I got it now. Changed the question accordingly, please check now whether it is clear or not.  
  > **username_1**: Ok, let's put it in that way. What would be an example of an answer you're looking for? Are you looking for an answer like "What you need is just unlabelled text. It doesn't matter the size, but is probably better because of reasong X"? By the way, I wouldn't call "labels" a type of data. Labels are part of the data that you have or not. So, you can have labelled data or unlabelled data.  
  > **username_0**: @nbro The primary intention is to know what is all data needed for classical algorithms for generating word embeddings using bidirectional RNN. Yes, I am waiting for an answer like that. If an answer says the algorithm name along with data expected, then it would be great.  
  
  ---
  
  ## 0 answers
  
  