Answers.Body: <p>You don't need a powerful machine or a pre-trained model. All you need
  — as Neil Slater rightly says in his comment to your question — is a corpus of English texts to analyse. There are some corpora available for linguistic research, or you can collect your own.</p>
  <p>Then you need to split the texts into sentences, and tokenise them, and you're ready to calculate probabilities.</p>
  <p>In linguistics there are some commonly used co-occurrence measures, such as mutual information, log-likelihood, or t-score. These are all used to measure the associations between words, typically in a window around the target word, rarely within sentences (as it makes processing easier). Any textbook on statistics in corpus linguistics will tell you how to do that.</p>
  <p>The exact parameters depend on what the purpose of your analysis is, but you won't be able to do this without a corpus.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-10-16T07:44:45.097Z
Answers.Text: Okay I can train a model myself, do you know a 2010+ british corpus ? for exemple BBC (news, sport, 2, 3, ....), the time, channel 4, BT sport, sky sports, sky news, ... ?
Answers.UserDisplayName: CechMS
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-10-16T07:50:26.430Z
Answers.Text: I thought about a pretrained model because it can train on a powerfull machine, for example if I want to know if "coffee" can appear with "run with floods." As even if it can it's very rare to catch this match, I think big company can do it but I probably can't.
Answers.UserDisplayName: CechMS
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-10-16T08:24:00.127Z
Answers.Text: Have a look at the BNC (British National Corpus)
Answers.UserDisplayName: Oliver Mason
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-10-15T17:51:14.687Z
Answers.LastActivityDate: 2021-10-15T17:51:14.687Z
Answers.UserDisplayName: Oliver Mason
Body: <p>How can I know if two words are likely to appear in the same sentence in (British) English (or English in general to enhance the chance of getting a result).</p>
  <p>As I don't have access to a powerful machine, is there any relevant website? Or a pretrained model I can use? Or something else?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-10-15T10:36:42.723Z
Comments.Text: There isn't really such a thing as "English in general", you have to pick some kind of corpus. Some analysis might go a little beyond the corpus and generate probabilities for pairs that are not observed. But it would still need to be based on some database of sentences, and will give different results whether you include Shakespear or Wikipedia or 20th century newspaper editorials or Twitter etc.
Comments.UserDisplayName: Neil Slater
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-10-15T10:46:17.167Z
Comments.Text: I thought it was obvious that it means english you want. I prefer modern english.
Comments.UserDisplayName: CechMS
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-10-15T10:55:41.120Z
Comments.Text: It has to be based on an actual realised corpus. There is no corpus of "modern English". I am not asking you to name a corpus in the question, but trying to set expectations that you cannot get a result for *english in general*. Instead you may be able to get results trained from different corpuses. The numbers you get will be different depending on what sources are analysed by the model. If that is OK, and you are not looking for some "true" value for word pairs, then its fine.
Comments.UserDisplayName: Neil Slater
Tags: <natural-language-processing>
Title: Probability that two words appear in the same sentence
UserDisplayName: CechMS
ThreadText: # Probability that two words appear in the same sentence
  
  (Asked by: username_0 on 2021-10-15)
  
  How can I know if two words are likely to appear in the same sentence in (British) English (or English in general to enhance the chance of getting a result).
  
  As I don't have access to a powerful machine, is there any relevant website? Or a pretrained model I can use? Or something else?
  
  **Tags:** natural-language-processing
  > **Comments:**
  > **username_1**: There isn't really such a thing as "English in general", you have to pick some kind of corpus. Some analysis might go a little beyond the corpus and generate probabilities for pairs that are not observed. But it would still need to be based on some database of sentences, and will give different results whether you include Shakespear or Wikipedia or 20th century newspaper editorials or Twitter etc.  
  > **username_0**: I thought it was obvious that it means english you want. I prefer modern english.  
  > **username_1**: It has to be based on an actual realised corpus. There is no corpus of "modern English". I am not asking you to name a corpus in the question, but trying to set expectations that you cannot get a result for *english in general*. Instead you may be able to get results trained from different corpuses. The numbers you get will be different depending on what sources are analysed by the model. If that is OK, and you are not looking for some "true" value for word pairs, then its fine.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2021-10-15. Score: 2 | Accepted answer
  
  You don't need a powerful machine or a pre-trained model. All you need
  — as Neil Slater rightly says in his comment to your question — is a corpus of English texts to analyse. There are some corpora available for linguistic research, or you can collect your own.
  
  Then you need to split the texts into sentences, and tokenise them, and you're ready to calculate probabilities.
  
  In linguistics there are some commonly used co-occurrence measures, such as mutual information, log-likelihood, or t-score. These are all used to measure the associations between words, typically in a window around the target word, rarely within sentences (as it makes processing easier). Any textbook on statistics in corpus linguistics will tell you how to do that.
  
  The exact parameters depend on what the purpose of your analysis is, but you won't be able to do this without a corpus.
  
  > **Comments:**
  > **username_0**: Okay I can train a model myself, do you know a 2010+ british corpus ? for exemple BBC (news, sport, 2, 3, ....), the time, channel 4, BT sport, sky sports, sky news, ... ?  
  > **username_0**: I thought about a pretrained model because it can train on a powerfull machine, for example if I want to know if "coffee" can appear with "run with floods." As even if it can it's very rare to catch this match, I think big company can do it but I probably can't.  
  > **username_2**: Have a look at the BNC (British National Corpus)  
  
  ---
  
  