Answers.Body: <p>I would go so far as to say that unless the training examples include predicate data- that is, data about conditions leading up to a crime or non-crime-, then you cannot have enough information to predict the occurence of a crime from conditions or events that happen in advance of a potential crime not yet committed.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-10-30T12:04:00.187Z
Answers.Text: This doesn't seem highly likely, and in fact, I have seen people use this kind of data to predict the probability that a crime will occur. In fact, it is likely you already do this yourself. For instance, where would you rather be at 3am: An affluent suburb, or the "bad part" of your town?
Answers.UserDisplayName: John Doucette
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-10-30T14:53:17.943Z
Answers.Text: Are you saying that the training data can just include spatial data and other such information that is only available at the immediate time of the crimes?  I suppose that's true.  A crime map of a city and no other information would give a valid basis for estimating future likelihood of crimes at various places in the city; and adding the times that the crimes occurred would make it possible to predict, e.g., that it is risky to walk alone at 3am Sunday morning in a particular neighborhood.  So I agree.  I won't delete my answer because it plus these comments might be useful to someone.
Answers.UserDisplayName: S. McGrew
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-10-30T04:21:19.317Z
Answers.LastActivityDate: 2019-10-30T04:21:19.317Z
Answers.UserDisplayName: S. McGrew
Answers.Body: <p>It might be more informative to:</p>
  
  <ol>
  <li><p>Label each combination of location, type, and time of crime with a crime <em>rate</em>. For example, theft, in Crystal City, at 11pm at night, occurs 20 times per year, or 0.4 times per resident per year.</p></li>
  <li><p>Predict the crime <em>rate</em>, rather than individual events.</p></li>
  </ol>
  
  <p>This avoids the need to have explicit examples of "non-crime", and lets you instead directly learn something related to the probabilities of crimes being committed (the rate).</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-10-29T11:37:13.803Z
Answers.Text: That makes sense. Do not know if I have enough data for that. Many thanks.
Answers.UserDisplayName: schoon
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-10-30T08:22:30.507Z
Answers.Text: This is hardly useful. That rate will be 0 most of the time, which corresponds to having explicit examples of "non-crime".
Answers.UserDisplayName: jan-glx
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-10-30T12:00:00.410Z
Answers.Text: @jan-glx That will depend to a great degree on the timescale you look at. Using, say, the yearly rate, will yield non-zero statistics for most kinds of crime for the size of areas these surveys usually cover (neighborhoods; police districts). I have used this technique myself, and seen others use it specifically with crime data, with good effect.
Answers.UserDisplayName: John Doucette
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-10-31T12:47:39.773Z
Answers.Text: You are asking to bin the data in bins that are so wide that most of bins are non-zero. Possible, sure, but then one won't be able to make use of all the features: keyword description of the time, time of the crime, location they span a space so large, there won't possibly be as many crimes. the approach you describe is useful if you want to predict crime based on features of the area but not if you want to use features of the crime.
Answers.UserDisplayName: jan-glx
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-10-28T15:37:54.613Z
Answers.LastActivityDate: 2019-10-28T15:37:54.613Z
Answers.UserDisplayName: John Doucette
Body: <p>I am trying to predict crime. I have data with factors: location, keyword description of the crime, time crime occurred and so on. This is for crimes that occurred in the past. </p>
  
  <p>I would like to treat the prediction of crimes as a binary classification problem. In this model, the data I have collected would form the "positive" examples: they are all examples of a crime happening. However, I am unsure what to use for the negative examples. </p>
  
  <p>Obviously, most of the time there is no crime at the location, but can I use this as negative data? For example, if I know there was a crime at 7pm at location X, and no other crimes there, should I generate new negative data points for every hour <em>except</em> 7pm?</p>
  
  <p>Ideally, I want to create probabilities of crime based on a set of factors.</p>
  
Tags: <machine-learning><ai-design><applications><prediction>
Title: How do I predict the occurrence of rare events?
UserDisplayName: schoon
ThreadText: # How do I predict the occurrence of rare events?
  
  (Asked by: username_0 on 2019-10-28)
  
  I am trying to predict crime. I have data with factors: location, keyword description of the crime, time crime occurred and so on. This is for crimes that occurred in the past.
  
  I would like to treat the prediction of crimes as a binary classification problem. In this model, the data I have collected would form the "positive" examples: they are all examples of a crime happening. However, I am unsure what to use for the negative examples.
  
  Obviously, most of the time there is no crime at the location, but can I use this as negative data? For example, if I know there was a crime at 7pm at location X, and no other crimes there, should I generate new negative data points for every hour *except* 7pm?
  
  Ideally, I want to create probabilities of crime based on a set of factors.
  
  **Tags:** machine-learning, ai-design, applications, prediction
  
  ---
  
  ## 2 answers
  
  ### Answer by username_2 on 2019-10-28. Score: 3 | Accepted answer
  
  It might be more informative to:
  
  1. Label each combination of location, type, and time of crime with a crime *rate*. For example, theft, in Crystal City, at 11pm at night, occurs 20 times per year, or 0.4 times per resident per year.
  2. Predict the crime *rate*, rather than individual events.
  
  This avoids the need to have explicit examples of "non-crime", and lets you instead directly learn something related to the probabilities of crimes being committed (the rate).
  
  > **Comments:**
  > **username_0**: That makes sense. Do not know if I have enough data for that. Many thanks.  
  > **username_3**: This is hardly useful. That rate will be 0 most of the time, which corresponds to having explicit examples of "non-crime".  
  > **username_2**: @jan-glx That will depend to a great degree on the timescale you look at. Using, say, the yearly rate, will yield non-zero statistics for most kinds of crime for the size of areas these surveys usually cover (neighborhoods; police districts). I have used this technique myself, and seen others use it specifically with crime data, with good effect.  
  > **username_3**: You are asking to bin the data in bins that are so wide that most of bins are non-zero. Possible, sure, but then one won't be able to make use of all the features: keyword description of the time, time of the crime, location they span a space so large, there won't possibly be as many crimes. the approach you describe is useful if you want to predict crime based on features of the area but not if you want to use features of the crime.  
  
  ---
  
  ### Answer by username_1 on 2019-10-30. Score: 0
  
  I would go so far as to say that unless the training examples include predicate data- that is, data about conditions leading up to a crime or non-crime-, then you cannot have enough information to predict the occurence of a crime from conditions or events that happen in advance of a potential crime not yet committed.
  
  > **Comments:**
  > **username_2**: This doesn't seem highly likely, and in fact, I have seen people use this kind of data to predict the probability that a crime will occur. In fact, it is likely you already do this yourself. For instance, where would you rather be at 3am: An affluent suburb, or the "bad part" of your town?  
  > **username_1**: Are you saying that the training data can just include spatial data and other such information that is only available at the immediate time of the crimes?  I suppose that's true.  A crime map of a city and no other information would give a valid basis for estimating future likelihood of crimes at various places in the city; and adding the times that the crimes occurred would make it possible to predict, e.g., that it is risky to walk alone at 3am Sunday morning in a particular neighborhood.  So I agree.  I won't delete my answer because it plus these comments might be useful to someone.  
  
  ---
  
  