Answers.Body: <p>Although this question is slightly primarily opinion-based and too broad (and I will probably close it as such) and a good answer will necessarily depend on your background, I will list some of the main theoretical prerequisites that everyone should ideally be familiar with before diving into TensorFlow Probability (TFP).</p>
  
  <p>I am familiar with TFP, given that I've been using it for a project, but I've not used all of its functionalities, such as the bijectors. I've only used the Bayesian layers, distributions, etc., so I will try to give an answer based on my experience. </p>
  
  <p>You definitely need to be familiar with the basic concepts of probability theory, such as distributions, random variables, expectations, etc. A full university-level course in probability and statistics would definitely be helpful!</p>
  
  <p>If you want to use Bayesian layers, you also need to be familiar with Bayesian neural networks (BNNs). To understand BNNs, you need to understand the basic concepts of Bayesian inference (BI), given that most BNNs are based on BI. If you are already familiar with variational auto-encoders (VAEs), then it will be easier to understand BNNs. To understand BNNs, you should start reading the paper <a href="https://arxiv.org/pdf/1505.05424.pdf" rel="nofollow noreferrer">Weight Uncertainty in Neural Networks</a>. If this paper is difficult to follow, you should first read the paper <a href="https://arxiv.org/pdf/1601.00670.pdf" rel="nofollow noreferrer">Variational Inference: A Review for Statisticians</a> and then it may also be useful to read and be familiar with <a href="https://arxiv.org/pdf/1312.6114.pdf" rel="nofollow noreferrer">the VAE paper</a>. Therefore, you will need to be familiar with concepts such as Kullback–Leibler divergence and <a href="https://arxiv.org/pdf/1906.10652.pdf" rel="nofollow noreferrer">Monte Carlo sampling</a>.</p>
  
  <p>The blog post <a href="https://blog.tensorflow.org/2019/03/regression-with-probabilistic-layers-in.html" rel="nofollow noreferrer">Regression with Probabilistic Layers in TensorFlow Probability</a> is the first blog post you should read. If you don't understand it, then it probably means you need to learn its prerequisites (i.e. you need to be familiar with at least linear regression).</p>
  
  <p>TFP also provides several <a href="https://github.com/tensorflow/probability/tree/master/tensorflow_probability/examples" rel="nofollow noreferrer">implementations</a> of other more advanced concepts such as BNNs. The easiest example to follow is probably <a href="https://github.com/tensorflow/probability/blob/master/tensorflow_probability/examples/logistic_regression.py" rel="nofollow noreferrer">the logistic regression example</a>. LR is relatively easy to follow compared to the topics I mentioned above, but, of course, you need to be familiar with logistic regression.</p>
  
  <p>Of course, I cannot list all the theoretical prerequisites (and that's why your post is too broad!) and there are definitely others, but this is a start. Bear in mind that these are not trivial topics, so it is normal to get stuck.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-03-08T22:43:05.423Z
Answers.LastActivityDate: 2020-03-08T23:06:49.803Z
Answers.LastEditDate: 2020-03-08T23:06:49.803Z
Answers.UserDisplayName: nbro
Body: <p>I have some familiarity with the regular Tensorflow library and have been able to create a number of working models with it. There are more than enough resources out there to get up and running and answer most questions on the standard library.</p>
  
  <p>But I recently came across the video on some high-level capabilities of the Tensorflow Probability library, <a href="https://www.youtube.com/watch?v=BrwKURU-wpk" rel="nofollow noreferrer">TensorFlow Probability: Learning with confidence (TF Dev Summit '19)</a>, and I would like to learn it.</p>
  
  <p>The issue is that there are very few resources out there on TFP and given my lack of a formal background in math/statistics, I find myself aimlessly googling to get a grasp of what's going on in the docs. I'm more than willing to invest the time needed, but I just need to know where I can start in terms of resources I can access online.  Specifically, I'm looking to get the necessary domain knowledge needed to work with the library given the lack of courses/tutorials on the library itself.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-03-08T22:01:59.077Z
Comments.Text: I'm sorry. I believe that this question is off-topic here and primarily opinion-based, so I should close it. Can you ask a question that is objectively answerable? I am familiar with TFP. If you want, I can create a chat room for discussion.
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-03-08T22:10:16.103Z
Comments.Text: Could I edit the question to give it more general value?  I realize it's very open-ended but wasn't sure how to avoid that.  Really I'm just looking for like a list of topics that could be considered prerequisites for working with the library.
Comments.UserDisplayName: SuperCodeBrah
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-03-08T22:12:13.467Z
Comments.Text: Have you already followed any of their tutorials/articles? Anyway, are you asking for the theoretical prerequisites or for resources?
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-03-08T22:33:37.733Z
Comments.Text: Thanks for your edit.  Resources would be great, but high level topics would be a start.  On the tutorials, I'm mainly interested in getting a Gaussian mixture model to work, but even that tutorial is a little dense for what I currently know.  Before that, I tried a few implementations but hit one issue or another and feel like I just need a better grasp on the fundamentals.  I can probably brute force the tutorials (i.e., repetitive googling), but just knowing which areas to investigate would probably help me find courses/tutorials that lay out the broad ideas in a more rational manner.
Comments.UserDisplayName: SuperCodeBrah
Tags: <tensorflow><probability><tensorflow-probability>
Title: What are the prerequisites to start using the TensorFlow Probability library?
UserDisplayName: SuperCodeBrah
ThreadText: # What are the prerequisites to start using the TensorFlow Probability library?
  
  (Asked by: username_0 on 2020-03-08)
  
  I have some familiarity with the regular Tensorflow library and have been able to create a number of working models with it. There are more than enough resources out there to get up and running and answer most questions on the standard library.
  
  But I recently came across the video on some high-level capabilities of the Tensorflow Probability library, [TensorFlow Probability: Learning with confidence (TF Dev Summit '19)](https://www.youtube.com/watch?v=BrwKURU-wpk), and I would like to learn it.
  
  The issue is that there are very few resources out there on TFP and given my lack of a formal background in math/statistics, I find myself aimlessly googling to get a grasp of what's going on in the docs. I'm more than willing to invest the time needed, but I just need to know where I can start in terms of resources I can access online. Specifically, I'm looking to get the necessary domain knowledge needed to work with the library given the lack of courses/tutorials on the library itself.
  
  **Tags:** tensorflow, probability, tensorflow-probability
  
  > **Comments:**
  > **username_1**: I'm sorry. I believe that this question is off-topic here and primarily opinion-based, so I should close it. Can you ask a question that is objectively answerable? I am familiar with TFP. If you want, I can create a chat room for discussion.  
  > **username_0**: Could I edit the question to give it more general value?  I realize it's very open-ended but wasn't sure how to avoid that.  Really I'm just looking for like a list of topics that could be considered prerequisites for working with the library.  
  > **username_1**: Have you already followed any of their tutorials/articles? Anyway, are you asking for the theoretical prerequisites or for resources?  
  > **username_0**: Thanks for your edit.  Resources would be great, but high level topics would be a start.  On the tutorials, I'm mainly interested in getting a Gaussian mixture model to work, but even that tutorial is a little dense for what I currently know.  Before that, I tried a few implementations but hit one issue or another and feel like I just need a better grasp on the fundamentals.  I can probably brute force the tutorials (i.e., repetitive googling), but just knowing which areas to investigate would probably help me find courses/tutorials that lay out the broad ideas in a more rational manner.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2020-03-08. Score: 2 | Accepted answer
  
  Although this question is slightly primarily opinion-based and too broad (and I will probably close it as such) and a good answer will necessarily depend on your background, I will list some of the main theoretical prerequisites that everyone should ideally be familiar with before diving into TensorFlow Probability (TFP).
  
  I am familiar with TFP, given that I've been using it for a project, but I've not used all of its functionalities, such as the bijectors. I've only used the Bayesian layers, distributions, etc., so I will try to give an answer based on my experience.
  
  You definitely need to be familiar with the basic concepts of probability theory, such as distributions, random variables, expectations, etc. A full university-level course in probability and statistics would definitely be helpful!
  
  If you want to use Bayesian layers, you also need to be familiar with Bayesian neural networks (BNNs). To understand BNNs, you need to understand the basic concepts of Bayesian inference (BI), given that most BNNs are based on BI. If you are already familiar with variational auto-encoders (VAEs), then it will be easier to understand BNNs. To understand BNNs, you should start reading the paper [Weight Uncertainty in Neural Networks](https://arxiv.org/pdf/1505.05424.pdf). If this paper is difficult to follow, you should first read the paper [Variational Inference: A Review for Statisticians](https://arxiv.org/pdf/1601.00670.pdf) and then it may also be useful to read and be familiar with [the VAE paper](https://arxiv.org/pdf/1312.6114.pdf). Therefore, you will need to be familiar with concepts such as Kullback–Leibler divergence and [Monte Carlo sampling](https://arxiv.org/pdf/1906.10652.pdf).
  
  The blog post [Regression with Probabilistic Layers in TensorFlow Probability](https://blog.tensorflow.org/2019/03/regression-with-probabilistic-layers-in.html) is the first blog post you should read. If you don't understand it, then it probably means you need to learn its prerequisites (i.e. you need to be familiar with at least linear regression).
  
  TFP also provides several [implementations](https://github.com/tensorflow/probability/tree/master/tensorflow_probability/examples) of other more advanced concepts such as BNNs. The easiest example to follow is probably [the logistic regression example](https://github.com/tensorflow/probability/blob/master/tensorflow_probability/examples/logistic_regression.py). LR is relatively easy to follow compared to the topics I mentioned above, but, of course, you need to be familiar with logistic regression.
  
  Of course, I cannot list all the theoretical prerequisites (and that's why your post is too broad!) and there are definitely others, but this is a start. Bear in mind that these are not trivial topics, so it is normal to get stuck.
  
  
  ---
  
  