Answers.Body: <p>as far as I understand it, it's not the log-probability but the log-likelihood.
  the likelihood can be greater than 1 so negative loss is possible.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-09-19T11:08:30.053Z
Answers.LastActivityDate: 2022-09-19T11:08:30.053Z
Answers.UserDisplayName: His Royal Dudeness
Answers.Body: <p>I don't know if this comment will be helpful, but shouldn't the sum of the log determinant of the Jacobian (LDJ) have opposite sign in the forward and inverse pass? I'm not talking about the LDJ being sum of the positive scaling function in the forward and sum of the negative of the scaling function in the inverse, I'm talking about the LDJ itself. </p>
  
  <p>For example, from the change of variable formula, 
  <span class="math-container">$$p_z(z) \ dz = p_x(x) \ dx$$</span>
  differentiating w.r.t <span class="math-container">$\ dz$</span> gives,
  <span class="math-container">$$ p_z(z) = p_x(x) \left|\frac{dx}{dz}\right|$$</span>
  taking the log of both sides gives,
  <span class="math-container">$$ ln(p_z(z)) = ln(p_x(x)) +  ln\left(\left|\frac{dx}{dz}\right|\right)$$</span>
  Now, if we repeat this process but differentitating w.r.t <span class="math-container">$\ dx$</span>, 
  <span class="math-container">$$ p_x(x) = p_z(z) \left|\frac{dz}{dx}\right|$$</span>
  taking the log and rearranging for <span class="math-container">$ln(p_z(z))$</span> gives,
  taking the log of both sides gives,
  <span class="math-container">$$ ln(p_z(z)) = ln(p_x(x)) -  ln\left(\left|\frac{dz}{dx}\right|\right)$$</span></p>
  
  <p>A change of sign occurs when the Jacobian is reversed. Could this be your problem? Also, as @chris-cundy said you can have a probability density greater than 1 at a single point. Remember it is the integral over all space that cannot be greater than one, an individual point between greater than 1 is fine.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-05-13T10:59:13.767Z
Answers.LastActivityDate: 2020-05-13T10:59:13.767Z
Answers.UserDisplayName: AlphaBetaGamma96
Body: <p>I am trying to use RealNVP with some data I have (the input size is a 1D vector of size 22). <a href="https://arxiv.org/abs/1605.08803" rel="nofollow noreferrer">Here</a> is the link to the RealNVP paper and <a href="https://lilianweng.github.io/lil-log/2018/10/13/flow-based-deep-generative-models.html" rel="nofollow noreferrer">here</a> is a nice, short explanation of it (the paper is pretty long). My code is mainly based on <a href="https://github.com/tonyduan/normalizing-flows" rel="nofollow noreferrer">this</a> code from GitHub and below are the main piece that I am using (with slight adjustments). The problem is that the loss is getting negative, which in the definition of my code means that the log-probability of the my data is positive, which in turn means that the probabilities are bigger than 1. This is impossible mathematically, and I see no way how this can happen, from a mathematical point of view. I also couldn't find a mistake in my code. Can someone help me with this? Is there a mistake there? Am I missing something with my understanding of normalizing flows? Thank you! </p>
  
  <pre><code>class NormalizingFlowModel(nn.Module):
  
      def __init__(self, prior, flows):
          super().__init__()
          self.prior = prior
          self.flows = nn.ModuleList(flows)
  
      def forward(self, x):
          m, _ = x.shape
          log_det = torch.zeros(m).cuda()
          for flow in self.flows:
              x, ld = flow.forward(x)
              log_det += ld
          z, prior_logprob = x, self.prior.log_prob(x)
          return z, prior_logprob, log_det
  
      def inverse(self, z):
          m, _ = z.shape
          log_det = torch.zeros(m).cuda()
          for flow in self.flows[::-1]:
              z, ld = flow.inverse(z)
              log_det += ld
          x = z
          return x, log_det
  
      def sample(self, n_samples):
          z = self.prior.sample((n_samples,))
          x, _ = self.inverse(z)
          return x
  
  
  class FCNN_for_NVP(nn.Module):
      """
      Simple fully connected neural network to be used for Real NVP
      """
      def __init__(self, in_dim, out_dim):
          super().__init__()
          self.network = nn.Sequential(
              nn.Linear(in_dim, 32),
              nn.Tanh(),
              nn.Linear(32, 32),
              nn.Tanh(),
              nn.Linear(32, 64),
              nn.Tanh(),
              nn.Linear(64, 64),
              nn.Tanh(),
              nn.Linear(64, 32),
              nn.Tanh(),
              nn.Linear(32, 32),
              nn.Tanh(),
              nn.Linear(32, out_dim),
          )
  
      def forward(self, x):
          return self.network(x)
  
  
  class RealNVP(nn.Module):
      """
      Non-volume preserving flow.
  
      [Dinh et. al. 2017]
      """
      def __init__(self, dim, base_network=FCNN_for_NVP):
          super().__init__()
          self.dim = dim
          self.t1 = base_network(dim // 2, dim // 2)
          self.s1 = base_network(dim // 2, dim // 2)
          self.t2 = base_network(dim // 2, dim // 2)
          self.s2 = base_network(dim // 2, dim // 2)
  
      def forward(self, x):
          lower, upper = x[:,:self.dim // 2], x[:,self.dim // 2:]      
          t1_transformed = self.t1(lower)
          s1_transformed = self.s1(lower)
          upper = t1_transformed + upper * torch.exp(s1_transformed)
          t2_transformed = self.t2(upper)
          s2_transformed = self.s2(upper)
          lower = t2_transformed + lower * torch.exp(s2_transformed)
          z = torch.cat([lower, upper], dim=1)
          log_det = torch.sum(s1_transformed, dim=1) + torch.sum(s2_transformed, dim=1)
          return z, log_det
  
      def inverse(self, z):
          lower, upper = z[:,:self.dim // 2], z[:,self.dim // 2:]
          t2_transformed = self.t2(upper)
          s2_transformed = self.s2(upper)
          lower = (lower - t2_transformed) * torch.exp(-s2_transformed)
          t1_transformed = self.t1(lower)
          s1_transformed = self.s1(lower)
          upper = (upper - t1_transformed) * torch.exp(-s1_transformed)
          x = torch.cat([lower, upper], dim=1)
          log_det = torch.sum(-s1_transformed, dim=1) + torch.sum(-s2_transformed, dim=1)
          return x, log_det
  
  flow = RealNVP(dim=data.size(1))
  flows = [flow for _ in range(1)]
  prior = MultivariateNormal(torch.zeros(data.size(1)).cuda(), torch.eye(data.size(1)).cuda())
  model = NormalizingFlowModel(prior, flows)
  model = model.cuda()
  
  for i in range(10):
      for j, dtt in enumerate(my_dataloader_bkg_only):
          optimizer.zero_grad()
          x = dtt[0].float()
          z, prior_logprob, log_det = model(x)
          logprob = prior_logprob + log_det
          loss = -torch.mean(prior_logprob + log_det)
          loss.backward()
          optimizer.step()
      if i % 1 == 0:
          print("Saved")
          best_loss = logprob.mean().data.cpu().numpy()
          print(logprob.mean().data.cpu().numpy(), prior_logprob.mean().data.cpu().numpy(),
                    log_det.mean().data.cpu().numpy())
  </code></pre>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-02-02T02:41:25.657Z
Comments.Text: You might well have a bug, but in general there's no reason to not have a log-likelihood greater than 0. E.g. consider fitting a single 1-d point with a Gaussian with parameterised mean and variance. The maximum-likelihood solution is an infinitely sharp peak with infinite log likelihood.
Comments.UserDisplayName: Chris Cundy
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-02-02T03:19:23.370Z
Comments.Text: So how should I think about a probability bigger than 1? Assuming I have 2 1-d points (in my case I have 1M), and one of them has a probability of, say, 1.5, how can I make sense of that mathematically? Thank you!
Comments.UserDisplayName: JohnDoe122
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-02-02T03:54:16.897Z
Comments.Text: You are computing a probability density (flow models are density models), so you should interpret it as a probability density. Maybe this will help https://towardsdatascience.com/probability-concepts-explained-probability-distributions-introduction-part-3-4a5db81858dc
Comments.UserDisplayName: Chris Cundy
Tags: <objective-functions><pytorch>
Title: RealNVP gives wrong probabilities
UserDisplayName: JohnDoe122
ThreadText: # RealNVP gives wrong probabilities
  
  (Asked by: username_0 on 2020-02-02)
  
  I am trying to use RealNVP with some data I have (the input size is a 1D vector of size 22). Here is the link to the RealNVP paper and here is a nice, short explanation of it (the paper is pretty long). My code is mainly based on this code from GitHub and below are the main piece that I am using (with slight adjustments). The problem is that the loss is getting negative, which in the definition of my code means that the log-probability of the my data is positive, which in turn means that the probabilities are bigger than 1. This is impossible mathematically, and I see no way how this can happen, from a mathematical point of view. I also couldn't find a mistake in my code. Can someone help me with this? Is there a mistake there? Am I missing something with my understanding of normalizing flows? Thank you! 
  class NormalizingFlowModel(nn.Module):
  
      def __init__(self, prior, flows):
          super().__init__()
          self.prior = prior
          self.flows = nn.ModuleList(flows)
  
      def forward(self, x):
          m, _ = x.shape
          log_det = torch.zeros(m).cuda()
          for flow in self.flows:
              x, ld = flow.forward(x)
              log_det += ld
          z, prior_logprob = x, self.prior.log_prob(x)
          return z, prior_logprob, log_det
  
      def inverse(self, z):
          m, _ = z.shape
          log_det = torch.zeros(m).cuda()
          for flow in self.flows[::-1]:
              z, ld = flow.inverse(z)
              log_det += ld
          x = z
          return x, log_det
  
      def sample(self, n_samples):
          z = self.prior.sample((n_samples,))
          x, _ = self.inverse(z)
          return x
  
  
  class FCNN_for_NVP(nn.Module):
      """
      Simple fully connected neural network to be used for Real NVP
      """
      def __init__(self, in_dim, out_dim):
          super().__init__()
          self.network = nn.Sequential(
              nn.Linear(in_dim, 32),
              nn.Tanh(),
              nn.Linear(32, 32),
              nn.Tanh(),
              nn.Linear(32, 64),
              nn.Tanh(),
              nn.Linear(64, 64),
              nn.Tanh(),
              nn.Linear(64, 32),
              nn.Tanh(),
              nn.Linear(32, 32),
              nn.Tanh(),
              nn.Linear(32, out_dim),
          )
  
      def forward(self, x):
          return self.network(x)
  
  
  class RealNVP(nn.Module):
      """
      Non-volume preserving flow.
  
      [Dinh et. al. 2017]
      """
      def __init__(self, dim, base_network=FCNN_for_NVP):
          super().__init__()
          self.dim = dim
          self.t1 = base_network(dim // 2, dim // 2)
          self.s1 = base_network(dim // 2, dim // 2)
          self.t2 = base_network(dim // 2, dim // 2)
          self.s2 = base_network(dim // 2, dim // 2)
  
      def forward(self, x):
          lower, upper = x[:,:self.dim // 2], x[:,self.dim // 2:]      
          t1_transformed = self.t1(lower)
          s1_transformed = self.s1(lower)
          upper = t1_transformed + upper * torch.exp(s1_transformed)
          t2_transformed = self.t2(upper)
          s2_transformed = self.s2(upper)
          lower = t2_transformed + lower * torch.exp(s2_transformed)
          z = torch.cat([lower, upper], dim=1)
          log_det = torch.sum(s1_transformed, dim=1) + torch.sum(s2_transformed, dim=1)
          return z, log_det
  
      def inverse(self, z):
          lower, upper = z[:,:self.dim // 2], z[:,self.dim // 2:]
          t2_transformed = self.t2(upper)
          s2_transformed = self.s2(upper)
          lower = (lower - t2_transformed) * torch.exp(-s2_transformed)
          t1_transformed = self.t1(lower)
          s1_transformed = self.s1(lower)
          upper = (upper - t1_transformed) * torch.exp(-s1_transformed)
          x = torch.cat([lower, upper], dim=1)
          log_det = torch.sum(-s1_transformed, dim=1) + torch.sum(-s2_transformed, dim=1)
          return x, log_det
  
  flow = RealNVP(dim=data.size(1))
  flows = [flow for _ in range(1)]
  prior = MultivariateNormal(torch.zeros(data.size(1)).cuda(), torch.eye(data.size(1)).cuda())
  model = NormalizingFlowModel(prior, flows)
  model = model.cuda()
  
  for i in range(10):
      for j, dtt in enumerate(my_dataloader_bkg_only):
          optimizer.zero_grad()
          x = dtt[0].float()
          z, prior_logprob, log_det = model(x)
          logprob = prior_logprob + log_det
          loss = -torch.mean(prior_logprob + log_det)
          loss.backward()
          optimizer.step()
      if i % 1 == 0:
          print("Saved")
          best_loss = logprob.mean().data.cpu().numpy()
          print(logprob.mean().data.cpu().numpy(), prior_logprob.mean().data.cpu().numpy(),
                    log_det.mean().data.cpu().numpy())
  
  **Tags:** objective-functions, pytorch
  > **Comments:**
  > **username_1**: You might well have a bug, but in general there's no reason to not have a log-likelihood greater than 0. E.g. consider fitting a single 1-d point with a Gaussian with parameterised mean and variance. The maximum-likelihood solution is an infinitely sharp peak with infinite log likelihood.  
  > **username_0**: So how should I think about a probability bigger than 1? Assuming I have 2 1-d points (in my case I have 1M), and one of them has a probability of, say, 1.5, how can I make sense of that mathematically? Thank you!  
  > **username_1**: You are computing a probability density (flow models are density models), so you should interpret it as a probability density. Maybe this will help https://towardsdatascience.com/probability-concepts-explained-probability-distributions-introduction-part-3-4a5db81858dc  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_2 on 2022-09-19. Score: 0
  
  as far as I understand it, it's not the log-probability but the log-likelihood.
  the likelihood can be greater than 1 so negative loss is possible.
  
  
  ---
  
  ### Answer by username_3 on 2020-05-13. Score: 0
  
  I don't know if this comment will be helpful, but shouldn't the sum of the log determinant of the Jacobian (LDJ) have opposite sign in the forward and inverse pass? I'm not talking about the LDJ being sum of the positive scaling function in the forward and sum of the negative of the scaling function in the inverse, I'm talking about the LDJ itself.
  
  For example, from the change of variable formula,
  $$p\_z(z) \ dz = p\_x(x) \ dx$$
  differentiating w.r.t $\ dz$ gives,
  $$ p\_z(z) = p\_x(x) \left|\frac{dx}{dz}\right|$$
  taking the log of both sides gives,
  $$ ln(p\_z(z)) = ln(p\_x(x)) + ln\left(\left|\frac{dx}{dz}\right|\right)$$
  Now, if we repeat this process but differentitating w.r.t $\ dx$,
  $$ p\_x(x) = p\_z(z) \left|\frac{dz}{dx}\right|$$
  taking the log and rearranging for $ln(p\_z(z))$ gives,
  taking the log of both sides gives,
  $$ ln(p\_z(z)) = ln(p\_x(x)) - ln\left(\left|\frac{dz}{dx}\right|\right)$$
  
  A change of sign occurs when the Jacobian is reversed. Could this be your problem? Also, as @chris-cundy said you can have a probability density greater than 1 at a single point. Remember it is the integral over all space that cannot be greater than one, an individual point between greater than 1 is fine.
  
  
  ---
  
  