Answers.Body: <blockquote>
  <p>Based on my understanding, in common machine learning, we formulate our matrices, from vectors, as rows to be observations, columns to be predictors.</p>
  </blockquote>
  <p>The rows (or, in general, the first dimension of your tensor) are typically the observations. For example, in TensorFlow, the first dimension of the input tensor typically refers to the batch size, i.e. the number of observations. If you are using Pandas (a Python library to manipulate data), the rows are typically the observations and the columns are the predictors.</p>
  <p>However, in general, it does not really matter which convention you use, provided that you use one of the conventions consistently in your implementation (i.e. you choose one of the conventions and you stick with it throughout all your code, to avoid complexity), and make it clear in the documentation. So, you can have a matrix where either the rows or columns are observations and, consequently, the columns or, respectively, rows are the features (aka predictors or independent variables).</p>
  <p>Anyway, it is probably a good idea to be consistent with existing literature and implementations/libraries, so you should probably use the rows for the observations.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-09-08T12:01:52.343Z
Answers.LastActivityDate: 2020-09-08T12:11:52.723Z
Answers.LastEditDate: 2020-09-08T12:11:52.723Z
Answers.UserDisplayName: nbro
Body: <p>I attempt to understand the formulation of dictionary learning for this paper:</p>
  <ol>
  <li><a href="https://www.nextcenter.org/wp-content/uploads/2018/02/Depression-Detection-via-Harvesting-Social-Media-A-Multimodal-Dictionary-Learning-Solution.pdf" rel="nofollow noreferrer">Depression Detection via Harvesting Social Media: A Multimodal Dictionary Learning Solution</a></li>
  <li><a href="https://arxiv.org/pdf/1502.01094.pdf" rel="nofollow noreferrer">Multimodal Task-Driven Dictionary Learning for Image Classification</a></li>
  </ol>
  <p>Both papers used the exact formulation in two different domains.</p>
  <p>Based on my understanding, in common machine learning, we formulate our matrices, from vectors, as rows to be observations, columns to be predictors.</p>
  <p>Given a matrix, <span class="math-container">$A$</span>:</p>
  <p><span class="math-container">\begin{array}{lcccccc} 
  &amp; p_1 &amp; p_2 &amp;  p_3 &amp;  p_4 &amp;  p_5 &amp; \text { label } \\
  o_1 &amp; 1 &amp; 2 &amp; 3 &amp; 4 &amp; 1 &amp; 1 \\
  o_2 &amp; 2 &amp; 3 &amp; 4 &amp; 5 &amp; 2 &amp; 1 \\
  o_3 &amp; 3 &amp; 4 &amp; 5 &amp; 6 &amp; 2 &amp; 0 \\
  o_4 &amp; 4 &amp; 5 &amp; 6 &amp; 7 &amp; 3 &amp; 0
  \end{array}</span></p>
  <p>So, using a math notation and excluding the label, I can define this matrix, <span class="math-container">$A = [o_1, o_2, o_3, o_4]  ∈ R^{4×5}$</span>, as <span class="math-container">$A = [{(1, 2, 3, 4, 1), (2, 3, 4, 5, 2), (3, 4, 5, 6, 2), (4, 5, 6, 7, 3)}]$</span>, and in numpy:</p>
  <pre><code>import numpy as np
  
  A = np.array([[1, 2, 3, 4, 1],
                [2, 3, 4, 5, 2],
                [3, 4, 5, 6, 2],
                [4, 5, 6, 7, 3]])
  
  A.shape
  # (4, 5)
  </code></pre>
  <p>Am I right?</p>
  
Tags: <machine-learning><math><papers><data-preprocessing><notation>
Title: Do the rows of the design matrix refer to the observations or predictors?
UserDisplayName: Angus
ThreadText: # Do the rows of the design matrix refer to the observations or predictors?
  
  (Asked by: username_0 on 2020-09-07)
  
  I attempt to understand the formulation of dictionary learning for this paper:
  
  Depression Detection via Harvesting Social Media: A Multimodal Dictionary Learning Solution
  Multimodal Task-Driven Dictionary Learning for Image Classification
  
  Both papers used the exact formulation in two different domains.
  Based on my understanding, in common machine learning, we formulate our matrices, from vectors, as rows to be observations, columns to be predictors.
  Given a matrix, $A$:
  \begin{array}{lcccccc} 
  & p_1 & p_2 &  p_3 &  p_4 &  p_5 & \text { label } \\
  o_1 & 1 & 2 & 3 & 4 & 1 & 1 \\
  o_2 & 2 & 3 & 4 & 5 & 2 & 1 \\
  o_3 & 3 & 4 & 5 & 6 & 2 & 0 \\
  o_4 & 4 & 5 & 6 & 7 & 3 & 0
  \end{array}
  So, using a math notation and excluding the label, I can define this matrix, $A = [o_1, o_2, o_3, o_4]  ∈ R^{4×5}$, as $A = [{(1, 2, 3, 4, 1), (2, 3, 4, 5, 2), (3, 4, 5, 6, 2), (4, 5, 6, 7, 3)}]$, and in numpy:
  import numpy as np
  
  A = np.array([[1, 2, 3, 4, 1],
                [2, 3, 4, 5, 2],
                [3, 4, 5, 6, 2],
                [4, 5, 6, 7, 3]])
  
  A.shape
  # (4, 5)
  
  Am I right?
  
  **Tags:** machine-learning, math, papers, data-preprocessing, notation
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2020-09-08. Score: 1 | Accepted answer
  
  > Based on my understanding, in common machine learning, we formulate our matrices, from vectors, as rows to be observations, columns to be predictors.
  
  The rows (or, in general, the first dimension of your tensor) are typically the observations. For example, in TensorFlow, the first dimension of the input tensor typically refers to the batch size, i.e. the number of observations. If you are using Pandas (a Python library to manipulate data), the rows are typically the observations and the columns are the predictors.
  
  However, in general, it does not really matter which convention you use, provided that you use one of the conventions consistently in your implementation (i.e. you choose one of the conventions and you stick with it throughout all your code, to avoid complexity), and make it clear in the documentation. So, you can have a matrix where either the rows or columns are observations and, consequently, the columns or, respectively, rows are the features (aka predictors or independent variables).
  
  Anyway, it is probably a good idea to be consistent with existing literature and implementations/libraries, so you should probably use the rows for the observations.
  
  
  ---
  
  