Answers.Body: <p>If I have understood the question is to generalise the column names to permit a function to accept dataframes where you don't have the same column name, or it's different from the previous dataframe column names. Here the column could be called <code>RNA1_Approved_Symbol</code> or <code>RNA2_Approved_Symbol</code> in addition a different column could be called <code>RNA2_ID</code> or <code>RNA_ID</code>. No problem providing the column order is identical for the columns in question. If the orders are not identical see the code below the <code>----</code> to correct that.</p>
  <p>If the order is identical the simple answer is <code>df.columns[i]</code>. Thus it's obtaining the column name on its position within the dataframe (<code>columns</code> command), where <code>i</code> is the column location.</p>
  <p>Example, to rename <code>RNA1_ID</code> to <code>RNA1_ID_new</code>, or <code>RNA2_ID</code> to <code>RNA2_ID_new</code></p>
  <pre><code>import re
  
  df1 = pd.DataFrame({&quot;perch&quot;: [5,6,7], &quot;RNA1_ID&quot;: [1,2,4], &quot;roach&quot;: [30,42,51]})
  df2 = pd.DataFrame({&quot;perch&quot;: [5,6,7], &quot;RNA2_ID&quot;: [1,2,4], &quot;roach&quot;: [30,42,51]})
  
  def renamecolumn(df):
      myname = df.columns[1]
      newname = re.sub(r'$','_new', myname)
      df = df.rename(columns={myname: newname})
      return df
  
  print (renamecolumn(df1))
  print (renamecolumn(df2))
  </code></pre>
  <p>Output,</p>
  <div class="s-table-container">
  <table class="s-table">
  <thead>
  <tr>
  <th>-</th>
  <th>perch</th>
  <th>RNA1_ID_new</th>
  <th>roach</th>
  </tr>
  </thead>
  <tbody>
  <tr>
  <td>0</td>
  <td>5</td>
  <td>1</td>
  <td>30</td>
  </tr>
  <tr>
  <td>1</td>
  <td>6</td>
  <td>2</td>
  <td>42</td>
  </tr>
  <tr>
  <td>2</td>
  <td>7</td>
  <td>4</td>
  <td>51</td>
  </tr>
  </tbody>
  </table>
  </div><div class="s-table-container">
  <table class="s-table">
  <thead>
  <tr>
  <th>-</th>
  <th>perch</th>
  <th>RNA2_ID_new</th>
  <th>roach</th>
  </tr>
  </thead>
  <tbody>
  <tr>
  <td>0</td>
  <td>5</td>
  <td>1</td>
  <td>30</td>
  </tr>
  <tr>
  <td>1</td>
  <td>6</td>
  <td>2</td>
  <td>42</td>
  </tr>
  <tr>
  <td>2</td>
  <td>7</td>
  <td>4</td>
  <td>51</td>
  </tr>
  </tbody>
  </table>
  </div>
  <p>I noted there was a fair number of <code>set_index</code> commands so the earlier the column name can be trapped inside a variable the better, because <code>set_index</code> will change the column order.</p>
  <hr />
  <p>Alternatively, the columns can be re-arranged to be in a given order if the original order between <code>df1</code> and <code>df2</code> is different. Here I'll move the column to the front.</p>
  <pre><code>mycols = df1.columns.tolist()
  mycols = mycols[-2:] + mycols[:-2]
  df1 = df1[mycols]
  print (df1)
  </code></pre>
  <div class="s-table-container">
  <table class="s-table">
  <thead>
  <tr>
  <th>-</th>
  <th>RNA1_ID</th>
  <th>roach</th>
  <th>perch</th>
  </tr>
  </thead>
  <tbody>
  <tr>
  <td>0</td>
  <td>1</td>
  <td>30</td>
  <td>5</td>
  </tr>
  <tr>
  <td>1</td>
  <td>2</td>
  <td>42</td>
  <td>6</td>
  </tr>
  <tr>
  <td>2</td>
  <td>4</td>
  <td>51</td>
  <td>7</td>
  </tr>
  </tbody>
  </table>
  </div>
  <p>Thus <code>RNA1_ID</code> has been moved from the second to the first column. If <code>df2</code> was in the last column repeat as above use <code>-1</code> instead of <code>-2</code> and both columns will now be at the start of the dataframe. Modify the core dataframe generalisation renaming accordingly, viz.</p>
  <pre><code>myname = df.columns[0]
  </code></pre>
  <p>... sorted.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-02-22T20:58:21.687Z
Answers.LastActivityDate: 2023-02-22T21:13:06.633Z
Answers.LastEditDate: 2023-02-22T21:13:06.633Z
Answers.UserDisplayName: M__
Body: <pre class="lang-py prettyprint-override"><code>def correlations(group):
      # Remove columns
      group = (
          group.reset_index()
          .drop(columns=[&quot;RNA1_Approved_Symbol&quot;], level=0)
          .set_index(&quot;RNA1_ID&quot;)
      )
      # compute correlations
      correlation = group.T.corr()
  
      # rename pair column
      correlation = correlation.reset_index().rename(
          columns={&quot;RNA1_ID&quot;: &quot;RNA1_ID_new&quot;}
      )
  
      # Remove NAs
      corr_result = (
          correlation.melt(value_name=&quot;Correlation&quot;, id_vars=[&quot;RNA1_ID_new&quot;])
          .set_index([&quot;RNA1_ID&quot;, &quot;RNA1_ID_new&quot;])
          .dropna()
      )
      return corr_result
  
  
  def correlation_1(data):
      ## Create Pivot table
      pivot_library = data.pivot(
          index=[&quot;RNA1_Approved_Symbol&quot;, &quot;RNA1_ID&quot;], columns=[&quot;RNA2_ID&quot;]
      )
  
      # Measure correlations between four id's
      return (
          pivot_library.groupby(&quot;RNA1_Approved_Symbol&quot;)
          .apply(correlations)
          .reset_index()
          .dropna()
      )
  </code></pre>
  <p>I am using the above function to calculate the correlation</p>
  <pre class="lang-py prettyprint-override"><code>data = df1
  correlations = correlation_1(df1)
  </code></pre>
  <p>I want to automate the above function when I am using different data frame where there is <code>RNA2_Approved_Symbol</code> instead of <code>RNA1_Approved_Symbol</code> and <code>RNA2_ID</code> instead of <code>RNA1_ID</code>. Like using index 1 and index 2.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-02-21T19:38:44.117Z
Comments.Text: Hi @Riya, pretty sure I could solve this. Could you supply a test data set for your new dataframe please? Actually, I think I get the question and there's no need of a dataset. Hope to have a response end of week.
Comments.UserDisplayName: M__
Tags: <python><r><pandas>
Title: Automate the def function in pandas for correlation
ThreadText: # Automate the def function in pandas for correlation
  
  (Asked by: username_0 on 2023-02-21)
  
  ```py
  def correlations(group):
      # Remove columns
      group = (
          group.reset_index()
          .drop(columns=["RNA1_Approved_Symbol"], level=0)
          .set_index("RNA1_ID")
      )
      # compute correlations
      correlation = group.T.corr()
  
      # rename pair column
      correlation = correlation.reset_index().rename(
          columns={"RNA1_ID": "RNA1_ID_new"}
      )
  
      # Remove NAs
      corr_result = (
          correlation.melt(value_name="Correlation", id_vars=["RNA1_ID_new"])
          .set_index(["RNA1_ID", "RNA1_ID_new"])
          .dropna()
      )
      return corr_result
  
  def correlation_1(data):
      ## Create Pivot table
      pivot_library = data.pivot(
          index=["RNA1_Approved_Symbol", "RNA1_ID"], columns=["RNA2_ID"]
      )
  
      # Measure correlations between four id's
      return (
          pivot_library.groupby("RNA1_Approved_Symbol")
          .apply(correlations)
          .reset_index()
          .dropna()
      )
  
  ```
  
  I am using the above function to calculate the correlation
  
  ```py
  data = df1
  correlations = correlation_1(df1)
  
  ```
  
  I want to automate the above function when I am using different data frame where there is `RNA2_Approved_Symbol` instead of `RNA1_Approved_Symbol` and `RNA2_ID` instead of `RNA1_ID`. Like using index 1 and index 2.
  
  **Tags:** python, r, pandas
  
  > **Comments:**
  > **username_1**: Hi @username_0, pretty sure I could solve this. Could you supply a test data set for your new dataframe please? Actually, I think I get the question and there's no need of a dataset. Hope to have a response end of week.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2023-02-22. Score: 1 | Accepted answer
  
  If I have understood the question is to generalise the column names to permit a function to accept dataframes where you don't have the same column name, or it's different from the previous dataframe column names. Here the column could be called `RNA1_Approved_Symbol` or `RNA2_Approved_Symbol` in addition a different column could be called `RNA2_ID` or `RNA_ID`. No problem providing the column order is identical for the columns in question. If the orders are not identical see the code below the `----` to correct that.
  
  If the order is identical the simple answer is `df.columns[i]`. Thus it's obtaining the column name on its position within the dataframe (`columns` command), where `i` is the column location.
  
  Example, to rename `RNA1_ID` to `RNA1_ID_new`, or `RNA2_ID` to `RNA2_ID_new`
  
  ```
  import re
  
  df1 = pd.DataFrame({"perch": [5,6,7], "RNA1_ID": [1,2,4], "roach": [30,42,51]})
  df2 = pd.DataFrame({"perch": [5,6,7], "RNA2_ID": [1,2,4], "roach": [30,42,51]})
  
  def renamecolumn(df):
      myname = df.columns[1]
      newname = re.sub(r'$','_new', myname)
      df = df.rename(columns={myname: newname})
      return df
  
  print (renamecolumn(df1))
  print (renamecolumn(df2))
  
  ```
  
  Output,
  
  | - | perch | RNA1\_ID\_new | roach |
  | --- | --- | --- | --- |
  | 0 | 5 | 1 | 30 |
  | 1 | 6 | 2 | 42 |
  | 2 | 7 | 4 | 51 |
  
  | - | perch | RNA2\_ID\_new | roach |
  | --- | --- | --- | --- |
  | 0 | 5 | 1 | 30 |
  | 1 | 6 | 2 | 42 |
  | 2 | 7 | 4 | 51 |
  
  I noted there was a fair number of `set_index` commands so the earlier the column name can be trapped inside a variable the better, because `set_index` will change the column order.
  
  ---
  
  Alternatively, the columns can be re-arranged to be in a given order if the original order between `df1` and `df2` is different. Here I'll move the column to the front.
  
  ```
  mycols = df1.columns.tolist()
  mycols = mycols[-2:] + mycols[:-2]
  df1 = df1[mycols]
  print (df1)
  
  ```
  
  | - | RNA1\_ID | roach | perch |
  | --- | --- | --- | --- |
  | 0 | 1 | 30 | 5 |
  | 1 | 2 | 42 | 6 |
  | 2 | 4 | 51 | 7 |
  
  Thus `RNA1_ID` has been moved from the second to the first column. If `df2` was in the last column repeat as above use `-1` instead of `-2` and both columns will now be at the start of the dataframe. Modify the core dataframe generalisation renaming accordingly, viz.
  
  ```
  myname = df.columns[0]
  
  ```
  
  ... sorted.
  
  
  ---
  
  