Answers.Body: <p>Yes it's easy you load everything into a dataframe, so each variable is one column and perform supervised machine learning. That will solve all questions.</p>
  <p><strong>Hypothesis 1</strong> The target is 'pre' 'post' <strong>if there is only one drug in the trial</strong>. If there is more than one drug in the trial the drugs would become the target, e.g. if you wanted to know the efficacy between 'drug A' and 'drug B'. What you want to know is the effective of the drug. This would help hypothesis 2.</p>
  <p><strong>Hypothesis 2</strong> Estimated cell size is the training target. This is more complicated because I'm not sure what x and y mean: but if x and y are cell measurements then an estimate of cell surface area should be made. If x and y can become a single variate of cell surface area than that can form categorical data ... super size, big, medium large, medium, medium-regular, regular etc ...
  This could then become the training target and probably what you are looking for.</p>
  <p>The ML feature selection will assess cell size (if 'pre' and 'post' is the training target; or), or 'pre'-'post' (if 'cell size estimation' is the training target). In either case the expected result is it should provide the strongest weight in the analysis.</p>
  <p><strong>Finally</strong> Finally and importantly, the more variables you can input the better, e.g. the gender of the patient, age of the patient as categorical data (bins), smoking, drinking ... anything you can think of, the more the better. ML should ignore anything that has not impact on 'pre' / 'post' or 'estimated cell size'.</p>
  <p>The ML algorithm is more complicated but start with naive Bayes and see how it goes. Beyond this it does start to get complicated however.</p>
  <hr />
  <p>To address the comments,</p>
  <ul>
  <li>Load everything into a dataframe</li>
  </ul>
  <p>A dataframe is a basic unit of operation in <code>Python</code> (<code>pandas</code>) or <code>R</code>. Python also uses <code>numpy</code> arrays.
  Everything, means everything - all the data you have, metadata all files. For the calculation the columns involving patient id and patient number, this might be better excluded. I'm not sure - it's complicated because you'll have duplication of records - I would do the calculations with and without this data. You can delete patient id (it's the same as patient number - from a calculation point of view). The x, y is a special case.</p>
  <p>Thus all the various tables you describe get loaded into one dataframe, this includes all the patient data such as gender, weight, age, social factors ... it's just like one giant excel table, with rows and columns. The columns are labelled according to the variable e.g. Drug therapy = 'pre' and 'post' data. The dataframe will feed directly to the ML algorithm - a good reason for using it.</p>
  <p><em><strong>x</strong></em> and <em><strong>y</strong></em></p>
  <p>Personally I don't think splitting this data into two columns is a good idea. It will confuse the algorithms - its needs to be one column of data - a single variate. To achieve this will be doable. I would use <strong>π</strong> to calculate the cell surface area. How that is done for an elongated circle - I don't know - but someone will have done it somewhere, you just need to look it up. Thus 2<strong>π</strong><em>r</em><sup>2</sup> if its perfectly round where <em>r</em> = diameter/2 ... elongation must have a derivation its just finding it.</p>
  <p>This will absolutely make a difference in your calculations - what I would do is make it to a categorical variable by plotting it and constructing a histogram - otherwise you are into transformations and thats complicated. Categorical variables will also permit logistic regression ML and that can be very useful, because it places each variable into positive or negative weights.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-03-30T00:19:31.483Z
Answers.Text: Thanks for your reply. I am not sure if i understand the first line of your response. What do you mean by "load everything into a dataframe, so each variable is one column"? Are you talking about the metadata file or the other files? and which varibales are you talking about?
  
  There are a few things that i should clarify too:
  1 - There is only on drug in the experiment.
  2 - x and y are coordinates of the each cell detected in the image of tissues in a 2D image.
Answers.UserDisplayName: eh329
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-03-30T14:51:12.730Z
Answers.Text: @EHR answered above below the -----
Answers.UserDisplayName: M__
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-03-29T23:50:07.777Z
Answers.LastActivityDate: 2023-03-30T14:51:00.350Z
Answers.LastEditDate: 2023-03-30T14:51:00.350Z
Answers.UserDisplayName: M__
Body: <p>We came across a project in our lab that no one exactly knows how to approach. Since, I know a little bit of Python programming, this project was assigned to me.</p>
  <p>There is a data from a randomised controlled clinical trial with 60 participants. Half (30) are control and the other half are actual patients (case). From each group, biopsy samples were taken before (pre) and after (post) treatment with drug A that targets cell type B in the tissue.</p>
  <p>In each biopsy sample, the B cells were detected and classified either as “normal” or “malignant”.</p>
  <p>Associated metadata for each biopsy are displayed here (I have only included 12 records for this file):</p>
  <pre><code>name    patient_id  arm treatment
  111     0       control     pre
  112     1       control     pre
  113     2       control     pre
  121     0       control     post
  122     1       control     post
  123     2       control     post
  211     75      case        pre
  212     76      case        pre
  213     77      case        pre
  221     75      case        post
  222     76      case        post
  223     77      case        post
  </code></pre>
  <p>●   name: spreadsheet file name</p>
  <p>●   patient_id: patient identity number</p>
  <p>●   arm: trial arm (‘case’ or ‘control)</p>
  <p>●   treatment: treatment condition (‘pre’ or ‘post’ treatment)</p>
  <p>Other files (I have only included 20 records for each file: 10 normal and 10 malignant) contain cell detection results from a single biopsy, and each row in a spreadsheet represents an individual detected cell:</p>
  <p>●   x: x coordinate</p>
  <p>●   y: y coordinate</p>
  <p>●   label: classified label (‘normal’ or ‘malignant’)</p>
  <p>Just showing an example, file 111 looks like this:</p>
  <pre><code>x   y   label
  730 724 normal
  1962 450 normal
  1511 817 normal
  1244 455 normal
  2529 397 normal
  1878 262 normal
  2248 369 normal
  2007 273 normal
  1531 878 normal
  1729 834 normal
  931  1270 malignant
  1282 314 malignant
  1630 839 malignant
  1543 460 malignant
  2493 237 malignant
  1311 744 malignant
  1999 366 malignant
  737 1361 malignant
  2252 448 malignant
  2620 398 malignant
  </code></pre>
  <p>The rest can be found here, but probably they will not be necessary: <a href="https://www.mediafire.com/file/ka7r59kf0swnbnd/OtherFiles.rar/file" rel="nofollow noreferrer">https://www.mediafire.com/file/ka7r59kf0swnbnd/OtherFiles.rar/file</a></p>
  <p>I am trying to answer 3 questions here (if you can think of other questions, please let me know):</p>
  <ol>
  <li>identifying post-treatment morphological changes due to the effect of drug A.</li>
  <li>proposing measures to quantify the changes.</li>
  <li>Using appropriate statistical analysis tools to provide insight whether the changes are due to chance or not.</li>
  </ol>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-03-30T14:59:37.590Z
Comments.Text: By morphological changes, do you mean the spatial arrangement of cells, or bulk properties of the sample? Since you have coordinates, you could look into "cellular neighborhoods", basically looking at each cell and a small number of its physically nearest neighbors. This could help show if cells are clumping together by type or if they're evenly spread out.
Comments.UserDisplayName: Nuclear Hoagie
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-03-30T15:40:06.127Z
Comments.Text: @Nuclear Hoagie I was initially going for spatial arrangment, but at this point i am just trying to come up with the right questions first. Because, i would like to know how to aaproach this problem. I think i should consider both of them.
Comments.UserDisplayName: eh329
Tags: <python><drugs><meta-analysis>
Title: Analysing the Effect of a Drug on the Morphological Changes of a Cell Type
ThreadText: # Analysing the Effect of a Drug on the Morphological Changes of a Cell Type
  
  (Asked by: username_0 on 2023-03-29)
  
  We came across a project in our lab that no one exactly knows how to approach. Since, I know a little bit of Python programming, this project was assigned to me.
  
  There is a data from a randomised controlled clinical trial with 60 participants. Half (30) are control and the other half are actual patients (case). From each group, biopsy samples were taken before (pre) and after (post) treatment with drug A that targets cell type B in the tissue.
  
  In each biopsy sample, the B cells were detected and classified either as “normal” or “malignant”.
  
  Associated metadata for each biopsy are displayed here (I have only included 12 records for this file):
  
  ```
  name    patient_id  arm treatment
  111     0       control     pre
  112     1       control     pre
  113     2       control     pre
  121     0       control     post
  122     1       control     post
  123     2       control     post
  211     75      case        pre
  212     76      case        pre
  213     77      case        pre
  221     75      case        post
  222     76      case        post
  223     77      case        post
  
  ```
  
  ● name: spreadsheet file name
  
  ● patient\_id: patient identity number
  
  ● arm: trial arm (‘case’ or ‘control)
  
  ● treatment: treatment condition (‘pre’ or ‘post’ treatment)
  
  Other files (I have only included 20 records for each file: 10 normal and 10 malignant) contain cell detection results from a single biopsy, and each row in a spreadsheet represents an individual detected cell:
  
  ● x: x coordinate
  
  ● y: y coordinate
  
  ● label: classified label (‘normal’ or ‘malignant’)
  
  Just showing an example, file 111 looks like this:
  
  ```
  x   y   label
  730 724 normal
  1962 450 normal
  1511 817 normal
  1244 455 normal
  2529 397 normal
  1878 262 normal
  2248 369 normal
  2007 273 normal
  1531 878 normal
  1729 834 normal
  931  1270 malignant
  1282 314 malignant
  1630 839 malignant
  1543 460 malignant
  2493 237 malignant
  1311 744 malignant
  1999 366 malignant
  737 1361 malignant
  2252 448 malignant
  2620 398 malignant
  
  ```
  
  The rest can be found here, but probably they will not be necessary: <https://www.mediafire.com/file/ka7r59kf0swnbnd/OtherFiles.rar/file>
  
  I am trying to answer 3 questions here (if you can think of other questions, please let me know):
  
  1. identifying post-treatment morphological changes due to the effect of drug A.
  2. proposing measures to quantify the changes.
  3. Using appropriate statistical analysis tools to provide insight whether the changes are due to chance or not.
  
  **Tags:** python, drugs, meta-analysis
  
  > **Comments:**
  > **username_1**: By morphological changes, do you mean the spatial arrangement of cells, or bulk properties of the sample? Since you have coordinates, you could look into "cellular neighborhoods", basically looking at each cell and a small number of its physically nearest neighbors. This could help show if cells are clumping together by type or if they're evenly spread out.  
  > **username_0**: @username_1 I was initially going for spatial arrangment, but at this point i am just trying to come up with the right questions first. Because, i would like to know how to aaproach this problem. I think i should consider both of them.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2023-03-29. Score: 1
  
  Yes it's easy you load everything into a dataframe, so each variable is one column and perform supervised machine learning. That will solve all questions.
  
  **Hypothesis 1** The target is 'pre' 'post' **if there is only one drug in the trial**. If there is more than one drug in the trial the drugs would become the target, e.g. if you wanted to know the efficacy between 'drug A' and 'drug B'. What you want to know is the effective of the drug. This would help hypothesis 2.
  
  **Hypothesis 2** Estimated cell size is the training target. This is more complicated because I'm not sure what x and y mean: but if x and y are cell measurements then an estimate of cell surface area should be made. If x and y can become a single variate of cell surface area than that can form categorical data ... super size, big, medium large, medium, medium-regular, regular etc ...
  This could then become the training target and probably what you are looking for.
  
  The ML feature selection will assess cell size (if 'pre' and 'post' is the training target; or), or 'pre'-'post' (if 'cell size estimation' is the training target). In either case the expected result is it should provide the strongest weight in the analysis.
  
  **Finally** Finally and importantly, the more variables you can input the better, e.g. the gender of the patient, age of the patient as categorical data (bins), smoking, drinking ... anything you can think of, the more the better. ML should ignore anything that has not impact on 'pre' / 'post' or 'estimated cell size'.
  
  The ML algorithm is more complicated but start with naive Bayes and see how it goes. Beyond this it does start to get complicated however.
  
  ---
  
  To address the comments,
  
  * Load everything into a dataframe
  
  A dataframe is a basic unit of operation in `Python` (`pandas`) or `R`. Python also uses `numpy` arrays.
  Everything, means everything - all the data you have, metadata all files. For the calculation the columns involving patient id and patient number, this might be better excluded. I'm not sure - it's complicated because you'll have duplication of records - I would do the calculations with and without this data. You can delete patient id (it's the same as patient number - from a calculation point of view). The x, y is a special case.
  
  Thus all the various tables you describe get loaded into one dataframe, this includes all the patient data such as gender, weight, age, social factors ... it's just like one giant excel table, with rows and columns. The columns are labelled according to the variable e.g. Drug therapy = 'pre' and 'post' data. The dataframe will feed directly to the ML algorithm - a good reason for using it.
  
  ***x*** and ***y***
  
  Personally I don't think splitting this data into two columns is a good idea. It will confuse the algorithms - its needs to be one column of data - a single variate. To achieve this will be doable. I would use **π** to calculate the cell surface area. How that is done for an elongated circle - I don't know - but someone will have done it somewhere, you just need to look it up. Thus 2**π***r*2 if its perfectly round where *r* = diameter/2 ... elongation must have a derivation its just finding it.
  
  This will absolutely make a difference in your calculations - what I would do is make it to a categorical variable by plotting it and constructing a histogram - otherwise you are into transformations and thats complicated. Categorical variables will also permit logistic regression ML and that can be very useful, because it places each variable into positive or negative weights.
  
  > **Comments:**
  > **username_0**: Thanks for your reply. I am not sure if i understand the first line of your response. What do you mean by "load everything into a dataframe, so each variable is one column"? Are you talking about the metadata file or the other files? and which varibales are you talking about?  There are a few things that i should clarify too: 1 - There is only on drug in the experiment. 2 - x and y are coordinates of the each cell detected in the image of tissues in a 2D image.  
  > **username_2**: @EHR answered above below the -----  
  
  ---
  
  