Answers.Body: <p>Since in your specific environmental predictors the number of features are not that large and most likely you're not having a large training dataset, you'd better focus on tree-based XBGoost hybrid model to flexibly estimate the two parameters of the Chapman-Richards growth equation where the relation between parameters and predictors are nonlinear, complex and unknown. XGBoost and other tree-based models capture interactions implicitly by splitting the data sequentially so the effect of one variable can change depending on the value of another variable, which is a form of non-linear interaction. You can also tune hyperparameters such as learning rate, maximum tree depth and number of estimators to prevent overfit satisfying your specification to extrapolate beyond the range of available data and have a well-performed and overall-interpretable model with a best <span class="math-container">$R^2$</span> or <span class="math-container">$\text{MSE}$</span> metrics.</p>
  <p>Comparatively, uninterpretable neural network or nonlinear polynomial regressor are prone to overfit and the latter needs additional explicit feature engineering and regularization.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-10-25T12:58:54.577Z
Answers.Text: Thank you! Surprisingly I have a very large dataset (remote sensing!) so the way I'm thinking of implementing this is: 
  
  1 - K-means clustering to divide the data into clusters of similar environmental conditions
  2 - fit chapman-richards to each cluster using gradient descent/nelder mead
  3 - use XGBoost to predict the k values (fit in step 2) for each cluster from the environmental predictors
  4 - use the trained XGBoost model to make new k values for new data.
  
  I would also appreciate advice on that approach as I have not found many examples of hybrid models
Answers.UserDisplayName: Ana Catarina Vitorino
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-10-25T18:44:24.777Z
Answers.Text: @AnaCatarinaVitorino Since you’re working with remote sensing data, pre-processing steps like dimensionality reduction on high-dimensional predictors (e.g., via PCA) can help improve the computational efficiency without sacrificing model accuracy, and still they cannot compare with imagenet or speech data. Your hybrid method is solid, just ensure to add cluster label to your feature space once k-means clustering is done so your later xgboost learning will automatically take these Categorical label predictor into account. You’d spend sometime on the 3 hyper parameters tuning.
Answers.UserDisplayName: cinch
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-10-25T04:52:37.010Z
Answers.LastActivityDate: 2024-10-25T04:52:37.010Z
Answers.UserDisplayName: cinch
Body: <p>I am trying to fit a Chapman-Richards growth curve:</p>
  <p><span class="math-container">$$
  B = A*(1-e^{-kt})
  $$</span></p>
  <p>Where B is the biomass of a forest, A is the asymptote, k is the growth rate, and t is forest age. I expect the growth rate to be a function of multiple environmental predictors.</p>
  <p>I have fit this curve using Nelder-Mead gradient descent with reasonable success. I considered k to be a linear combination of the environmental predictors. I understand that the linear combination is an oversimplification, and may explain the not-so-great predictive power. <br />
  Using XGBoost the R squared values are much better... but it is useless to me since I ultimately need a growth equation to extrapolate beyond the range of the data.</p>
  <p>While looking at alternatives to improve the fit of the curve, I came across:</p>
  <ul>
  <li>neural networks</li>
  <li>hybrid models (someone used XGBoost to estimate a parameter of a process based model, which I have not yet found out how to implement in my case)</li>
  <li>hyperparameter tuning (so far, not helpful)</li>
  </ul>
  <p>What would you recommend I focus on to find the best possible parameter combination for my growth curve?</p>
  
Tags: <neural-networks><gradient-descent><hyperparameter-optimization>
Title: Options for fitting a growth curve - process-based, hybrid, or neural networks?
UserDisplayName: Ana Catarina Vitorino
ThreadText: # Options for fitting a growth curve - process-based, hybrid, or neural networks?
  
  (Asked by: username_0 on 2024-10-25)
  
  I am trying to fit a Chapman-Richards growth curve:
  
  $$
  B = A\*(1-e^{-kt})
  $$
  
  Where B is the biomass of a forest, A is the asymptote, k is the growth rate, and t is forest age. I expect the growth rate to be a function of multiple environmental predictors.
  
  I have fit this curve using Nelder-Mead gradient descent with reasonable success. I considered k to be a linear combination of the environmental predictors. I understand that the linear combination is an oversimplification, and may explain the not-so-great predictive power.   
  Using XGBoost the R squared values are much better... but it is useless to me since I ultimately need a growth equation to extrapolate beyond the range of the data.
  
  While looking at alternatives to improve the fit of the curve, I came across:
  
  * neural networks
  * hybrid models (someone used XGBoost to estimate a parameter of a process based model, which I have not yet found out how to implement in my case)
  * hyperparameter tuning (so far, not helpful)
  
  What would you recommend I focus on to find the best possible parameter combination for my growth curve?
  
  **Tags:** neural-networks, gradient-descent, hyperparameter-optimization
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2024-10-25. Score: 0 | Accepted answer
  
  Since in your specific environmental predictors the number of features are not that large and most likely you're not having a large training dataset, you'd better focus on tree-based XBGoost hybrid model to flexibly estimate the two parameters of the Chapman-Richards growth equation where the relation between parameters and predictors are nonlinear, complex and unknown. XGBoost and other tree-based models capture interactions implicitly by splitting the data sequentially so the effect of one variable can change depending on the value of another variable, which is a form of non-linear interaction. You can also tune hyperparameters such as learning rate, maximum tree depth and number of estimators to prevent overfit satisfying your specification to extrapolate beyond the range of available data and have a well-performed and overall-interpretable model with a best $R^2$ or $\text{MSE}$ metrics.
  
  Comparatively, uninterpretable neural network or nonlinear polynomial regressor are prone to overfit and the latter needs additional explicit feature engineering and regularization.
  
  > **Comments:**
  > **username_0**: Thank you! Surprisingly I have a very large dataset (remote sensing!) so the way I'm thinking of implementing this is:   1 - K-means clustering to divide the data into clusters of similar environmental conditions 2 - fit chapman-richards to each cluster using gradient descent/nelder mead 3 - use XGBoost to predict the k values (fit in step 2) for each cluster from the environmental predictors 4 - use the trained XGBoost model to make new k values for new data.  I would also appreciate advice on that approach as I have not found many examples of hybrid models  
  > **username_1**: @username_0 Since you’re working with remote sensing data, pre-processing steps like dimensionality reduction on high-dimensional predictors (e.g., via PCA) can help improve the computational efficiency without sacrificing model accuracy, and still they cannot compare with imagenet or speech data. Your hybrid method is solid, just ensure to add cluster label to your feature space once k-means clustering is done so your later xgboost learning will automatically take these Categorical label predictor into account. You’d spend sometime on the 3 hyper parameters tuning.  
  
  ---
  
  