Body: <p>We are working on the subject above, where a sequence of <span class="math-container">$n$</span> glass frames forms an example with an associate target that is a video of the glass future state.
  We would like to know if there exists an algorithm (ConvLSTM, Trasformer, etc…), that could be trained and that it is able to predict glasses breaking videos. In this moment, we are testing SimVPv2 (on MMNIST).</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-03-15T16:47:35.657Z
Comments.Text: How do the glass images look like, are there like initial fractures somewhere, giving a hint that it may break in near future?
Comments.UserDisplayName: NikoNyrh
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-03-15T16:53:32.723Z
Comments.Text: Thank you! Yes, the glasses have already fractures. One of the initial tasks is to classify them between "broken" and "intact". Then a regression algorithm could be used to estimate the degree of rupture. The final task is to have a predictor that will give the next frame of the glass given a degradation video (perhaps with a ConvLSTM or a Transformer algorithm).
Comments.UserDisplayName: Filippo Portera
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-03-17T10:48:26.990Z
Comments.Text: At present we are trying OpenSTL-SimVPv2 on MMNIST. We are unable to run in parallel the VPTR learner, due to our 32 GB RAM limit.
Comments.UserDisplayName: Filippo Portera
Tags: <computer-vision><long-short-term-memory><vision-transformer><video-prediction>
Title: Glass Degradation Video Prediction
UserDisplayName: Filippo Portera
ThreadText: # Glass Degradation Video Prediction
  
  (Asked by: username_0 on 2024-03-07)
  
  We are working on the subject above, where a sequence of $n$ glass frames forms an example with an associate target that is a video of the glass future state.
  We would like to know if there exists an algorithm (ConvLSTM, Trasformer, etc…), that could be trained and that it is able to predict glasses breaking videos. In this moment, we are testing SimVPv2 (on MMNIST).
  
  **Tags:** computer-vision, long-short-term-memory, vision-transformer, video-prediction
  > **Comments:**
  > **username_1**: How do the glass images look like, are there like initial fractures somewhere, giving a hint that it may break in near future?  
  > **username_0**: Thank you! Yes, the glasses have already fractures. One of the initial tasks is to classify them between "broken" and "intact". Then a regression algorithm could be used to estimate the degree of rupture. The final task is to have a predictor that will give the next frame of the glass given a degradation video (perhaps with a ConvLSTM or a Transformer algorithm).  
  > **username_0**: At present we are trying OpenSTL-SimVPv2 on MMNIST. We are unable to run in parallel the VPTR learner, due to our 32 GB RAM limit.  
  
  ---
  
  ## 0 answers
  
  