Answers.Body: <p>As per your requirements, I would suggest that you start with any simple CNN network.</p>
  <blockquote>
  <p>CNNs take advantage of the hierarchical pattern in data and assemble
  more complex patterns using smaller and simpler patterns. Therefore,
  on the scale of connectedness and complexity, CNNs are on the lower
  extreme.</p>
  </blockquote>
  <p>Here is a Keras example:</p>
  <pre><code>model = models.Sequential()
  model.add(layers.Conv2D(32, kernel_size=(3, 3), activation='relu', input_shape=image_shape))
  model.add(layers.MaxPooling2D((2, 2)))
  model.add(layers.Conv2D(64, kernel_size=(3, 3), activation='relu'))
  model.add(layers.MaxPooling2D((2, 2)))
  model.add(layers.Conv2D(64, kernel_size=(3, 3), activation='relu'))
  model.add(layers.Flatten())
  model.add(layers.Dense(64, activation='relu'))
  # output layer
  model.add(layers.Dense(1))
  </code></pre>
  <p>where <code>image_shape</code> is the resolution and number of channels of images (e.g. 128x128x3 for RGB images). I also suggest you downscaling the images to a lower resolution. You will also have to crop the images as they must all be the same <code>image_shape</code>.</p>
  <p>Also take a look at the <a href="https://www.tensorflow.org/api_docs/python/tf/keras/layers/MaxPool2D" rel="nofollow noreferrer">MaxPooling2D</a>  and <a href="https://www.tensorflow.org/api_docs/python/tf/keras/layers/BatchNormalization" rel="nofollow noreferrer">BatchNormalization</a> layers.</p>
  <p>Since you only have real and CGI images, this becomes a binary classification problem. Therefore you can have a single output (0 - CGI, 1 - real). Such problems can be solved with <a href="https://www.tensorflow.org/api_docs/python/tf/keras/losses/BinaryCrossentropy" rel="nofollow noreferrer">BinaryCrossentropy</a> loss.</p>
  <pre><code>model.compile(loss=losses.BinaryCrossentropy(from_loits=True), optimizer='adam')
  </code></pre>
  <p>Finally, you can fit your model</p>
  <pre><code>history = model.fit(train_images, train_labels, epochs=1000, validation_data=(test_images, test_labels))
  </code></pre>
  <p>You can find a complete example <a href="https://www.tensorflow.org/tutorials/images/cnn" rel="nofollow noreferrer">here</a>.</p>
  <p>Please note that depending on your data, the model can become biased if your dataset is unbalanced. That is, if all of your CGI images have text, and only a small fraction of the real images also have text, they might be misclassified. Therefore, I recommend that you <a href="https://cs231n.github.io/understanding-cnn/" rel="nofollow noreferrer">visualize</a> your model to better understand what it has learned. Here is an <a href="https://arxiv.org/pdf/1902.10178.pdf" rel="nofollow noreferrer">example</a> of such a problem we faced at our university.</p>
  <p>There are also more advanced CNN architectures such as <a href="https://en.wikipedia.org/wiki/Residual_neural_network" rel="nofollow noreferrer">ResNet</a>, <a href="https://arxiv.org/abs/1409.1556" rel="nofollow noreferrer">VGG</a> or <a href="https://arxiv.org/abs/1506.02640" rel="nofollow noreferrer">YOLO</a>. You can also extend your model with time series (i.e. video) using <a href="https://en.wikipedia.org/wiki/Long_short-term_memory" rel="nofollow noreferrer">LSTM</a> or <a href="https://en.wikipedia.org/wiki/Gated_recurrent_unit" rel="nofollow noreferrer">GRU</a> architecture.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-10-24T17:01:43.170Z
Answers.Text: Thanks. I guess now i need to collect a few hundreds of samples for dataset. One qq: what's the rationale behind the node size 32,64,64? If I add another layer, it should go 128?
Answers.UserDisplayName: Mary
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-10-24T17:02:22.340Z
Answers.Text: And is there any relationship between the number of layers and dataset size? Jut some intuition?
Answers.UserDisplayName: Mary
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-10-25T11:28:02.983Z
Answers.Text: As far as I know, there is no relationship between the number of layers and dataset size. More layers mean more  hidden features. ResNet and VGG networks use up to 100 hidden layers.
Answers.UserDisplayName: Aray Karjauv
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-10-25T11:49:30.193Z
Answers.Text: 32, 64 and 64 are the number of filters. Filters present hidden features of the images. The first layer will usually be activated by straight lines (so the layer should be small), whereas the last layers will be activated by some abstract features and colors. Basically, you decide how many filter layers to contain.
Answers.UserDisplayName: Aray Karjauv
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-10-26T00:48:53.783Z
Answers.Text: Thanks. One last question: Do you know of any open dataset for this?!
Answers.UserDisplayName: Mary
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-10-26T11:02:24.677Z
Answers.Text: I haven't come across such a dataset
Answers.UserDisplayName: Aray Karjauv
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-10-23T21:36:18.933Z
Answers.LastActivityDate: 2020-10-24T09:48:05.107Z
Answers.LastEditDate: 2020-10-24T09:48:05.107Z
Answers.UserDisplayName: Aray Karjauv
Body: <p>Given a video, I'm trying to classify whether it is a graphical (computer-generated) or realistic scene. For instance, if it contains computer-generated graphics, credit, moving bugs, blue screen, etc. it will be computer-generated graphics, and if it is a realistic scene captured by camera, it will be a realistic scene.</p>
  <p>How can we achieve that with AI? Do we have any working solutions available?</p>
  <p>Some examples of graphical scenes:</p>
  <p><a href="https://i.sstatic.net/xiantm.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/xiantm.png" alt="enter image description here" /></a></p>
  <p><a href="https://i.sstatic.net/4TxE5m.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/4TxE5m.png" alt="enter image description here" /></a></p>
  <p><a href="https://i.sstatic.net/SxOUAm.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/SxOUAm.jpg" alt="enter image description here" /></a></p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-10-23T06:56:18.670Z
Comments.Text: Why not you try ELA(Error Level Analysis). Using ELA you can identifyedited scenes and non edited scenes.
Comments.UserDisplayName: Hiren Namera
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-10-23T10:34:03.850Z
Comments.Text: Since neural networks are very good at finding patterns, you can try to just train a neural network (e.g. CNN) on you examples given the labels whether it is real or CGI.  If you want then to understand what your NN has learned, you can [visualize its layers](https://cs231n.github.io/understanding-cnn/)
Comments.UserDisplayName: Aray Karjauv
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-10-23T14:10:10.780Z
Comments.Text: Should it be CNN only, or LSTM with multiple frames?! I need it to be lightweight.
Comments.UserDisplayName: Mary
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-10-23T14:25:32.677Z
Comments.Text: It depends. If you have a series of images (i.e. a video), you can use LSTM to improve stability. But I would suggest starting simple. To make it lightweight, you can also downscale the images to smaller resolution and make them b&w. You can also use maxpooling layers/strides in your CNN
Comments.UserDisplayName: Aray Karjauv
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-10-23T19:15:28.360Z
Comments.Text: Umm I'm not yet much into architecting deep layers. If you could post a sample in python using Keras, it would be great.
Comments.UserDisplayName: Mary
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-10-24T12:33:59.290Z
Comments.Text: I posted my answer. Feel free to ask me if anything is unclear. If the answer is helpful, don't forget to accept it.
Comments.UserDisplayName: Aray Karjauv
Tags: <deep-learning><classification><ai-design><object-detection><scene-classification>
Title: How can I determine whether a video's frame is realistic (was recorded by a camera) or contains computer-generated graphics?
UserDisplayName: Mary
ThreadText: # How can I determine whether a video's frame is realistic (was recorded by a camera) or contains computer-generated graphics?
  
  (Asked by: username_0 on 2020-10-23)
  
  Given a video, I'm trying to classify whether it is a graphical (computer-generated) or realistic scene. For instance, if it contains computer-generated graphics, credit, moving bugs, blue screen, etc. it will be computer-generated graphics, and if it is a realistic scene captured by camera, it will be a realistic scene.
  
  How can we achieve that with AI? Do we have any working solutions available?
  
  Some examples of graphical scenes:
  
  [![enter image description here](https://i.sstatic.net/xiantm.png)](https://i.sstatic.net/xiantm.png)
  
  [![enter image description here](https://i.sstatic.net/4TxE5m.png)](https://i.sstatic.net/4TxE5m.png)
  
  [![enter image description here](https://i.sstatic.net/SxOUAm.jpg)](https://i.sstatic.net/SxOUAm.jpg)
  
  **Tags:** deep-learning, classification, ai-design, object-detection, scene-classification
  > **Comments:**
  > **username_1**: Why not you try ELA(Error Level Analysis). Using ELA you can identifyedited scenes and non edited scenes.  
  > **username_2**: Since neural networks are very good at finding patterns, you can try to just train a neural network (e.g. CNN) on you examples given the labels whether it is real or CGI.  If you want then to understand what your NN has learned, you can [visualize its layers](https://cs231n.github.io/understanding-cnn/)  
  > **username_0**: Should it be CNN only, or LSTM with multiple frames?! I need it to be lightweight.  
  > **username_2**: It depends. If you have a series of images (i.e. a video), you can use LSTM to improve stability. But I would suggest starting simple. To make it lightweight, you can also downscale the images to smaller resolution and make them b&w. You can also use maxpooling layers/strides in your CNN  
  > **username_0**: Umm I'm not yet much into architecting deep layers. If you could post a sample in python using Keras, it would be great.  
  > **username_2**: I posted my answer. Feel free to ask me if anything is unclear. If the answer is helpful, don't forget to accept it.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2020-10-23. Score: 2 | Accepted answer
  
  As per your requirements, I would suggest that you start with any simple CNN network.
  
  CNNs take advantage of the hierarchical pattern in data and assemble
  more complex patterns using smaller and simpler patterns. Therefore,
  on the scale of connectedness and complexity, CNNs are on the lower
  extreme.
  
  Here is a Keras example:
  model = models.Sequential()
  model.add(layers.Conv2D(32, kernel_size=(3, 3), activation='relu', input_shape=image_shape))
  model.add(layers.MaxPooling2D((2, 2)))
  model.add(layers.Conv2D(64, kernel_size=(3, 3), activation='relu'))
  model.add(layers.MaxPooling2D((2, 2)))
  model.add(layers.Conv2D(64, kernel_size=(3, 3), activation='relu'))
  model.add(layers.Flatten())
  model.add(layers.Dense(64, activation='relu'))
  # output layer
  model.add(layers.Dense(1))
  
  where image_shape is the resolution and number of channels of images (e.g. 128x128x3 for RGB images). I also suggest you downscaling the images to a lower resolution. You will also have to crop the images as they must all be the same image_shape.
  Also take a look at the MaxPooling2D  and BatchNormalization layers.
  Since you only have real and CGI images, this becomes a binary classification problem. Therefore you can have a single output (0 - CGI, 1 - real). Such problems can be solved with BinaryCrossentropy loss.
  model.compile(loss=losses.BinaryCrossentropy(from_loits=True), optimizer='adam')
  
  Finally, you can fit your model
  history = model.fit(train_images, train_labels, epochs=1000, validation_data=(test_images, test_labels))
  
  You can find a complete example here.
  Please note that depending on your data, the model can become biased if your dataset is unbalanced. That is, if all of your CGI images have text, and only a small fraction of the real images also have text, they might be misclassified. Therefore, I recommend that you visualize your model to better understand what it has learned. Here is an example of such a problem we faced at our university.
  There are also more advanced CNN architectures such as ResNet, VGG or YOLO. You can also extend your model with time series (i.e. video) using LSTM or GRU architecture.
  
  > **Comments:**
  > **username_0**: Thanks. I guess now i need to collect a few hundreds of samples for dataset. One qq: what's the rationale behind the node size 32,64,64? If I add another layer, it should go 128?  
  > **username_0**: And is there any relationship between the number of layers and dataset size? Jut some intuition?  
  > **username_2**: As far as I know, there is no relationship between the number of layers and dataset size. More layers mean more  hidden features. ResNet and VGG networks use up to 100 hidden layers.  
  > **username_2**: 32, 64 and 64 are the number of filters. Filters present hidden features of the images. The first layer will usually be activated by straight lines (so the layer should be small), whereas the last layers will be activated by some abstract features and colors. Basically, you decide how many filter layers to contain.  
  > **username_0**: Thanks. One last question: Do you know of any open dataset for this?!  
  > **username_2**: I haven't come across such a dataset  
  
  ---
  
  