Question: When one tries to look up concepts such as overfitting and underfitting, the most common thing that pops up is polynomial regression. Why is polynomial regression often used to demonstrate these concepts? Is it just because it can be easily visualised like the graphs here:

<https://scikit-learn.org/stable/auto_examples/model_selection/plot_underfitting_overfitting.html>

But then most ml algorithm such as kmeans clustering can also be used. Then why is it usually polynomial regression only? Are there any other similar algorithms that could be used?

Answer: 
For padding in CNN's there is a useful answer in this link:<https://stats.stackexchange.com/questions/246512/convolutional-layers-to-pad-or-not-to-pad>

In Q-learning an action is taken in current state, and a next state is obtained. Then state-action value of the current state-action pair is updated by using best state-action value of obtained next state. If you have discrete state-action space, or a few states you can simply record each state-action pair in a table, and then update it by Q-learning, but if the space is continuous, or there are so many state-action pairs then you should use function approximation methods like neural networks. For example by extracting current state's visual features by CNN, and then updating weights to better approximate value of this perceived state, and taken action pair.
We only use zero padding on the edge of the image to be able to compute our convolution on 'edgy' pixels. It is usually just to keep 'round' values of dimension.

For example if your input is 64x64, with strides = 2x2, you'd expect a 32x32 output, but without padding, you would get 31x31. Padding is really not a big deal and you can remove it without having much consequences on your results (at least according to my experience), it's more a reason of 'good looking' dimensions (multiples of 2).

To get back to the use of CNN in policy decisions, your agent uses the state value to decide which action to take. CNN is used to transform a 256x256 input into a 30 or 20 output features that is concatenated into the state of the network. So a corresponding state could be [agent\_position, agent
\_speed, features\_from\_image] instead of [agent\_position, agent
\_speed, image], and then your network / Q algorithm takes its decision from this state. So CNN is a kind of subpart of the network.