Is it possible to have both Python3.6 and Python3.8 versions installed at same time in windows machine
Yes. You can have both versions installed on single machine. All u need to do is to download Python3.6 from its Official site, set your Interpreter to python3.6 and u r all set.
I am going through a Django tutorial but it's an old one. The videos were all made using Django 1.11 and Python 3.6. Problem is I have installed python3.8 in my machine. So I was trying to create virtualenv with python version 3.6. But as python 3.6 is not available in my machine, I couldn't do that. At this point I was wondering even if it is actually possible to have both python 3.6 and python 3.8 in a machine at same time. 
Kindly someone help me with this problem or point me to the right resource to understand more on this problem.