I have noticed that both of the instructions tk.Tk() and tk.Frame make a new window, so what is the difference between them? and what is the advantage of using one over the other ?
I have noticed that both of the instructions tk.Tk() and tk.Frame make a new window

That is not correct. tk.Frame will not make a new window, except for the fact that any widget will force the creation of a root window if you haven't already created the root window.
Widgets in a tkinter application exist in a hierarchy, and that hierarchy must have a root window. The root window is special in that it doesn't have a parent. All other widgets must have a parent. Every application must have an instance of tk.Tk, and except for very rare circumstances you should never have more than one instance of tk.Tk.
tk.Frame is a frame: a widget with a border and not much else. Like all other widgets (except tk.Tk), it must have a parent. 
The advantage to using tk.Tk is that your application must have an instance of it. If you don't create one, one will be created for you. The zen of python says explicit is better than implicit, so you should always explicitly create it.
The advantage to using tk.Frame is that it makes it easy to collect widgets into groups and be able to manage them as a group (add a border, lay them out as a group, etc).
difference between tk.Tk() and tk.Frame