Switch Between Windows of the Same Application Again I'm new to Mac.
I searched and found Command + ` does what I want. But I'm wondering how would I do it with a mouse? I tried hovering over the icon in the dock, but nothing happens.
You have multiple possibilities : 


Right-clic or Long-left-click on the dock. It will open a contextual menu with at the top the list of all open windows for the given app
Use a mission control shortcut for the action "Application Windows". You can set this in System Preferences > Keyboard > Shortcut > Mission Control.