How do I get rid of my encrypted iPhone backups to start from scratch? Is there a way I can delete all my encrypted iPhone backups for which I forgot the password, so that I can start backing up from fresh? 

At the moment I'm stuck in this loop (below) which I can't seem to exit
To delete the backups, go to iTunes>Preferences>Devices. All iPhone backups will be listed here. Highlight them, then hit Delete. 

But if you're using a Mac, I would first try to see if you can find the password in your Keychain (from discussions.apple.com):


  
  Go to the "keychain access " app.
  Under "Keychains" select "Login"
  Under "Category" select "Passwords"
  Scroll down and look for "iPhone Backup"
  Double click and a window opens
  Check the box that says "Show Password"
  A window pops up asking for your OS X account password
  Type in that password and voila!