How do I get the Calendar app to stop asking me for my gmail password? Since upgrading to Mavericks, the Calendar app will pester me for my gmail password every half hour or so, even though it already has the correct password stored.

I've tried changing the password, setting up 2-step verification and creating an app-specific password, changing the app-specific password... I even tried deleting all of the relevant passwords from the Keychain Access app and then entering a new app-specific password into Calendar, and still Calendar continually pesters me to re-enter the password.

I'd blame google, but since Calendar (and specifically Mavericks Calendar) is the only app that has trouble with the Google calendar, that says that the problem is on Apple's side.  Has anyone experiencing similar problems found a workaround?
It turns out the issue for me is that OSX has created 2 entries in the Calendar app for my Gmail account (even though I only have the account entered once under the Internet Accounts system pref).

The Calendar app won't let me delete the duplicate since it thinks the duplicate is managed under the Internet Accounts pref. Deleting and re-adding the gmail account in Internet Accounts didn't help either.  After deleting the Internet Account, the duplicate still sticks around in the Calendar app is not deletable since the Calendar app can't delete it, and there is no corresponding Internet Account for that calendar.

I got around the issue by un-checking the "Enable this account" checkbox for the duplicate in the Calendar app (since it won't let me delete it).  That at least makes it stop checking for calendar updates so I don't get pestered for passwords.