Swearing by acts of worship is haraam but there is no expiation for it
Question

                I swore by my fasting that I would never do that, but now I want to do that thing. What is the fatwa on this oath?.
Praise be to Allah.
      It is not permissible to swear an oath except by one of the 
      beautiful names of Allah or one of His sublime attributes, because the 
      Prophet (blessings and peace of Allah be upon him) said: “Whoever wants to 
      swear (an oath), let him swear by Allah or else remain silent.” 

    Narrated by al-Bukhaari, 2679 

    The Prophet (blessings and peace of Allah be upon him) 
    regarded swearing by anyone or anything other than Allah as minor shirk, 
    because it involves venerating something or someone other than Allah, 
    because an oath can only be sworn by something that is venerated. The 
    Prophet (blessings and peace of Allah be upon him) said: “Whoever swears by 
    something other than Allah has committed an act of shirk.”

    Narrated by Abu Dawood (3251); classed as saheeh by 
    al-Albaani in Sunan Abi Dawood. 

    Swearing an oath by fasting, prayer or any other act of 
    worship is swearing by something other than Allah, so it is haraam. 

    It says in Tabyeen al-Haqaa’iq (3/109): 

    Swearing by an act of worship is not an oath because it is 
    swearing by something other than Allah, may He be exalted. End quote. 

    Ibn al-Humaam said in Fath al-Qadeer (5/71): 

    Swearing by acts of worship is swearing by something other 
    then Allah or His attributes, so it is not an oath. End quote. 

    The one who has done that has to repent to Allah, regret what 
    he has done and resolve never to go back to it again. 

    An oath sworn by something other than Allah is not a binding 
    oath, so it does not have to be fulfilled and no expiation has to be offered 
    for it. 

    Ibn Hazm said in al-Muhalla (9/125): 

    The one who swears by something other than Allah has not 
    sworn and it is not an oath; it is invalid and nothing has to be done with 
    regard to it except asking Allah for forgiveness and repenting only. End 
    quote. 

    Shaykh al-Islam Ibn Taymiyah (may Allah have mercy on) said: 

    Swearing by created things such as the Ka‘bah, the kngels, 
    shaykhs, kings, parents, parents’ graves and so on is an oath that is not 
    binding and no expiation has to be offered for it, according to scholarly 
    consensus. Rather it is forbidden according to scholarly consensus, and this 
    prohibition means that it is haraam according to the more sound of the two 
    opinions. End quote. 

    Al-Fataawa al-Kubra, 3/222 

    Based on this, their is nothing wrong with you doing what you 
    swore not to do if it was permissible, and you do not have to offer any 
    expiation, but you have to repent from this haraam oath. 

    And Allah knows best.