Front Derailleur stays on smallest chain ring I am trying to get an old mountain bike working.

I have read and watched numerous videos on the front derailleur but have yet to find anyone having this same issues except for someone who had something caught in the bike chain.

When I shift the chain will move to the middle chain ring but when I release the shift lever the derailleur moves back to the smallest chain ring.  I can move the derailleur by hand and it will lift the chain all the way to the biggest chain ring but again as soon as I let go of the derailleur it moves back to the smallest chain ring again.  

So the best I can tell is the spring brings the derailleur back in towards the bike and the cable pulls the derailleur out away from the bike.  I am curious what is suppose to hold the derailleur out away from the bike once you shift? 

Is the derailleur broken or is this just a tuning issue?  

I am new to bike maintenance so hopefully I have worded everything correctly and you can understand what I am asking.   

Update 7/23/2016


Thanks for all of the input.  The issue does appear to be in the shift lever, I believe mine are called index shifters.  The info stamped on the shift/brake levers say Shimano ST-MO50L Japan 22.2  I unhooked the derailleur cable and removed the shift lever from the bike and was able to spray WD40 into the shifter and get it clicking again.  I would like to remove the cover from the shifter to give it a better cleaning but it is not clear how to do this.  i have searched for diagrams/instructions on line but have not come up with anything.

Update 7/25/2015

Found the below diagram on line:



Uh Oh, guess I am not suppose to take the cover off of the shift lever.  I already took the screw out but have put it back and still seems to be working.
If this is an older bike with a standard friction shifter, you need to adjust the friction.



On the shaft where the lever pivots is an adjustment knob of some sort.  Simply tighten that knob until the lever is "stiff" operating but can still be moved with reasonable ease.

If, on the other hand, you have an indexed shifter then the procedure to correct this problem depends on the specific type you have (and how repairable that particular type is).  A picture of your shifter would be most helpful if it is not the standard friction style.