 After spending his first years as a senior with the reserves, he was loaned to Toulouse FC with which he first appeared in Ligue 1, making his debut in the competition on 14 August 2004 in a 4–1 away win against RC Strasbourg.Returned to "L'OL" for the 2005–06 season, Clerc went on to become and important first-team member as the club won three national championships in a row, contributing with 62 league appearances combined in that timeframe.He battled with Anthony Réveillère for first-choice status during his tenure, appeared in 20 UEFA Champions League games but was also banned for four matches and fined €150,000 by the French Professional Football League, after he incurred in an illegal transfer agreement to Olympique de Marseille in 2006.