After serving a three match ban, Hartley made his return for the club on 26 January 2010, in a 1–1 draw against Gillingham. Hartley would make forty-one appearances in all competitions, scoring three times. At the start of his Hartlepool career, Hartley started playing for Pools at left-back under Chris Turner before moving back to his natural role as a centre-half in the 2010–11 season forming a solid partnership with Sam Collins. First-team coach Mick Wadsworth praised Hartley for his progress and stated that he was developing into a "top, top centre-half". Hartley spent the start of the season on the bench for six matches out of ten and then scored his first goal of the season, in a 2–0 win over Peterborough United on 9 October 2010. His performances attracted the interest of clubs from the Football League Championship. Then, on 1 January 2011, Hartley scored again with a volley, in a 4–2 win over Oldham Athletic. At the end of the season, Hartley remained in talks with the club over a new contract and later confirmed the signing of a new contract on 18 May 2011. Hartley would make forty-four appearances in all competitions in his second season. In the 2011–12 season, Hartley continued his partnership with Collins at the start of the season. He scored his first goal of the 2011–12 season, on 19 August 2011, in a 2–2 draw against Stevenage. In a 1–1 draw against Leyton Orient, Hartley was the victim of mistaken identity after he was booked by the referee, when in fact Evan Horwood had committed the foul. The club would win the appeal, which manager Wadsworth was relief over this. He scored again, on 30 December 2011, in a 3–1 loss against Sheffield United and another three weeks later, on 20 January 2012, in a 2–2 draw against Sheffield Wednesday. In the last game of the season, Hartley scored his fourth goal of the season, in a 3–2 loss against Charlton Athletic. Hartley would make forty-seven appearances in all competitions, scoring four times. Though he lost out for the club's supporters' Player of the Year award, Hartley would win Players' Player of the Year. At the start of the 2012–13 season, Hartley was named Hartlepool captain in October 2012 by boss Neale Cooper. Hartley then scored his first goal of the season, in a 2–1 win over Notts County on 2 February 2013, which he and James Poole were described as "Hartley-Poole double-act" and two weeks later, on 16 February 2013, he scored again, in a 2–1 win over Leyton Orient.