Mitch Garbutt opened the scoring for Leeds, but Dorn's treble and tries from Denny Solomona and Joel Monaghan made it 30-6 to Cas at the break.
Solomona added his second after a mistake from Ash Handley, who soon crossed for Leeds, but Gadwin Springer and Ben Crooks extended the lead.
Grant Millington took Cas past 50 points after fine work from Luke Gale.
The defeat was the 11th of the season for the reigning Super League champions and the second time they had conceded 50 points this campaign following their 56-12 loss to Widnes in February.
Rhinos had few answers to deal with Tigers' veteran full-back Dorn in his first game back following two months out with a hamstring injury.
The 33-year-old collected Junior Moors' offload to cross for his first before collecting Gale's kick into space to score under the posts.
The Australian then completed his hat-trick just after the 30-minute mark as Mike McMeeken's pass left him with a simple finish.
Castleford move above Wakefield and Widnes to sixth in the table.
Leeds Rhinos: Golding; Handley, Watkins, Keinhorst, Hardaker; McGuire, Lilley; Galloway, Burrow, Garbutt, Jones-Buchanan, Ferres, Cuthbertson.
Replacements: Mullally, Sutcliffe, Achurch, Walters.
Castleford Tigers: Dorn; Flynn, Crooks, Monaghan, Solomona; McShane, Gale; Patrick, Milner, Jewitt, McMeeken, Moors, Massey.
Replacements: Cook, Millington, Springer, Tickle.
Referee: Richard Silverwood

Crews were called to the market town's High Street just before 14:30 GMT.
A number of people have been evacuated from nearby properties amid concerns the car could be fuelled by Liquid Petroleum Gas (LPG).
A fire service spokeswoman said: "We don't get many vehicle fires involving LPG but but it is explosive so we do take precautions."

RAF Wittering in Cambridgeshire was initially named after the nearby town of Stamford.
Spitfires, Hurricanes, the V-Bomber Force and Harriers have been based at the airfield. It is now home to the RAF's A4 Force and trainee pilots.
A centennial memorial has been unveiled at All Saints' Church in Wittering.
Luke Dorn scored a first-half hat-trick as Castleford Tigers thumped Super League's bottom side Leeds Rhinos.
[NEXT_CONCEPT]
Firefighters have been tackling a car blaze in the centre of Skipton.
[NEXT_CONCEPT]
An airfield established during World War One as a Royal Flying Corps base for anti-Zeppelin fighters has marked 100 years of active service.