A tractor pulled up outside the building, in Morledge, at 07:10 GMT and deposited the dirt, police said.
A note left inside the abandoned vehicle said the action was a "personal protest against the severely flawed judicial system".
The 49-year-old man was arrested on suspicion of criminal damage and later released on bail.
Updates on this story and more from Derbyshire
The note left in the tractor read: "What a shame to have to resort to this, today is a personal protest against the severely flawed judicial system that has consistently failed to listen to simple truth and reason.
"Sorry for any inconvenience caused (to the general public)."
BBC Radio Derby reporter Nigel Cash said: "The pile immediately behind the trailer is probably a couple of feet high and couple of feet wide.
"It looks as if whoever has dumped it there may have deliberately spread it out."
Police removed the tractor and spreader after a few hours.
They said the man was bailed while their investigation continues and they "consult with the Crown Prosecution Service".
Anyone who witnessed the incident or has any information has been asked to contact the force.

Media playback is not supported on this device
Calvin Wellington, Elliot Jenkins, Zak Williams, Josh Ralph, Caleb Aekins and Daniel Brown are the newcomers.
Wales start against Papua New Guinea in Port Moresby on Saturday, 28 October.
Further group games follow against Fiji on Sunday, 5 November, in Townsville and Ireland in Perth on Sunday, 12 November.
The squad features 12 players from Super League clubs, nine players from Championship clubs, 13 players from League 1 clubs and four players from NRL and Australian clubs.
"The train-on squad is a mixture of those who have achieved with us and newcomers who have come to our attention," said Kear.
"We want to be able to pick the best squad for the World Cup.
"Ben Flower, Ollie Olds and Rhys Evans are all currently injured. We're giving them every opportunity to recover, go through their rehab and put their hands up for selection."
WALES: Larne Patrick (Castleford Tigers), Zak Williams (Coventry Bears), Courtney Davies, Steve Parry, Lewis Reece (all Gloucestershire All Golds), Danny Ansell (Hunslet), Sam Hopkins (Leigh Centurions), Michael Channing, Ben Evans, Dalton Grant, Elliot Kear, Rhys Williams (all London Broncos), Josh Ralph (Newcastle Knights), Matty Barron (Newcastle Thunder), Joe Burke (Oldham), Caleb Aekins, Daniel Brown (both Penrith Panthers), Ant Walker (Rochdale Hornets), Craig Kopczak (Salford Red Devils), Matty Fozard (Sheffield Eagles), Morgan Evans, Connor Farrer, Andrew Gay, Christiaan Roets (all South Wales Ironmen), Regan Grace, Elliot Jenkins, Morgan Knowles, Ben Morris, Calvin Wellington (all St Helens), Rhodri Lloyd (Swinton Lions), Jake Emmitt, Dan Fleming (both Toronto Wolfpack), Rhys Evans (Warrington Wolves), Ollie Olds (West Brisbane Panthers), Gil Dudson, Lloyd White (both Widnes Vikings), Ben Flower (Wigan Warriors), Phil Joseph (Workington Town).
Wales World Cup group matches:
Papua New Guinea v Wales - Saturday, 28 October, Port Moresby
Fiji v Wales - Sunday, 7 November, Townsville
Ireland v Wales - Sunday 12 November, Perth.

Calvin Trevena and Ann Varran, both 51, were found between Chiverton Cross and St Agnes in Cornwall on the 2 August.
Police found a damaged white S4 Audi estate on the B3277 and the bodies of the two walkers nearby.
Ryan Morrish, 27, from Redruth, has been charged with two counts of causing death by dangerous driving and one of perverting the course of justice.
He is due to appear at Truro Magistrates' Court on 28 March.
A man has been arrested and bailed after muck was spread across the steps of Derby Crown Court.
[NEXT_CONCEPT]
Wales coach John Kear has named six uncapped players in his 32-man training squad for the 2017 Rugby League World Cup.
[NEXT_CONCEPT]
A man has been charged with causing the deaths of two walkers by dangerous driving.