About 100 people were involved in the incident towards the end of Didcot Town's Southern Football League game against Hereford FC.
Didcot said it was "very disappointed", while Hereford - leading 2-1 at the time of the abandonment - said it was "deeply concerned".
BBC commentator Bill Rennells said "fists were flying everywhere".
A Thames Valley Police spokesman said: "A 19-year-old man from Hereford was arrested on suspicion of a public order offence, and a 19-year-old man from Hereford was arrested on suspicion of common assault."
Both men have been bailed until April.
About 800 people attended Saturday's match, with the majority supporting the visiting Hereford team.
Didcot manager Andy Ballard called it a "very bizarre day".
He added: "I do have to commend the referee, he made a fantastic decision to call the game off.
"He couldn't guarantee the safety of the players, the supporters and everybody in the ground, so he made a very tough call, but I think it's the right one."
Hereford FC were formed in 2015 after former Football League club Hereford United were wound up in London's Royal Court of Justice the previous year.
They are the best supported team in the Southern League, with an average home attendance this season in excess of 2,700.
In a statement, the club said: "It goes without saying that Hereford FC does not condone unlawful behaviour of any sort and club officials will be working very closely with the local police, Didcot Town, the stewards who were in attendance, the FA and the Southern League to establish the relevant facts, secure all available evidence and conduct a thorough investigation."
Mr Rennells, a journalist since the age of 14,  said: "It's something I've never ever seen before in my 71-year career. There were people being held back, and some people trying to stop it."

Quins overcame the loss of lock George Merrick to injury to score five first-half tries and secure a bonus point, with Stade managing a solitary reply.
Tim Visser's intercept try came as Stade rallied early in the second half.
Clement Daguin and Willem Alberts tries gave Stade faint hope that Marland Yarde's late try quashed emphatically.
It was an impressively dynamic first-half performance in front of watching England boss Eddie Jones, while it was their defensive mettle under more intense pressure in the second period that would have stood out.
Quick shifts and sharp set-piece work set up Joe Marler's try from a rolling maul, and after Merrick's injury stoppage, a two-try blast from Joe Marchant and Visser established control.
Sekou Macalou returned a skewed Yarde kick for Stade's opener, but two bludgeoning James Chisholm scores extinguished any momentum the Top 14 side had before the break.
Gonzalo Quesada's side threatened to breach a creaking Quins defence in the second half before Visser pounced to increase Quins' buffer.
The visitors did find some consolation with scores from Daguin and Alberts but they came too late to influence the result, particularly when Yarde's score took Quins beyond 40 points.
Harlequins: Brown; Yarde, Marchant, Roberts, Visser; Swiel, Dickson; Marler, Buchanan, Collier, Merrick, Horwill (capt), Robshaw, Wallace, Chisholm.
Replacements: Gray, Evans, Jones, Twomey, Luamanu, Mulchrone, Jackson, Marchant.
Stade Francais: Camara; Vuidarvuwalu, Bosman, Williams, Sinzelle; Plisson, Daguin; Taulafo, Burden, Slimani, Pyle, Pape (capt), Macalou, Lakafia, Parisse.
Replacements: Bonfils, Zhvania, Melikidze, De Giovanni, Alberts, Coville, Fartass, Millet.
Referee: Ben Whitehouse.
Two men were arrested after a violent pitch invasion caused a football match to be abandoned.
[NEXT_CONCEPT]
Last season's beaten finalists Harlequins kicked off the European Challenge Cup with a dominant home victory against Stade Francais.