Ulster Unionist James Cooper lost out to Sinn Fein's Michelle Gildernew by a majority of just 53 votes.
Sinn Fein's victory followed a recount of the votes and it was the last Northern Ireland Westminster seat to be declared on Friday night.
Mr Cooper said there had been malpractice at some polling stations in the constituency
This followed reports that some votes were cast after the permitted time of 2200 BST on Thursday.
"We do not accept the outcome of the contest in Fermanagh south Tyrone," he said.
"We are going to fight our corner and we are not going to quit on this."
However, Mr Cooper would not say whether his party intended to bring their fight to the courts.
"The issue of how we handle a re-run is another political consideration.
"There are also matters as to whether or not it would be a re-run or whether certain votes that were cast would be discounted and those matters still need to be resolved," he said.
Mr Cooper, who is strongly in favour of the Good Friday Agreement, said that there was "clear and irrefutable evidence of electoral malpractice" in some parts of the constituency".
"The result of this may well be that my party will be considering the legal implications of the manner in which this elections was conducted in various stations throughout County Fermanagh and Tyrone," he said.
Former Ulster Unionist MP Ken Maginnis had held the seat for 18 years, before announcing his retirement from Westminster earlier this year.
The DUP had selected the social development minister Maurice Morrow to contest the seat, but he withdrew in favour of Independent Unionist Jim Dixon.
Mr Dixon was seriously injured in the 1987 IRA bombing in Enniskillen, County Fermanagh.
He was also standing as an anti-Agreement candidate, which split the unionist vote in the constituency.
In previous elections the DUP had given gave Ken Maginnis a free run rather than letting a nationalist take the seat.
In the 1998 assembly elections, Sinn Fein topped the poll with 1,200 first preference votes ahead of the Ulster Unionists.
Fermanagh South Tyrone result