Media playback is not supported on this device
Pompey showed more vim and vigour in attack than the Premier League side and took the lead when Gary Roberts tapped in Marc McNulty's cross.
The Cherries came alive in the last 20 minutes and equalised through Josh King, who headed in from eight yards.
Pugh made it 2-1 the 83rd minute when his header went in off Adam Webster.
For 70 minutes, Bournemouth gave the impression of a side who viewed the FA Cup as a distraction.
Eddie Howe made 10 changes to the starting line-up that drew against Sunderland, but the replacements did little to prove to their manager they are worth a regular spot in the first team.
The double substitution of Pugh and Matt Ritchie in the 62nd minute changed the dynamic of the Bournemouth attack, as the Premier League side finally gained control on the pitch.
They levelled matters through King, who provided the deftest of touches on Eunan O'Kane's free-kick from the right. Irishman O'Kane was one of the shining lights for the Cherries.
With Bournemouth eager to capitalise on their dominance, Lewis Grabban drove forward on the right and drove in a shot that was saved by Ryan Fulton. However, the loose ball fell to the fast-arriving Pugh, who nodded in via Webster.
Since winning the FA Cup in 2008, Portsmouth have suffered relegations, points deductions and two administrations.
The club is now run by a supporters' trust, and today those who have backed them could be heartened that there may be better days ahead.
Pompey will perhaps rue not making the most of their golden spell in the first half, when they had the Cherries pressed up against a wall.
Aside from the goal, McNulty forced a great save from Adam Federici and scorer Roberts fired wide from six yards with just the keeper to beat.
Paul Cook's side are seven points adrift of an automatic promotion spot in League Two - but repeat displays will surely see them in the shake-up in May.
Portsmouth manager Paul Cook:
"I'm very proud of our football club. Eventually Bournemouth's quality came to the fore.
"We have big days in front of us.
"You don't have to win to get clapped off. We didn't win but we were clapped off.
"We turned up today after losing last week to Oxford."
Bournemouth manager Eddie Howe:
"Give credit to the way Pompey played, they could have gone direct but played very well. We're glad we're through.
"We were poor in the first half. You have to give them a lot of credit for the way they played.
"The players showed real character. Once we made the substitutions we were the better team.
"The tempo of the game was high and we didn't deal with it very well. As the second half wore on we created more chances."
Pompey face Morecambe away on Tuesday, and the Cherries are back in Premier League action at Crystal Palace on the same day.
Substitute Marc Pugh's late header ended the FA Cup hopes of League Two side Portsmouth as Bournemouth advanced into the fifth round.