Clinically-taken first-half goals from Duncan and Abdi Sharif had earned the Reds a healthy advantage 45 minutes into the tie at the Totally Wicked Stadium.
But it was a different story after the break, with Bradley Lethbridge halving the visitors’ deficit before setting up Joe Hancott for an equaliser.
Liverpool were not to be denied, however, and Duncan was the hero for Barry Lewtas’ side when he forced home Paul Glatzel’s cross from close range to finally defeat Portsmouth and set up a fourth-round clash with Accrington Stanley.
Duncan had threatened a breakthrough on nine minutes.
Glatzel found a pocket of space in the Portsmouth half and capitalised on it with a pass into the run of his strike partner to the left, whose shot across goal was dragged wide of the far post.
There was an even flow to the early exchanges but it was the young Reds creating the cleaner opportunities in front of the watching Academy graduate Trent Alexander-Arnold.
Glatzel was inches away from netting with 12 on the clock but, having contorted his upper body well to meet Sharif’s lofted delivery from the left, headed the ball against the upright.
Liverpool did not have to wait much longer, though.
On the quarter-hour, Jake Cain’s rudimentary clearance quickly turned into a dangerous counter via a defensive error, allowing Duncan to scamper through, sidestep a series of challenges in composed fashion and convert for 1-0.
Lethbridge had established himself as the obvious Portsmouth danger man and twice went close to producing an equaliser in the aftermath of the opener.
First, he struck a fierce, 20-yard drive from left to right narrowly wide of the far post and then Vitezslav Jaros had to race from his goalline and block when the forward forged a sight of goal with a clever touch inside the box.
But it was Liverpool who scored next.
Duncan was the architect, astutely collecting a kick out by his goalkeeper, rolling a marker and sliding an assist between the Portsmouth defence for Sharif to clinically thread a finish beyond Leon Pitman.
The visitors’ first-half performance suggested the tie was far from over and the Reds pressed for a third goal, Pitman tipping away a smacked effort from Elijah Dixon-Bonner just before the hour.
And the save proved pivotal as two goals in six minutes hauled Portsmouth level.
Lethbridge buried a penalty into the top left corner after Duncan committed a handball in the defensive wall from a free-kick, and the scorer turned provider when his persistence kept a loose ball alive and Hancott clipped his teammate’s lobbed cross into an unguarded net.
Liverpool were understandably deflated but regained their grasp of the match and, finally, the lead which would provide progress from the third round.
In the 92nd minute, a short corner routine saw Glatzel spin around a defender and send the ball into the centre, where Duncan made no mistake from yards out to seal a dramatic victory.
Liverpool: Jaros, Walls, Boyes (Savage, 83), R. Williams, Clayton, Clarkson, Dixon-Bonner (Bearne, 79), Cain, Glatzel, Duncan, Sharif.
Unused substitutes: Brookwell, Kelly, O’Rourke.