United manager Greg Abbott shuffled his starting pack following the disappointing Tuesday night result against Tranmere with Chris Chantler, Peter Murphy and debut boy Jake Jervis coming into the starting line-up at the expense of Alessio Bugno, Mike Edwards and Paul Thirlwell.
"The keyword for us throughout last week was response," he said. "We wanted to see the players get out there and show us the determination and energy which we felt was missing from Tuesday night. They did that and they got their victory for it ... in the end.
"Just once I'd love to be able to sit back and enjoy a win. Yet again we've gone through the mill, but we've kept going and I'm delighted for the players. We've got the first league win of the season under our belt and our record at the moment isn't too bad."
The Cumbrians started in determined mood and an early McGovern cross was flicked on by Berrett inside the area. Robson went for a first-time volley but it spun off his boot and fell invitingly for Jervis. The big striker set himself but he was off balance and he skied his shot over the bar.
The blue tide swept forward and Robson went close with a rasping free kick from 30-yards which whisked just past the angle of post and bar.
Cadamarteri was next to work some space and his persistence led to an opening for Jervis as he made a run across the first defender. His glanced header was accurate but Andersen was in the right place to claim it.
The deserved goal came on 13 minutes when McGovern once again did the donkey work to create an opening. He robbed the ball off Harley and measured his cross perfectly. New-boy Jake Jervis did the rest with a beautifully weighted header past the full length dive of the Pompey stopper.
"I seem to have a knack of scoring on my debut," Jervis explained. "I've done it at a few of the clubs I've been with on-loan and it's always good to get a goal.
"I saw the ball coming across and you just try to react to it. When you get the connection you try to put it as far away from the keeper as you can, and I managed to do that."
The visitors registered their first effort of note on 28 minutes when the awkward Izale McLeod took matters into his own hands but his 25-yard shot was pulled well wide in the end.
Luke Rodgers did the same when he tried his luck under pressure from Livesey, and Carlisle reasserted themselves as they closed ranks in midfield and recaptured the passing momentum.
Matt Robson found space on the left just before the break and it needed a brave block from Long to stop a shot from angling in. Cadamarteri raced onto the loose ball and his pass to Berrett opened a path to the edge of the box for Noble. Another snapshot brought another block and Berrett was unlucky when his thunderbolt whistled straight into Andersen's midriff.
However, the moment of the half belonged to McLeod when he found himself one-on-one with Collin. He took his eye off the ball, swung his leg at thin air and landed in a heap accompanied by cheers from the Warwick Road End fans behind the goal.
The Cumbrians came out of the second half traps at full tilt and Jervis brought a fine save out of Andersen with a curling shot which was heading for the back of the net. The Portsmouth reprieve was short-lived when Danny Livesey stepped away from his marker at the resulting corner and popped his header over the keeper to double the lead.
Robson and Jervis had efforts go just wide as the home side tried to kill things off but Portsmouth started to find space as they threw men forward. Compton was first to test the target and Walker brought the best out of Collin as the visitors grew in confidence in front of goal.
The Cumbrian goal lived a charmed life on 75 minutes when Obita dropped his shoulder and let on fly as he bore down on the target. The ball rattled the underside of the crossbar, bounced on the line and was helped on its way to safety by the attentive Peter Murphy.
Collin needed strong hands as the clock ticked into the last ten minutes when Howard crunched a shot through a gaggle of bodies and it appeared that Portsmouth were left with a mountain to climb when Kevin Long was shown a second yellow card moments later.
But the incident galvanised the troubled south coast outfit and they regained a foothold in the game when Harris finished off an untidy move.
Carlisle responded and a break from Jervis brought a fine save from Andersen as the pair went toe-to-toe. Unfortunately for him James Berrett reacted first and he kept his cool to pass the ball into the net.
Now into time added on, the action kept coming and Clifford stabbed home a scramble in the area to send the sizeable travelling support into celebration mode.
But substitute Paddy Madden sealed the points with the last kick of the game when he followed up on a parried save from a McGovern shot to score his first goal of the season on his return from a frustrating injury niggle.
"A lot of what we saw was fantastic and there was certainly plenty of excitement out there," Abbott concluded. "I'd have preferred it to end 2-0 if you really push me on it, because things like that are really tough to watch as a manager.
"You ask me what I've learned about the group from that last ten minutes - I can tell you I've learned absolutely nothing. It was about passion and energy, and credit to them for seeing it through."
United - Jervis (13), Livesey (47), Berrett (90+3), Madden (90+5)
Pompey - Harris (88), Clifford (90+3)
United - Noble (58), Robson (61), McGovern (74)
Pompey - Harley (40), Long (45 & 84 - sent off), Connolly (87)
Carlisle United - Collin, Simek, Livesey, Murphy, Chantler, McGovern, Noble, Berrett, Robson, Cadamarteri (Madden 76), Jervis. Subs - Gillespie, Thirlwell, Edwards, Potts, Beck, Symington.
Portsmouth - Andersen, Dumbuya, Harley, Long, Walker, Howard, McLeod (Clifford 77), Rodgers (Harris 77), Compton, Connolly, Webster (Obita 35). Subs - Eastwood.
Ref: Eddie Ilderton
Att: 5,120 (647 away fans)