Carlisle captain Danny Grainger left it late to earn his side a point but his injury time strike will be long remembered - as much for the fact that it came off his right boot as for the quality of the finish.
Portsmouth have now gone five league games without a victory and they thought they'd done enough as a ten-minute cameo appearance from substitute Marc McNulty helped them to overturn Grainger's first strike of the day, a thundering first half penalty spot finish.
United looked bright from the kick off and a burst from Gilliead allowed Kennedy to step in and take over. He fed the ball through the back line for Gilliead to chase but calm defending from Stevens took the ball to safety behind the goal.
But Pompey should have taken the lead on 5 minutes when Ben Davies dropped his shoulder out wide. His delivery was perfect and Lavery appeared to have time and space on his hands as he raced in. His header was firm but Gillespie held his ground and made a superb save as it angled at the roof of his net.
Davies trotted across to take the set piece and it caused problems as Bennett ducked under it at the near post. It eventually fell for Lavery and Grainger was needed on the line as it bounced high off the surface and past the stopper.
Portsmouth settled into their stride and Gillespie was in fine form as he completed a triple-save at the feet of Enda Stevens when he tricked his way past two challenges to repeatedly stab the ball at the target.
It was impressive stuff from the away side and Raynes had to stretch to clear a lobbed cross from Evans when he hooked it over his shoulder in search of Hollands and Lavery. A long clearance then found Webster in his own half and he strode forward almost 40 yards to drill a powerful shot over the bar.
Burgess was next to go close when he glanced another teasing Davies corner beyond the far post and Raynes and Ellis had to block as McGurk and Evans tried their luck from distance.
Portsmouth were well organised and they made it difficult for the hosts to get out of their own half as they squeezed the space and worked hard to stop the home passing game from finding its feet.
A fantastic sliding challenge from Atkinson prevented McGurk from finishing off a cut-back from Evans and the defensive heroics were made to count for something almost immediately as Danny Grainger overlapped on the left flank. He angled towards the penalty area and match official Scott Duncan had no hesitation in pointing to the spot as Burgess lunged in to stop the Carlisle man in his tracks. Grainger dusted himself off and swept an unstoppable spot kick past Murphy to give the hosts the half time lead.
A diving defensive header from Ellis from a Davies cross welcomed the second half opening whistle, but the United response was swift as Miller skipped clear on the right. His cross had to be dealt with as Ibehre and Gilliead steamed in.
The unexpected lead took the Cumbrians into the second half in confident mood and skill from Gary Dicker sparked a flowing move from back to front which involved Miller, Kennedy and Joyce. A spirited run from the latter created space for Grainger and he sent a perfect pass to Gilliead as he screamed for it inside the box. Murphy did well to go full length and tip the shot away from the bottom corner.
More good work from Gilliead took him to the byline and Ibehre was handed his first chance of the day when he made a run for the near post. The volley was difficult and he was disappointed to see it spin high into the Warwick.
Controversy reigned on the hour when Davies was felled by Atkinson, as the referee deemed the incident to have taken place outside the box. The away players immediately took issue and they were left with even more to say when the resulting set piece was drilled into the wall.
Pompey felt justice was done with their next attack when an injection of pace opened the Carlisle right flank up. A fizzed low cross brought Caolan Lavery to the near post and his touch left Gillespie with no chance.
And the game was turned on its head 60 seconds later when a loose pass allowed substitute Marc McNulty to pounce. He had just one thing on his mind as streaked past a challenge and drilled the ball into the back of the net.
The sides exchanged blows for the remainder of the game and Antony Sweeney raised hopes when he jumped highest for a Gilliead cross. It was on target but Murphy was well positioned to gather.
Gilliead sliced a late Wyke knock down from a Dicker cross into the night sky and Miller executed a wonderful block at close range from Tollitt as Portsmouth thought they'd wrapped up the points.
But the final say went the way of the Cumbrians as a patient build-up left Danny Grainger with time on his hands on the left wing. He stepped inside and spotted a gap and brought the crowd to its feet with a magnificent right-footed strike which literally arrowed into the top corner.
United - Grainger (44 pen, 90+4)
Pompey - Lavery (66), McNulty (68)
United - Dicker (85)
Pompey - Lavery (90)
Carlisle United - Gillespie, Atkinson, Raynes (Asamoah 73), Ellis, Miller, Dicker, Joyce (Sweeney 78), Kennedy, Grainger, Gilliead, Ibehre (Wyke 73). Subs - Hanford, Hery, Brough, McQueen.
Portsmouth - Murphy, Davies, Stevens, Webster, Burgess, McGurk (McNulty 65 (Atangana 74)), Doyle, Lavery, Bennett, Evans (Tollitt 65), Hollands. Subs - Boco, Chaplin, Clarke, Bass.
Referee - Mr Scott Duncan
Attendance - 5,503 (953 away fans)