Conway's low drive found the bottom corner with seconds remaining in the match to help the Bluebirds come from behind to secure the win.
Kenny Miller had earlier handed Cardiff the advantage on the quarter-hour mark, planting the ball past Pompey goalkeeper Stephen Henderson from Peter Whittingham's cross after the hosts spent the majority of the early stages in their visitors' half.
Darcy Blake, Miller and Joe Mason had good chances to extend the Bluebirds' lead in an opening period dominated by the Welsh outfit, but Pompey equalised seven minutes before half-time through Marko Futacs, who capitalised on an error by home keeper David Marshall to slot home.
The visitors went in front four minutes after the break, with Greg Halford firing beyond Marshall, but the lead lasted just 20 minutes before Mark Hudson levelled once again ahead of Conway's winner.
Cardiff's first chance of the match came after six minutes, but Aron Gunnarsson could only fire wide when he was played through on goal by Whittingham as the hosts started on the front foot.
Don Cowie also tried his luck without success for the Bluebirds in the early stages before the home side made the most of their dominance to take the lead in the 15th minute.
Whittingham's corner was half-cleared by the fist of Henderson but the in-form playmaker delivered the ball back into the area where Miller was on hand to find the back of the net on the half-volley.
The Pompey keeper was called into action moments later when he had to be alert to hold onto Mason's effort.
The visitors had to wait until the 22nd minute for their first sight of goal, but Dave Kitson was unable to beat Marshall, while Kelvin Etuhu saw his effort saved by the home keeper two minutes later as Portsmouth began to stage a revival.
Chances were coming thick and fast for both sides, with Miller being denied a second by Henderson before the Scottish striker shot wide when well placed inside the box.
Blake also missed the target at the end of a barnstorming run from halfway, as Cardiff cranked up the pressure in the lead-up to half-time.
But despite all the hosts' opportunities, Pompey levelled after 38 minutes.
Marshall spilled Erich Huseklepp's cross into the path of Hungarian striker Futacs, who gleefully seized on the chance to notch his third goal for the south-coast club.
The equaliser came against the run of play but three minutes after half-time the visitors turned the game on its head.
Liam Lawrence's corner was not properly cleared by the Cardiff defence and Halford was on hand to beat Marshall on the turn and hand Portsmouth the lead.
Cardiff countered, with Andrew Taylor forcing a save out of Henderson and Conway shooting wide from inside the area, before the promotion-chasing hosts got themselves back on level terms.
Conway provided the assist, crossing for captain Hudson who nodded past Henderson.
Clear-cut chances were at a premium in the final quarter, and the game seemed to be meandering towards a draw, before the Bluebirds dug deep to find a late, late winner.
A quick free-kick found its way to Conway, whose low effort found the bottom-right corner to earn Cardiff three crucial points.
Source: PA