Craig Conway opened the scoring with his first City goal as the winger slid Jon Parkin's pull back into the Oxford net via a Damien Batt deflection.
Simon Clist tapped in the equaliser when Jon-Paul Pittman's effort was parried by visiting keeper Tom Heaton.
But Whittingham's strike and a Jarvis header sealed Cardiff's extra-time win.
Jon Parkin could have sealed a passage through in normal time for the under-strength Championship side.
The 29-year-old should have tested Oxford keeper Clarke but instead he headed a first-half Conway cross over the crossbar from close range.
And the big Bluebirds striker thought he had given the visitors victory late on in normal time.
His goal-bound effort from Whittingham's cross looked to have gone in but referee Andy D'Urso judged that the ball had not crossed the line.
Just before Parkin's late goal-mouth drama, teenage Cardiff midfielder Ibrahim Farah almost scored on his debut but his long-range shot hit the bar.
And Oxford, League Cup winners in 1986 after beating Queens Park Rangers 3-0 at Wembley, could have taken the lead when Deane Smalley's header hit the Cardiff crossbar.
But Whittingham's long-range strike gave Cardiff the advantage as his left footed effort ensured Cardiff made the second round for the ninth time in 10 years.
And Jarvis rounded off Cardiff's triumph in the final minute with a goal on his Cardiff debut.
Cardiff keeper Heaton's long kick was headed over Oxford goalie Clarke by the 19-year-old striker from just outside the penalty area.