It looked so good for Cardiff when Aron Gunnarsson and Peter Whittingham scored early in the second half but Derby deservedly hit back through goals from Jordon Ibe and Craig Bryson.
Cardiff's interim coaches Danny Gabbidon and Scott Young, in charge following the exit of Ole Gunnar Solskjaer on Friday, made four changes and the Welsh club came close to scoring in the second minute when Gunnarsson headed narrowly wide from a free kick.
The home side almost took the lead when a lovely exchange between Will Hughes and Chris Martin sent the striker in on goal only for David Marshall to make an excellent block before a minute's applause marked the 10th anniversary of the death of former Derby manager Brian Clough.
He would have approved of the way Derby were moving the ball through midfield, although it was a low cross from Cyrus Christie in the 27th minute that picked out Jamie Ward whose shot from 15 yards was well saved by Marshall.
Anthony Pilkington curled a shot over from 20 yards but Cardiff were struggling to have any sustained possession and Derby went close again when Bryson headed a Ward cross just wide.
The only concern for head coach Steve McClaren was that his team had failed to turn their superiority into goals but Gabbidon and Young had a lot to sort out at half-time to try and disrupt Derby's flow.
Whatever they said clearly worked because Cardiff stunned Derby with two goals in the space of five minutes to silence the the home fans.
Pilkington set up Gunnarsson for a shot that was deflected behind in the 51st minute and from the corner, Matthew Connolly headed the ball back across for Gunnarsson to hook it past Lee Grant from six yards.
McClaren responded by sending on Liverpool's on-loan winger Ibe but, in the 55th minute, his team conceded again when Pilkington went past two defenders on the right to set up Whittingham who lashed a shot into the top right corner from 15 yards.
Derby needed to reply quickly and Ibe got them back in the game in the 61st minute when Christie played the ball across the face of the Cardiff box and the teenager drilled a low shot into the bottom left corner of Marshall's goal.
Cardiff were on the back foot again and the pressure finally told in the 84th minute when Johnny Russell cut in from the right and found Bryson who unleashed an unstoppable shot from 20 yards which hit the inside of a post and flashed past Marshall.
The visitors were hanging on but despite five minutes of added time, Derby could not force a winner.