"But I'll say this: When they're replacing you with Ramon Sessions for defensive purposes, that might be a sign you have work to do. As good as Irving was on offense, he was a horrifying, flaming train wreck on defense. Synergy rated him the worst defensive player in the league with at least 300 plays defended, opposing point guards ripped him for a 19.0 PER according to 82games.com, and the Cavs gave up 5.0 points per 100 possessions more with him on the court -- even though his usual replacement, Sessions, was himself among the league's most flammable point guards."
Hard to say he is top 3-5 in NBA when his team is so bad and he is possibly the worst defender in the NBA.
Luke Walton gets minutes for them. Cavs have the worst bench in the NBA and aside from Varejao who's been unbelievable this year and an extremely raw rookie in Waiters their starters aren't that much better.
Who cares about defense at the PG position? He simply isn't athletic enough to stay in front of the elite 1's in this League. Not many players in this league can.