^ This!
Just handing over a gem to the most devious and untrustworthy person seems like way too much of a risk in order to gain another gem. At least for Ronan he was just hired to get a stone and bring it back, he didn't actually give him the Aether, for example, in order to find it
They could explain the color thing if it turned purple when removed from the scepter but the stupidity of willfully giving it to Loki would be hard to reconcile to me.