We have several cases of stronger characters doing less damage with charged attacks. For example, Tenshinhan, with a move stronger than the Kamehameha (according to Roshi himself), and a power level higher than Roshi's ever was, only destroyed the World Martial Arts Tournament ring. Roshi meanwhile had destroyed:
1. a mountain
2. the moon
So, I say it's a similar situation to that. It's all about proper ki control. Maybe Baby has it while Goku and Cell didn't.