Yep, I agree. It's the personal use that's the problem. But that doesn't change the fact that some people are idiots and can't use common sense or self control. If everyone used common sense and self control, there would be no laws and no regulations, no need for airbags, seatbelts, etc.. But that's not the real world.
If you have a large problem with distracted drivers, it makes sense to minimize the possible distractions, not give people more things to take their attention away from the road.
Just like if you have a problem with an overactive and hyper kid, you don't give him a bowl of candy and say "just have 1." Or if you have a problem with people entering cockpits of planes and hijacking them, you don't leave the door open and unlocked, allow guns, knives, etc. on the plane.
It's a situation where the possible harm is greater than the inconvenience of a few. I've seen far too many accidents where there was contact after someone crossed over the centerline because they were not paying attention. I've also avoided far too many for my liking (like the one I mentioned above). Distracted driving is a big problem in our society today. Possibly even worse than drunk driving at this point.