you don't know where it's been

An object encountered in an unfamiliar place (such as outdoors) could be dirty or contaminated by germs, and should therefore not be handled through bodily contact. Chiefly said to command children not to interact with foreign objects.