We study the magnetic properties of the metallic kagome system by a combination of muon spin relaxation (), magnetic susceptibility, and scanning tunneling microscopy (STM) measurements in single crystal specimens with Co concentrations . In the undoped antiferromagnetic compound FeSn, we find possible signatures for a previously unidentified phase that sets in at , well beneath the Neel temperature , as indicated by a peak in the relaxation rate observed in zero field (ZF) and longitudinal field (LF) measurements, with a corresponding anomaly in the ac and dc susceptibility, and an increase in the static width in ZF-µSR measurements. No signatures of spatial symmetry breaking are found in STM down to 7 K. Related to the location and motion of muons in FeSn, we confirm a previous report that about of the implanted muons reside at a field-cancelling high symmetry site at , while an onset of thermal hopping changes the site occupancy at higher temperatures. In , where disorder eliminates the field-cancellation effect, all the implanted muons exhibit precession and/or relaxation in the ordered state. In , we find canonical spin glass behavior with freezing temperature ; the ZF- and LF-µSR time spectra exhibit results similar to those observed in dilute alloy spin glasses CuMn and AuFe, with a critical behavior of at and as . The absence of spin dynamics at low temperatures makes a clear contrast to the spin dynamics observed by in many geometrically frustrated spin systems on insulating kagome, pyrochlore, and triangular lattices. The spin glass behavior of CoSn doped with dilute Fe moments is shown to originate primarily from the randomness of doped Fe moments rather than due to geometrical frustration of the underlying lattice.