Direct nanoscale observation on the nucleation and growth of hydrogenated amorphous and microcrystalline silicon on graphite substrates was made using scanning tunneling microscopy, atomic force microscopy, and Raman scattering spectroscopy. Nucleation of hydrogenated silicon clusters is initiated through the nucleation sites created by reactive hydrogen species coming from the source gas plasma. The difference in spatial distribution of nucleated clusters at the initial stage of deposition between a-Si:H and mu c-Si:H is ascribed to the difference in the number density of nucleation sites which results in difference in the diffusion length of a SiH3 radical at the initial stage of deposition on the graphite substrate. The RMS roughness of mu c-Si:H films is larger than that of a-Si:H when the film thickness is larger than 10 Angstrom, which is opposite to the behavior at the initial nucleation stage on the graphite substrate.