The contrast variation method in polarized neutron scattering is used to determine the magnetic structure of fine particles in the ferrofluid. The "contrast" in the scattering power was varied by mixing D2O with the water solvent of the ferrofluid. Then the diffracted intensity of polarized neutrons is sensitive to the "contrast", which enables to determine the microscopic structure of dispersed particles in the ferrofluid.