After connecting the power of homodyne and spectrum analyzer(this can remove the ground loop noise), I measured noise spectrum of homodyne again. During this measurement, there is only an infrared beam incident and has a power value of 1.3mW.
The result is shown in the attached figure.