In this paper, stray lights in optical system of remote sensing satellite camera is simulated. For this purpose, stray lights, sources of production and their destructive effects on the image quality is studied. Moreover, mechanisms to deal with these unwanted lights are presented. Then, the effect of optical housing and surface coatings on the performance of stray light in satellite camera, has been studied by the results of the simulation. Finally an optimal design includes a set of different techniques for improving the performance of the considered camera was selected. Simulation results show that by this method, at large angles, stray lights can be completely eliminated whereas at angles near the FOV, stray lights intensities can be very much reduced namely up to 100000 times of their original intensity without almost no serious disturbance in the camera’s field of view.