To accurately detect even large out-of-focus by using a small focus detection sensor in time-division system focus detection.

Plural pairs of pupil areas 71/72 and 73/74 being the pairs of pupil areas symmetric with respect to the optical axis 76 of a photographing optical system and having different distances from the optical axis 76 of the photographing optical system to the center-of-gravity position of the pupil areas are set in the vicinity of the exit pupil of the photographing optical system, and the pair of pupil areas used for focus detection out of plural pairs of pupil areas 71/ 72 and 73/74 is wholly switched. Just after starting the focus detection or when out-of-focus amount being the result of previous focus detection is equal to or above a specified value, the pupil area is switched to the pair of pupil areas 71/ 72 whose center-of-gravity position is near to the optical axis 76 of the photographing optical system, and when the out-of- focus amount is smaller than the specified value, the pupil area is switched to the pair of pupil areas 73/74 whose center-ofgravity position is far from the optical axis 76 of the photographing optical system. Then, subject light is made to pass and intercepted in a time division manner in every pupil area of the pair of pupil areas so as to perform the focus detection by using luminous flux from a subject passing through the pair of pupil areas.