An image processing apparatus, an image display system, an image processing method, and a program by which it is possible to display an optimum three dimensional image when display is switched from a two dimensional tomographic image to a three dimensional image are provided. The processor (14) outputs a tomographic image display signal representing a two dimensional tomographic image included in a first tomographic image group based on first imaging data obtained by imaging a subject, extracts a second tomographic image group having a smaller interval between tomographic images than the first tomographic image group, on the basis of second imaging data acquired in imaging corresponding to the imaging for acquiring the first imaging data, if a display switching signal indicating switching from display of the two dimensional tomographic image to display of a three dimensional image is acquired, and outputs a three dimensional image display signal representing a three dimensional image generated on the basis of the extracted second tomographic image group.