Mid-infrared (MIR) photothermal microscopy/spectroscopy represents a new, ultrasensitive, superresolution infrared absorption technique. The origin of its signal contrast is complex, containing competing contributions from photothermal-induced changes to specimen refractive indices and volumes as well as those of the local environment. In this study, we investigate the interplay between abovementioned contributions to observed contrast in MIR photothermal microscopy images and spectra. This entails size-dependent measurements on individual polystyrene and polymethylmethacrylate nanoparticles with radii below 100 nm and as low as 25 nm. From this, we now establish that sizable photothermal contrast arises from the interference between light scattered from a specimen’s local dielectric medium and backreflected light from an air/substrate interface. This is particularly important for small specimens embedded in media. Attesting to this, we observe, for the first time, a predicted, small nanoparticle size crossover in signal contrast between particles embedded in air versus a dielectric medium with a refractive index greater than one.