This is an interesting issue that's isolated to Outlook 2007. The 2007 version actually has limited HTML support and uses a proprietary Word rendering engine to display HTML. Whereas other versions of Outlook tend to use Internet Explorer to render email.
Adding a caption below any small image in a mixed Text & Image module causes an issue where the caption actually jumps above the text, instead of displaying correctly below the image.
There are two ways around this issue.