You can install this extension.
It gives me a preview both in Windows Explorer and in Adobe Bridge (which was unexpected).