Installing and running joomla 1.0.12, Virtuemart 1.0.9, Windoze dev box (yeah I know, severs are linux, and PHP 5.1.2.
I read all the info I could and followed the instructions. The initial result was that the image was not being shown at all. First problem was not having the "GD" installed (which is a library called gif draw, see http:/www.libgd.org/Main_Page for info & downloads). After editing my php.ini file to enable the GD section, it was producing an image source stream, but not displaying.
I downloaded the core patches, which enable the CAPTCHA in the Joomla default functions. Oh, I recommend not enabling the administrator login part until you have the normal site working (or you are locked out).
After this still wasn't generating the image. I read a post about turning off debug mode to make it work, but that didn't do it.
I finally had to edit the code because the error output (seen by pasting the URL of the image into a new browser page) was complaining about cant rewrite headers. I had to change the source hn_captcha.class.php, commenting out line 660 which tries to set a header content type.
After that one line fix it worked perfectly, in both IE 7 and Firefox 2.0.
Again, great module!