The first issue could be the result of incorrect regional settings. For example, the modem may be expecting a North American dial tone, and instead is getting an Australian dial tone (as an example) and it is unable to detect it (or vice versa). Some modems may offer a way to change regional settings to work in other countries around the world.
The second possible cause is an incompatibility. The modem may not be fully TAPI compliant, and perhaps is not properly signalling to IVM that a dial tone has been detected.
For your second problem, I am not sure. Sometimes your phone company may require a special dial string for transferring calls (different to the default !,) and it must be changed in order for it to function correctly. Try a blind transfer and see if that works. If it does, I'm inclined to think either the phone service does not support confirmed transfers, or the dial string for confirmed transfers is different.
Edit: Nevermind, I just saw your other topic and you specifically noted that you were using Vonage and the correct dial string codes. I don't use Vonage any more so I can't test confirmed transfers, and I'm not sure what the success rate is for other Vonage users.