Only a thought but if you used insulated ducting maybe it would keep the air warm until it actually hits the chimney then it may rise straight out of chimney before it starts to cool and cause the waters in the air to freeze.
Turning your fan up more may help as well as it might force the air out quicker, you could always attach a dimmer switch onto the mains wire for the fan and then turn it up bit by bit to see how it goes.