i had the same issue, mine is much better now ive included the below:
- Insulated ducting
- built cardboard box around fan, seal gaps with duct tape
- duct tape all connections
- add a silencer to the end of your fan
I can run my fan at 100% now and can barly hear anything, the main noise issue now are the circulating fans within my grow tent but they arent too loud so i can deal with that.