If you have found traces of them and actual worms themselves, I would suggest spending time everyday inspecting your plants and removing the worms, poop, and damaged buds until harvest.
So far I've pulled about 30 worms from my plants and they caused a considerable amount of damage. But two...