if anything id see it as a plus.
they dig tunnels, which would make the soil more loose allowing your roots to grow through the soil easier.
the ants shouldnt touch the plant.
they just like the soft soil.
i also heard THC kills bugs , but im not positive thats true because my outside grow...