Ants won't directly harm your plants, but they can farm aphids. They literally find them in your yard, carry them to your plant, and line them up in rows. Then they feed on the aphids poop.
Wash your plants regularly with soap and water and apply a horticultural oil (neem, sesame, tea tree, etc) to the leaves every few weeks. The ants will get annoyed and look for other plants to farm on.