ants invading should i worry

so i go to check on the girls this morning and there is an ant colony moved in over night and has a hole and they are carrying little white things and they are crawling on my plants. i think the rain we just had has somthing to with this infestation can i spray the rim of th pots with poison or what should i do

Get some ant traps.
I don't know if I'd be spraying poison close to my plants.

"Diatomaceous Earth" will take care of them. Ants are notorious for farming aphids so you do not want to let them have access to your plants. DANGER! DANGER!