Petri,
Waters ph has little effect on the soil itself. Soil isn't inert material. It has a cation exchange capacity. Plus buffers and amendments.
If you want proof, and have a soil kit. Take 2 samples of the same soil. One dampened with plain water. The other sample dampened with ph adjusted water. Leave the samples for half an hour, or so. Then test the samples. They'll be very close to one another.
Has it been in the same pot for 2 months, out of curiosity? Is the plant root bound?
How big is the pot?
How big is the plant?
You could use sulphur to bring the ph down. Be careful using sulphur. It takes time to activate (days +). Or mulch with some peat moss.
There's a few options out there.