I can see that happening from just weed. If I smoke too much it feels a bit like a bad acid trip - it sucks, because I never used to have this problem. After having a few bad trips on acid and from staying up too long on speed, I can get that same feeling from weed. I don't hallucinate or anything, and what it really boils down to is an enhanced anxiety attack. That's probably what you're having too - bad anxiety, with extra detachment from reality brought on by the weed. The following ones that you've had are just anxiety attacks IMO.
The lesson I've learned: moderation. If you get the urge to smoke again, try a tiny hit and stop there. If you feel fine, then leave it at that. Nothing wrong with being a lightweight if you enjoy yourself.
I really don't think you've done anything permanent. Just do some deep breathing, and clear your mind if you feel anxiety coming on. You'll get past this.