Oh man...."Find God". Yeah, join a murderous group of delusional moron's, that's the key to relieving your guilt.
Unfortunately guilt is an emotion that you have learned. It's a behavior that was engrained in you by your father and through your upbringing. I'm sure that you can trace back these feelings of guilt back as long as you can remember.
This is assuming your guilt is unfounded and greatly exaggerated. If you got drunk one night 10 years ago and killed a little girl crossing the street while driving intoxicated than that's a whole 'nother story. Your guilt would have obvious origins other than childhood, but barring any tragedy, it's likely a learned behavior.
It's very difficult to change your ways of thinking and it does require help. You should seek professional help to work on your issues and to help you find happiness. I don't believe in everything a psychologist has to say and that's not the purpose of meeting with one. It's to talk to a professional listener.
Turning to God and joining the church is the same as joining the Klan to relieve your guilt. Associating yourself with and partaking in the delusional ramblings of a murderous organization isn't going to relieve any guilt. Sort of a step in the wrong direction.
If you don't like the idea of speaking to a professional, perhaps you can find some way to communicate your thoughts of guilt to us nameless people online. Not that any advice given will be meaningful, but for the opportunity to discuss with anonymity what exactly it is that makes you feel this guilt.
Just a thought...