Beating a dead horse

I don't mean to keep bringing this up but here is my dilemma with this whole "food allergy" thing.

1. He symptoms were never really bad. I bet most people would have just let it go with how mild they were so if this were a real "allergy" wouldn't it be much worse?
2. Why would the allergy start when he was almost 7 weeks old? My diet hadn't changed much
3. If anything it seems more of an "intolerance" than an "allergy" (from what I understand an intolerance has more digestive system reactions where an allergy has all kinds of problems like face swelling/rashes, etc) so if I were to take him to an allergist for testing, is an intolerance enough to show up on their tests?
4. How do I know if things are better with the diet when the symptoms weren't that bad to begin with? I know no bloody stool is a good sign but he didn't have that much or very frequently, I could count the times on one hand. The fussiness I have a feeling is just what a baby does. How do I know what fussiness is normal and what isn't?

I guess I have noticed a little less painful shrieks...I just don't know. He still occasionally pushes and grunts hard sounding uncomfortable (like today right after nursing) so does that mean it could have been something in my diet that day or do all babies do that? Part of me doesn't really feel there is much of an allergy. I guess these are the things I should be talking to the doctor about huh? That's what I'll be doing Wednesday.