Talking boxing with casuals

Every now and then a big fight comes along that grabs the Joe Publics imagination, and in conversation with casual boxing fans/ general sports fans,us as boxing fans go into slightly more detail when analysing the "hyped up" fighters strengths and weaknesses, and then giving your predicted outcome.
Do you ever find that the question along the lines of " have you ever boxed?",often comes along.
Then when you answer no(if the answer is no,such as my case), you get a look as if to say "then WTF you going on about then".
Just an observation that got me thinking whether or not its just my paranoia or is quite common?

just your paranoia aja

there is an assumption from many people that you have to have done something in boxing/sport/life it to understand it or comment on. I can understand why people think that, but when you listen to the rubbish spouted by pundits, sometimes maybe the better understanding comes from the appreciative observer... and in the case of boxing, appreciative observers who haven't taken too many shots to the head.

Must admit, I am also like this when watching my son playing for his team on a Saturday.
Some fat lump shouting at my kid telling him exacly where he should be(not the manager though, I can accept that), but can't help thinking, and stating on one occasion when the red mist twisted my mellow, that if you knew that much about football you would be way too busy on a Saturday morning to be here.