Not that my sort of wargaming is wrong; but that I am not even a wargamer..
The one is just a logical extension of the other though Mike. “What you’re doing isn’t valid, therefore you are not valid,” is just a harsher version of “you’re doing it wrong, therefore you are wrong.”
It is, however, just silly. Especially in the context you’re experiencing it. Because sci-fi likely dwarfs historicals, fantasy certainly dwarfs historicals. Thus, I guess I would not challenge why you feel as you do, but I would discourage you from giving their criticism much weight as it makes so little sense.
Perhaps things are different in the UK scene. Here in the US historicals have to fight to get any visibility what-so-ever compared to fantasy and sci-fi, forget board gaming and card games which have seen huge upticks in the last couple decades while miniatures have had a slow decline that sometimes stabilizes.