Biased, but i think if you have the knowledge of all 8 billion items in this game, what class they're for, and how they're referenced to in the item schema i think u can get away with having a slight advantage.
also cosmetics in general have the habit of just breaking spy disguises agnostic of no hats mod and they're allowed in current whitelist (and nobody complains even in hl), this game is so broken that trying to patch out everything that confers a minor advantage is probably a futile endeavor. i think the approach of identifying shit that is clearly busted (scope removal, spy sound script, hud shadows etc.) and banning that explicitly is fine, and here i would consider this as squarely in the not busted category especially since you should probably be taking off cosmetics on offclasses regardless because of disguise kit.