I'm gonna avoid writing a full blown nerd essay here, but I have a few inklings of things I think need to be addressed.
1. This game has always been *ours* it is made up by a select group of people who enjoy the format. Trying to cater to imaginary people doesn't get us anywhere. If we were actively having conversations with large groups of non-6s playing people (such as a pub community or a lot of exclusively HL players were suddenly very interested), or Valve about what they'd like to see, with a clear path towards more support - it would be different, but none of that exists so trying to interject any of these things into a debate about what sort of whitelist we ought to have is a red herring.
Personally I'm in favor of a very strict whitelist simply to remove as many extraneous mechanics from the game as possible because I don't feel that they add anything particularly exciting. What does a soldier spamming a choke so as to be able to toot a horn that gives a temporary boost really add to the experience of playing competitive TF2 besides the momentary joy of doing something silly? But I realize most people don't see it that way.
2. Part of the reason off-classes have become so limited in their usage relative to say 5-6 years ago is actually less a function of how players think and more a function of our shrinking community. It used to be entirely possible to play nearly full-time pyro/heavy in lobbies and not be completely hated by the entire community, because there were so many lobbies going on, if people really had trouble dealing with it they could just move into another lobby, but the lobby with your offclassing fan would still fill. That doesn't happen anymore, we're in much tighter pug groups with a much more limited player set, so if you main heavy the whole game and upset too many people, they'll never play with you again, and then the person who gleefully plays the fat man never gets to play again and the player base has less and less experience in dealing with say, a heavy standing on spire.
It's also a function of the community's penchant for pre-arranging scrims. When you set up scrims in advance with the same small selection of teams over and over again there is a major incentive to not piss off that team such that they'll refuse to scrim you because you run x off-class more than they'd like; ultimately ending up as a team that can't find scrims. Since you're playing nice with your scrim partners, there is a clear incentive to not practice doing things like dragging a sentry out to second, so when it comes time for a match, having never practiced it, people don't do it. Particularly because again, that team they're playing may be a scrim partner on every other week of the season and you don't want to make them hate you. Whereas, in the past, one could find scrims very readily even if you had a reputation for, say, dragging your sentry from last all the way to gullywash mid, because ultimately if the other team refused to play from that point on, you could just go find another scrim in 5-10 minutes, and eventually that team would forget what you'd done to them because they would be playing bunches of other teams every day.
Obviously those 2 points don't necessarily apply to prem/invite teams of the past, but those players of that time, would've come up through that system and thus gained similar experience in playing and dealing with off classes too, and they still ran offclasses of different varieties far more often due to that experience they gained when they were still relatively green.