Well, if you really wanted to find out reasons, you should go to list of streams for those games.
For example, right at this moment, I clicked on SM64, which has 1.7k viewers (TF2 at 200~). Further inspection shows that 1.6k of those viewers are watching a speed runner, who has 16,963 YT subscribers and 71,674 Twitch.tv followers.
Pokemon Snap: 2,736 viewers, all but 9 from a channel with 82,038 YT subscribers and 26,139 Twitch.TV followers.
The top 5 games have thousands upon thousands of players, yet can only get 20-82k. It sounds big, but when you compare that to the games population as well as consider the fact that those are some of the most popular games currently being played around the world, that's low.
How about a newer game: GTAV
Around 7.5k total viewers, most popular viewer atm has 2.7k viewers, 142,885 Twitch.tv followers and 35,981 YT subscribers.
Let's compare to the TF2 ones you listed.
ExTv: 72,405 YT subscribers and 6,028 Twitch.tv followers.
TF.TV: 10,158 YT subscribers and 5,433 Twitch.Tv followers.
TF2 streams are mainly focused on being viewable to the non-casual audience. TF2's main audience is traders and casuals, don't forget that. TF2 does not have the backing others games do.
Seems like, with this all said, the game is doing as well as it should be. You need popular players to praise and play your game to get views, and even then it's still hard to pull viewers in. There are only 2 TF2 people that have the popularity to bring in viewers 24/7 that I know of: stabby (47,813 YT and 10k Twitch) and STAR_ (304,284 YT subs and 16k Twitch). People like playing video games more than watching them, that simple. Special things like SaltyBet and Speed Runs get views for doing something out of the box, other games are just popular, and guess what? Popular people will go to popular games for more popularity.