I think it's possible to setup Mumble such that people could listen in but be muted, but Mumble is pretty archaic so I'm not surprised if he hasn't gotten around to it. He's also more focused on the site, at the moment. There's only so much enigma to be able to do stuff.
I'm not here to dictate your choice of where to watch the game, but stream viewership is very important; it's the only number that gets pushed to sponsors. After being professional, of course. Which, at the moment, is probably a bad idea for TF2, if only because of that one time.