I haven't played tf2 at all in awhile because my gpu assploded but I don't remember tf2mix being that bad when I last played it two months ago. If you want tf2mix to be less circlejerky you're gonna have to man up and be a captain to keep friends from using tf2mix as an inhouse pug organizer. Circlejerkimg is not really something that can be stopped by banning people because it's a pretty hard thing to prove and enforce. If you do a good job in game you may even impress the people in the circlejerk and they may pick you the next time you play.
Not playing tf2mix won't fix it from being a circlejerk.