This is how hitboxes in TF2 interact. If you walk into someone and get inside their hitbox, you'll be pushed out so the hitboxes don't overlap. It's most noticable on scout because you can get into their hitbox quickly before the game realises due to the fast movement speed. You can actually use this to your advantage on scout on low ping by tapping w when you're at very close range next to someone to repeatedly jitter out of them. Because it's you that's being displaced it makes your model really jittery and hard to hit, although this is mostly impractical in actual games.
The interaction does change with ping though. I think because the game takes more time to realise you're inside someone you can actually get through the middle of them and you'll get pushed out in a different direction which is why you can sort of slide through other players on scout at higher ping. You can't use this to your advantage too well because you kind of get teleported randomly and aren't looking in the direction of the player when you come out the other side.