While I’m not agreeing with, or disagreeing with anyone on this I think that the discussion of tiers is viewing a MU between based on knowing what to do, when to do it, and how to execute it properly. Then you add in the players on top of that layer to see who sinks or swims to the potential values based on human abilities. Tiers change however as new techniques and options are explored as you noted. Tiers are measured based on the information and ideas on hand at the time they were created. What may seem like the ideal route could turn out to have a counter not yet found that someone could introduce by going against the accepted “correct” way to play the match, or simply introducing something new into it. In addition you can win matches by playing them “wrong” and putting the opponent on tilt. Playing to the player not the character.
Showing players playing the match is a double edged sword. On one hand it’s the only way to really show how a match would play out at all as it’s the most true testing grounds, the real world. However it’s also the least scientific as it adds so many additional variables that cannot be controlled. Different players add different styles, knowledge, reactions, habits, and more. In addition you have the Online vs Offline which is also split again with Side By Side vs Head To Head offline play which changes styles and abilities to react to things.
I’ve talked with David about Hakan’s potential and hes told me that despite how much he talks about it people just can’t wrap their heads around it until he sits them down and shows them things in person. Not necessarily “beats them in a match” but actually shows them how these tools can be used in a match.
Which brings me to the last thing, something I agree with you on, posting a match where a player wins/loses isn’t entirely indicative of the matchup’s difficulty. Not just because players abilities can change day to day and that add tons of variables to the equation. It’s more important to look at where/when the players are interacting and how. Not the final result.
It’s kind of like when you hear “So and so got bodied, he lost 4-10 in a FT10” but then you go and see that every single match came down to the last round and often the rounds were ending where both players were on their last 10% health or something. Yeah, obviously the better player won, and consistently, but with all matches being extremely close you shouldn’t really call it a bodying. Even if you only care just on who won or lost to make your point it can be presented in two very different ways to give two very different views.
If you listed a match as being lost 4 to 10 it sounds pretty awful. He only won 28.5% of the matches played. However if you do it by rounds the number suddenly looks much more palatable. If every match went down to round 3 then 4 to 10 would be 18 to 24. That would be 42.8% of the rounds played. A huge jump just based on how the data is presented.
Ultimately though I think that it’s best to concentrate more on analyzing how the match up is being played, and where the losses/wins are occurring over concentrating strictly on the score. The final score IS important though.