Better Playtesting: The Fallacy of Bad Match-Ups

While playtesting we learn a lot about our lists.  We learn what strengths it has, what weaknesses it has, and most importantly, how it plays against opposing lists and codices.

Unfortunately, that last part is a gigantic potential stumbling block.   Why, you ask?

Practice makes perfect

Let’s assume that you play 20 playtest games with your Vanilla Marines against a gauntlet Space Wolves list.  You go 8-12.  This is bad news.  There are a few things you can conclude from this playtesting…

1.  Perhaps your list is bad.  This is possible.  However, if other folks have had great success with very similar lists, or if your list trounces other gauntlet lists, this probably isn’t true.  This is especially the case if you’re experienced and you understand how to build a competitive list.

2.  Perhaps Space Wolves are a bad match up.  It could very well be the case that the peculiar aspects of Spaces Wolves units and rules are such that they have an inherent advantage over a particular codex that is very difficult to overcome.  Again this is possible, but among two 5^th Edition books (besides Nids) the results shouldn’t be so pronounced that you lose 60% of the time with a tuned list against a stock list.  My personal feeling is that two equal players with two modern codices, one playing a tuned list and the other a stock list should be at worst 50/50 even in a ‘bad matchup.’  I don’t have any statistics to back that up, just a gut reaction.

3.  Perhaps your playtest partner is a better player than you.  This is unlikely to be true unless your other gauntlet match ups go similarly.  If that is the case, there really isn’t a mystery anymore.

4.  You play the match up incorrectly.  Now this is interesting and gets to the crux of the matter.  The key lesson to take from playtesting is to understand the key aspects of the match up, so that when you are faced with the match up in a tournament, you have a plan for victory.

Let’s say you play the supposed 20 playtest games, and that it just so happened that in all your wins, you got early pressure on his Long Fangs while in your losses you got no pressure on his Long Fangs and they went to town.  It is highly likely that early pressure may be the key to the match up in that case.  For the purposes of this example, let’s say that Long Fang pressure is the key.  After identifying that, you play another 20 playtest games.  Suddenly, instead of going 8-12, you go 15-5.  Congratulations, you “solved” the match up.  Now repeat the same formula for every other list in your gauntlet and proceed to kick ass at your next tourney.

Obviously, it will rarely be so simple, and the solution will rarely produce such a wild swing in results.  Most often it won’t be one thing that you have to change to win, it will be several things that you have to do differently to pull the nose up.  So it can be rather hard to do in practice and requires a lot of time spent playing, which is a definite grind.

But the rewards are there.  Imagine our hypothetical playtesters never considered option 4.  Imagine that, instead, they decided (incorrectly as it were in our example) that option 1 or 2 was true.  They would have either radically altered the list in order to try to improve one particular match up (which may hurt them against other lists by unbalancing their list) or they may have abandoned the list and army entirely.  That would be quite a shame to abandon a viable army which you like simply because you failed to identify a simple play change.

What should you take away from this?  Primarily that the solution to a bad match up in playtesting is often solvable by changing your approach to the game, rather than changing your list.  It requires a lot of time and effort to get the answers 100% right, but if you’re committing to scientific playtesting you’re ostensibly doing it because you’re willing to put in work to get results.  Lastly, even in abbreviated playtesting with a small sample size of matches you can learn valuable information about how to approach a particular match up so long as you carefully (and unemotionally) analyze the games you do play.  Happy playtesting!

Thoughts?  Comments?  Questions?