Thursday, January 5, 2012

40k Theory: Evaluating in the Dark

Nurglitch poses a good question, how do we determine whether something unique is also good?

Before I answer that, let address one more point he makes.  He indicates that there are units which mathhammer on paper very well, but are useless on the table top.  I presume he is talking about things like Plasma Cannon Devastators, which kill marines by the handful on paper but no one in his right mind would actually play in a tournament.  

Doesn't take an expert to evaluate these guys.
Both of these questions have a simple solution: playtesting.  If a new codex has a quirky unit that defies mathhammer but you suspect it may still be good, by all means playtest it.  Frankly, while I can think of several units that mathhammer well, but in actuality rather suck, I struggle to come up with a list if units that mathhammer poorly but play well on the table top.  The closest I can come up with are force multipliers like Sanguinary Priests, or Haemonculi or Vulcan etc.

Playtesting isn’t really the answer anyone is looking for, though.  We crave theorycraft, so let’s theorycraft.

For Imperial armies we can see they all have a familiar skeleton for the most part.  Even the most radical departure from Codex: Space Marines (GK I suppose?) still fit into the framework.  Certain things will be true across the platform.  Barring a tremendous points discount, Heavy Bolter Devastators won’t be worth the FoC slot.  OK, we can handle those kinds of comparisons with ease.  Something completely different like Mephiston?  I would argue he mathhammers rather well, but besides that point, an experienced 3+ armor save player can make an educated guess how he will fit in with the known framework of a 3+ list.  Subsequent playtesting will reveal whether Mephiston is better or worse than predicted, but I can’t imagine a sober, thoughtful analysis by an experienced MEQ player would be too far off of reality in either direction.

For Xenos armies, it is a bit trickier; since we don’t have a quick plug and play mental framework for ‘how it’s supposed to be,’ especially when the book gets a radical re-design like Necrons.  A lot of Necron wargear is geared for close combat, but on the surface Necrons appear to be garbage at close combat.  So just how good is all that close combat stuff then?  Is Necron close combat greater than the sum of its parts?  My guess is mathhammer can answer that, honestly.

How about weird stuff like a IC who gives you a 4+ chance to seize the initiative?  Or wargear that makes the first turn always nightfighting?  That stuff IS hard to quantify in numbers or theory.  Having the first turn, for example, is only as good as it is important to the army.  We know from experience the Dark Eldar benefit greatly from that.  Space Wolves don’t particularly rely on it.  With a new codex we don’t have the experience to make a perfectly accurate snap decision.  But for things like that, we can make assumptions based on things we CAN mathhammer, like the defensive ability of the army.  Or the type of long range firepower that benefits from getting the first turn.  And so on.

To sum up, nothing will replace actually playing with the stuff to come up with a good evaluation.  But the gut instinct of an experienced player, combined with mathhammer can give us a good starting point, even if it isn’t perfectly accurate.  A good player may not be able to tell you how deep the water is just by sticking his toe in, but he should be able to tell you the temperature.


  1. Good points but a good way to think about things that are different from the norm is pretty easy. Particuarly Necrons theorying is this -

    "Scarabs are too good" - Only until you realize your enemy can lure them out with some bait, ignore them until then, and then blast/template them apart. People don't believe this, that blasts and templates are the norm of a good army, but they are. If not there, they usually can spam some good s6+ weaponry once the Scarabs come out to play. And with WS 2, even counter charging them is viable in some cases.

    "Wraiths are good" This is a case of where you gotta look at the whole of your army and tell me this straight in the face again. Necrons have plenty of AV 13, Missiles don't like to shoot at AV 13, but Missiles are found everywhere in many lists. So, then, where do the Missiles go? Oh that's right, the T4 guys with 3+ saves, instagibbing any failed saves, just like Marines...Silly me I almost forgot that.

    Those are just a few examples, but I feel ya 100%

  2. I think Smurfy understands what I'm getting at here. There's 'good by the numbers' and then there's 'good in context'. Play-testing is nice, but I find it tends to under-determine the utility of unique units. Without a pre-existing theoretical framework with which to inform your experience, that experience is pretty worthless: sometimes good, sometimes misleading.

  3. @Nurglitch You don't believe in mathhammer to determine how good a unit is. You don't like real world game play to figure it out, either. Now you have me curious as to what method you would say works.

  4. @Nikephoros: I think you're mischaracterizing what I've said. I think that statistics and game experience under-determine all of the quantifiable behaviour in the game.

    Smurfy touches on the method I would say works, which contextualizes both the predicted statistical results and the player's ability to engage their opponent.

    For example, Smurfy notes that Scarabs are regarded as "too good" in isolation, where they are free to attack any vehicle. But Scarabs do not exist in isolation. A Necron player's opponent can either attack them first, or attack them after they have been exposed to attack by going after a sacrificial vehicle.

    We need to quantify the choices that a player should make given the choices that her opponent should make. So when I make a choice given my opponent's choice in the previous player-turn, I have both a notion of how that choice might play out in terms of dice results to be expected, and how it exposes me to my opponent's actions the following player-turn.

    Therefore to evaluate whether something unique is also good, we should look at its capabilities and the potential responses by opponents, putting both our predicted values for behaviour (mathhammer) and game-play in context for objective consideration.

  5. The "my unit can do this" and the "well i counter that with this" debates are circular and endless. The only real way to settle them is to play games on the table top and see how they turn out in reality.

    So we start with mathhammer, then move to theorycraft for proof of concept, and then settle it with actual game play.

  6. Actually the "If my opponent does a, then I do x, but if I do x, then my opponent does b, and if my opponent does b, then I do y, etc, ad nauseum" discussions don't have to be circular and endless. The game only lasts between 5-7, and your opponent's ability to respond is finite.

    If my choice is x or y and my opponent's choices are a or b, then instead of attempting to second guess my opponent, I can simply determine the best choice regardless of my opponent's response.

    Which is where play-testing can be misleading if uninformed by mathematical strategy, because a choice may cause me to lose a game despite being the best choice, and losing might make me think that was the wrong choice, and hindsight might make me trust my judgment more than I should.

    I think we start with the rules, particularly the basic mission rules and work our way backwards to evaluate game-play.