We Are Watching You… Metrics Part 2

This is a follow on from Vic’s post ‘Monthly Metrics : 2015’

The game includes a Metrics option that sends data back to our secret Caribbean server every five rounds. What does it send? Only the information that’s been used to compile the following. It’s purely game data and couldn’t reach out beyond the game software package even if it wanted to.

We’re not part of the NSA and Edward Snowden has, I’m pretty sure, never heard of us despite his familiarisation with all things Russian.

blog_33

 

The information is really useful to us in balancing the game and getting an idea of how it’s being played. As developers we strive to do better and having hard data to base decisions around is a huge plus.

The forum is great for bugs and observations but having a few people state that this or that is unbalanced isn’t that meaningful simply because the sample size is too small. We do take it into account but it’s complimentary to the metrics data rather than being a reason for change itself.

If you haven’t already, I’d highly recommend switching on the Metrics setting. Doing so gives you a warm fuzzy ‘helping to improve the game’ feeling that’ll get you through the first few days of winter. Can’t vouch for what comes after that although hard liquor might help.

Here’s what we’ve ascertained to date. We’ve got data back from almost 600 different players and have over 1300 unique records covering about 440 games (each game sends data every 5 rounds hence the larger number of records than games).

First question is who is playing which side?

image (1)

Turns out pretty much everybody is playing the Germans. Looking at the game types played it’s obvious that the small group tackling the Soviet side are mainly those forced to do so because somebody has to in a PBEM game.

This is a bit disappointing although understandable as the Germans are the aggressors. The Soviet side is fun to play and is a very different experience. I’d encourage people to give it a go. Vic has given the German AI a wide range of mix and match plans. Enough to keep a Soviet player guessing for quite a while.

image (2)

PBEM games are running at roughly 1 in 10 of the total which is in line with the general trend for games. It’s a small segment of the market and the overwhelming majority of people are going to be enjoying a single player experience which is why Vic has put so much effort into providing a competent, kick you in the n*ts, AI.

However we’re happy to cater for those who prefer playing a meat ball rather than Silicon Slim and this will be our approach going forward into the next game – solid AI and multi-player enabled. We’re thinking of expanding the multi-player to allow more than two people to join in.

Options. Lots of observations here.

image (10)

 

There isn’t any one option that more than 30% of people choose. You’re a pretty diverse bunch that are generally happy to go with the default game settings. For the next game the more options we can provide, probably the better. No surprises there.

Decisions OFF (with simplified logistics) appears to be of little interest. Decisions LIMITED even more so. People appear to like the Decisions & Relationship part of the game more than we were expecting. A fair bit of work was put into providing alternatives for those that wanted a straight traditional hex based war game.

It’s great that people like the Decisions as that’s one of the core parts of the design and even more interesting that so few people wanted to stick with a tried and true approach. Might be stretching it a bit but it could be a small indication of how receptive people are for a bit of innovation in the war gaming space.

Imperial measurements would an indication of how many customers don’t speak metric. I suspect that the majority of USA based customers are capable of conversing in both languages but there is still a significant segment (roughly 1 in 5) of people who prefer imperial so that’s something we’d carry forward to the next game.

Easy Mode would be an indication of people who are learning the game and feel happier easing into the experience. There are, once again, enough there to make it worthwhile to continue offering this option.

Mild Winter? It appears hardly anyone wants a blizzard free experience. I’d say that a reason for this is that so few games have reached the point where the harsh winter conditions kick in (December). This might change once people get better at the game and see first hand just how debilitating severe cold can be.

Historical. The most popular choice by a smidgen. It’s a game that portrays an historical conflict and leans heavily on history for it’s sense of immersion. Not unexpected. If you’re looking for maximum replay value though this is best left off.

Geneva Convention. Here is the hot potato. The big experiment. Turned out well but it could have gone either way and ended up mired in flame wars and controversy. Pleased that it didn’t.

Portraying the Dark Side of the war involves treading a very fine line and it looks like we have managed to wobble our way across the chasm. An interesting side note is that both Vic and I had different views on where the line should be drawn. There were a number of aspects that I couldn’t, from my antipodean perspective, imagine people getting upset over that Vic, correctly, pointed out would likely end up doing the opposite. In the end we went with his European viewpoint. For the better.

About 1 in 4 people prefer to keep the Geneva Convention switched on. It’s there to accommodate differing preferences but I suspect that a lot of people have left it on simply because it’s the default setting. Whether that’s the case or not it’s important that people are given the option. If there was only one person who felt the need to use it then it should still be there. We aren’t in the business of causing offence.

What I would be interested in knowing is whether people thought having moral choices in the game added to the Operational Command experience? Might put a poll up at the end. Feel free to add further thoughts in the comments.

Adding this aspect to a future game would be worthwhile only if people thought that it added something. Metrics can’t measure that.

Now we come to the AI.

image (4)

 

AI think speed. Fewer people preferred the default ‘Normal’ speed which is puzzling. 35% of you opted for SLOW which would indicate a desire for an optimum AI opponent. A larger number (45%) went for FAST. Either because they couldn’t abide the between turn wait for the AI to do it’s thing or because they are running it on older hardware. A modern processor makes a big difference with turn wait.

For those that chose SLOW the benefit to the AI isn’t all that significant. It’s helps but nowhere enough as upping the level of the AI difficulty.

Which is where everybody has been voting for the same party – The AI Normal party. If this keeps going they’ll take over the world. There are quite a few features in the higher AI levels that are aimed at increasing the difficulty in ways that don’t involve straight bonuses. For those who have already rolled Panzers into Kremlin Square, have a crack at Challenging AI.

image (5)

 

Strategies. We’ll start with the Germans because that appears to be what everybody is playing. Support Hitler, the default option, has a solid lock on the prize for the dominant strategy. Why? Maybe because it’s the default and it’s easier just to bang through the pre-start turn or perhaps because it gives you the most PP’s, both upfront and later on.

Military Independence is a very difficult, hard scrabble, type of experience. No support from Hitler, ridicule from your men if you attempt to curry favour with him, bare minimum PP’s and the chance of getting fired if you really mess up your relationships. Offsetting this is the ability to fight the war on your terms. Nor will there be any interference from the Führer. For those that want to experience the challenge of command and understand that delegation is a necessity, it’s an interesting alternative.

image (6)

For the Soviets it’s all about Moscow. Which is, from my point of view, puzzling. The idea of the Soviet strategy is to throw a spanner into the works of the Germans. You can, given the right circumstances, turn a German win into a Draw or a Draw into a loss. Choosing Moscow as your strategy kind of defeats this. The Germans, if they follow the ‘Support Hitler’ strategy, as most seem to be doing, will likely end up focusing on Moscow (not always but probable) by the time of Hitler’s final conference in November.

image (7)

 

It’s a case of having all your eggs in one basket. Moscow falls and the Germans get their own objective and you lose yours. Personally I’d opt for one of the other two.

Finally we come to scores. I’ve opted to use Scores rather than Victory Points as the Scores are adjusted for difficulty levels and should, theoretically, all be on an equal footing.

What you are looking at here are the averages of the German player’s scores separated into the three versions of the game that are currently out there. It’s inclusive of all PBEM and Single Player games but excludes games where a Player has taken the Soviet side. This covers the bulk of the metrics data.

image (8)

 

The Release version (v15) isn’t any different to the Beta 101 version which is to be expected as there were few changes that affected the balance. There were a few items such as beefing up the SS Motorised Divisions, adding additional starting hexes to the Baltic region for the Germans around Memel but this was offset by a reduction in the AGN starting fuel stockpile.

Overall little change which is reflected in the graph. It’s only when we get to round 20 and beyond that there is some divergence. It’s easy to see why if we look at the following.

image (9)

 

By round 20 most games have finished. There is a paucity of data going forward which makes the reliability of it less certain.

It’s this big drop off at round 20 that led us to make the balancing tweaks in v17, Beta 102. The arrival of Soviet reinforcement armies was spaced out to prevent the big wall in AGC that tended to stop a lot of players in their tracks. Soviet AI fortification numbers were severely cut back and a few key Soviet Command Cards had their initial PP costs increased.

As everybody are, in general terms, playing the Germans on Normal AI difficulty, we felt that there is scope to smooth out the difficulty curve a touch. Whether we have overshot is something that only further metrics will reveal.

You can see in the Average German Scores graph above that it appears to be working (orange line, v17 Beta 102) although the bar graph below it highlights that we really don’t have enough data to make an informed judgement yet.

PBEM balance is a slightly different problem in that any balancing changes we make to the Soviet AI are irrelevant. As PBEM tends to attract the more competitive players (although don’t let that put you off if you’re considering it) we may have to introduce a separate set of balancing changes specifically for this group.

As we don’t have enough valid data to make a decision on this Vic has taken on the challenge of test driving the Soviet side in a PBEM game with the latest 102 Beta version to get a feel for the situation himself. In the absence of hard data, feel and feedback are the fall back option.

There you go. Metrics. Toggle it on and join the rebellion. We’ll change the world. A tiny virtual part of it perhaps, but every little bit helps.

And before you go a quick poll on the hot potato. Feel free to make add additional feedback in the comments below.

Moral Decisions (Geneva Convention Option) & the Dark Side of the War : Do you feel that this...

Loading ... Loading ...

Cheers,
Cameron

This entry was posted in DC:Barbarossa, Game Design. Bookmark the permalink.

11 Responses to We Are Watching You… Metrics Part 2

  1. MaxG says:

    In my view the first turn in the game is not realistic and tilted heavily in favor of Soviets side. Historically, by June 26th Germans captured Yakobstadt and were just north of Minsk, but in the game its next to impossible even to capture Vilnus by the end of the first turn. When the game starts, all Soviet divisions are fully entrenched (100 or more) which is in contrary with their presumed offensive posture. I think all Soviet troops should have entrenchment 0 at the beginning of the first turn.

  2. Cameron Harris says:

    Hi MaxG,

    Soviet Divisions suffer a -50 Entrenchment penalty on the first turn due to their unpreparedness and the fact that few Soviets were manning defences on the opening day of the campaign.

    This penalty gradually dissipates over time.

    If you look at the first active campaign turn you can see that the vast majority of Soviet units are at entrenchment levels of around 40, not 100. The exceptions are those in adverse terrain or cities.

    The also suffer a heft Blitzkrieg penalty which makes it very difficult for the Soviets to do much in the first few turns other than pull back where, and if, possible.

    Cheers,
    Cameron

  3. MaxG says:

    Cameron, how about allowing German player to set focuses and artillery support on a pre-turn June 18th? Isn’t that reasonable that this kind of preparations can be done on the eve of the invasion?

  4. JagdFlanker says:

    i havn’t screwed around with the game for more than a few turns in yet to get a feel for it and get myself organized, but every time i start a new game all the options are always reset to default which stinks since there’s settings i ALWAYS want every game no matter what, like no geneva and slow AI turns.

    i understand that these settings are set this way for the initial time a new player starts the game for the first time, but is there any way you can implement a way to save the settings at game start so they are always set to a player’s own default or last game start, like how already when i turn off the music in game and the music is always turned off every game from that point on?

    this would likely make for more accurate metrics as well

  5. vic says:

    Hi JagdFlanker,

    You are not the first one to mention this minor incovenience. I’ll look into it.

    Best wishes,
    Vic

  6. Cameron Harris says:

    Hi MaxG,

    The very large Blitzkrieg bonus that applies across the board to all soviet units on the early turns isn’t solely a result of surprise.

    A large part of it is from the efforts of the Theatre based artillery and specialised assets, hence their unavailability on the opening day for general use.

    Cheers,
    Cameron

  7. MaxG says:

    Cameron, I can see reasoning behind artillery support for the first turn (not completely buying into it still), but what about focus assignment? Why not to do it in a pre-turn as soon as German player finds out campaign goals?

  8. Cameron Harris says:

    Hi MaxG,

    Same reason. All those Nebelwerfers, Stug Bn’s, etc. are contributing to the Blitzkrieg bonus on the first day.

    Cheers,
    Cameron

  9. MaxG says:

    Cameron-Vic,
    What I find confusing is a lack of clarity in the manual on how ‘HQ Combat mod’ and ‘HQ Morale mod’ applied to the individual units in a battle. The HQ-related info in the manual emphasizes several times that all HQ-related bonuses determined based on the distance between HQ and a subordinate unit at the beginning of the turn. However, as you move the unit to or away from HQ during the turn you can see that ‘HQ Combat mod’ and ‘HQ Morale mod’ change based on a current distance from an HQ. So which bonuses are actually used for a combat resolution: current ones or those that unit had at the beginning of the turn?

  10. Cameron Harris says:

    Hi Max,

    Values current at the time of combat are the ones used.

    Cheers,
    Cameron

  11. Andrej says:

    So, as the Soviets, I managed to kill 1400ish (reaching parity in active medium tanks, we both have some 480 mediums left) German medium Tanks by the 5. August, vs. an AI with some boni.

    The big exploitable thing was the AIs presponsity to send its tanks after weak units. I was using small, beaten up units as bait. AI tanks smash 1 unit, move in, figure they can smash another one (and easily do so) and well, ooops, you are right next to 4 fresh infantry divisions and 4 tank units which are still on offensive posture. You usually dont get an encirclement from such lures, AI is too smart from that, but you can get a juicy full AP attack, coming from combined arms, with a high concentricity bonus.

    This Soviet tank concentrations do quite a number on German tanks, if they get the charge off.

    In reality, this was very difficult to do for the Soviets to pull off, because individual Armies had real issues communicating horizontally with each other and because the Luftwaffe was trying hard to blow up any concentration of Soviet armor they saw.

    I think part one could be approximated by Soviet units behaving less efficient if you mix and match units from different armies. This could for example be implemented by 2 units mistiming their attacks, so one of the attakcing units wastes its first turn or something like that.

    It could also be utilized via a modified initiative roll on the participating army commanders.

Leave a Reply