We Are Watching You… Metrics Part 2

This is a follow on from Vic’s post ‘Monthly Metrics : 2015’

The game includes a Metrics option that sends data back to our secret Caribbean server every five rounds. What does it send? Only the information that’s been used to compile the following. It’s purely game data and couldn’t reach out beyond the game software package even if it wanted to.

We’re not part of the NSA and Edward Snowden has, I’m pretty sure, never heard of us despite his familiarisation with all things Russian.

blog_33

 

The information is really useful to us in balancing the game and getting an idea of how it’s being played. As developers we strive to do better and having hard data to base decisions around is a huge plus.

The forum is great for bugs and observations but having a few people state that this or that is unbalanced isn’t that meaningful simply because the sample size is too small. We do take it into account but it’s complimentary to the metrics data rather than being a reason for change itself.

If you haven’t already, I’d highly recommend switching on the Metrics setting. Doing so gives you a warm fuzzy ‘helping to improve the game’ feeling that’ll get you through the first few days of winter. Can’t vouch for what comes after that although hard liquor might help.

Here’s what we’ve ascertained to date. We’ve got data back from almost 600 different players and have over 1300 unique records covering about 440 games (each game sends data every 5 rounds hence the larger number of records than games).

First question is who is playing which side?

image (1)

Turns out pretty much everybody is playing the Germans. Looking at the game types played it’s obvious that the small group tackling the Soviet side are mainly those forced to do so because somebody has to in a PBEM game.

This is a bit disappointing although understandable as the Germans are the aggressors. The Soviet side is fun to play and is a very different experience. I’d encourage people to give it a go. Vic has given the German AI a wide range of mix and match plans. Enough to keep a Soviet player guessing for quite a while.

image (2)

PBEM games are running at roughly 1 in 10 of the total which is in line with the general trend for games. It’s a small segment of the market and the overwhelming majority of people are going to be enjoying a single player experience which is why Vic has put so much effort into providing a competent, kick you in the n*ts, AI.

However we’re happy to cater for those who prefer playing a meat ball rather than Silicon Slim and this will be our approach going forward into the next game – solid AI and multi-player enabled. We’re thinking of expanding the multi-player to allow more than two people to join in.

Options. Lots of observations here.

image (10)

 

There isn’t any one option that more than 30% of people choose. You’re a pretty diverse bunch that are generally happy to go with the default game settings. For the next game the more options we can provide, probably the better. No surprises there.

Decisions OFF (with simplified logistics) appears to be of little interest. Decisions LIMITED even more so. People appear to like the Decisions & Relationship part of the game more than we were expecting. A fair bit of work was put into providing alternatives for those that wanted a straight traditional hex based war game.

It’s great that people like the Decisions as that’s one of the core parts of the design and even more interesting that so few people wanted to stick with a tried and true approach. Might be stretching it a bit but it could be a small indication of how receptive people are for a bit of innovation in the war gaming space.

Imperial measurements would an indication of how many customers don’t speak metric. I suspect that the majority of USA based customers are capable of conversing in both languages but there is still a significant segment (roughly 1 in 5) of people who prefer imperial so that’s something we’d carry forward to the next game.

Easy Mode would be an indication of people who are learning the game and feel happier easing into the experience. There are, once again, enough there to make it worthwhile to continue offering this option.

Mild Winter? It appears hardly anyone wants a blizzard free experience. I’d say that a reason for this is that so few games have reached the point where the harsh winter conditions kick in (December). This might change once people get better at the game and see first hand just how debilitating severe cold can be.

Historical. The most popular choice by a smidgen. It’s a game that portrays an historical conflict and leans heavily on history for it’s sense of immersion. Not unexpected. If you’re looking for maximum replay value though this is best left off.

Geneva Convention. Here is the hot potato. The big experiment. Turned out well but it could have gone either way and ended up mired in flame wars and controversy. Pleased that it didn’t.

Portraying the Dark Side of the war involves treading a very fine line and it looks like we have managed to wobble our way across the chasm. An interesting side note is that both Vic and I had different views on where the line should be drawn. There were a number of aspects that I couldn’t, from my antipodean perspective, imagine people getting upset over that Vic, correctly, pointed out would likely end up doing the opposite. In the end we went with his European viewpoint. For the better.

About 1 in 4 people prefer to keep the Geneva Convention switched on. It’s there to accommodate differing preferences but I suspect that a lot of people have left it on simply because it’s the default setting. Whether that’s the case or not it’s important that people are given the option. If there was only one person who felt the need to use it then it should still be there. We aren’t in the business of causing offence.

What I would be interested in knowing is whether people thought having moral choices in the game added to the Operational Command experience? Might put a poll up at the end. Feel free to add further thoughts in the comments.

Adding this aspect to a future game would be worthwhile only if people thought that it added something. Metrics can’t measure that.

Now we come to the AI.

image (4)

 

AI think speed. Fewer people preferred the default ‘Normal’ speed which is puzzling. 35% of you opted for SLOW which would indicate a desire for an optimum AI opponent. A larger number (45%) went for FAST. Either because they couldn’t abide the between turn wait for the AI to do it’s thing or because they are running it on older hardware. A modern processor makes a big difference with turn wait.

For those that chose SLOW the benefit to the AI isn’t all that significant. It’s helps but nowhere enough as upping the level of the AI difficulty.

Which is where everybody has been voting for the same party – The AI Normal party. If this keeps going they’ll take over the world. There are quite a few features in the higher AI levels that are aimed at increasing the difficulty in ways that don’t involve straight bonuses. For those who have already rolled Panzers into Kremlin Square, have a crack at Challenging AI.

image (5)

 

Strategies. We’ll start with the Germans because that appears to be what everybody is playing. Support Hitler, the default option, has a solid lock on the prize for the dominant strategy. Why? Maybe because it’s the default and it’s easier just to bang through the pre-start turn or perhaps because it gives you the most PP’s, both upfront and later on.

Military Independence is a very difficult, hard scrabble, type of experience. No support from Hitler, ridicule from your men if you attempt to curry favour with him, bare minimum PP’s and the chance of getting fired if you really mess up your relationships. Offsetting this is the ability to fight the war on your terms. Nor will there be any interference from the Führer. For those that want to experience the challenge of command and understand that delegation is a necessity, it’s an interesting alternative.

image (6)

For the Soviets it’s all about Moscow. Which is, from my point of view, puzzling. The idea of the Soviet strategy is to throw a spanner into the works of the Germans. You can, given the right circumstances, turn a German win into a Draw or a Draw into a loss. Choosing Moscow as your strategy kind of defeats this. The Germans, if they follow the ‘Support Hitler’ strategy, as most seem to be doing, will likely end up focusing on Moscow (not always but probable) by the time of Hitler’s final conference in November.

image (7)

 

It’s a case of having all your eggs in one basket. Moscow falls and the Germans get their own objective and you lose yours. Personally I’d opt for one of the other two.

Finally we come to scores. I’ve opted to use Scores rather than Victory Points as the Scores are adjusted for difficulty levels and should, theoretically, all be on an equal footing.

What you are looking at here are the averages of the German player’s scores separated into the three versions of the game that are currently out there. It’s inclusive of all PBEM and Single Player games but excludes games where a Player has taken the Soviet side. This covers the bulk of the metrics data.

image (8)

 

The Release version (v15) isn’t any different to the Beta 101 version which is to be expected as there were few changes that affected the balance. There were a few items such as beefing up the SS Motorised Divisions, adding additional starting hexes to the Baltic region for the Germans around Memel but this was offset by a reduction in the AGN starting fuel stockpile.

Overall little change which is reflected in the graph. It’s only when we get to round 20 and beyond that there is some divergence. It’s easy to see why if we look at the following.

image (9)

 

By round 20 most games have finished. There is a paucity of data going forward which makes the reliability of it less certain.

It’s this big drop off at round 20 that led us to make the balancing tweaks in v17, Beta 102. The arrival of Soviet reinforcement armies was spaced out to prevent the big wall in AGC that tended to stop a lot of players in their tracks. Soviet AI fortification numbers were severely cut back and a few key Soviet Command Cards had their initial PP costs increased.

As everybody are, in general terms, playing the Germans on Normal AI difficulty, we felt that there is scope to smooth out the difficulty curve a touch. Whether we have overshot is something that only further metrics will reveal.

You can see in the Average German Scores graph above that it appears to be working (orange line, v17 Beta 102) although the bar graph below it highlights that we really don’t have enough data to make an informed judgement yet.

PBEM balance is a slightly different problem in that any balancing changes we make to the Soviet AI are irrelevant. As PBEM tends to attract the more competitive players (although don’t let that put you off if you’re considering it) we may have to introduce a separate set of balancing changes specifically for this group.

As we don’t have enough valid data to make a decision on this Vic has taken on the challenge of test driving the Soviet side in a PBEM game with the latest 102 Beta version to get a feel for the situation himself. In the absence of hard data, feel and feedback are the fall back option.

There you go. Metrics. Toggle it on and join the rebellion. We’ll change the world. A tiny virtual part of it perhaps, but every little bit helps.

And before you go a quick poll on the hot potato. Feel free to make add additional feedback in the comments below.

Moral Decisions (Geneva Convention Option) & the Dark Side of the War : Do you feel that this...

Loading ... Loading ...

Cheers,
Cameron

This entry was posted in DC:Barbarossa, Game Design. Bookmark the permalink.

11 Responses to We Are Watching You… Metrics Part 2

Leave a Reply