{"id":1278,"date":"2016-01-11T07:37:03","date_gmt":"2016-01-11T07:37:03","guid":{"rendered":"http:\/\/www.vrdesigns.net\/?p=1278"},"modified":"2016-03-04T09:35:29","modified_gmt":"2016-03-04T09:35:29","slug":"we-are-watching-you-metrics-part-2","status":"publish","type":"post","link":"https:\/\/www.vrdesigns.net\/?p=1278","title":{"rendered":"We Are Watching You\u2026 Metrics Part 2"},"content":{"rendered":"

This is a follow on from Vic\u2019s post \u2018Monthly Metrics : 2015\u2019<\/a><\/p>\n

The game includes a Metrics option that sends data back to our secret Caribbean server every five rounds. What does it send? Only the information that\u2019s been used to compile the following. It\u2019s purely game data and couldn\u2019t reach out beyond the game software package even if it wanted to.<\/p>\n

We\u2019re not part of the NSA and Edward Snowden has, I\u2019m pretty sure, never heard of us despite his familiarisation with all things Russian.<\/p>\n

\"blog_33\"<\/a><\/p>\n

 <\/p>\n

The information is really useful to us in balancing the game and getting an idea of how it\u2019s being played. As developers we strive to do better and having hard data to base decisions around is a huge plus.<\/p>\n

The forum is great for bugs and observations but having a few people state that this or that is unbalanced isn\u2019t that meaningful simply because the sample size is too small. We do take it into account but it\u2019s complimentary to the metrics data rather than being a reason for change itself.<\/p>\n

If you haven\u2019t already, I\u2019d highly recommend switching on the Metrics setting. Doing so gives you a warm fuzzy \u2018helping to improve the game\u2019 feeling that\u2019ll get you through the first few days of winter. Can\u2019t vouch for what comes after that although hard liquor might help.<\/p>\n

Here\u2019s what we\u2019ve ascertained to date. We\u2019ve got data back from almost 600 different players and have over 1300 unique records covering about 440 games (each game sends data every 5 rounds hence the larger number of records than games).<\/p>\n

First question is who is playing which side?<\/p>\n

<\/p>\n

\"image<\/a><\/p>\n

Turns out pretty much everybody is playing the Germans. Looking at the game types played it\u2019s obvious that the small group tackling the Soviet side are mainly those forced to do so because somebody has to in a PBEM game.<\/p>\n

This is a bit disappointing although understandable as the Germans are the aggressors. The Soviet side is fun to play and is a very different experience. I\u2019d encourage people to give it a go. Vic has given the German AI a wide range of mix and match plans. Enough to keep a Soviet player guessing for quite a while.<\/p>\n

\"image<\/a><\/p>\n

PBEM games are running at roughly 1 in 10 of the total which is in line with the general trend for games. It\u2019s a small segment of the market and the overwhelming majority of people are going to be enjoying a single player experience which is why Vic has put so much effort into providing a competent, kick you in the n*ts, AI.<\/p>\n

However we\u2019re happy to cater for those who prefer playing a meat ball rather than Silicon Slim and this will be our approach going forward into the next game – solid AI and multi-player enabled. We\u2019re thinking of expanding the multi-player to allow more than two people to join in.<\/p>\n

Options. Lots of observations here.<\/p>\n

\"image<\/a><\/p>\n

 <\/p>\n

There isn\u2019t any one option that more than 30% of people choose. You\u2019re a pretty diverse bunch that are generally happy to go with the default game settings. For the next game the more options we can provide, probably the better. No surprises there.<\/p>\n

Decisions OFF (with simplified logistics) appears to be of little interest. Decisions LIMITED even more so. People appear to like the Decisions & Relationship part of the game more than we were expecting. A fair bit of work was put into providing alternatives for those that wanted a straight traditional hex based war game.<\/p>\n

It\u2019s great that people like the Decisions as that\u2019s one of the core parts of the design and even more interesting that so few people wanted to stick with a tried and true approach. Might be stretching it a bit but it could be a small indication of how receptive people are for a bit of innovation in the war gaming space.<\/p>\n

Imperial measurements would an indication of how many customers don\u2019t speak metric. I suspect that the majority of USA based customers are capable of conversing in both languages but there is still a significant segment (roughly 1 in 5) of people who prefer imperial so that\u2019s something we\u2019d carry forward to the next game.<\/p>\n

Easy Mode would be an indication of people who are learning the game and feel happier easing into the experience. There are, once again, enough there to make it worthwhile to continue offering this option.<\/p>\n

Mild Winter? It appears hardly anyone wants a blizzard free experience. I\u2019d say that a reason for this is that so few games have reached the point where the harsh winter conditions kick in (December). This might change once people get better at the game and see first hand just how debilitating severe cold can be.<\/p>\n

Historical. The most popular choice by a smidgen. It\u2019s a game that portrays an historical conflict and leans heavily on history for it\u2019s sense of immersion. Not unexpected. If you\u2019re looking for maximum replay value though this is best left off.<\/p>\n

Geneva Convention. Here is the hot potato. The big experiment. Turned out well but it could have gone either way and ended up mired in flame wars and controversy. Pleased that it didn\u2019t.<\/p>\n

Portraying the Dark Side of the war involves treading a very fine line and it looks like we have managed to wobble our way across the chasm. An interesting side note is that both Vic and I had different views on where the line should be drawn. There were a number of aspects that I couldn\u2019t, from my antipodean perspective, imagine people getting upset over that Vic, correctly, pointed out would likely end up doing the opposite. In the end we went with his European viewpoint. For the better.<\/p>\n

About 1 in 4 people prefer to keep the Geneva Convention switched on. It\u2019s there to accommodate differing preferences but I suspect that a lot of people have left it on simply because it\u2019s the default setting. Whether that\u2019s the case or not it\u2019s important that people are given the option. If there was only one person who felt the need to use it then it should still be there. We aren\u2019t in the business of causing offence.<\/p>\n

What I would be interested in knowing is whether people thought having moral choices in the game added to the Operational Command experience? Might put a poll up at the end. Feel free to add further thoughts in the comments.<\/p>\n

Adding this aspect to a future game would be worthwhile only if people thought that it added something. Metrics can\u2019t measure that.<\/p>\n

Now we come to the AI.<\/p>\n

\"image<\/a><\/p>\n

 <\/p>\n

AI think speed. Fewer people preferred the default \u2018Normal\u2019 speed which is puzzling. 35% of you opted for SLOW which would indicate a desire for an optimum AI opponent. A larger number (45%) went for FAST. Either because they couldn\u2019t abide the between turn wait for the AI to do it\u2019s thing or because they are running it on older hardware. A modern processor makes a big difference with turn wait.<\/p>\n

For those that chose SLOW the benefit to the AI isn\u2019t all that significant. It\u2019s helps but nowhere enough as upping the level of the AI difficulty.<\/p>\n

Which is where everybody has been voting for the same party – The AI Normal party. If this keeps going they\u2019ll take over the world. There are quite a few features in the higher AI levels that are aimed at increasing the difficulty in ways that don\u2019t involve straight bonuses. For those who have already rolled Panzers into Kremlin Square, have a crack at Challenging AI.<\/p>\n

\"image<\/a><\/p>\n

 <\/p>\n

Strategies. We\u2019ll start with the Germans because that appears to be what everybody is playing. Support Hitler, the default option, has a solid lock on the prize for the dominant strategy. Why? Maybe because it\u2019s the default and it\u2019s easier just to bang through the pre-start turn or perhaps because it gives you the most PP\u2019s, both upfront and later on.<\/p>\n

Military Independence is a very difficult, hard scrabble, type of experience. No support from Hitler, ridicule from your men if you attempt to curry favour with him, bare minimum PP\u2019s and the chance of getting fired if you really mess up your relationships. Offsetting this is the ability to fight the war on your terms. Nor will there be any interference from the F\u00fchrer. For those that want to experience the challenge of command and understand that delegation is a necessity, it\u2019s an interesting alternative.<\/p>\n

\"image<\/a><\/p>\n

For the Soviets it\u2019s all about Moscow. Which is, from my point of view, puzzling. The idea of the Soviet strategy is to throw a spanner into the works of the Germans. You can, given the right circumstances, turn a German win into a Draw or a Draw into a loss. Choosing Moscow as your strategy kind of defeats this. The Germans, if they follow the \u2018Support Hitler\u2019 strategy, as most seem to be doing, will likely end up focusing on Moscow (not always but probable) by the time of Hitler\u2019s final conference in November.<\/p>\n

\"image<\/a><\/p>\n

 <\/p>\n

It\u2019s a case of having all your eggs in one basket. Moscow falls and the Germans get their own objective and you lose yours. Personally I\u2019d opt for one of the other two.<\/p>\n

Finally we come to scores. I\u2019ve opted to use Scores rather than Victory Points as the Scores are adjusted for difficulty levels and should, theoretically, all be on an equal footing.<\/p>\n

What you are looking at here are the averages of the German player\u2019s scores separated into the three versions of the game that are currently out there. It\u2019s inclusive of all PBEM and Single Player games but excludes games where a Player has taken the Soviet side. This covers the bulk of the metrics data.<\/p>\n

\"image<\/a><\/p>\n

 <\/p>\n

The Release version (v15) isn\u2019t any different to the Beta 101 version which is to be expected as there were few changes that affected the balance. There were a few items such as beefing up the SS Motorised Divisions, adding additional starting hexes to the Baltic region for the Germans around Memel but this was offset by a reduction in the AGN starting fuel stockpile.<\/p>\n

Overall little change which is reflected in the graph. It\u2019s only when we get to round 20 and beyond that there is some divergence. It\u2019s easy to see why if we look at the following.<\/p>\n

\"image<\/a><\/p>\n

 <\/p>\n

By round 20 most games have finished. There is a paucity of data going forward which makes the reliability of it less certain.<\/p>\n

It\u2019s this big drop off at round 20 that led us to make the balancing tweaks in v17, Beta 102. The arrival of Soviet reinforcement armies was spaced out to prevent the big wall in AGC that tended to stop a lot of players in their tracks. Soviet AI fortification numbers were severely cut back and a few key Soviet Command Cards had their initial PP costs increased.<\/p>\n

As everybody are, in general terms, playing the Germans on Normal AI difficulty, we felt that there is scope to smooth out the difficulty curve a touch. Whether we have overshot is something that only further metrics will reveal.<\/p>\n

You can see in the Average German Scores graph above that it appears to be working (orange line, v17 Beta 102) although the bar graph below it highlights that we really don\u2019t have enough data to make an informed judgement yet.<\/p>\n

PBEM balance is a slightly different problem in that any balancing changes we make to the Soviet AI are irrelevant. As PBEM tends to attract the more competitive players (although don\u2019t let that put you off if you\u2019re considering it) we may have to introduce a separate set of balancing changes specifically for this group.<\/p>\n

As we don\u2019t have enough valid data to make a decision on this Vic has taken on the challenge of test driving the Soviet side in a PBEM game with the latest 102 Beta version to get a feel for the situation himself. In the absence of hard data, feel and feedback are the fall back option.<\/p>\n

There you go. Metrics. Toggle it on and join the rebellion. We\u2019ll change the world. A tiny virtual part of it perhaps, but every little bit helps.<\/p>\n

And before you go a quick poll on the hot potato. Feel free to make add additional feedback in the comments below.<\/p>\n

\n\t
\n\t\t

<\/p>\n\t\t

<\/p>\n\t\t

Moral Decisions (Geneva Convention Option) & the Dark Side of the War : Do you feel that this...<\/strong><\/p>