Double-Breaks Far too Frequent

[rbodleyscott]

The tooltip is said to be the result of 1000 trials' result. The ideal way is to do more than 1000 to see whether the result is in accordance with the tooltip figures.

But that may mean we need script to do it other than manually testing more than 1000 times.

That code only tests the Win:Draw:Lose ratios, it does not test CT results.

Sorry, but I don't know about the structure of computer games, but you seem to be saying that there are a number of different RNG's operating in the game at the same time. Have I understood that correctly? If so, do they operate autonomously from each other? I would presume that they do. So, it is possible in some matches that a majority of these separate RNG's are tending to favour one player over the other, I guess. Maybe this is what we are picking up on?

No, there is only one RNG function used in the battles.

[MVP7]

The tooltip is said to be the result of 1000 trials' result. The ideal way is to do more than 1000 to see whether the result is in accordance with the tooltip figures.

But that may mean we need script to do it other than manually testing more than 1000 times.

That code only tests the Win:Draw:Lose ratios, it does not test CT results.

Sorry, but I don't know about the structure of computer games, but you seem to be saying that there are a number of different RNG's operating in the game at the same time. Have I understood that correctly? If so, do they operate autonomously from each other? I would presume that they do. So, it is possible in some matches that a majority of these separate RNG's are tending to favour one player over the other, I guess. Maybe this is what we are picking up on?

Speaking as a programmer, there is no way the RNG is broken or favoring one side. RNG also doesn't mean that two sides will have equal amount of luck over a sample group that is not very large. It's less likely that the double breaks happen equally to both sides than it is for them to be spread unevenly.

https://en.wikipedia.org/wiki/Gambler%27s_fallacy
https://en.wikipedia.org/wiki/Hindsight_bias
https://en.wikipedia.org/wiki/Extension_neglect

[melm]

That code only tests the Win:Draw:Lose ratios, it does not test CT results.

Sorry, but I don't know about the structure of computer games, but you seem to be saying that there are a number of different RNG's operating in the game at the same time. Have I understood that correctly? If so, do they operate autonomously from each other? I would presume that they do. So, it is possible in some matches that a majority of these separate RNG's are tending to favour one player over the other, I guess. Maybe this is what we are picking up on?

No, there is only one RNG function used in the battles.

Aren't there two?

One is the random variable X, that determines whether you win or draw or lose that battle, which is related with relative POA.
And the other is random variable Y, which mimics two six-sided dice to justify whether you pass CT or not, condition on the r.v. X result that you lose current battle(melee or impact).

[rbodleyscott]

Sorry, but I don't know about the structure of computer games, but you seem to be saying that there are a number of different RNG's operating in the game at the same time. Have I understood that correctly? If so, do they operate autonomously from each other? I would presume that they do. So, it is possible in some matches that a majority of these separate RNG's are tending to favour one player over the other, I guess. Maybe this is what we are picking up on?

No, there is only one RNG function used in the battles.

Aren't there two?

One is the random variable X, that determines whether you win or draw or lose that battle, which is related with relative POA.
And the other is random variable Y, which mimics two six-sided dice to justify whether you pass CT or not, condition on the r.v. X result that you lose current battle(melee or impact).

There are lots of places in the scripts where RNG is used - e.g. shooting casualties, close combat resolution, cohesion tests, decisions whether to evade, evade and pursuit distances, AI decisions etc. etc. etc.

But they all access the same Rand() function.

If there was a logical error, it would be in the scripts that use the Rand() function, not in the Rand() function itself.

The CohesionTest() function in /Data/scripts/MoraleTools.BSF is pretty straightforward, especially the part that determines double drops, and anyone with any programming experience can read it for themselves, and see that it works as advertised.

[melm]

There are lots of places in the scripts where RNG is used - e.g. shooting casualties, close combat resolution, cohesion tests, decisions whether to evade, evade and pursuit distances, AI decisions etc. etc. etc.

But they all access the same Rand() function.

If there was a logical error, it would be in the scripts that use the Rand() function, not in the Rand() function itself.

The CohesionTest() function in /Data/scripts/MoraleTools.BSF is pretty straightforward, especially the part that determines double drops, and anyone with any programming experience can read it for themselves, and see that it works as advertised.

One more quick question.

Does Rand() generates uniform distributed number between 0 and 1, or it generates normal distributed number as N(0,1), i.e. standard normal distribution?

[rbodleyscott]

One more quick question.

Does Rand() generates uniform distributed number between 0 and 1, or it generates normal distributed number as N(0,1), i.e. standard normal distribution?

Rand(x,y) generates a uniformly distributed integer between x and y inclusive.

That is why (to mimic an approximation to a normal distribution) the Cohesion Test uses Rand(1,6) + Rand(1,6).

[stockwellpete]

Right, I have created a test scenario with 30 close order war band units facing off against another 30 close order war band units. And I am just doing the impact phase. A few results already (30 results per set) . . .

Set 1 - 3 fragmented, 4 disrupted, 1 cohesion test passed (on a bad defeat), RNG neutral
Set 2 - 3 fragmented, 6 disrupted, 1 cohesion test passed, RNG neutral
Set 3 - 6 fragmented, 3 disrupted, 2 cohesion tests passed, RNG favoured Side A (2 frags) Side B disadvantaged (4 frags, 3 disrupts)

[rbodleyscott]

Pete, when you have completed doing that set of samples, it would be worth comparing a set for 30 Heavy Infantry Offensive Spearmen vs 30 Heavy Infantry Offensive Spearmen.

[stockwellpete]

Right, I have created a test scenario with 30 close order war band units facing off against another 30 close order war band units. And I am just doing the impact phase. A few results already (30 results per set) . . .

Set 1 - 3 fragmented, 4 disrupted, 1 cohesion test passed (on a bad defeat), RNG neutral
Set 2 - 3 fragmented, 6 disrupted, 1 cohesion test passed, RNG neutral
Set 3 - 6 fragmented, 3 disrupted, 2 cohesion tests passed, RNG favoured Side A (2 frags) Side B disadvantaged (4 frags, 3 disrupts)

Set 4 - 2 fragmented, 5 disrupted, 5 cohesion tests passed, RNG neutral
Set 5 - 3 fragmented, 3 disrupted, 0 cohesion tests passed, RNG neutral
Set 6 - 5 fragmented, 0 disrupted, 3 cohesion tests passed, RNG favoured Side B (1 frag) Side A disadvantaged (4 frags)
Set 7 - 2 fragmented, 5 disrupted, 2 cohesion tests passed, RNG neutral
Set 8 - 1 fragmented, 2 disrupted, 3 cohesion tests passed, RNG neutral
Set 9 - 1 fragmented, 2 disrupted, 3 cohesion tests passed, RNG neutral
Set 10 - 2 fragmented, 5 disrupted, 1 cohesion test passed, RNG favoured Side B (1 frag, 1 disrupted) Side A disadvantaged (1 frag 4 disrupted)

I am going to stop now for a while or I will start going doo-lally.

Very interesting though. RNG favouring one side in 3 of the sets (twice significantly, once moderately). There have been 28 fragmented and 35 disrupted results out of 300 impact combats. 21 cohesion tests were passed out of 84 (28+35+21) which is exactly 25%. I think this figure is low. And double drops are happening just under 10% of the time. Also fragmented results are happening nearly as often as disrupted results, which is something that surprised me a bit. I was expecting disrupted results to be about twice the number of fragmented outcomes.

I will do another 300 later today.

[stockwellpete]

Pete, when you have completed doing that set of samples, it would be worth comparing a set for 30 Heavy Infantry Offensive Spearmen vs 30 Heavy Infantry Offensive Spearmen.

Yes, OK. How many should I do for each test? Would a 1,000 be sufficient?

[rbodleyscott]

Pete, when you have completed doing that set of samples, it would be worth comparing a set for 30 Heavy Infantry Offensive Spearmen vs 30 Heavy Infantry Offensive Spearmen.

Yes, OK. How many should I do for each test? Would a 1,000 be sufficient?

If you can stand it.

[TheGrayMouser]

Oh for the love of Ares, Mars, Loki and other assorted gods of war and misfortune, not this tired old theme again. “Im a tactical genius and should have won if not for rng” said every wargamer at least once in their career... It’s ok to gripe about it here and there , ie grognard griping, but this wanting to change rules to “enhance” so called skill based play (chess anyone?) or investigate code for perceived rng injustice is just crazy. S**t happens, soldier on.

You are missing the point and misrepresenting the argument entirely. No-one is claiming that they are an unlucky player, but we are saying that that, in some games, the RNG favours one side or the other too much and it spoils them. Nobody is asking for FOG2 to be like chess (yawn!).

I’m not misrepresenting anything, your talking out of both ends, just read what you wrote: “no one is claiming they are unlucky, just that...rng favors one or the other to much...”.

[stockwellpete]

I’m not misrepresenting anything, your talking out of both ends, just read what you wrote: “no one is claiming they are unlucky, just that...rng favors one or the other to much...”.

One of us certainly is. I'll repeat my argument for you. Nobody, as far as I know, is claiming that they are a persistently unlucky player. It is generally accepted that good and bad luck evens itself out for each of us over a period of time. But that does not mean that luck cannot play an important role in a certain proportion of the matches we participate in. And in some of these matches luck can be the decisive factor and it often spoils our enjoyment of the game.

[TheGrayMouser]

Ok Pete, my original post was not personal nor directed at any one person, it was meant as a proverbial glass of cold water splashed into this current series of complaints about rng.

The same theme came up over and over in fog1 and it was tedious. This is all about personal taste. Since You don’t like certain outcomes of certain games you are advocating rules changes.

Instead of claiming I misrepresent the issues you must understand my raised eyebrows rise around the fact that most people if they think they are lucky don’t complain about it too much, it’s only when they are unlucky that” something is wrong”. Of course if you are the spokes person for everyone whom altruistically wants a reduced role of rng, forgive me.

The game needs a certain amount of volitability otherwise it is just...deterministic, I suppose. Warbands and lancers fit the bill nicely.

[stockwellpete]

Right, I have created a test scenario with 30 close order war band units facing off against another 30 close order war band units. And I am just doing the impact phase. A few results already (30 results per set) . . .

Set 1 - 3 fragmented, 4 disrupted, 1 cohesion test passed (on a bad defeat), RNG neutral
Set 2 - 3 fragmented, 6 disrupted, 1 cohesion test passed, RNG neutral
Set 3 - 6 fragmented, 3 disrupted, 2 cohesion tests passed, RNG favoured Side A (2 frags) Side B severely disadvantaged (4 frags, 3 disrupts)

Set 4 - 2 fragmented, 5 disrupted, 5 cohesion tests passed, RNG neutral
Set 5 - 3 fragmented, 3 disrupted, 0 cohesion tests passed, RNG neutral
Set 6 - 5 fragmented, 0 disrupted, 3 cohesion tests passed, RNG favoured Side B (1 frag) Side A severely disadvantaged (4 frags)
Set 7 - 2 fragmented, 5 disrupted, 2 cohesion tests passed, RNG neutral
Set 8 - 1 fragmented, 2 disrupted, 3 cohesion tests passed, RNG neutral
Set 9 - 1 fragmented, 2 disrupted, 3 cohesion tests passed, RNG neutral
Set 10 - 2 fragmented, 5 disrupted, 1 cohesion test passed, RNG favoured Side B (1 frag, 1 disrupted) Side A moderately disadvantaged (1 frag 4 disrupted)

I am going to stop now for a while or I will start going doo-lally.

Very interesting though. RNG favouring one side in 3 of the sets (twice significantly, once moderately). There have been 28 fragmented and 35 disrupted results out of 300 impact combats. 21 cohesion tests were passed out of 84 (28+35+21) which is exactly 25%. I think this figure is low. And double drops are happening just under 10% of the time. Also fragmented results are happening nearly as often as disrupted results, which is something that surprised me a bit. I was expecting disrupted results to be about twice the number of fragmented outcomes.

I will do another 300 later today.

Set 11 - 3 fragmented, 6 disrupted, 4 cohesion tests passed, RNG neutral
Set 12 - 2 fragmented, 2 disrupted, 3 cohesion tests passed, RNG neutral
Set 13 - 2 fragmented, 6 disrupted, 2 cohesion tests passed, RNG favoured Side B (1 frag, i disrupt) Side A moderately disadvantaged (1 frag and 5 disrupted)
Set 14 - 0 fragmented, 3 disrupted, 3 cohesion tests passed, RNG neutral
Set 15 - 2 fragmented, 3 disrupted, 1 cohesion test passed, RNG favoured Side B (2 disrupted) Side A moderately disadvantaged (2 frags and 1 disrupted)
Set 16 - 5 fragmented, 7 disrupted, 3 cohesion tests passed, RNG favoured Side B (1 frag, 3 disrupts) Side A severely disadvantaged (4 frags, 4 disrupted)
Set 17 - 2 fragmented, 5 disrupted, 0 cohesion tests passed, RNG neutral
Set 18 - 3 fragmented, 4 disrupted, 3 cohesion tests passed, RNG favoured Side B (1 frag, 1 disrupted) Side A moderately disadvantaged (2 frags, 3 disrupted)
Set 19 - 2 fragmented, 6 disrupted, 2 cohesion tests passed, RNG favoured Side B (0 frags, 3 disrupted) Side A moderately disadvantaged (2 frags, 3 disrupted)
Set 20 - 2 fragmented, 7 disrupted, 5 cohesion tests passed, RNG favoured Side A (0 frags, 4 disrupted) Side B moderately disadvantaged (2 frags, 3 disrupted)
Set 21 - 2 fragmented, 3 disrupted, 3 cohesion tests passed, RNG neutral
Set 22 - 3 fragmented, 1 disrupted, 5 cohesion tests passed, RNG neutral
Set 23 - 2 fragmented, 1 disrupted, 4 cohesion tests passed, RNG neutral
Set 24 - 7 fragmented, 4 disrupted, 0 cohesion tests passed, RNG favoured Side B (2 frags, 2 disrupted) Side A severely disadvantaged (5 frags, 2 disrupted)
Set 25 - 5 fragmented, 3 disrupted, 0 cohesion tests passed, RNG favoured Side A (1 frag, 2 disrupted) Side B severely disadvantaged (4 frags, 1 disrupted)

Just 8 and a bit more sets to do now.

[MikeC_81]

You can stop now Pete. There is nothing wrong with the code. I am work so I can't post in depth but will do so later tonight

Edit ie how the data lines up with expected values.

[Morbio]

I am sorry, but Pete's argument that games should be closer on average otherwise there is too much RNG is a complete crock, because of the snowballing effect once one player gains an advantage for whatever reason. Mostly when this does not happen it is the Pyrrhic situation where one side wins on one wing and the other on the other wing.

Well, I haven't actually made my argument in that way, have I, Richard? What I have said is that the RNG appears to be impacting on a significant number of matches disproportionately in favour of one player (maybe in around of a third of matches). In the other matches the RNG is fairly neutral. I would have expected some of those 8 war band quarter-final match-ups in the Themed Event to be closer. But the sequence of results was 70-6, 56-20, 47-13, 54-21, 39-7, 40-11, 40-12 and 55-29. So none of them can be described as "close matches" even though the players participating in them were fairly evenly matched.

Unless I'm misunderstanding what Pete is trying to say then this is how it should be. There will be edge cases and sometimes the results will favour one player, sometimes the other and sometimes it will be neutral. If you consider 2 people tossing a coin two times with heads being the winner. If this was repeated a large number of times then on average the result will favour one person with 2 wins 25% of the time, it will be 1-1 50% of the time and will favour the other person 25% of the time. Therefore it is expected that luck will favour one of the people 50% of the time. This is not disproportionate, it is the natural spread of the event. I've used a simple example here to make it easy to understand. If a similar event was held with 2 people tossing a coin 4 times then you'd still have a spread of results and a significant percentage of those result would naturally favour one side or the other. This isn't unusual, this is expected. Indeed, as the number of events increases then it will be less likely for an equal split, but equally the number of edge events of all the results favouring one side would reduce too.

The first thing to state about the data set Pete lists is that it is small, randomness evens itself out over 100's,1,000's or more events, so a set of 8 matches shouldn't be taken as a proof of anything and I'd caution even taking it as an 'indication' of anything as well.

The next thing to consider, and I posted on this previously, is that a wide margin isn't, by itself, and proof of RNG biased luck. There are other factors that come into the equation, e.g. player skill, terrain and army composition to name but three and I'm sure other people can name more. To reiterate one of my earlier posts, I've been in plenty of close battles, whereby one piece of brilliant play, or maybe one mistake can have a catastrophic effect... the line is breached, the flank attacks start, chain cohesion drops follow and a close match ends with a wide margin. This is probably exacerbated by mistakes by me or my opponent (as I'm not a top player and tend to be playing others of similar level) in not having the reserves in the right place.

There are mathematical statistical tools that can analyse cause and effect for sets of data with multiple factors to consider, but to do this needs large data sets. So, unless someone wants to spend weeks simulating impact and melee and trying to control the variables, or better still the developer do something automated, then posting a small set of data and drawing conclusions is pointless.

For sure RNG has an effect, and Mike C has explained numerous times how this can be mitigated, but it isn't the only and probably not the most significant factor otherwise the same people wouldn't be consistently top of the rankings.

[MikeC_81]

You don't need such huge sample sizes. You can plot the expected means of results and CLT tells you that out of whack results will become apparent exceedingly quickly.

Just eyeballing the figure confirms this.

[stockwellpete]

You can stop now Pete. There is nothing wrong with the code. I am work so I can't post in depth but will do so later tonight

Edit ie how the data lines up with expected values.

OK, if I stop now I have done 750 impact combats.

70 led to a fragmented unit (9.3%)
96 led to a disrupted unit (12.8%)
584 led to no cohesion loss (77.9%)

And there were 59 passed cohesion tests out of 225 "red results" (70+96+59) which represents 26.2% of the total. These 225 "red results" constitute exactly 30% of the sample. Set 16 was the most extraordinary sequence with 15 of the 30 combats producing a red result. The lowest number of "red results in a set was 6.

The final thing is that the RNG was most definitely favouring Side B throughout the test.

[stockwellpete]

The first thing to state about the data set Pete lists is that it is small, randomness evens itself out over 100's,1,000's or more events, so a set of 8 matches shouldn't be taken as a proof of anything and I'd caution even taking it as an 'indication' of anything as well.

Yes, I accept that it is a small set of results, but I just used it to illustrate the point I was making previously. I do have knowledge of what happened in 4 of those 8 matches and RNG was most definitely a factor in those 4.

The next thing to consider, and I posted on this previously, is that a wide margin isn't, by itself, and proof of RNG biased luck. There are other factors that come into the equation, e.g. player skill, terrain and army composition to name but three and I'm sure other people can name more. To reiterate one of my earlier posts, I've been in plenty of close battles, whereby one piece of brilliant play, or maybe one mistake can have a catastrophic effect... the line is breached, the flank attacks start, chain cohesion drops follow and a close match ends with a wide margin. This is probably exacerbated by mistakes by me or my opponent (as I'm not a top player and tend to be playing others of similar level) in not having the reserves in the right place.

Yes, of course. But my argument is that sometimes the vagaries of the RNG can overwhelm the skill factor and decide the game. It is obvious to me that this is the case from the statistics I have been compiling today. There were some incredible sequences including 3 fragmented results to the same side in just 6 combats. If you were to suffer that in a game then you've had it, contingency plan or not. But that doesn't mean that I think the code is "broken". It does mean that I try and think about ways in which the game design might mitigate the impact of the RNG. The idea that has come to me in the last few days is that maybe war bands should not suffer a -1 penalty for cohesion tests if they are fighting other war bands. In my sample today, only 26% of cohesion tests were passed by the war bands, which I think is a very low figure. If they did not have a -1 penalty then what would the figure be? And then from that idea about war bands, would it be relevant to other types of shock units fighting each other, and then what about shock units fighting against different type of shock units? I don't know the answers myself. But I don't think war bands losing the -1 penalty for cohesion tests when fighting other war bands would cause any serious knock-on effects elsewhere in the game. After all, their impact and melee characteristics would not be affected when fighting other units.

There are mathematical statistical tools that can analyse cause and effect for sets of data with multiple factors to consider, but to do this needs large data sets. So, unless someone wants to spend weeks simulating impact and melee and trying to control the variables, or better still the developer do something automated, then posting a small set of data and drawing conclusions is pointless.

For sure RNG has an effect, and Mike C has explained numerous times how this can be mitigated, but it isn't the only and probably not the most significant factor otherwise the same people wouldn't be consistently top of the rankings.

Yes, so we agree RNG has an effect. I cannot really see how it can be argued otherwise. We may then just differ on how much of an effect it has. In some games I feel the RNG can be decisive, particularly if it favours one player right at the start, or favours them at a crucial juncture. But I am not making a massive leap from that to then conclude that the RNG is so powerful that it can propel quite modest players to the top of the rankings. This would be to overstate the role of the RNG enormously.

The last bit of your paragraph I do not agree with though. This point is always made when discussing luck or RNG. People say how can it be luck when so-and-so always finishes near the top of the table? My answer to that is simple. They always finish near the top because they are one of the best players - and because luck evens itself out in the long run. There are no permanently lucky or unlucky players.