Why Cronk's Test is Inconclusive - The Stats

Quite a bit of argument going on in this thread: http://www.pathofexile.com/forum/view-thread/292103/page/1. Someone stated those who disagree with poor stats should show their own evidence. OP stated those who wanted to disagree should start their own thread. So here goes:

OP dropped x number of items and stated that it's impossible for witches to drop so few items so it can't be RNG. I really don't know who's right or who's wrong, but I do know more testing needs to be done. Why is his test inconclusive? Because I believe his sample size is too small.

In order to prove this to myself, I first set out to replicate his test on excel, and this is something you can easily do yourself too. Assuming x drops and assuming equal weightage between classes (represented arbitrarily by 1 - 6), each drop should be: =RANDBETWEEN(1,6)

Repeating this test x times either via macro or just pulling the formula down, I would have replicated this test in an environment with equal weightage.

Since I don't know the weightage for what constitutes as "Neutral" drops per his test, I've used his only set of data which doesn't include neutral drops which I've set out below:

+++++++++++++++
97 items found (25.5% of all Equipment items)

Str - 15 (4%)
Str/Dex - 17 (4.5%)
Str/Int - 15 (4%)
Dex - 23 (6%)
Dex/Int - 17 (4.5%)
Int - 10 (2.5%)
+++++++++++++++

So, repeating my RANDBETWEEN=(1,6) 97 times on my excel, on the very first try, I get the following:

FIRST TRY
Class 1 - 11
Class 2 - 12
Class 3 - 18
Class 4 - 18
Class 5 - 17
Class 6 - 21

Already in my first try, one of the classes outperforms the rest, class 1 appears 11 times which is very close to Cronk's Int result of 10 times.

SECOND TRY
Class 1 - 16
Class 2 - 18
Class 3 - 10
Class 4 - 18
Class 5 - 10
Class 6 - 25

Woo hoo! Now 2 classes are being nerfed badly... GGG doing some really bad stuff with RNG huh...

THIRD TRY
Class 1 - 10
Class 2 - 16
Class 3 - 22
Class 4 - 19
Class 5 - 15
Class 6 - 15

Yet again, my uniform distribution is not uniform in my test of 97 class only drops. Yet again, 1 class severely outperforms the rest.

This, my friends, is RNG in action.

Cronk, I appreciate the work you're doing, but I think your test remains inconclusive and I believe you shouldn't state that your sample size is alright given the above.

As much as I admire your satiristic, but understandable premise, TL;DR, I have to say that you have misinterpreted Cronk's formula already by assuming premature rolls with no governing mechanisms supporting it upon a third layer of factors!

Layer 1 : RNG

Layer 2 : Distribution

Layer 3 : Alternating the drop rate formula within the RNG, within the distribution, accordingly.

Good job of showing him your point about the sample size.

Oh boy, I can't believe I'm getting into this. But, here goes:

You're ignoring the fact that (first list below) pure int items exist in a much smaller quantity than say, Str/Dex. His actual drop rate (second list) isn't that far off from the potential item list. Based on this, their RNG is fine. What's not fine, is that there are only 19 int items dropping from 1-23. I'm not sure how to express this properly, but I'm trying to say that because the items seem to be weighted evenly and because there are fewer "witch" items, witches get fewer opportunities at drops.

I think that's what hes trying to say.

"

Up to required-level 23 there are 194 Equipment Items that have a chance to drop. The class distribution of those items is as follows:

Str - 27 (14%)
Str/Dex - 34 (17.5%)
Str/Int - 24 (12.5%)
Dex - 31 (16%)
Dex/Int - 24 (12.5%)
Int - 19 (10%)
Neutral - 35 (18%)

So you can see that the Witch is the most under-dropping character before you even analyse the 381 actual Equipment drops, which were as follows:

Str - 53 (14%)
Str/Dex - 91 (24%)
Str/Int - 55 (14.5%)
Dex - 66 (17%)
Dex/Int - 38 (10%)
Int - 33 (8.5%)
Neutral - 43 (11%)

"

Tabx wrote:

I'm not sure how to express this properly, but I'm trying to say that because the items seem to be weighted evenly and because there are fewer "witch" items, witches get fewer opportunities at drops.

I think that's what hes trying to say.

"

Up to required-level 23 there are 194 Equipment Items that have a chance to drop. The class distribution of those items is as follows:

Str - 27 (14%)
Str/Dex - 34 (17.5%)
Str/Int - 24 (12.5%)
Dex - 31 (16%)
Dex/Int - 24 (12.5%)
Int - 19 (10%)
Neutral - 35 (18%)

So you can see that the Witch is the most under-dropping character before you even analyse the 381 actual Equipment drops, which were as follows:

Str - 53 (14%)
Str/Dex - 91 (24%)
Str/Int - 55 (14.5%)
Dex - 66 (17%)
Dex/Int - 38 (10%)
Int - 33 (8.5%)
Neutral - 43 (11%)

Fucking /thread

"

Tabx wrote:

Oh boy, I can't believe I'm getting into this. But, here goes:

You're ignoring the fact that (first list below) pure int items exist in a much smaller quantity than say, Str/Dex. His actual drop rate (second list) isn't that far off from the potential item list. Based on this, their RNG is fine. What's not fine, is that there are only 19 int items dropping from 1-23.

If he was including weapons in this list, which I believe he was, it seems fine to me. If we're thinking along the stereotypical class lines of strength = melee and int = caster, then it makes since there there are far fewer caster weapons available than melee weapons.

Why?

Simple. Casters don't rely entirely upon their weapon to do damage. Melee characters do.

Consider something like an FP witch, who gets her base damage from skills, some bonuses from gear, some from passives, and so on. She doesn't have to hunt high and low for a good wand in order to be useful. To a physical cleave marauder, a good weapon can make all the difference in the world, so it makes sense to throw more weapons at him. He needs them.

"

realatomicpanda wrote:

If he was including weapons in this list, which I believe he was, it seems fine to me. If we're thinking along the stereotypical class lines of strength = melee and int = caster, then it makes since there there are far fewer caster weapons available than melee weapons.

Why?

Simple. Casters don't rely entirely upon their weapon to do damage. Melee characters do.

Consider something like an FP witch, who gets her base damage from skills, some bonuses from gear, some from passives, and so on. She doesn't have to hunt high and low for a good wand in order to be useful. To a physical cleave marauder, a good weapon can make all the difference in the world, so it makes sense to throw more weapons at him. He needs them.

Hello again, RAP,

I think you are missing the context Cronk was heading after in the very beginning: The lesser quantity of Witch items reduces the drop rate, regardless if you're depending on wands for damage or not. It decreases the drop rate for the weapon (you still want a good wand, no?), and so as you level, and you do more than 4 places as Cronk did, the numbers will remain the same, as in, you gain much more of everything else besides those wands, which is the entire point behind Cronk's research -- to reveal the drop-rate-bias in RNG when it comes to Witch gear.

Deceptionist, perhaps my point was misunderstood. The premise of Cronk's argument is that his test should show equal weightage if there's no biasness in POE's RNG for item drops. I merely was stating that even in a real bias-free environment, drops can still favour one over the other by mere coincidence, ie RNG.

Also Tabx/Deceptionist my test only replicated Cronk's third set of data where Int actually has more items than the hybrid classes yet he turned out lesser drops. So the point about Int having fewer weapons etc doesn't really factor in here (though that would definitely make for a good discussion in another thread)

"

dlrr wrote:

Deceptionist, perhaps my point was misunderstood. The premise of Cronk's argument is that his test should show equal weightage if there's no biasness in POE's RNG for item drops. I merely was stating that even in a real bias-free environment, drops can still favour one over the other by mere coincidence, ie RNG.

Tabx, my test only replicated Cronk's third set of data where Int actually has more items than some of the other classes yet he turned out lesser drops.

Cronk did not say that in an unbiased formula, the generated numbers would end up with the same results per variable; Cronk was explaining that there is a probabilistic nature behind [i.e 1,2,3,4,5,6], and so upon that probabilistic nature, there will be (on average) a very distinctive pattern that will emerge upon rolling each number for, say, 100x.

You misunderstood Cronk's premise. That was when he was explaining against those who cried "small sample". Cronk's formula goes beyond a random number generator, but the very mechanics that govern the random number generator in PoE, while also notifying everyone that the smaller amount of Witch gear is what causes the RNG to be unfavorable towards the Witch.

It's not RNG. It's the mechanics that govern the RNG.

In his third test he specifically showed Int had more items than hybrids. Your bolded statement misses that.

You're using different words to say the same thing. You're saying it should show a pattern. I've shown that it doesn't always have to be the case. I don't think I'll be replying to you on this anymore (though of course I might change my mind later) as it's clear neither of us would budge.