Some statistical gymnastics so the zero-drop-gang can also move on

Seeing the large commotion that was caused by zero-drop situation on the forums/reddit, I became a bit curious as to how legitimate the whole uproar really was. Best thing to do in such cases is try to simulate what happened, and see whether the numbers match.

Some poll was spread around to gauche how many people got zero drops, which had some interesting results: https://docs.google.com/forms/d/e/1FAIpQLSem7DHcmGhx8kNX2dSakbOGm8dHzoN2ya2r6zW23nmeKfQQZw/viewanalytics

Now, I would say that people who received zero drops and watched many hours are probably more likely to vote on this poll than people who received a few drops, simply because they are more upset, so in this sense the poll could be somewhat biased, though it's not really clear by what extent.

So I decided to simulate the experiment in Python, and gather some resulting statistics from it. You can check this out in this jupyter notebook on google's colaboratory platform (very cool stuff btw!): https://colab.research.google.com/drive/1-dZUJwHc734mgwKCNqqipt8HHkmQcO_N

It contains some comments about what assumptions were made, but the TLDR is this: We assume 12000 people were eligible for drops (this is what I heard, feel free to correct me, it's very easy to change in the experiment), then we try to estimate the hours that users have watched by fitting an appropriate distribution to the poll's hour numbers (which will be biased towards the zero-drop/8+hour folks) and sampling it, then we perform Bernoulli trials (aka biased coin-flips) to simulate the amount of drops users got based on those sampled amounts of hours watched per user.

I've attached the results from this to the thread in the form of images, but you can also see these results in the notebook I linked (and even play with them if you copy it over to a notebook of your own, if you re-run it, it might show slightly different results).



As you can see, quite a few people could potentially end up without any drops, roughly around 17-18%, which amounts to 2000 people if we assume 12000 people were eligible for drops via the twitch linking procedure (which is a rather small part of the overall audience that viewed the stream). 38.7% of the 248 people that voted on the poll had zero drops, roughly around 100 people. Likely half of these cared so much about the issue that they felt like creating a lot of noise, aka a very loud but small minority (as usual with similar outrages), but these numbers do match what is visible on reddit and the forums.

Now clearly the poll shows that 38.7% of the people voted had zero drops, while the experiment finds a value of around 17-18%. This is very likely caused by bias that the poll intrinsically generates, because people who have watched many hours and received no drops are far more upset and thus far more likely to vote on a poll like this. Another factor could be that some people may have thought they had linked their account, but did not actually do this properly. This 38.7% is not a good indication anyway, because the sample size is incredibly small.. Around 250 people voted on the poll, which is tiny compared to the 12000 people who were eligible for drops. The experiment also shows that the largest chunk of people probably got around 1-2 drops. When I initially did not consider hours watched and automatically assumed everyone had watched the full 16 hours, there still was good chunk of 5-6% of all 12000 people who would not get a single drop.

Obviously there could've also been something wrong with Twitch, which means that this 12000 number should've been larger in the first place. Unfortunately this can only be confirmed by Twitch/GGG.

Anyway I hope people will see that it's not all that weird to not receive any drops, it's unfortunately the nature of randomness, and as ARPG fans you should be all too familiar with that :) If any statistics experts have comments on the validity of this experiment, feel free to share them as well.

Yes, I am procrastinating a thing or two....

EDIT: I've updated the script to take into account the higher drop chance on the second day (just averaged the drop chance over both days), check the new plots in the notebook.

This experiment, noble as it is, fails to capture the most important part of all of this: the complainers don't want to understand probability, they want to be outraged.

Not an experiment you don't have a hypothesis you can test.

Also your numbers are incorrect. Drop chance was doubled second day.

"

mightybirdPOE wrote:

Not an experiment you don't have a hypothesis you can test.

Also your numbers are incorrect. Drop chance was doubled second day.

Well it's just a simulation though, more or less giving an overview of how the drops would be distributed after the event. Could you elaborate on why a hypothesis would be necessary here?

For the second point, it's hard to exactly find out which portion of the people watched that second day. If we just average the drop chance over the two days, the zero drop chance becomes 8-9% ish, so it does have a significant impact.

EDIT: Now that I think of it, I could do something with a hypothesis to show the statistical significance of the differences between the poll and the simulation.

"

innervation wrote:

This experiment, noble as it is, fails to capture the most important part of all of this: the complainers don't want to understand probability, they want to be outraged.

Probably the truth unfortunately, but it seems to be a minority in the global scheme of things.

"

innervation wrote:

the complainers don't want to understand probability, they want to be outraged.

Perfectly stated.
Great job OP, though I imagine it will fall on deaf ears, as Innervation points out.

nah wont move on because it's a funny meme.

but you can keep trying.. the more you try to make people move on it will be worse but hey keep it up it might get mentioned every exilecon every year..

"

Xystre wrote:

nah wont move on because it's a funny meme.

but you can keep trying.. the more you try to make people move on it will be worse but hey keep it up it might get mentioned every exilecon every year..

Well it was kinda hilarious to see the chaos erupt, admittedly :D

Was mostly just curious about the numbers that could be behind it.

I do think one mistake from GGG was to mention anything about an average viewing time to get a drop, because averages are really bad at describing distributions, so it may have given a lot of false impressions.

"

DoubleU wrote:

"

innervation wrote:

the complainers don't want to understand probability, they want to be outraged.

Perfectly stated.
Great job OP, though I imagine it will fall on deaf ears, as Innervation points out.

Thank you!

NOT UH I DESERVE DROPS FOR LITERALLY DOING NOTHING :_; GGG'S FAULT WAAAAAAAA

"

Xystre wrote:

nah wont move on because it's a funny meme.

but you can keep trying.. the more you try to make people move on it will be worse but hey keep it up it might get mentioned every exilecon every year..

The name '0 drop gang' is genuinely funny in a silly/comical way - specifically because of the absurdity of using the utterly insignificant common cause of 'we didn't get free pixels' to form a group generally considered significant insofar as gangs tend to be powerful/feared.

The name is great.

The rest of the humor lies in laughing at these people though, not with them.