Some Statistics from Arena's Leaderboard

[suxerz]

Ever since arena been implemented, I've been really curious to see the distribution of games played for each deck. Now that zanz has implemented the leader board, I've decided to compile the data and thought I would share this here. I've compiled the data from all 500 ranks in platinum league. If anyone is interested, I put the raw data in this googlesheet (https://spreadsheets.google.com/spreadsheet/ccc?key=0ApI-50p2LmlcdFUzUDhDdFB1bVFnSG5pbkt4YzR0WlE&hl=en_US). Some data might be missing due to the rank refreshed when I was gathering the data.


I'm not really expert in this statistic stuff but I think this should be a good summary to have an overview of the data. It shows the relation between the number of games played with the deck age and rank as well.

Bronze League

_{*(taken at appox. 7/16/2011 21:00 GMT)}

Silver League

_{*(taken at appox. at 7/16/2011 21:30 GMT)}

Gold League

_{*(taken at appox. at 7/16/2011 22:00 GMT)}

Platinum League

_{*(taken at appox. at 7/16/2011 06:00 GMT)}



From the above, noticed how the number of decks in each row scattered. I found this to be quite interesting, so this table below might give a better focus. I focused on the gap between the minimum and maximum games and grouped it by deck age. (Red -> Yellow -> Blue represents the data scales from highest to lowest difference)

Bronze League

_{*(taken at appox. 7/16/2011 21:00 GMT)}

Silver League

_{*(taken at appox. at 7/16/2011 21:30 GMT)}

Gold League

_{*(taken at appox. at 7/16/2011 22:00 GMT)}

Platinum League

_{*(taken at appox. at 7/16/2011 06:00 GMT)}


Hence, I'm wondering if this sample justify that some decks got insanely more hits than the others? i.e. RNG clearly favours these guys . Maybe I'm just being paranoid, but if this imbalance really does exist, is there a "fairer" alternative formula that zanz should use rather than total RNG. On the other hand, probably the point that I'm missing lies in how many people actually play the arena per day (specifically platinum league for this case).


_{*edit: added the rest of the leagues' stats.}

[wavedash]

Very cool stats, thanks.

I think there's something wrong with the frequency that day-zero decks come up in the Arena. It is incredibly slow getting a deck going, and statistically speaking, it is a very real possibility to only get 3-4 plays on your first day, as it has happened to me time and time again.

[Essence]

I think it has more to do with what time of day you submit your decks in than much else -- on day zero, that is.  the fact that there is, somewhere, an 10-day-old deck that's gotten less than 4 plays per day, that's just crazy.  OTOH, a 1-day old deck with 49 plays is just batsh!t insane, too. 

Personally, I think it would be cool (and appropriate) if the formula to determine which deck got played against at any given point in time was based on how long it's been since the deck was played last.

[zanzarino]

Uhm... good job, but platinum really is the wrong place to start with statistics.

First of all, it is the only league that is not effectively T500 yet. It still is T450, I think; that means that a deck that falls to, let's say, rank460 is already out and it won't get picked anymore.
Platinum also has still a few old decks from the close-to-beta times when platinum had only 250 decks or less.; those decks got lots of games if in the top 200, no games if the rank was >250.

In a few words statistics for decks older than 7 days mean nothing because the platinum league was - ans still is - transitioning.

Try another league, or try platinum again in a few weeks, it should be much more uniform.

[suxerz]

Thanks for the reply zanz. Yeah, I agree platinum probably won't be a good place to do this, yet. It was my first choice since I'm in that league.
I'll try other league next.

[suxerz]

^{[shameless bump!]}

Since I'm quite free today and the next couple of days, I would like to revisit this stats again. I'm really interested to find whether it's true that higher ranking decks get more hits than the rest of the decks in each league. Sort of a Mythbuster mission - whether it's a myth, fact or plausible.

However, I'm not entirely confident in my analysis, thus I would like to hear some feedback on this (especially from those math/stats wiz).
Bear in mind that I'm not looking for anything that's too complicated - My only tools will be MS excel.

To make things a bit easier to understand, I'll be referring to this googlesheet (https://spreadsheets.google.com/spreadsheet/ccc?key=0ApI-50p2LmlcdFUzUDhDdFB1bVFnSG5pbkt4YzR0WlE&hl=en_US) on some points below. You should be able to play around with the last tab (pivot table), but I really hope no one will "bork" it.

Okay, so here goes my wall of text. >.<

^{[start wall of text]}


DATA POOL
In order to get a good result, stats from 1 day is not enough. I haven't decided how large the data pool should be (how many days), but I think it should be around 3-4 days.

GATHERING DATA
Since the leaderboard only shows the total games of each deck, I need to figure the exact number of hits for that particular day. To get these numbers I need to collect the data for every 24 hours; i.e. I collect the first data on 1st Aug at 12:00 GMT, the second on 2nd Aug 12:00 GMT, etc. Bottom line, the time should be specific.

FORMULA
Next, is the formula. Specifically, how to calculate the exact number of hits for that particular day. The basic formula is fairly easy; i.e. I just need to calculate total hits from Day 2 -- total hits from Day 1. To anyone who is familiar with excel functions, I'm using the vlookup function (see Tab 2, column L. "Day_1" is a named range referring to Tab 1)

On the other hand, there are some challenges/obstacles that I have identified so far:

User can reset his deck; e.g. in Day 1 the user has a total games of 10, but he reset his deck in Day 2 and got 5 hits on that day.
Solution: I can't use the basic formula. I would just need to take "5" as the total number of hits for Day 2.
Similar to above, users who were not in the leaderboard in Day 1 can submit a new deck in Day 2 and enter the leaderboard.
Solution: same as above.
The time when a deck was reset/submitted. These can happen within the 24 hours. I'm not sure whether this would have any significant effect or not. Also, I don't think there's a way to overcome this "challenge".
Solution: dunno... >.<

Based on the above, I've concluded as per below. You can also find the formula in Tab 2, column M. _{(btw, this is not a psedocode. I'm not a programmer >.<)}

   If deck's age=0 (deck reset) OR name not found in Day 1 (new user in leaderboard),
      Take total number of hits for Day 2
   Else
      total number of hits for day 2 - total number of hits for day 1



RESULT
Displaying the result is fairly easy using Pivot table. The tricky part is to figure out which are the most relevant values. For starters, I'm going with rank vs total number of hits. I'm displaying the rank in range of 1-24, 25-49, etc. Then, I have min, max, sum and average of hits within the rank range.

Below is the result for Silver league. The data was taken at around 5:00 GMT July, 24 . So, I can say that from 5:00 GMT July, 23 till 5:00 GMT July, 24, people around the world played a total of 5,956 games in Silver league.


You can also view this in the googlesheet link I've provided earlier. Take note that the pivot table option may take a while to load. Make sure you select on of the cells in the table and wait. The only filter you should change is the League as shown below.


_{[end wall of text]}
Again, I would very much appreciate any feedback about this, specifically the analysis/method/math/formula/orwhutevaitis that I mentioned above. Also, I would rather we do not begin to discuss the result from this one day result... yet.

[Essence]

I think that this is a great project, but we need data from ~4-9 more days before we can start making confident judgments.  That said, from the data you've shown here, it certainly looks like the RNG is heavily predisposed to favoring the top 150 decks, and slightly less so the decks ranked below 350.

[suxerz]

Yeah, 4-9 sounds better. Though it might not be really important, I'm thinking it'll be best if I could get the data when less people play arena, so that the leaderboard won't change much while I'm copy-pasting. Any idea when is the "ideal" time?

But more importantly, I need to make sure whether my method is correct. So, does anyone see any problem with it?

[zanzarino]

Here I am again messing with your good work

You are calculating "rank vs total number of hits" which is useful, but only if there are 500 decks total.

The problem is that you are not thinking in four dimensions (the top 500 decks change over time)
There are actually about 2,000 decks in silver league, only the top 500 get to play against users. Decks that are not very successful get a few plays and are kicked out of T500: those "hits" disappear from your statistics.

That means that the decks in the bottom part of the leader board actually get the same amount of hits/hour, but are more likely to be kicked out of top500.

The "age vs total number of hits" thing that you did before is actually more useful.

[suxerz]

Tee hee.. not a problem, zanz. At least you saved me from doing this 7 days straight. 

Yeah, I guess you're right. I identified the time when a deck was reset/submitted as a potential problem but didn't realized it has bigger effect. I believe this also prove why the mid ranks (200-300) in all four leagues have significant lower "hits" (especially bronze and silver) - they were pushed out of the ranks as soon as they get few losses without any wins.

Erm.. now I'm not sure what I should do. Maybe I should just do the "age vs total number of hits" thingy.

[FSK]

The Leaderboard should also show the Oracle card spun, in addition to the mark.

I suspect that most of the top decks have a lucky spin.