Decks Used Against Arena with Stats 1.32

[ColorlessGreen]

Double posting AGAIN because I can.

There's a number of improvements I want to make to the study. I had intended to wait until 1.4 and just make the improvements while making the new study, but I'm not really holding my breath on that happening anytime soon, and since the FG study is coming back to life, some of these improvements will help a lot with consistency across studies, so I'm going to make some of the changes now. (Note that "now" means sometime within the next month or two.)


Things I will be changing:
-Instead of just reporting UEI, I will be reporting UEI(333) and UEI(250). What this means is UEI assuming 333 turns are played per hour (i.e. 10.81 seconds/turn) as well as UEI assuming 250 turns are played per hour (i.e. 14.40 seconds/turn). This will permit an easy way for people to say "this deck plays faster than that deck" without having to do any math on their own. Of course, anyone who wants to is still able to convert into actuals by finding their own time/turn for a deck.

-As determined by me, and me alone, decks which I consider to be the same deck will be grouped into one result. Individual modifications of a given deck with their individual tests will be shown in a spoiler under the deck. I will listen to arguments about why your modification adding an extra air dragon makes it a whole different deck, but it's ultimately my decision what qualifies as the same deck. Basically, I'm tired of the top five decks being mono air plus or minus an eagle eye.


Things I want to change but will not be changing now:
-Adding a spoiler under each test with multiple testers showing the exact details of each tester's test, along with the date the test was taken. This will allow people to see what tests go into making up the overall results for a deck, as well as how long ago these tests were done. I think this is of great value, but I do not think it is of enough value for me to go through all >12k tests in the thread again to re-collect individual data. This change will happen in 1.4.


Things I haven't made my mind up about yet:
-Saying a big ol' "Screw You" to democracy and overruling the "how do I sort this thread" poll from a few months ago. WR is an absolutely pointless way to sort things, and I would prefer to sort by SSPH (note that SSPH sort and UEI sort turn out almost exactly the same) since this is what people using this thread (by and large) actually care about. However, before I make up my mind to just throw away the results of the democratic process, I wanted to discuss it a bit. I reserve the right to do whatever the hell I feel like at the end of the discussion, but I will absolutely give full and honest consideration to anyone who wants to argue that we should continue being sorted by WR.

-How to handle the groupings of similar decks. In any case, the best individual deck will be the deck shown. Option 1 is to show the results for the individual test of the best deck and show similar decks in a spoiler underneath. Option 2 is to show the average results across all tests for nearly-identical decks and show all individual results in a spoiler underneath.


Anyone who has any thoughts on any part of this, please let me know. The more input the better. Also, if there's anything else you'd like changed about the way the study operates, please let me know that as well.

Thanks everybody!

[Chapuz]

Things I haven't made my mind up about yet:
-Saying a big ol' "Screw You" to democracy and overruling the "how do I sort this thread" poll from a few months ago. WR is an absolutely pointless way to sort things, and I would prefer to sort by SSPH (note that SSPH sort and UEI sort turn out almost exactly the same) since this is what people using this thread (by and large) actually care about. However, before I make up my mind to just throw away the results of the democratic process, I wanted to discuss it a bit. I reserve the right to do whatever the hell I feel like at the end of the discussion, but I will absolutely give full and honest consideration to anyone who wants to argue that we should continue being sorted by WR.

Sort it however you like, every option is understandable and it won't take long to look for the deck I want.

[rob77dp]

-SNIP-

Things I haven't made my mind up about yet:
-Saying a big ol' "Screw You" to democracy and overruling the "how do I sort this thread" poll from a few months ago. WR is an absolutely pointless way to sort things, and I would prefer to sort by SSPH (note that SSPH sort and UEI sort turn out almost exactly the same) since this is what people using this thread (by and large) actually care about. However, before I make up my mind to just throw away the results of the democratic process, I wanted to discuss it a bit. I reserve the right to do whatever the hell I feel like at the end of the discussion, but I will absolutely give full and honest consideration to anyone who wants to argue that we should continue being sorted by WR.

-snip-

Anyone who has any thoughts on any part of this, please let me know. The more input the better. Also, if there's anything else you'd like changed about the way the study operates, please let me know that as well.

Thanks everybody!

I tried to toot this horn back when you asked for opinions.  Overall win-rate is simply not the goal when grinding.  In PvP, overall win-rate is a STUPENDOUS way to sort your decks.  I fully support the move to SSPH or UEI (as you stated, the sort order is nearly identical for these in Arena results).  100%, fully, and totally support - I like it a lot.
/ranting-and-raving

  Thanks for this change, CG.  (I really do not understand how WR won that pole.  Do. Not. Grok.)

[Shantu]

-How to handle the groupings of similar decks. In any case, the best individual deck will be the deck shown. Option 1 is to show the results for the individual test of the best deck and show similar decks in a spoiler underneath. Option 2 is to show the average results across all tests for nearly-identical decks and show all individual results in a spoiler underneath.

Showing 3 copies of the same deck really is pointless and confusing.. I believe you should list only one deck, and put possible variations in a spoiler, something like:

Speed Poison Kamikaze
[deck]
stats table

Spoiler for Possible variations:

-1 Shard of Sacrifice, +1 Bone Tower
-1 Bone Tower, +1 Arsenic
etc etc

The point is, showing 3 extremely similar deck images where the difference is not apparent is confusing and annoying.
I'd also avoid adding stats for each mods because there will be enough of them as is. If the test results are similar for the mods (say, +/- 10% SSPH or UEI?) I think you could just average them since arena is extremely random.

Sorry for my rambling..

[ColorlessGreen]

-How to handle the groupings of similar decks. In any case, the best individual deck will be the deck shown. Option 1 is to show the results for the individual test of the best deck and show similar decks in a spoiler underneath. Option 2 is to show the average results across all tests for nearly-identical decks and show all individual results in a spoiler underneath.

Showing 3 copies of the same deck really is pointless and confusing.. I believe you should list only one deck, and put possible variations in a spoiler, something like:

Speed Poison Kamikaze
[deck]
stats table

Spoiler for Possible variations:

-1 Shard of Sacrifice, +1 Bone Tower
-1 Bone Tower, +1 Arsenic
etc etc

The point is, showing 3 extremely similar deck images where the difference is not apparent is confusing and annoying.
I'd also avoid adding stats for each mods because there will be enough of them as is. If the test results are similar for the mods (say, +/- 10% SSPH or UEI?) I think you could just average them since arena is extremely random.

Sorry for my rambling..

Things I will be changing:
-As determined by me, and me alone, decks which I consider to be the same deck will be grouped into one result. Individual modifications of a given deck with their individual tests will be shown in a spoiler under the deck. I will listen to arguments about why your modification adding an extra air dragon makes it a whole different deck, but it's ultimately my decision what qualifies as the same deck. Basically, I'm tired of the top five decks being mono air plus or minus an eagle eye.

I am *definitely* solving the problem. I am just not sure about whether to report the best deck with an average of all nearly-identical deck's stats or the best deck with only the best deck's stats (and the other modification's stats hidden in spoilers).


Showing the best deck with an average prevents the problem of one lucky 50 game test modification outweighing a slightly less good 500 game test modification and keeping us constantly playing whack-a-mole with trying to flesh out low-size tests for accurate results.

Code: [Select]

(best deck)
W:450 L:50

spoiler
best deck W:91 L:9
mod1 W:90 L:10
mod2 W:90 L:10
mod3 W:90 L:10
mod4 W:89 L:11


Showing the best deck with the best deck's stats more accurately represents the results of the best deck instead of it being drawn down in rankings by less good modifications.

Code: [Select]

(best deck)
W:91 L:9

spoiler
mod1 W:90 L:10
mod2 W:90 L:10
mod3 W:90 L:10
mod4 W:89 L:11


Unsure which is the lesser evil.


Regarding the tolerance for similar results, if a modification is different enough that it has a significant difference in UEI/SSPH results, that would be different enough to be considered a separate entry.

[the dictator]

First I definitely support sorting by SSPH (or UEI, though rather SSPH).

Regarding the groupings of similar decks, I support that, on the issue of what to list on top (best deck vs. average) I would say:
Use the average, but don't include data for modification that are clearly inferior (in UEI/SSPH that is), this way you have the best of both:
More data for the archetype, thus (hopefully) more accurate data, thus a better comparison to other decks, without someone testing their own (bad) modification dragging down the overal rating of said deck, but without having to make up a new entry in the overal list for the inferior deck modifications.

Basically, the difference between:

Code: [Select]

<Best deck>
W: 450 L:50

spoiler
Best deck - W: 91 L: 9
mod 1 - W: 90 L: 10
mod 2 - W: 90 L: 10
mod 3 - W: 90 L: 10
mod 4 - W: 89 L: 11
end spoiler

<>--<>--<>--<>--<>--<>--<>--<>--<>

<different deck>
W: 400 L:50

spoiler with modifications

<>--<>--<>--<>--<>--<>--<>--<>--<>

<inferior modification of earlier best deck>
W: 200 L: 50

spoiler
inferior deck - W: 80 L: 20
other inferior mod 1 - W: 60 L: 15
other inferior mod 2 - W: 60 L: 15

This one basically starts a second grouping, for the inferior modifications of the first grouping.
Instead I would suggest

Code: [Select]

<Best deck>
W: 450 L:50

spoiler - possible modifications
   Best deck - W: 91 L: 9
   mod 1 - W: 90 L: 10
   mod 2 - W: 90 L: 10
   mod 3 - W: 90 L: 10
   mod 4 - W: 89 L: 11
   nested spoiler - failed modifications (or something less demeaning)
      inferior deck - W: 80 L: 20
      other inferior mod 1 - W: 60 L: 15
      other inferior mod 2 - W: 60 L: 15
   end nested spoiler
end spoiler


[eaglgenes101]

Maybe we try using the power of statistics?

I think it's a reasonable assumption to assume that performances of decks are approximately normal-distrubited against arena AI's we aren't curb-stomping. (For decks we can curb-stomp AI's with, losses are rarities, TTW becomes floored by the limits of the deck, and its payoff distribution becomes skewed. ) 

So we can use ANOVA to determine which decks arre similar performance-wise, and group decks with similar constructions and similar performances together, and report the average of the best group.

[Ungaros]

I think the averages should be displayed for each group unless there is an outlier variant which would be its own group.

Also, I'd love to see the poll done again for rating order (winrate vs UEI vs SSPH). But instead of picking 1 of the choices, perhaps we could rate the choices or pick 2 choices. Because I imagine the people who like UEI also like SSPH and vice versa, but their votes were separated between the two.

[Elbirn]

First off, as I haven't said anything here in months, nor thanked you since you took over, lemme say that handing this study off to you, CG, was probably the best thing possible for this thread. I had no idea what I was doing starting this up, but you've turned it into quite the study. Thankies. [/asskissing]

The only thing I really want to comment on is Win Rate Vs. SSPH Vs. UEI.
At first this post was going to be me going "Hurr just sort by Win Rate, winning more games and winning reliably = more spins and more monet and more and more and more", until it finally clicked; that thing I haven't been getting all this time. Time exists and is a factor. In discussions of efficiency, it matters.

And Lo, the light shined down upon me, and as enlightenment filled my very being, I knew that I had been blessed with knowledge, CAN I GET AN AMEN?!

But yeah. Point of my rambling being my opinion, which is thus. SSPH and UEI rankings would be virtually identical. Like, I can't find a case where they wouldn't be, and thus it doesn't matter.  I think we can all agree that we need only favor one of the two over flat win rate. However I'd like to go on record as supporting UEI. Grinders are searching for Rares, electrum, or both; but when you get right down to it it's about acquiring value. It all comes back to cold hard in the end.

So how do we sort when SSPH and UEI are nearly equivalent? When future users and statisticians peruse this catalogue, their question becomes, "This deck is better than that deck, but by how much"? And to me, "XXXX.XX Electrum per hour" gives a more definitive answer to that question than "X.XX Spins per hour".

[the dictator]

(snip)
But yeah. Point of my rambling being my opinion, which is thus. SSPH and UEI rankings would be virtually identical. Like, I can't find a case where they wouldn't be, and thus it doesn't matter.  I think we can all agree that we need only favor one of the two over flat win rate. However I'd like to go on record as supporting UEI. Grinders are searching for Rares, electrum, or both; but when you get right down to it it's about acquiring value. It all comes back to cold hard in the end.
(snip)

UEI is mostly a function of SSPH, since the rares you spin is where most of your electrum income comes from, but not all of it. A deck with slightly less spins per hour, that has roughly the same winrate, but a better EM rate (more electrum per win), gives a better UEI, but a worse SSPH. This will hardly ever matter, since they will both be quite close, thus sorting isn't that important (2.95 vs. 3 Spins per hour is pretty much equal, because of the errors everywhere else in the calculations/errors in the data). The only case where it could make a real difference is with decks really trying for EM, especially high HP EMs, and especially against platinum and silver. (because those have higher rewards compared the the value of the rares in the special spin slot). Ideally, as pointed out by CuCN, people grinding bronze or silver are after Rares, not Electrum, thus SSPH would be most important there, while people grinding gold or platinum are usually after Electrum, with Rares being a secondary, so UEI would be the best sorting factor here. For consistency across whole thread, we have to chose 1, and my personal feeling is that the upper half of the arena has a closer relationship between UEI and SSPH, thus I would suggest sorting by SSPH, that way they are both as good as possible (barring some 500 hp em decks in platinum section, but people specifically looking for those should be able to find them no matter what the sorting variable is).

I think the averages should be displayed for each group unless there is an outlier variant which would be its own group.

You nailed what I was trying to say in my full post in a single sentence, thanks for that.
So, to use your words to describe my idea: I think the averages should be displayed for each group, unless there is an outlier variant, which would be its own SUBGROUP. The averages of the best subgroup should be displayed, while other subgroups should be listed as part of the SAME entry, since they are still the same 'deck', just badly modified. That way, you won't end up with two entries for the same 'deck', but you also won't have trouble with bad modifications dragging down the ranking of the group as a whole. You could even make subgroupings based on other criteria as well: Mono Air Pure Rushes. Mono Air CC Rushes. Inferior Modifications (3 groupings, first 2 would be included in the averages of the Mono Air Rush group, the 3rd shouldn't).

[Zawadx]

"SuperPsiontal
by Zawadx, tested by Zawadx

Spoiler for deck:

Hover over cards for details, click for permalink

Deck import code : [Select]

61o 61o 61o 61o 61o 61o 61o 61t 61t 61t 61t 61t 61t 622 622 622 622 63a 63a 63a 63a 63a 63a 63a 80e 80l 80l 80l 80l 80l 8pu

50 games tested against Bronze.

General	Speed
Wins: 47 94% Losses: 3 6% EM: 7 14.89%	TTW: 11.72 TTL: 12 Average 11.74

Notes:

Tests were done and forum output was generated with help of the 1.32 arena statsheet."

At present, this deck would be grouped as an Upped deck. However, it is only a fifth of a fully upped deck, and as such can't be expected to compete with them. Could we please create a subsection for semi unupped decks (having upto 6 ups)? These decks are important stepping stones for a newbie, and are closer to unupped decks than upped decks. So, new grouping please (they would only be for Bronze/Silver, so it shouldn't be too hard).

[Keolino]

At present, this deck would be grouped as an Upped deck. However, it is only a fifth of a fully upped deck, and as such can't be expected to compete with them. Could we please create a subsection for semi unupped decks (having upto 6 ups)? These decks are important stepping stones for a newbie, and are closer to unupped decks than upped decks. So, new grouping please (they would only be for Bronze/Silver, so it shouldn't be too hard).

First, for that, we would need at least about 10 tested decks (about 5 for bronze, and about the same for silver), before it would make sence to list them as a seperate group. I thought about this myself already, and gold could be probably added for some decks like semi-upped pestal (2 eclipses, 2 vampires). Anyway, because of a lack of motivation as well as lack of viable decks which would be worth testing I never went to doing it myself.