*Author

Offline Zawadx

  • Legendary Member
  • ******
  • Posts: 3197
  • Country: aq
  • Reputation Power: 53
  • Zawadx brings all the vitality and activity of a Life Nymph.Zawadx brings all the vitality and activity of a Life Nymph.Zawadx brings all the vitality and activity of a Life Nymph.Zawadx brings all the vitality and activity of a Life Nymph.Zawadx brings all the vitality and activity of a Life Nymph.Zawadx brings all the vitality and activity of a Life Nymph.Zawadx brings all the vitality and activity of a Life Nymph.Zawadx brings all the vitality and activity of a Life Nymph.Zawadx brings all the vitality and activity of a Life Nymph.Zawadx brings all the vitality and activity of a Life Nymph.
  • Officially Eisoptrophobic
  • Awards: Slice of Elements 8th Birthday CakeSlice of Elements 7th Birthday CakeWeekly Tournament WinnerCompetition - Won War as a Worthless WarriorWeekly Tournament WinnerSlice of Elements 6th Birthday CakeWar #8 Winner - Team FireBrawl #3 Winner - Divine LightSlice of Elements 5th Birthday Cake
Re: [1.32] False Gods Stats Thread https://elementscommunity.org/forum/index.php?topic=53573.msg1126606#msg1126606
« Reply #60 on: February 18, 2014, 09:23:40 am »
Also in this case TTL has minor impact, cause many of your losses are actually skips. Only FGs which can be beat regularly are added. This means that the differences in TTL and TTW have minor effect, as there are few actual 'losses'.

Note that UEI was made to compare decks, and as such was designed to as objective as possible. Playstyles could really mess that up.
When you play the game of thrones, you win or you die. There is no middle ground.

Offline CuCN

  • Hero Member
  • *****
  • Posts: 1756
  • Reputation Power: 25
  • CuCN is a proud Wyrm taking wing for the first time.CuCN is a proud Wyrm taking wing for the first time.CuCN is a proud Wyrm taking wing for the first time.CuCN is a proud Wyrm taking wing for the first time.CuCN is a proud Wyrm taking wing for the first time.
  • Toxic
  • Awards: Slice of Elements 5th Birthday CakeSlice of Elements 4th Birthday Cake
Re: [1.32] False Gods Stats Thread https://elementscommunity.org/forum/index.php?topic=53573.msg1126607#msg1126607
« Reply #61 on: February 18, 2014, 09:35:49 am »
Also in this case TTL has minor impact, cause many of your losses are actually skips. Only FGs which can be beat regularly are added. This means that the differences in TTL and TTW have minor effect, as there are few actual 'losses'.

Not true, because TTL is important in determining which FGs should be skipped and which should not.

Offline Keolino

  • Sr. Member
  • ****
  • Posts: 553
  • Reputation Power: 6
  • Keolino is a Spark waiting for a buff.
  • +/Gl!it)c?h%§"#
Re: [1.32] False Gods Stats Thread https://elementscommunity.org/forum/index.php?topic=53573.msg1126609#msg1126609
« Reply #62 on: February 18, 2014, 10:24:13 am »
Not true, because TTL is important in determining which FGs should be skipped and which should not.

That also means, that you will have to skip a special god (or not), solely dependent on the question if you play the god till you lose or quit turn four or five, because you were gonna lose anyway. That means, that the optimal skiplist changes with each person and even changes with your mood (do I feel like playing long losses which may be a win with 0-5% chance right now or will I get to the next god?)
What's the speed of dark?

Offline Jangoo

  • Sr. Member
  • ****
  • Posts: 877
  • Reputation Power: 0
  • Jangoo hides under a Cloak.
  • New to You
Re: [1.32] False Gods Stats Thread https://elementscommunity.org/forum/index.php?topic=53573.msg1126613#msg1126613
« Reply #63 on: February 18, 2014, 11:08:09 am »
Keolino, as Zawadx already has pointed out, what you are describing as "losing" is actually "skipping".
Losing = getting bashed down to 0 HP
Skipping = making an educated guess and giving up
This issue has always been a weak point in these studies. So far there have been two ways of dealing with it:

1. No skipping while testing
Testers are not to skip at all but play it out to the death hence generating actual TTL-values.
The asumption here is that as long as you are not actually dead, you can still win.
Thinking about current top-decks like PoisonDials or Rol/Hope, this often enough holds true: Just before you die, you do draw that Fractal to beef up your shield. Or you do draw that sundial and it doesn't get deflagged, hence starting a chain of cards on which you ride to victory.
Problems: Obvious instances of skipping drag down the deck. E.g. if the FG just miracled and there is no way that he could die of poison before you deck out, then why on earth should you be forced to sit there and roll in 10 more losing-turns?

2. Educated skipping
Testers are trusted to know their deck and the FGs so that they can make that educated guess.
Groundrules for skipping based on common knowledge of deck- and god-performance are commonly set in the framework of a study.
Problems: Testers may (must lol) not be trusted because their choice is too subjective.


The inverse statement of recording TTW but not TTL because TTL is too subjective is that "winning is not affected by the player", that it is "objective".
Perhaps this is not as problematic as it used to be because modern decks have a much more straightforward gameplan. So it is true: The TTW will probably not vary all that much. But still: E.g., apparently there are players that consider Rainbow not worthwhile with PDials while others state that they frequently win ... how is that possible if winning if objective?
Most importantly when it comes to winning, what does happen frequently is that players blow it and turn a possible win into a certain loss! So we are not even talking about a couple turns more or less to win. This instance is simply highly subjective, human error with a vast impact on deck-performance.

As CuCN has pointed out, the TTL will then have an impact on the skip-selection.
An error that is completely unknown in its extent (beacuse no stats were taken) will likely be reproduced and multiplied in this process: A deck that most certainly does have much shorter TTL than TTW might get kicked out in favor of a deck that does have much longer TTL than TTW. Why? Because for both decks TTL=TTW has been assumed.
Assuming that TTL=TTW for reasons of eliminating subjectivity is kind of like pitting different race-cars against each other, having different people drive them, assuming that this does not matter because when it comes to winning they will all do equally well and then finally disallowing the usage of gear 7 and 8 because not all drivers might be able to handle a 7th or 8th gear. ;)


Now Keolino, you were asking me for clear solutions. I don't have any other solutions than the two approaches described above which worked out satisfactory when conjoined.
In the older FGei-thread I had already had a talk with CG in order to find out how this system would work and I get the idea behind this approach now and hope it works out well. Eliminating actual time loses a lot but it also wins a lot. Personally, I still think that eliminating subjectivity is simply not entirely possible as long as there actually are subjects (humans) testing the decks. So CGs mission statement to eliminate it "as much as possible" (my read: some things just aren't possible) would be crucial here: TTW vs TTL is certainly a critical, objective aspect of a decks performance. Hence, eliminating it for the sake of objectivity defeats the purpose.


« Last Edit: February 18, 2014, 11:11:44 am by Jangoo »

Offline Keolino

  • Sr. Member
  • ****
  • Posts: 553
  • Reputation Power: 6
  • Keolino is a Spark waiting for a buff.
  • +/Gl!it)c?h%§"#
Re: [1.32] False Gods Stats Thread https://elementscommunity.org/forum/index.php?topic=53573.msg1126616#msg1126616
« Reply #64 on: February 18, 2014, 11:47:35 am »
Because skips are completely excluded from the data, I have to be strict: a skip is only a skip for the purpose of this study if it's based solely on the name of the opposing god. Quitting because of a misplay or a bad starting hand counts as a loss.

^^from the OP

So what you said about "skipping" and "losing" isn't true. Or, at least, not in this thread.

About your first option: No quitting before dying? So you tell testers that they can't quit a false god they are losing to anyway. Specially with Poison Dials, those gonna be terrible games. Thinking of fighting Ferox with 3 empatic bonds and 5 creatures already out, and you with currently dealing 12 damage (growing because of arsenic) Average-he plays one creature per turn, so gets three more regeneration. Now, there can be an average of 15 turns left which I still have to play till are actually die. (Asuming that I am getting not pissed enough to just stop playing Sundials and SoSacs to die faster)... Right now I like collecting stats because I can play pretty normal, with quitting once there is no hope to win left. If I would have to play all those extra turns for my stats to be accepted, then I'd rather record no stats at all. (Actually, I would still record, but probably not as much as now) There are already not many testers as it is now, and even if I may stay, other may stop recording, and it would get more difficult for potential new testers to like recording stats and stay motivated.

Further now you have those TTL, right. Time to calculate. Fine. finished. Now somebody plays, and he quits if he isn't gonna win. Then what did I calculate the TTL for, if no player in a normal game actually plays till he actually lost??


About the second option, as I said already, that wouldn't work because different players stop at different turns, and it would get pretty subjective.
What's the speed of dark?

Offline Leodip

  • Sr. Member
  • ****
  • Posts: 786
  • Country: it
  • Reputation Power: 11
  • Leodip is taking their first peeks out of the Antlion's burrow.Leodip is taking their first peeks out of the Antlion's burrow.
  • Go home Homura, you're drunk.
  • Awards: Slice of Elements 5th Birthday CakeSlice of Elements 4th Birthday Cake
Re: [1.32] False Gods Stats Thread https://elementscommunity.org/forum/index.php?topic=53573.msg1126630#msg1126630
« Reply #65 on: February 18, 2014, 02:54:53 pm »
Damn, playing melancholy, I'm hatin' it. I'll finish the tests (3 per god, I'll do tons of skipping in order to test the ones I lack) and then forget about the deck. Never, ever gonna play it. It's not even a valid deck for a FG prediction, I'll just delete the code.

Nothing against the creator of the deck, it's only that I hate decks THAT susceptible to starting hands.

Offline the dictatorTopic starter

  • Hero Member
  • *****
  • Posts: 1261
  • Country: nl
  • Reputation Power: 17
  • the dictator is a Blue Crawler starting to think about his first run.the dictator is a Blue Crawler starting to think about his first run.the dictator is a Blue Crawler starting to think about his first run.
  • There are no dumb questions, only dumb answers
  • Awards: Slice of Elements 5th Birthday CakeWeekly Tournament Winner
Re: [1.32] False Gods Stats Thread https://elementscommunity.org/forum/index.php?topic=53573.msg1126632#msg1126632
« Reply #66 on: February 18, 2014, 03:00:09 pm »
First of all, could you please stop referring to abandoning a lost game as skipping. For the purposes of this thread, a skip only refers to quitting a game at the stats, based on the name of the opposing god (as pointed out by Keolino) and I wouldn't want people to get confused.

What Zawadx meant, wast that a god with a high amount of losses will end up marked as a skip in the recommendation, so the duration of those games doesn't matter afterwards, because they are set to 1 turn (or whatever turns to skip is set at).
Though, as pointed out by CuCN, the turns to lose does influence the place where it ends up in the skip order, so it might change the recommendation in the first place.

All right, now that that is clarified:
Yes, your point is valid, factoring in turns to lose would increase the accuracy of the tests, IF we can figure out a meaningful reduce tester bias from the TTL recorded.

Your point is that winrate and turns to win are not objective either, so there is no reason record those and NOT TTL. But I do disagree with you here: to get the most objective measurement possible, one should try to reduce the subjectivity in the data wherever possible. Currently there isn't a more objective way to get more accurate winrates are TTWs aside from getting much more testers playing a lot more games per deck (I will get back to this later).

Turns to Lose on the other hand is much more dependent on play style, as pointed out already. The only way to get accurate data on this would be restrict the style of all testers, to ensure the data from the different decks and different testers is accurately comparable, which raises a few other problems:

First, what would be the 'correct' style. Playing until 0 hps, or until you 'know' you are going to lose, there is good arguments for both: the former because it's less subjective, the second because it more accurately represents the behavior someone farming.
Second, how to classify that style. Playing until 0 hps can be influenced my making 'mistakes' during play, like forgetting to play the next sundial/SoSac with PDials. Playing until you know the match is lost is even more questionable, because how would 'lost' be classified: until you are absolutely certain you can't win (aka, a miracle played with not enough damage out until deckout to try again), note, this hardly ever happens, at least for most decks. So you want to classify how to be 'reasonably' certain, after which subjectivity from testers comes in again, and much larger subjectivity than in the TTW.

The real problem though, is what Keolino pointed out last: I want to make the data gathering as easy and unrestricted as possible, to (hopefully) maximize the number of testers and tested games, which should help reduce tester bias, and yes, that would include reducing errors in TTL tracking. But, the way I see it, the real problem with any restriction imposed during testing will likely put of potential testers.

The way I have the study currently set up (or at least tried to), is that someone farming can take stats as a side activity, without having to change their preferred play style or skiplist, and the data is completely usable. The only limitation is that a deck or modification needs 3 played games against each god to get they own separate listing, but stats without those 3 played games are still included in the overall data and count towards the stats of the modification and deckgroup, the modification (or deckgroup) is simply hidden until it meets those requirements.

Damn, playing melancholy, I'm hatin' it. I'll finish the tests (3 per god, I'll do tons of skipping in order to test the ones I lack) and then forget about the deck. Never, ever gonna play it. It's not even a valid deck for a FG prediction, I'll just delete the code.

Nothing against the creator of the deck, it's only that I hate decks THAT susceptible to starting hands.

To help you out games required to complete the data set for Melancholy: Decay (3), Eternal Phoenix (1), Ferox (1), Fire Queen (3), Gemini (2), Incarnate (1), Jezebel (1), Morte (1), Octane (1), Osiris (2).
« Last Edit: February 18, 2014, 03:05:12 pm by the dictator »
[18:28:55] Acsabi44: grinding is about pwning the AI as fast as possible
Back from a long break.
:water War Veteran - 4 times
Because inverted art is awesome - Looking for a tool for gather stats for arena, look no further

Offline eaglgenes101

  • Hero Member
  • *****
  • Posts: 1964
  • Country: us
  • Reputation Power: 29
  • eaglgenes101 is a proud Wyrm taking wing for the first time.eaglgenes101 is a proud Wyrm taking wing for the first time.eaglgenes101 is a proud Wyrm taking wing for the first time.eaglgenes101 is a proud Wyrm taking wing for the first time.eaglgenes101 is a proud Wyrm taking wing for the first time.
  • The rising all-'rounder of Elements
  • Awards: Slice of Elements 6th Birthday CakeSlice of Elements 5th Birthday CakeSlice of Elements 4th Birthday Cake
Re: [1.32] False Gods Stats Thread https://elementscommunity.org/forum/index.php?topic=53573.msg1126633#msg1126633
« Reply #67 on: February 18, 2014, 03:02:42 pm »
My policy would be, abandon the game only if you're > 99.5% sure it's going to be a loss.
My 3 game-modification principles:
1. If it ain't broke, don't wreck it.
2. Simple fixes for simple problems.
3. Remember to fill in the holes.

Offline the dictatorTopic starter

  • Hero Member
  • *****
  • Posts: 1261
  • Country: nl
  • Reputation Power: 17
  • the dictator is a Blue Crawler starting to think about his first run.the dictator is a Blue Crawler starting to think about his first run.the dictator is a Blue Crawler starting to think about his first run.
  • There are no dumb questions, only dumb answers
  • Awards: Slice of Elements 5th Birthday CakeWeekly Tournament Winner
Re: [1.32] False Gods Stats Thread https://elementscommunity.org/forum/index.php?topic=53573.msg1126634#msg1126634
« Reply #68 on: February 18, 2014, 03:07:43 pm »
My policy would be, abandon the game only if you're > 99.5% sure it's going to be a loss.

While that works as a error bar (or any other arbitrary percentage for that matter), I doubt anyone feels like calculating the odds of losing every turn just too see if they match the requirement (aside from the fact that most people probably don't even know how to correctly calculate those odds).
[18:28:55] Acsabi44: grinding is about pwning the AI as fast as possible
Back from a long break.
:water War Veteran - 4 times
Because inverted art is awesome - Looking for a tool for gather stats for arena, look no further

Offline Jangoo

  • Sr. Member
  • ****
  • Posts: 877
  • Reputation Power: 0
  • Jangoo hides under a Cloak.
  • New to You
Re: [1.32] False Gods Stats Thread https://elementscommunity.org/forum/index.php?topic=53573.msg1126641#msg1126641
« Reply #69 on: February 18, 2014, 04:02:24 pm »


That is a classic out-of-the-frying-pan-into-the-fire situation.  :(

The only thing I can think of to at least alleviate this problem a little bit, is to reduce the TTL in the default, to say, e.g., TTL = 0,7xTTW.
This is of course also arbitrary, not reflecting actual farming-experience and hence mistreating certain decks but it would at least come a bit closer to reality and not put off decks that rely on fast (and likely frequent) losing right away.

In my experience, losing almost always happens notably faster than winning, be proper losing or abandoning.


Offline the dictatorTopic starter

  • Hero Member
  • *****
  • Posts: 1261
  • Country: nl
  • Reputation Power: 17
  • the dictator is a Blue Crawler starting to think about his first run.the dictator is a Blue Crawler starting to think about his first run.the dictator is a Blue Crawler starting to think about his first run.
  • There are no dumb questions, only dumb answers
  • Awards: Slice of Elements 5th Birthday CakeWeekly Tournament Winner
Re: [1.32] False Gods Stats Thread https://elementscommunity.org/forum/index.php?topic=53573.msg1126646#msg1126646
« Reply #70 on: February 18, 2014, 04:47:49 pm »
Yes, it is indeed about choosing the lesser between 2 evils.

TTL = 70% TTW is a workable solution, though it isn't really any different from TTL= 100% TTW, at least not until we get more data. For now, it will keep set at 100%, but in I will make a note in the opening post that tracking turns to lose would be appreciated to improve the accuracy of the formula in the future. In this case the style doesn't really matter, since we will be averaging over all decks, modifications and testers, so the styles get nicely averaged in deciding the percentage to use, assuming people play when testing as they play when farming, which is what I have been aiming for in the way the study is set up, so that seems not to unreasonable an assumption.

Regarding the last part: losing or abandoning happens faster if you are playing a defending style of deck, where the defense fails before you can kill successfully. For a rush deck, losing happens slower, because you lose when you get outrushed, which, in case of a proper rush, happens when your own rush fails for some reason, likely a faildraw slowing you down.

Granted, most FG decks are defending decks (control, stall, denial, etc), because 200 hp's and double draw is practically impossible to outrush, but a lot of FG decks do rely on a speedy damage in some way, usually because they can only stall for a limited amount of turns.
« Last Edit: February 18, 2014, 04:50:02 pm by the dictator »
[18:28:55] Acsabi44: grinding is about pwning the AI as fast as possible
Back from a long break.
:water War Veteran - 4 times
Because inverted art is awesome - Looking for a tool for gather stats for arena, look no further

Offline Keeps

  • Full Member
  • ***
  • Posts: 440
  • Reputation Power: 9
  • Keeps is a Spark waiting for a buff.
  • New to Elements
Re: [1.32] False Gods Stats Thread https://elementscommunity.org/forum/index.php?topic=53573.msg1126762#msg1126762
« Reply #71 on: February 20, 2014, 03:44:59 am »
Simple testing procedure best practices:
One must remove as much bias as possible.

Everyone here talking about TTL as being subjective but TTW is not is fooling themselves. 
In past studies the best results had
A tester had to 'know' the deck inside and out, and multiple testers were needed to reduce TTW and TTL bias, and games had to be played to completion regardless of how obvious the win or loss would be, if a person is more likely to drop out of the game sooner, that's great for that player, but we aren't testing players, we are testing decks.

Also, as pointed out already, to figure out skips to get the most optimum EI, TTL is a requirement.

Final thing,
CuCN is right, in fact on this forum, he is the only person I have seen consistently get statistics, testing methods, equations, etc all correct.  He has (re)schooled me a couple of times, these being tools I used to use all the time in my past as an engineer, but I've gotten sloppy with as a manager these days.  If CuCN makes a point, I seriously recommend thinking really hard about the direction you are going and consider his point.  He also agrees you can't get skips without TTL. 

 

anything
blarg: