[OUTDATED] The Return of FGei (for v1.32x)

[rob77dp]

Spoiler for Quotes from pella / UTL:

I notice after looking again at the thread, that Lionheart is winnable by decking him if you shield up and do a minimum of card draws.  I wasn't playing that way in my losses against Lionheart, so the stats for him should probably be ignored, sorry.

Actually, those losses should be counted, and they will.

Losses against FGs are part of the learning process.  While losing to them, we learn how to beat them.  Those losses are important, and they should be reflected in the overall stats.  Thank you for your contribution.

I strongly disagree with that stance, Pella. FGei is a tool for comparing FG decks with each other. How can we compare them if we don't try to play them at their best potential? Furthermore, the "losses due to learning process" is a slippery slope. At how big of a mistake will you consider the stats unusable? There's no way to keep this consistent if you just allow certain errors to be played. Thus, for this to work, every deck has to be played as well as possible, and if there's a mistake that significantly affects the winrate of a certain god, the data on that god should be thrown out.

I think in summary Pella is putting down that human error is part of using a deck and playing Elements vs FG's and UTL is indicating that FGei is more of a "property" of a particular deck and hence FGei should strive to remove human error where noticed and possible.  Is that accurate?

If so, I will have to agree with UTL - FGei should be something a player can go to and say "ceteris paribus, X deck will earn me faster than Y deck" without worrying about +/- %errors or things of that nature.


--Rob

[CuCN]

Spoiler for Quotes from pella / UTL:

I notice after looking again at the thread, that Lionheart is winnable by decking him if you shield up and do a minimum of card draws.  I wasn't playing that way in my losses against Lionheart, so the stats for him should probably be ignored, sorry.

Actually, those losses should be counted, and they will.

Losses against FGs are part of the learning process.  While losing to them, we learn how to beat them.  Those losses are important, and they should be reflected in the overall stats.  Thank you for your contribution.

I strongly disagree with that stance, Pella. FGei is a tool for comparing FG decks with each other. How can we compare them if we don't try to play them at their best potential? Furthermore, the "losses due to learning process" is a slippery slope. At how big of a mistake will you consider the stats unusable? There's no way to keep this consistent if you just allow certain errors to be played. Thus, for this to work, every deck has to be played as well as possible, and if there's a mistake that significantly affects the winrate of a certain god, the data on that god should be thrown out.

I think in summary Pella is putting down that human error is part of using a deck and playing Elements vs FG's and UTL is indicating that FGei is more of a "property" of a particular deck and hence FGei should strive to remove human error where noticed and possible.  Is that accurate?

If so, I will have to agree with UTL - FGei should be something a player can go to and say "ceteris paribus, X deck will earn me faster than Y deck" without worrying about +/- %errors or things of that nature.


--Rob

Human error happens while getting used to a deck, but it's too hard to control for, especially when multiple testers are involved. And (at least as I see it) someone who uses FGei to choose a deck will be playing it for more games than each tester's data has, so the learning period will make up a smaller proportion of that time.

[Pella]

Okay, I'm considering the pros and cons of both sides of this.  I'd like to restate both of them as objectively as possible, and see what y'all say.

For removing erroneously played duels from the statistics
The statistics should be as free of errors as possible.  Clearly, we can't remove all errors, but where a tester made an obvious mistake while playing, the data from those duels should be thrown out.  This keeps the FGei statistics for all decks as comparable as possible.

For leaving erroneously played duels in the statistics
Errors are a part of human nature, and they are a part of daily play.  Even an experienced player will misclick something occasionally.  Leaving small errors in the statistics helps the final statistics to be as close to reality as possible.  Otherwise, players using FGei as a reference may get a false impression of the quality of a deck.

I will entertain discussion below.

[UnderneathTheLens]

Different players make different errors. Players have different playing levels. If you have a veteran testing one deck, and a noob testing another one, or even the same deck, the statistics are simply not comparable, which completely undermines the system.

In addition, the "reality" that you are talking about is personalized. What I mean is that EVERYONE is going to make different errors and have different playing quirks. Somebody may forget to chain Phase Shields. Someone might let their HP for Poison Dials go down below 40 because he wasn't counting the damage. Somebody else might use too many SoBs in Don't Cut Yourself. The possible variations in errors are endless. While you may have good intentions in trying to simulate a learning curve, it's going to be completely different for everyone, and what ends up happening is data that reflects the experience of a single person, and not a  general experience, as someone trying to play the decks ideally would reflect.

[andretimpa]

If we have many different people testing different decks (each person contributing around 100 games each time) should (in the long run) mitigate some of these problems. Another option is only allowing experienced players to test. For the FGei to reflect the quality of the deck, it is enough that all decks be test in similar conditions. I do believe also that statistics should be recorded only after a player got the hang of the deck.

An example about the deck I tested that I don't know if it affects FGei, but in Swallow you can either aim for a faster game (reducing TTW) or deliberately slow the game down to get an EM (increasing electrum gain from the match but increasing TTW and hence the time that it takes between spins). Because of this I played my first games aiming for EMs and the last ones aiming for speed.

[Pella]

UnderneathTheLens, you make some good points.  I think everything you wrote can be boiled down into this:

what ends up happening is data that reflects the experience of a single person

If all the data for a particular deck come from one tester, this is definitely true.  If, as is ideal, we have three or more testers contributing data to each deck, then the data "even out", creating a more balanced picture.  This is one of the main principles I studied in four different college-level statistics courses.  Over time, and with larger and larger amounts of data, everything tends toward the centre, contingent on a "margin of error".

*thinks about what he just wrote*

Statisticians hate error.  They know it exists, and they do everything they can to prevent it.  In the end, all they can do is minimize it and report results based on their best-faith effort.  Your position is that we have a known error; therefore, we should remove it from the data so our results will be closer to accurate.

I guess I had to think it out and write it out in my own terms before I understood your position the way you do.  Now that I have, I agree.  I will remove the losses from Lionheart's data for the Firecell FG OTK deck.  Thank you all for your input.

And this, boys and girls, is the value of calm discourse--the sharing and discussing of opinions.  This is the lost art of "debate", a word that has become maligned in recent decades. 

If we have many different people testing different decks (each person contributing around 100 games each time) should (in the long run) mitigate some of these problems. Another option is only allowing experienced players to test. For the FGei to reflect the quality of the deck, it is enough that all decks be test in similar conditions. I do believe also that statistics should be recorded only after a player got the hang of the deck.

After one "gets the hang of the deck", I can see.  I won't require it, though.  As for allowing only experienced players to test, I disagree.  Many who come looking for decks are inexperienced.  Having a mix of players test a deck balances things like the time and TTW.

An example about the deck I tested that I don't know if it affects FGei, but in Swallow you can either aim for a faster game (reducing TTW) or deliberately slow the game down to get an EM (increasing electrum gain from the match but increasing TTW and hence the time that it takes between spins). Because of this I played my first games aiming for EMs and the last ones aiming for speed.

This is another perfect example of things balancing each other.  Since this deck can be played two ways, and it was tested both ways, the overall statistics can be considered accurate.  Players who choose to play the deck only one way or the other will experience an income that differs from the FGei, but that's all right.  OVERALL, Swallow will produce X per hour of play, and that's the goal of the project.  If you conduct further testing on this deck, I would urge you to follow a similar model again. 

[Bloodshadow]

I haven't played Elements in more than a year now, but I hope my comment can still be relevant.

Guys, you need to distinguish between systematic error and random error. Systematic error are mistakes in your experimental setup; they can and should be removed. Random error, on the other hand, cannot be removed (I could go into quantum mechanics and chaos theory but I doubt anyone would want me to) and is considered a part of your data; that's what the uncertainty or standard deviation or what not are for.

The thing is, bad luck (e.g. bad draws) is random error and should indeed be included in testing the performance of a deck against False Gods. But playing a deck badly, on the other hand, is systematic error. If you include that, your experimental setup is botched, and the results will no longer be very accurate, depending on how significant the systematic error is. So if this is supposed to be a purely statistical thing under ideal conditions (or at least as ideal as possible), I think systematic error like playing the deck badly should be removed.

[CuCN]

An example about the deck I tested that I don't know if it affects FGei, but in Swallow you can either aim for a faster game (reducing TTW) or deliberately slow the game down to get an EM (increasing electrum gain from the match but increasing TTW and hence the time that it takes between spins). Because of this I played my first games aiming for EMs and the last ones aiming for speed.

I would suggest testing these strategies separately, as one might be more efficient than the other. If that is the case, someone looking for a FG deck should be given the information of which way of playing the deck produces better results.

[Pella]

I haven't played Elements in more than a year now, but I hope my comment can still be relevant.

Guys, you need to distinguish between systematic error and random error. Systematic error are mistakes in your experimental setup; they can and should be removed. Random error, on the other hand, cannot be removed (I could go into quantum mechanics and chaos theory but I doubt anyone would want me to) and is considered a part of your data; that's what the uncertainty or standard deviation or what not are for.

The thing is, bad luck (e.g. bad draws) is random error and should indeed be included in testing the performance of a deck against False Gods. But playing a deck badly, on the other hand, is systematic error. If you include that, your experimental setup is botched, and the results will no longer be very accurate, depending on how significant the systematic error is. So if this is supposed to be a purely statistical thing under ideal conditions (or at least as ideal as possible), I think systematic error like playing the deck badly should be removed.

That is pretty much the conclusion to which I came.  You stated it much more eloquently, though. 

An example about the deck I tested that I don't know if it affects FGei, but in Swallow you can either aim for a faster game (reducing TTW) or deliberately slow the game down to get an EM (increasing electrum gain from the match but increasing TTW and hence the time that it takes between spins). Because of this I played my first games aiming for EMs and the last ones aiming for speed.

I would suggest testing these strategies separately, as one might be more efficient than the other. If that is the case, someone looking for a FG deck should be given the information of which way of playing the deck produces better results.

That sounds like a good idea.  It will be more tedious, of course, but ultimately more useful.

[andretimpa]

An example about the deck I tested that I don't know if it affects FGei, but in Swallow you can either aim for a faster game (reducing TTW) or deliberately slow the game down to get an EM (increasing electrum gain from the match but increasing TTW and hence the time that it takes between spins). Because of this I played my first games aiming for EMs and the last ones aiming for speed.

I would suggest testing these strategies separately, as one might be more efficient than the other. If that is the case, someone looking for a FG deck should be given the information of which way of playing the deck produces better results.

Yeah. I have a hunch that aiming for speed may be more efficient against FG, while going for EMs is better in plat. (actually I started playing like if I was in plat during the test, which was how I was used to, but then realized that I could get spins quicker changing the play style).

Another thing to think about is that after a while you start to taylor your strategy to the FG you are facing, but I guess this qualifies as getting the hang of the deck.

[Keeps]

I think it's nice someone put up my Malignant Balls deck.  However, this time around there is too much 'systematic'/'player' error for the FGei type study.  I've tried all these decks posted.  Don't cut yourself, I tend to cut myself on, I like the deck, but my play style just doesn't jive with it.  While Pdials, I play well, I prefer the 35 card version.  My favorite for FGs is my own though; however, what you can and can not do against FGs, or what tricks work with which FGs is experienced based often with a deck.  Also mistakes happen too, I had an easy game against Miracle, normally really easy for this deck, then I decked myself out with SoBrs not paying attention, even after playing it so many times.  Half these decks have a bit more meta in them then previous FGei studies making it hard to test, and ultimately, a positive sign for this game as the depth is getting more complex than in the past.

[ColorlessGreen]

I think it's nice someone put up my Malignant Balls deck.  However, this time around there is too much 'systematic'/'player' error for the FGei type study.  I've tried all these decks posted.  Don't cut yourself, I tend to cut myself on, I like the deck, but my play style just doesn't jive with it.  While Pdials, I play well, I prefer the 35 card version.  My favorite for FGs is my own though; however, what you can and can not do against FGs, or what tricks work with which FGs is experienced based often with a deck.  Also mistakes happen too, I had an easy game against Miracle, normally really easy for this deck, then I decked myself out with SoBrs not paying attention, even after playing it so many times.  Half these decks have a bit more meta in them then previous FGei studies making it hard to test, and ultimately, a positive sign for this game as the depth is getting more complex than in the past.

This is an argument for a larger number of tests, not an argument against tests being useful. Player error and such, when handled over a sufficiently large number of tests, will either effectively disappear or (if it does not disappear) can be considered an actual factor in the performance of the deck - simplicity of playstyle is an attribute that is to be considered.

Additionally, bear in mind that, as FGEI was originally described (see the link in the OP) when it was first released, it is a personal stat. It represents how well the player who used it does with that particular deck. Anyone who looks to stats taken by another person (particularly one other person) should bear that in mind.

An average of a large number of stats across a number of people can be presumed to be vaguely representative of a true average performance of the deck. This is what we need for this study.

More testers.