[OUTDATED] The Return of FGei (for v1.32x)

[ColorlessGreen]

The proper way is not to track skips at all...  but to play all the games through as best as you can.
Then, have the program or sheets calculate the FGs to skip to give the highest FGei.
But none of the tests have worked that way.

Actually, many of the tests have worked that way, just not this one. I agree that it's basically the optimal way to do things.

Just as a general status update, I do not intend to make any major revisions to the methodology (or spreadsheets) of this study until 1.4 is released, as there is reasonable evidence that things will change quite a bit with regards to FG when that happens. This will continue as it is for the foreseeable future. Assuming the new 1.4 study is under my management, I intend to bring it back much closer to how the previous studies were handled.

[Keeps]

Oh, I guess the few I've missed flitting in and out with weren't.
I remember one way back in the 1.2x days that people were recording skips.
I also thought when instosis was tested a lot of the comments were about how it had obvious skips making it get the FGei
The main stats sheets, and statmasta has fields to record skips, and putting that together,
Also several conversations in the past involved were about really knowing the decks so they could record skips correctly. 

So, I just assumed any of the FGei's I wasn't ever privy to especially the year I stopped playing  were ran like those.

[ColorlessGreen]

The big main original FGEI study

1. NO SKIPPING ALLOWED - It is important that the data collected is as clean and raw as possible.  Testers may have a very good idea of which false gods will be skipped for the purposes of actual farming, but we want as much information as possible at this point.  We will go in and take a look at the data in the future and determine which false gods should be skipped based off the information collected.

That thread also includes a list of "skip stats", with the raw stats modified to account for optimal skipping per the procedure linked in the OP (though I make no comment on the validity of that particular procedure at this time).

The statmasta spreadsheet allows people to record statsskips because statmasta (and actually originally FGEI) is supposed to be personal results, i.e. a method for a given person to know how well a thing works for them with their play style.

Assuming I manage the new 1.4 FGEI study, this will be a rule again.

[Keeps]

Cool for the future and the big study that's good.

It also explains why I remember certain things otherwise, so I'm not insane...  not being insane is good.

[the dictator]

I'm currently testing my own version of voodoo bravery, gathering data with the Statmasta sheet, but this one tracks Time to win/lose/skip, instead of turns, which not only seems more accurate, but necessary if you want to calculate electrum gain per hour.

wrong. The time to win differs from player to player MUCH more than from deck to deck. even if we don't do anything else, my brother needs about *1,2 the time that I do, with the same deck, to play each turn. Because he likes to think a little more and plays generally slower than me (Not only elements, but many games). If both of us would record time to win instead of turns, you wouldn't be able to put them into the same thread. (That aside, I need about ten-twenty more seconds per game if I am sleepy or not really motivated to grind fast.

--> So the more logical as well more accurate way is, to decide on a normal turns per hour speed (which the community did long in the past) And calculate the Electrum/Hour from that, asuming that all decks need for each turn the same amount of time.

Interestingly, it turns out this is not what the google doc file is doing either, it does require you to add in the matchdurations, and uses those to calculate FGei(cn).

[Sevs]

I added 100 more games of data to the "Don't cut yourself" tab on the gdoc, but I have no idea where "1) Select 'Integrate data'.  2) Select 'Remove IN USE indicator'." are so I guess I will leave that up to someone else.

[Faro]

added Beat the Bug in a new tab in the spreadsheet.
but I need some help from you guys, i'm a really slow tester ....

I added 100 more games of data to the "Don't cut yourself" tab on the gdoc, but I have no idea where "1) Select 'Integrate data'.  2) Select 'Remove IN USE indicator'." are so I guess I will leave that up to someone else.

you find them in the "testing" menu

[Jangoo]

EDIT: partly removed for reasons of misreading stuff

The proper way is not to track skips at all...  but to play all the games through as best as you can.
Then, have the program or sheets calculate the FGs to skip to give the highest FGei.

Yes, exactly.
In a proper study, there is no need to skip while play-testing a deck if you can artificially work in the "advised" skips afterwards.
       |
       v

That thread also includes a list of "skip stats", with the raw stats modified to account for optimal skipping per the procedure linked in the OP (though I make no comment on the validity of that particular procedure at this time).

Hehehe. Yes, the procedure was pretty debatable.
Procedures of working in the "proper" skips get less and less debatable the more games for each god you actually have. 
Hence, finding a lot of decent testers who will play a lot of decent games is of utmost importance if you want a reliable index.

The statmasta spreadsheet allows people to record statsskips because statmasta (and actually originally FGEI) is supposed to be personal results, i.e. a method for a given person to know how well a thing works for them with their play style.

Also several conversations in the past involved were about really knowing the decks so they could record skips correctly. 

It will never be any other way than different people playing differently and making the best, or worst (or likely something in between) of the material they have at hand. There will also never be an ultimate index or farmsheet that can extract "the one true independent deck-value" from a set of games played by actual humans. That is simply not possible. Not in elements, not anywhere else where humans still play a role in a process.
If you test a deck as an individual person, you will likely know your deck and play (and possibly skip) as you wish ... the outcome is indeed an individual performance index. But if people know what level of player you are, this will already tell them quite a lot.

If you test a deck in a study, you have to decide what you are trying to get:

1. A high-performance index? ("as good as this deck gets")
 -> select only the best of the best players, they should make no mistakes
2. An average-performance index? ("Joey and his mother driving a ferrari")
 -> select anybody but be aware that the index generated shows something specially different

"As good as it gets" tends to be more interesting since it will tell you how good the deck really is once you have eliminated human faultiness as much as possible. Having these indexes lets you compare decks in a competitive, "professional" way: Which deck is best?
 That having said a more balanced "average performance" index can be very attractive ... Just figure having stats from like 1000 games by like 8 different people and working these into a single index! People would know that this is not "as good as it gets" but knowing the average performance for pretty damn sure lets you make more than educated guesses what YOU, the person deciding on whether to use this deck or not, can expect from it. From a different viewpoint "the best" deck might just be the one that most people perform well with. 

[ColorlessGreen]

Even if somebody takes generally longer than another, he will still be a lot faster with a fast-paced deck.

This is 100% the opposite of what surveying the various studies over the years that have recorded both time per win and turns per win has shown. When I looked into it, the variation between person A and person B both playing essentially identical decks was far greater than the variation between person A playing one type and person A playing another type. What target you're facing has some impact on things, as FG is more likely to result in full-field battles than, say, ai3 is, but within a given target, the variation is relatively tiny (not nonexistent, but much smaller than variation between people).

Personal stats are far more useful for the person who created them, but aggregate stats are far more useful to the community as a whole. People consistently play at wildly different speeds even with the same deck. Short of having one person doing all the tests, it is far more useful to aggregate the data while removing as much subjectivity as possible (by removing, among other things, time per turn) and allow people to either take their own personal stats for additional accuracy, or even to just apply modifiers when they find that they personally play a deck slower than another deck. As an example of the latter, if a given person believes they play deck X 10% slower than deck Y, they have simply to reduce the total value of deck X by 10% when comparing to deck Y.

Reporting based on a standardized number of turns rather than how many games some arbitrary group of people could finish in an hour allows people who are particularly concerned with this and don't believe that the time per turn is actually very close between decks to simply sit down and very quickly play a couple of games with the deck to determine their personal time per turn and calculate a profit per hour stat from the turn-based reported efficiency index. Reporting based on time does not allow this flexibility whatsoever unless the people who recorded times also concurrently record and report turns.

[Jangoo]

This is 100% the opposite of what surveying the various studies over the years that have recorded both time per win and turns per win has shown. When I looked into it, the variation between person A and person B both playing essentially identical decks was far greater than the variation between person A playing one type and person A playing another type. What target you're facing has some impact on things, as FG is more likely to result in full-field battles than, say, ai3 is, but within a given target, the variation is relatively tiny (not nonexistent, but much smaller than variation between people).

Not sure I understand what you mean. Lets take Keolino and his brother (as mentioned on the page before). Keolino says his brother his ~20% slower than he himself is. Are you saying that his brother might just be even slower when he plays a deck generally considered to be fast? I would think that he may be slow playing CCYB but he will still be notably faster when playing Rol/Hope ... the reference of course is he himself. But this CCYB-Rol/Hope speed-relation will probably also be valid for Keolino himself and the majority of all other players.
Hence we would have two different players, one fast one slow, playing two different decks, one fast one slow. Outcome would be: Rol/Hope is a faster deck.    ? ?

Reporting based on a standardized number of turns rather than how many games some arbitrary group of people could finish in an hour allows people who are particularly concerned with this and don't believe that the time per turn is actually very close between decks to simply sit down and very quickly play a couple of games with the deck to determine their personal time per turn and calculate a profit per hour stat from the turn-based reported efficiency index. Reporting based on time does not allow this flexibility whatsoever unless the people who recorded times also concurrently record and report turns.

Ah, now I get it. I was so fixated on the perennial TTW, TTL -thing since it's also mentioned here and wildly mixed into the debate.
Hence, I completely missed out on turns per hour .
That of course is an entirely different thing than TTW etc. and, you are right of course, fully valid if handled correctly. It's simply another way of putting deck-speed. As long as these values end in "per time-unit" an actual efficiency is possible to calculate.

 

[ColorlessGreen]

This is 100% the opposite of what surveying the various studies over the years that have recorded both time per win and turns per win has shown. When I looked into it, the variation between person A and person B both playing essentially identical decks was far greater than the variation between person A playing one type and person A playing another type. What target you're facing has some impact on things, as FG is more likely to result in full-field battles than, say, ai3 is, but within a given target, the variation is relatively tiny (not nonexistent, but much smaller than variation between people).

Not sure I understand what you mean. Lets take Keolino and his brother (as mentioned on the page before). Keolino says his brother his ~20% slower than he himself is. Are you saying that his brother might just be even slower when he plays a deck generally considered to be fast? I would think that he may be slow playing CCYB but he will still be notably faster when playing Rol/Hope ... the reference of course is he himself. But this CCYB-Rol/Hope speed-relation will probably also be valid for Keolino himself and the majority of all other players.
Hence we would have two different players, one fast one slow, playing two different decks, one fast one slow. Outcome would be: Rol/Hope is a faster deck.    ? ?

I'm saying that when I did a survey of the studies with data, I found a bunch of stuff like:
Person A w/ Rush A: 10.2 sec/turn
Person A w/ Domination A: 10.7 sec/turn
Person B w/ Rush A: 16.0 sec/turn
Person B w/ Domination A: 17.5 sec/turn
(numbers are arbitrary, not actual, just illustrating my point)

Basically, individual was a faaaaar bigger factor than deck archetype.  Again, note that I'm talking time per turn. Turns to win is very important, naturally, and from TTW we calculate aggregate time per game (and games/wins per hour) based on an average time/turn (which, at least for UEI, is fixed at about 11 sec/turn for various reasons which I don't feel like repeating here, though I still eventually want to actually run a study and get a less arbitrary average).

Also, note that I'm not saying there is no variation between decks, just that it's a lot smaller than people generally seem to think it is, and that it's substantially smaller than the variation between players from what I've seen. The huge variation between players means that aggregating data based on this heavily skews decks based on the playstyle, attention, and computer speed of the person who did that individual test and makes it extremely difficult to accurately compare decks played by different people.

In the example above, say Person A only tested Deck B (10.7 sec/turn) and Person B only tested Deck A (16 sec/turn). Deck B would be regarded as significantly faster and better than Deck A due to a difference in playstyle/computer/etc, while in reality (according to the numbers above), Deck A is faster. Abstracting away time/turn solves this problem. It does create the additional problems of time/turn not being captured between decks, but this is a smaller problem (and is easier to personalize by just finding a personal time/turn) than the playstyle/computer/etc problem.

[Jangoo]

Ah yes. Now I get it.
Now I also get what you meant when referring to drawn out battles in FG-farming. I have always suspected that deck-speed, while still playing some role, is marginal in AI3- and, to some degree, arena-farming. Whenever I play against these AIs, decks are generally in the "rushy" segment anyways because an endless stallbow is not considered valid in the first place. And AIs inferior to FGs don't tend to put up an endless fight. Either they finish you fast (Higher end Arena, with some stoneskin-antimatter-stallcrap-lets-deck-him-out exceptions) or you finish them fast.
Also, I have noticed that e.g. the AI3-studies go for a pretty minimal player-deck-allocation. Usually it's rather much "Person A does this deck, person B this one and person C takes the remaining two decks ...". Here, your observations of course imply some serious flaw in study-design and possibly render the results "doubtable". 

I would still think that working in an estimated "slow/fast-player-percentage" as in "person B is a slow player so we will boost his stats by ... erm ... how about 20%" is problematic. It is still pretty damn subjective and arbitrary and, as an element of surprise, might just not do a certain deck justice one day.
Here as well, the only proper way would be increasing the number of different players testing a single deck.
When reading earlier, I was actually thinking how awesome a study would be where every tester needs to test every deck over an equal amount of games.
Unless the number of tested deck is reduced to a chosen few from the start (archetype-top-decks), this would probably never be possible to realize.