FG Farming Efficiency Study: Design Phase

[kirchj33]

I think a FG farming efficiency study would be highly helpful and would like to try to organize such a thing.  The TTW study (http://(http://elementscommunity.org/forum/index.php/topic,17903.0.html)) for AI3 has been extremely useful to the community and I think a study that focuses more on what is relevant to FG could be extremely helpful as well. 

While things like TTW, Time per match, Score/Hour, Electrum/Hour are important, the difficulty in designing a study for FG would be not so cut and dry.

Please critique the following study ideas in a positive manner and I will move forward with its implementation.

Q1.  Which decks would be tested?

A.  While there are many great decks out there, I feel only those regarded as the best farmers should be included.  Right now I feel those decks are: RoL/Hope, Shakar's, CCYB, Timebow, Eternal Chaos, Lonestalker, and perhaps Liquid Antimatter or Antimatter You.

Q2.  You didn't mention any unupped decks, why is that?

A.  While it may be practical to test AI3 decks unupped for beginners, I feel FG decks should be optimized for performance as nobody REALLY wants to farm false gods with an unupped deck if they don't have to, let alone test it.

Q3.  How would you keep track?

A.  This is up for some debate, and to me, presents the biggest challenge of all when comparing decks.  In my opinion, TTW does not matter, nor does total time, but the only thing that really matters is Score/Hour, Electrum Gain or Loss/Hour (adjusted for cards sold), and Cards Won/Hour.  TTW can give some indication of how long your matches might be, so should be listed, but not considered as a factor for how good or bad a farmer XY&Z decks are.  Please discuss.

Q4.  How would you test each deck?

I could be convinced to test each of the first six I listed for 5 or maybe 10 hours.  Ideally, I would get 50 total hours on each deck.  So hopefully, 5 reliable members of the community help test 10 hours for each of those six decks.  So 60 hour time commitment from each person total for 5 people.  Ideally, each person contributes equal amounts.  Is this too much to expect?  Should it be narrowed down to 2 testers, 10 hours each?  Please discuss.

Q5.  How do you know which FG's to fight with each deck?

I think I could tell you off-hand which FG's to skip for each of the first three decks, and would need to spend a bit more time with the last three.  These could then be formulated into a list, debated a bit, and then each tester would have a standard list to go off of.  Another way would be to let each tester use their own discrepancy, in order to simulate what the farming population would do.


I think this could be beneficial to both new players, in that they would have a guide to go off of for what the advantages for each deck are, and veterans, so they can really compare what kind of card production and score commitment each deck will be.  Please discuss for a few days and then I will try to get the most community helpful study produced.

[DSSCRA]

You could use the The FalseGod-efficiency-index (FGei) for testing (link below).




http://elementscommunity.org/forum/index.php/topic,14626.0.html

[willng3]

A1:  Those decks are all solid choices, but the only real issue I see with choosing them specifically is that there are sooooo many variations of each one to choose from.  You'd probably have to contact some people who have extreme experience with each to find out which one is best to run stats on.  Unfortunately many people are still inclined to choose whatever variation they feel is best, regardless of what stats may show for the deck...which means this information would go ignored and unused by them.  Also, some decks have already been tested and analyzed in advance; this information could either simply be added or you could do a different statistical study with each, if needed.

A2:  Not doing unupped decks is fine.  From what I understand Theonlyrealbeef is already working on a similar study for unupped decks anyways so this wouldn't interfere in the least.

A3:  Eh, TTW isn't really as big with FGs as, say, AI3, you're right.  It's hard to get accurate TTW values anyways with so many FGs stalling you and other rushing; if you get a win versus Hermes 40 times and Miracle 0 times because you never encountered him for one trial then this is obviously not going to give you an accurate measuring.  TTW should be included, but not a major priority.  I like the focus on Electrum/hour / Score/hour, but unfortunately because the number of identically spins is not the same each time done, your Electrum/hour may not be as consistent as it should be.  I would advise not selling cards until the very end of testing and only making a note of how many cards won during each trial.  There /is/ a method for finding the base Electrum earned without the spins being included which you could use to give users an idea of the range of Electrum they can expect to receive, however.

A4:  I do not like the idea of measuring each entry based on a time limit.  The problem with doing this is that people will have to stay seated for 5-10 hours straight, grind non-stop, and not get distracted by anything.  Unless the person times him/herself and pauses every time they have to get up to go the bathroom or something, this is going to be very tedious and either way there's a great chance of it being inaccurate.  I would instead gauge it based on number of games played; say 500-1000.  There are a few reasons for this.  Firstly, it takes a bit of focus off of time because you're no longer focusing on a time limit, but more so on the time taken to complete a total of games.  Secondly, for decks such as Timebows which will likely take 20+ minutes for each game you are much more likely to miss a certain FG during testing, meaning that your stats will not be as accurate if you focus on a total time spent.  By gauging based on total number of games you will get far more productive stats.

EDIT:  Basically, you could use the link posted above.

A5:  I personally wouldn't skip FGs while doing these stats only because you have people who like skipping certain FGs when there is little hope to begin with and then there are others who just don't like losing without a fight.  For the FGs where it would be easiest to just skip them you could create a separate set of stats showing what would have happened had the FG simply been skipped (this would be done by simply omitting the game's data from the calculations and instead replacing it with time spent values of 0, and the remainder being treated as a normal loss).  This would take even more work though, so it might not be the best idea.


One last suggestion:  I highly advise waiting to start this until after 1.27 has been released.  For one, the AI is getting an intelligence increase and there /should/ be 4 more FGs to fight.  That might also affect the total number of games needed...something to consider at least.

In saying all of this I /might/ be able to assist with this, but it ultimately depends on what my schedule entails in the future.

[TheonlyrealBeef]

As for Question 4, I'm afraid you'll need a lot more testing to get anything close to reliable data. Some decks are extremely inconsistent (I'm looking at you Shak'ars) and Card spins have the same problem, we want the stats to be able to filter those things out.

As for Question 2: unupped decks, yeah, I was of busy with it, but it's kind of on a halt right now. With ai getting improved, and new fgs being added, I want to wait for 1.27 to start retesting the top 3, and add more stats (seconds and turns per game, including drawn-out losses, excluding insta-quits and more). I also don't have as much time as I had when I started it (I'll leave the juicy details of why out of it).

@ Question 3 (encompasses question 5) (just typing it in the order answers pop in my mind): Basically wins/losses, seconds and turns per game (including lost ones IMO), cards won, electrum won and even score gain/loss (some people care about score ). EMs are also important (kind of part of the electrum and score won). If you keep track of everything and note the fg, enough testing could actually show what fg is worthwhile to farm, and which ones aren't for a farmer. Not sure if it's beyond the scope of this study, but it would be awesome if we could get enough games for it (I'm basically talking about 1000 games about now, and for several testers... >_>, though more reliable testers => less games per tester for good stats).

For Question 1 fully upgraded decks (not Liquid Antimatter) that are widely used. I suggest hopping in chat when there're 20+ people and asking frequent, known users of those decks to attempt to agree on one specific variation to test.

[zse]

First I would split anti-FG decks into 2 different categories:

• Decks that try to win as many games as possible within certain amount of time, disregarding how many times they lose.
  Rush Hope/RoL, Shakar's, Retrovirus decks.
• Decks that try to win as many games as possible compared to the number of losses, disregarding how long time it takes to play one game.
  Control Hope/RoL, CCYB, Lone Stalker, Dune Scorpion decks.

From my experience it basically takes minimum 200 games total, or some 8 to 10 games against each FG, to get somewhat reliable results on how well given deck works in a long run. Anything less is just too much luck dependent.

[kirchj33]

First I would split anti-FG decks into 2 different categories:

• Decks that try to win as many games as possible within certain amount of time, disregarding how many times they lose.
  Rush Hope/RoL, Shakar's, Retrovirus decks.
• Decks that try to win as many games as possible compared to the number of losses, disregarding how long time it takes to play one game.
  Control Hope/RoL, CCYB, Lone Stalker, Dune Scorpion decks.

I think this is a really good idea, and could be listed into categories as such, although the stats should tell the story as well.

A1:  Those decks are all solid choices, but the only real issue I see with choosing them specifically is that there are sooooo many variations of each one to choose from.  You'd probably have to contact some people who have extreme experience with each to find out which one is best to run stats on.  Unfortunately many people are still inclined to choose whatever variation they feel is best, regardless of what stats may show for the deck...which means this information would go ignored and unused by them.

I will begin to post the consensus decks based upon conversations with others and my experiences as to what I think should be used when testing each of the top 6 decks mentioned. 

Also, some decks have already been tested and analyzed in advance; this information could either simply be added or you could do a different statistical study with each, if needed.

I will begin to do some data mining soon to see what is out there and how up to date it is.

A3:  Eh, TTW isn't really as big with FGs as, say, AI3, you're right.  It's hard to get accurate TTW values anyways with so many FGs stalling you and other rushing; if you get a win versus Hermes 40 times and Miracle 0 times because you never encountered him for one trial then this is obviously not going to give you an accurate measuring.  TTW should be included, but not a major priority.  I like the focus on Electrum/hour / Score/hour, but unfortunately because the number of identically spins is not the same each time done, your Electrum/hour may not be as consistent as it should be.  I would advise not selling cards until the very end of testing and only making a note of how many cards won during each trial.  There /is/ a method for finding the base Electrum earned without the spins being included which you could use to give users an idea of the range of Electrum they can expect to receive, however.

A4:  I do not like the idea of measuring each entry based on a time limit.  The problem with doing this is that people will have to stay seated for 5-10 hours straight, grind non-stop, and not get distracted by anything.  Unless the person times him/herself and pauses every time they have to get up to go the bathroom or something, this is going to be very tedious and either way there's a great chance of it being inaccurate.  I would instead gauge it based on number of games played; say 500-1000.  There are a few reasons for this.  Firstly, it takes a bit of focus off of time because you're no longer focusing on a time limit, but more so on the time taken to complete a total of games.  Secondly, for decks such as Timebows which will likely take 20+ minutes for each game you are much more likely to miss a certain FG during testing, meaning that your stats will not be as accurate if you focus on a total time spent.  By gauging based on total number of games you will get far more productive stats.

EDIT:  Basically, you could use the link posted above.

You could use the The FalseGod-efficiency-index (FGei) for testing (link below).




http://elementscommunity.org/forum/index.php/topic,14626.0.html

I would definitely use this index after looking at it more closely.  I remember seeing it awhile ago and didn't put much thought to it because it seemed complicated at first glance.  In reality, it's fairly basic and doesn't seem to have been utilized in other data as much as it should.  It seems pretty all-inclusive and data could be held which both represents winning percentage (god-by-god) and electrum/score speed.

A5:  I personally wouldn't skip FGs while doing these stats only because you have people who like skipping certain FGs when there is little hope to begin with and then there are others who just don't like losing without a fight.  For the FGs where it would be easiest to just skip them you could create a separate set of stats showing what would have happened had the FG simply been skipped (this would be done by simply omitting the game's data from the calculations and instead replacing it with time spent values of 0, and the remainder being treated as a normal loss).  This would take even more work though, so it might not be the best idea.


One last suggestion:  I highly advise waiting to start this until after 1.27 has been released.  For one, the AI is getting an intelligence increase and there /should/ be 4 more FGs to fight.  That might also affect the total number of games needed...something to consider at least.

In saying all of this I /might/ be able to assist with this, but it ultimately depends on what my schedule entails in the future.

At first, I was thinking that I would definitely skip for purposes of the study, but on second thought, if categories were used, as zse suggests, we could have both decks for skipping, and decks for not.  Your suggestion about 1.27 is a great one, and would give time to properly prepare the study and move forward when the time is right.

[Jangoo]

Great project Kirch ... I am totally looking forward to the results!


Concerning Skipping:

For me, the thing is that there is skipping and "skipping".
In the first case, the deck is clearly not designed to beat, e.g., Octane and you skip him right away.
In the second case, you are watching your FGei/game-speed, start playing a match and sooner or later find out that this is not going to go all that well ... so you skip those time-consuming turns needed to finally bring you down, mark the match as a loss and move on.

If you were to be strictly scientific, you would indeed have to run two seperate studies: One for win-rate and one for FGei, because you never know if by some miracle you had won that "skipped" match or you would have quickly "skipped" tons of matches to save time ...
In order to manage both stats at once, very proficient gameplay with the deck in question is required to make the right choice and be quick about it.
You will still get some inaccuracies because that's what the choice between perhaps still winning and decisively saving time entails.