[Old] New voting system in development: Discuss

[Submachine]

I dislike using stars, because it provides too much opportunity for trolling, corruption, and bias, but...it might work in this system.

I just wanted to say. People don't often  vote other people's ideas, and sometimes if they do, they give 1 or 2 stars.

[Mnasyllus]

I just read the title, not where it was posted, and got excited.  My first thought was a change to the awful vote system in arena was coming.

I don't see how this change really affects much.  1000's of cards are made every week with ... what? ... 3 making it into the game now?

[OdinVanguard]

I think a star based system is good as a barrier to get into crucible or advance to forge, but it is still too simplistic.
To make having a card advance meaningful, the system should somehow encourage refinement and development.

I more or less like the crucible as it is right now, but I think the proposed system will be nice in that it will limit the amount of time and energy needed on behalf of the curators (which I'm certain can be quite intensive if each element has a dozen or more cards in it).

So making a "star" system to enter crucible is good.

Somewhere along the line, development and refinement should be worked into the voting system.
It would be nice if we could vote on what aspects of a card still need work at some point.

For instance instead of "pick your 3 favorite cards" at the Forge level, the polling would be "rate each card in terms of -art -balance -novelty -thematics" etc. The ratings on each category would get summed to give an overall rating.
Then the barrier to advance would be a minimum threshold on each aspect and the X best overall that meet those minimums get promoted while the Y lowest overall get demoted. Lastly, if a card is failing minimums and no changes or discussion get made then it can be archived (i.e. assume the author abandoned it).

So something like:
Smithy -> Crucible = minimum star level or curator sponsor
Crucible -> Forge = Favorite card overall (same as it is now) -Top X promote, bottom Y demote, middle float
Forge -> Armory = -minimum rating in each core aspect,
              -X top cards meeting minimums advance,
              -the Y lowest that met minimums demote
              -cards in the middle are retained
              --any cards that don't meet minimums get archived if no changes get made before next pass
Armory = ongoing favorite overall standing

At the forge level, if a card fails the minimum and no changes or discussion get made before the next N voting cycles then it can be assumed to be abandoned and get archived.

The number of cards in the forge level should be kept fairly small since it would become the "refine, tweak, and polish" level and voting would be most complex there.

[Pineapple]

I'm a bit hesitant to support the star system because of the existence of noobs/trolls/illiterates, but with this community I doubt that'll be a problem, and even if it would then the problem would be just as prevalent in poll voting. So we'll just have to try it out and see.

I approve of the change from archiving to demoting. IMO it is currently way too easy for strong ideas to be archived, and then way too difficult for ideas still up-to-date to be revived in a meaningful manner.


As OldTrees said, outflow must equal inflow. We cannot allow leniency (in outflow) without enforcing strictness (in inflow).

Suggestion #1:

Instead of having a set point value your card idea thread rating has to exceed (e.g. 3.99) in order to move from the Smithy to the Crucible, allow the Crucible to hold only X highest rated threads.
Specifically, after the cards with the lowest Crucible poll votes are kicked out of the Crucible, look at the Crucible card with the lowest thread rating. Then, accept up to X cards in the Crucible submission thread, but only the ones with the highest thread rating, and only if that rating is stronger than the weakest card currently in the Crucible.

This will not only keep the Crucible from being bloated, but also maintain an implicit standard of quality for cards moving through the system.


As I said before, demotion > archiving. That's why I would also suggest disabling the Crucible Archive.

Suggestion #2:

Instead of sending cards that leave the Crucible into the Crucible Archive, send them back into the Smithy, so they have another chance of getting into the Crucible if the card is good enough. Of course, there should be a long delay from when you leave the Crucible before you can resubmit your card for approval into the Crucible, say 2 rotations.

If this mechanic is implemented in tandem with the mechanic in Suggestion #1, then we can give card ideas who are demoted due to poor circumstance a second chance without prioritizing them over fresh, new card ideas that outrank them in thread rating.

[ddevans96]

I think a star based system is good as a barrier to get into crucible or advance to forge, but it is still too simplistic.
To make having a card advance meaningful, the system should somehow encourage refinement and development.

...

Somewhere along the line, development and refinement should be worked into the voting system.
It would be nice if we could vote on what aspects of a card still need work at some point.\

For instance instead of "pick your 3 favorite cards" at the Forge level, the polling would be "rate each card in terms of -art -balance -novelty -thematics" etc. The ratings on each category would get summed to give an overall rating.
Then the barrier to advance would be a minimum threshold on each aspect and the X best overall that meet those minimums get promoted while the Y lowest overall get demoted. Lastly, if a card is failing minimums and no changes or discussion get made then it can be archived (i.e. assume the author abandoned it).\

...

Agreed with all this. Maybe instead of the Forge, your proposal could even be at the suggested added level, to potentially make the system flow more smoothly. So say Crucible > New Level > Forge > Armory.

Also liking Pineapple's suggestions, especially the second one.

[Blacksmith]

Happy to see that so many responded to this. Now it's time to answer things....

Spoiler for quotes and comments.:

Spoiler for Hidden:

I'm happy to see CI&A section alive and active.

I like the new voting system, but I'd reduce the number of cards in crucible to a max of 20.

Nice job CC's! Thank you and your opinion is noted.


 

Spoiler for Hidden:

I like the new voting system, but I'd reduce the number of cards in crucible to a max of 20.

Regardless of my lack of knowledge on the current inflow of ideas, I think 20 is too limiting.

@OP

Question: what happens to the current archives? Are they scrapped? Left alone for posterity?Atm I think artimes idea is very good. The old archive stays just that people may at any time suggest card to go back into the rotating system.

My main problem with the proposed system has to do with what it's based on: stars. Specifically, variability. We can assume that if it's already in the new crucible or forge that it did have four stars at one point, but in the time it takes for things to be moved from the smithy to the crucible, it can easily change from four or five to two. I see you point. Hopefully enough people vote so that radical changes won't happen to often.

To sum it up, record keeping. At the moment, the card designer rankings are dependent on how far cards go. Would this new system mean that they should start anew? Or would they just fluctuate? Also dependent on record keeping is people's portfolios, which would have a similar fluctuating situation.
Hm I actually haven't thought of the rankings. I suppose we keep the old ranking and from now on you get a point when your card gets archived or reach the second highest lv.

Spoiler for Hidden:

I dislike using stars, because it provides too much opportunity for trolling, corruption, and bias, but...it might work in this system.

An added level, either immediately before or after Forge, might be a good idea, because we'll probably have a large amount of cards in both Crucible and Forge. And in each level, move between 1/4 and 1/3 cards up or down, allowing reentry (after proof of either card redesign or change of community opinion) back into the Crucible.

Sounds like a decent system. Thank you

Spoiler for Hidden:

I dislike using stars, because it provides too much opportunity for trolling, corruption, and bias, but...it might work in this system.

I just wanted to say. People don't often  vote other people's ideas, and sometimes if they do, they give 1 or 2 stars.Hopefully this will get 10% of the viewers to vote on the card.

Spoiler for Hidden:

I just read the title, not where it was posted, and got excited.  My first thought was a change to the awful vote system in arena was coming.

I don't see how this change really affects much.  1000's of cards are made every week with ... what? ... 3 making it into the game now?11 actually. So still not that many. But Zanz said himself that looking at the Cia section gives him many ideas so even if only 11 been added it might have given him ideas for many mores + that cleaning the old section a bit will make him able to watch the new cards that people like and not the ones he seen 2 years ago and didn't pick.

Spoiler for Hidden:

I think a star based system is good as a barrier to get into crucible or advance to forge, but it is still too simplistic.
To make having a card advance meaningful, the system should somehow encourage refinement and development.

I more or less like the crucible as it is right now, but I think the proposed system will be nice in that it will limit the amount of time and energy needed on behalf of the curators (which I'm certain can be quite intensive if each element has a dozen or more cards in it).

So making a "star" system to enter crucible is good.Kay

Somewhere along the line, development and refinement should be worked into the voting system.Agreed.
It would be nice if we could vote on what aspects of a card still need work at some point.

For instance instead of "pick your 3 favorite cards" at the Forge level, the polling would be "rate each card in terms of -art -balance -novelty -thematics" etc. The ratings on each category would get summed to give an overall rating.
Then the barrier to advance would be a minimum threshold on each aspect and the X best overall that meet those minimums get promoted while the Y lowest overall get demoted. Lastly, if a card is failing minimums and no changes or discussion get made then it can be archived (i.e. assume the author abandoned it).

So something like:
Smithy -> Crucible = minimum star level or curator sponsor
Crucible -> Forge = Favorite card overall (same as it is now) -Top X promote, bottom Y demote, middle float
Forge -> Armory = -minimum rating in each core aspect,
              -X top cards meeting minimums advance,
              -the Y lowest that met minimums demote
              -cards in the middle are retained
              --any cards that don't meet minimums get archived if no changes get made before next pass
Armory = ongoing favorite overall standing
To be honest I think that would be a bit complicated. With for example 20 cards to choose from you have to make 80 decisions, a bit to much I think.

At the forge level, if a card fails the minimum and no changes or discussion get made before the next N voting cycles then it can be assumed to be abandoned and get archived.We could add that if a card stays in the same voting with no changes twice in a row it get archived? This will encourage changes so the cards won't get archived.

The number of cards in the forge level should be kept fairly small since it would become the "refine, tweak, and polish" level and voting would be most complex there.Is a third a good number between Lv1 and lv 2 you think?
I agree that low tweaking and changing might be a problem. Maybe add that the lower half of the cards moving to the next lv (lowest vote) will be moved to a middle stage or what you should call it before they can enter the next voting system. In this so called middle stage you get community response and make changes before you can proceed.

Spoiler for Hidden:

I'm a bit hesitant to support the star system because of the existence of noobs/trolls/illiterates, but with this community I doubt that'll be a problem, and even if it would then the problem would be just as prevalent in poll voting. So we'll just have to try it out and see.

I approve of the change from archiving to demoting. IMO it is currently way too easy for strong ideas to be archived, and then way too difficult for ideas still up-to-date to be revived in a meaningful manner.Kay


As OldTrees said, outflow must equal inflow. We cannot allow leniency (in outflow) without enforcing strictness (in inflow).Yeah we have to find a good balance even.

Suggestion #1:

Instead of having a set point value your card idea thread rating has to exceed (e.g. 3.99) in order to move from the Smithy to the Crucible, allow the Crucible to hold only X highest rated threads.
Specifically, after the cards with the lowest Crucible poll votes are kicked out of the Crucible, look at the Crucible card with the lowest thread rating. Then, accept up to X cards in the Crucible submission thread, but only the ones with the highest thread rating, and only if that rating is stronger than the weakest card currently in the Crucible.Hm thats another way to do it. I agree that we have no clue of how many cards that will get 4 stars or higher and this seems to be a reasonable system to do it. But I don't think it should be added before we know how the all above 4 star system, work. We might be forced to have a test period of a month or 2 before adding a thing like this.

This will not only keep the Crucible from being bloated, but also maintain an implicit standard of quality for cards moving through the system.Yeah I can see the benefits! and I think it's good.


As I said before, demotion > archiving. That's why I would also suggest disabling the Crucible Archive.

Suggestion #2:

Instead of sending cards that leave the Crucible into the Crucible Archive, send them back into the Smithy, so they have another chance of getting into the Crucible if the card is good enough. Of course, there should be a long delay from when you leave the Crucible before you can resubmit your card for approval into the Crucible, say 2 rotations.I definitely agree with you and what Odin said about it being important that cards needs feedback before getting archived.As it is now ( sorry for not adding this in main thread until now ) the only cards that are getting archived are cards that has stayed at the same place in rotating( the once in the middle with not enough votes for next lv and not to little to be kicked out ) twice without any changes. But the ones that are kicked out of Crucible are moved to archive yes since there is no lower lv to send them back to.
So yeah adding those cards to the smithy sounds reasonable. If they has made major changes I think they can join the voting system next week again. If they fail 2 times they get archived.

If this mechanic is implemented in tandem with the mechanic in Suggestion #1, then we can give card ideas who are demoted due to poor circumstance a second chance without prioritizing them over fresh, new card ideas that outrank them in thread rating.

Spoiler for Hidden:

I think a star based system is good as a barrier to get into crucible or advance to forge, but it is still too simplistic.
To make having a card advance meaningful, the system should somehow encourage refinement and development.

...

Somewhere along the line, development and refinement should be worked into the voting system.
It would be nice if we could vote on what aspects of a card still need work at some point.\

For instance instead of "pick your 3 favorite cards" at the Forge level, the polling would be "rate each card in terms of -art -balance -novelty -thematics" etc. The ratings on each category would get summed to give an overall rating.
Then the barrier to advance would be a minimum threshold on each aspect and the X best overall that meet those minimums get promoted while the Y lowest overall get demoted. Lastly, if a card is failing minimums and no changes or discussion get made then it can be archived (i.e. assume the author abandoned it).\

...

Agreed with all this. Maybe instead of the Forge, your proposal could even be at the suggested added level, to potentially make the system flow more smoothly. So say Crucible > New Level > Forge > Armory.

Also liking Pineapple's suggestions, especially the second one.Your opinion is noted. Happy you like them

Pewh that took a while... finally done 

But feel free to continue to leave feedback!!!

[OldTrees]

Cards that fall out of level 1 (crucible) could return to level 0 (smithy) in order to allow reentry.

[Blacksmith]

Cards that fall out of level 1 (crucible) could return to level 0 (smithy) in order to allow reentry.

Yeah Pineapple thought about that too and I like the idea.

[Espithel]

I would say that the real problem of stars is that everyone uses them differently, and everyone considers different cards different ratings. I rate 4 if I like a card, rate 3 if other people do, don't rate if I don't.
Some people rate 3, 4, 5 on different circumstances,
Some people rate 1 or 5,
Some people rate 3 or 5,
Some rate 1 or 2, and there's no real criteria for each rating.

It would be simpler if it was a binary Y/N vote..

Otherwise, I'm a noob so I can't really say anything. ;-;

[andretimpa]

Would a thumb up/down system be more consistent then?

Stars = 5*up/(up + down)

[Blacksmith]

I would say that the real problem of stars is that everyone uses them differently, and everyone considers different cards different ratings. I rate 4 if I like a card, rate 3 if other people do, don't rate if I don't.
Some people rate 3, 4, 5 on different circumstances,
Some people rate 1 or 5,
Some people rate 3 or 5,
Some rate 1 or 2, and there's no real criteria for each rating.

It would be simpler if it was a binary Y/N vote..

Otherwise, I'm a noob so I can't really say anything. ;-;

Very true^

If this new system gets through we will add some criteria for each star rating in the read this first post ( hopefully people will read that ). And I think that many people that reads the Cia section will get proficient at making accurate ratings and the ones who just pass by maybe doesn't even know about the new system and don't rate at all.
But yes I'm sure bad ratings will be made but hopefully they aren't enough to affect things to much.

And if it doesn't work well we could always try Pineapples idea that the X highest rating proceeds every week.

Would a thumb up/down system be more consistent then?

Stars = 5*up/(up + down)

Thats another alternative. People make pools asking Yes/No if the card should go to Lv1.

[ARTHANASIOS]

During Forum Brawl #2, there was a Forum Expert task at Round 4 about "Rework Voting in the CI&A section in a way that would better show a card's quality". There may be some quite interesting suggestions in there if you have both the time and patience to read them all. The link is HERE!