RNA Lab based on "Point Payment"

1

As you know, EteRNA is going to be released to the public in a few days. (Wohoo!) However, as good as evolving into a public phase is, I think this event calls for a change in the way we play this game, as it will get exposed to all kinds of gamers, users and trolls that might look forward to cause trouble just to get attention.

So far, I think the section that is more vulnerable to unfair use would be the RNA Lab and I have seen 3 proposals in the news post that seem like a good start, so I decided it would be a good idea to move my proposal here, in order to get more feedback.

My idea is this:

As the main incentive of the game is going to be getting your design synthesized, a lot of players will get eager to enter the RNA Lab and even create fake accounts to favor their designs; which in turn will create more troubles, such as an overflow of design proposals (an overwhelming number of entries might discourage players from reviewing and voting) and of course, that a design might get chosen unfairly.

In order to solve this, I propose using the point system that already exists to “buy” the right to enter the lab, post solutions and vote for designs, so players that invest more time in the game or are better at solving puzzles will have a greater chance to propose designs. Also, this system can lead to some interesting mechanics, for example:

  • You have to pay 5000 points in order to unlock the Lab. As the player won’t have enough points to make a proposal immediately, this will encourage him/her to check out how things work in the lab first.

  • Voting for a proposal costs a certain amount of points, that increases each time. For example, for the first vote you might have to “pay” 250 points, the second costs 500, the third 750, etc.

  • Of course, let the players know that they will get a reward if the design they choose gets synthesized and gets a good score, so they can understand this as an investment.

  • In order to make a design proposal, you have to “pay” 3000 points, but you are limited to 3 solution slots, as always.

The drawback to this idea is that it might conflict with the current Leaderboard, which uses the score to show the most proficient players. So, that’s it, what do you think?

2

Henry,
These are some great ideas!

We will indeed be locking the lab, and require people to have a minimum number of points to enter. Great idea!

Also I really like the idea of using points to buy votes and submissions. However I also agree with your analysis that it conflicts with the current Leaderboard. So this probably won’t happen for launch, however perhaps in the future.

Thanks for the input!
Alex

3

I’ve been thinking about this and just the concept of buying the right to vote is an anathema to me.
Within systems, membership usually gives you voting rights, and selected groups of members may have voting rights for specific and clearly definined items.
I don’t actually like the voting system in eterna because it seems so subjective for a science based endeavour.

Completely understand the anti-flooding requirements, and the possibility of lots of facefriends turning up to vote.

Paying a token ‘admin fee’ of 5-10 points is perhaps bearable. But why even bother?

Why not let people vote based on the number of puzzles they have solved or their points tally? Once you get a certain number of points you can vote on one design, and give people more votes on a scale based on points.

It is good to let people in to the lab just to vote, as observers, so that they can understand the environment and what happens. This may require some code work.

I don’t disagree with getting people to pay points to enter a design - this would slow people who cared about their rank down a bit, perhaps submitting fewer but more considered designs.

IMHO the ability and drive to design is different to the ability to solve puzzles, however that is a different conversation :slight_smile:

4

Actually, before coming up with this suggestion I thought on something very similar to your idea, SPM; but had to discard it because there were many ways in which it could be abused.

Plus, from a game designer’s viewpoint, it can even be considered a poor idea because it doesn’t give the player the possibility to make an intelligent choice because there are no risks involved.

(“Can I vote? Yes. Do I lose something to do it? No. Well, I can spend a lot of time reviewing designs, or just vote for my friend here/randomly. Easier alternative wins”)

Now, by adding a risk factor (“I’ll lose my points if this design doesn’t work”), the player will feel motivated to make a wiser choice, and will feel encouraged to review the designs carefully before voting. Also, we have to stress the point that the player’s points won’t go down the drain, but that in the end, if the choice was right, the player will end with more points that the ones that s/he risked.

The actual voting system in EteRNA has been good all this time, mostly because we acted in a very educated and civilized way. However, there’s no way it could have been “subjective” as not everyone of us has a degree on biology or natural science and from what I understand, we are still learning from the experiments. It was kind of fun to see how, at the start we thought that the less energy, the better, and almost everyone voted accordingly. When the first results proved a balance was better, the popularity of balanced designs increased. So, I think we are following the scientific method that way.

In fact, I took into account your last thought in this idea. If a player just wants to solve puzzles, s/he will have no motivation to spend points on the Lab, so his/her goal of becoming the top player by score alone will be easier. Players motivated to design RNA’s will have to risk a bit of the ability of being the top player, but they will receive another kind of rewards in the end.

5

Hey Henry, I couldn’t find you in the player lists. I’m really not trying to get into a barney here; however a few things:

it doesn’t give the player the possibility to make an intelligent choice because there are no risks involved.

Sorry – can’t see the logic in this, I make intelligent choices every day because I’m strategic, I calculate risk and (hopefully) minimize it over the long term. (I also do dumb, fun, reactive things, but hey, I’m human)

Making reactive choices in life is, of itself, high risk; one of the reasons perhaps why we like to play nice safe computer games. Some of the most intelligent people I know are base jumpers and climbers, but they do not just jump off tall buildings willy-nilly. They plan.

the player will… feel encouraged to review the designs carefully before voting

Not entirely clear about ‘’review’’ –usually if I have several designs to review I am able to compare them side by side in and out of context;
hey I really tried to look at each design, but by the time I clicked thru to the fourth design; if I was honest with myself, I was quite unable to effectively compare it to the first.
Comparing designs by evaluating the data shown in the lab was interesting, however Í wasn’t sure how relevant or useful it was.

…we acted in a very educated and civilized way. However, there’s no way it could have been “subjective” as not everyone of us has a degree on biology or natural science and from what I understand, we are still learning from the experiments. It was kind of fun to see how, at the start we thought that the less energy, the better, and almost everyone voted accordingly

Subjective is as you have described, it’s opposite is objective, ie based on evidence.

‘’We’’ may have thought that lots less (or lots more) energy is better but not everyone follows the crowd.

I submitted two designs each on a shortish stretch either side of the middle; my aim was to understand the opportunities and value of plus and minus energy fairly close to the mean. Of course I voted for myself.
Having seen the (certainly earlier) iteration of the lab interface I did not feel that I had enough information to submit a third design.
Agreed, very boring method! Baselining.

My results are still unclear – I have lots of points for one design, however I have no record of the other, I was expecting to see results on my personal, or the public lab page along the lines of:

Design one - rubbish don’t give up your day job
Design two - lots of points, but don’t get cocky

Second design was close to a winning design, (no idea how close, can’t see it.)

So if I have to pay 3000 points each for a design I want to see where my points went. Not happening now; also if I have to pay to vote for myself as well – perfectly normal practice, putting my money where my mouth is; I want to see the accounts :slight_smile:

Your last para – motivation
I have lots of motivation to help design RNA, and generally help the science by just solving puzzles if that is useful. Crowdsourcing is good. There are lots of interesting science crowdsourcing projects online.

(“Can I vote? Yes. Do I lose something to do it? No. Well, I can spend a lot of time reviewing designs, or just vote for my friend here/randomly. Easier alternative wins”)

I have no motivation to suck up to people in chat rooms to get votes for my designs, particularly as the “”design evaluation’” process as it stands does not really meet my standards for objective evaluation. People are people and will vote as such.

A combination of compute ‘‘mincer’’ and human votes may perhaps be optimal but not easy to code I suspect.

For me at the moment the tag is: designed by humans, voted on by (some) humans, then scored by nature.

Mmm, think I read a thesis on that once, or was it the titanic post implementation review?

I’m sure the game will be a huge success and I sincerely wish everyone well.

All the very best spmm

All my pre launch posts may be deleted prior to launch :slight_smile:

6

Here are some of my ideas about the Lab:

Votes

  • each player has the same number of votes (after unlocking the lab)

  • the votes of different players can have different values, though the default value is the same for each player

  • players can enhance the value of their votes by
    – solving challenge puzzles (for example +0.1 for each 10,000 points)
    – ranking high in competitions
    – designing high scoring RNAs in the lab
    – voting for high scoring RNAs

  • there is an upper limit for the value of players’ votes

  • the value can decrease when
    – a player’s rankings in competitions drop
    – a player designs low scoring RNAs in the lab
    – a player votes for low scoring RNAs

Design slots

  • each player starts with 1 design slot (after unlocking the lab)

  • players can achieve up to 3 design slots (by doing the same things they do to enhance the value of their votes)

7

Well, madde, that is a great idea, as it won’t get into conflict with the current leaderboard system! Plus, it could give way for more thrilling voting periods, different strategies and a better way to break a tie.

It’s only drawback might be that it can be a bit difficult to understand, but that’s nothing that a good tutorial or help page can’t solve.

@SPM: I can see myself in the player lists, rank #22 as for now. But I don’t quite understand what this has to do with the idea discussion.

Maybe you can’t see the logic on my points because you haven’t read on game design or AI. If you want to understand it, I suggest you to visit Extra Credits’ chapter on “Choice and conflict” at The Escapist.

You got it right, that can be thought as subjective, but my point is “How can we be objective? Not everyone is a RNA specialist.” However, if you propose a way in which we players can learn to be objective, I would be eager to learn more about it.

Being able to see your past designs that weren’t chosen or got low scores can actually be a good idea on its own; so players can learn from their own mistakes.

Anyways, if you see the current voting system can’t meet your expectations and my idea isn’t convincing enough too, you can always make a suggestion for yourself on how the game can be improved! That’s what this Forums are for!

8

The title says it all. Many people have been talking about the problems with the voting system in chat. Namely, it would be very easy for people to group together and agree to vote for one design regardless of its merit, and leave the rest of those playing fair out in the cold. Do you think the system should be reformed? Share your opinions and ideas so we can nip this in the bud.

Some suggestions were:
-Ban voting for someone who is in your group. This might prevent abuse, but also might disadvantage those who like group play
-Have the players simply vote to nominate a handful of designs, and let the team running the show make the final call
-Abandon voting all together. Democracy fails to choose designs based on how good they truly are. Have a heuristic-based decision making process.
-Weighted voting. Have players vote for both the best and the worst ideas so that you can gain AND lose points toward being selected. This could water-down any inflated votes.

What do you think?

This reply was created from a merged topic originally titled
Voting! Is the process broken and how do we fix it?.

9

Last week’s Lab 2/Round 2 was my first experience with eteRNA. I greatly enjoyed the puzzle and gaming aspects which led to what I hope was a reasonable design submission to the lab; all great fun!

The voting cycle proved quite frustrating to me for many of the reasons previously mentioned and seconded on this thread and others:
* concerns about fairness and ‘gaming the vote’;
* and adequacy of the design review tool set for quickly comparing and evaluating large numbers of designs.

I very much like the idea of building in additional incentives and disincentives for submitting and voting upon designs based upon points, pitting a capitalist-investment (or researcher-grant) model against a direct democracy model. Given the game’s domain, I imagine the former has more real-world relevance. In any case, it seems additional constraints, risks and rewards are required and personally, I think they’d add to the overall pleasure in playing.

One idea I haven’t yet come across is to separate the design submission from voting phases. Arriving late in the last round with many designs already having received significant votes had two primary effects:
* dampened enthusiasm that my submission would have any chance of consideration;
* after reviewing the top design at the time, I had to fight the urge to build a ‘magic christmas tree’ versus seeking a more balanced design since the former was so well rewarded with votes.

Separation of the design submission and voting phases would make the process more fair as all of the candidates would be visible and available for review from the design phase without any recorded crowd bias present when voting begins.

10

I would urge caution in changing the system before it has had time to display whether it works or not. A couple of days ago we thought there would be all Christmas Trees synthesized and that it was all over, and now we are only synthesizing one of them.

A fear of “mob rule” when a democracy is created is a very common phenomenon, especially among established nobility. We should not fall into that trap.

11

I’m looking over a few of the concerns and we talked about late submissions and just to recall the point scheme. You get a score based on your closest synthesized design.

You get 20 times the score of your best design. We had been talking about ways to cluster your designs so that closer designs would get more points and I can think of a few ways to incentivize that.

You also get 10x the designs from your votes. I think this does toss in some diversity of selection. There needs to be some way of rewarding the lower voting contributions to avoid any hive-mind mentality. Possibly weighting the clustering would help.

13

I think vote value should just be a function of past voting performance and not of challenge puzzle performance, since the whole point of the lab is that it takes knowledge that can’t be gained just by doing puzzles.

One simple solution is to make each player’s voting power equal to the average of the synthesis scores (written as decimals, e.g. 0.94 for 94/100) of the entries they’ve voted for that have been synthesized, or a default value (e.g. 0.50) if they haven’t voted for any entries that have been synthesized. This provides a natural upper limit to voting power (1.00) and obviously increases or decreases it based on how “successfully” the player votes. One could also consider variations of this such as using a recency-weighted average of synthesis scores or squaring the average so as to emphasize the votes of particularly discerning voters even more (i.e. twice the average score = four times the voting power).