During Eternacon 2017 the question came up regarding getting points for the labs. Do we all want to start receiving points for labs? The results of this conversation will help drive getting lab points or more gamification. Please comment if you have any thoughts.
1 Like
I’d like to see a relatively high amount of points for up to 5 to 20 lab submissions per Lab category - which would incent more gamer participation and broader participation. But with that limited amount of submissions to avoid incentivizing poor quality submissions just to get points.
Here are some opinions that I have about it:
- We need to decide on a way to score it that will not be abuse-able, which also means that players won’t be getting more points by behaving differently than others, say by posting designs of lower quality or a lot of similiar ones.
Another point brought by (I think) LFP6 is that we can’t always know for sure the quality of the design- Maybe I submitted a terrible design to learn from it? And what about running # experiments?
Solutions to those problems might be giving a high amount of points to the first couple of solutions (per @Gerry’s answer above) only to give a motivation boost for entering the lab in the first place, or maybe letting the devs decide on the top lab players and sorting the others by total lab score or by a formula, and scoring them based on their leaderboard position. - Until now, at least as far as I know, the points function like a counter of how many puzzles you solved- They can’t go down, you can earn every “point” only once, and no point sources were canceled. I don’t think that this “feature” is really important (or even intentional), and I am honestly against it, but as giving us points from labs cancels this, it should be considered whether or not we care about it.
I do think I like a permanent, injective (As in every point had a one time source) system for scoring, but I don’t think that a score counter is what we want. Maybe a counter of how many puzzles did we solve from each type or just badges for inijective stuff. - I think that other status items, like badges, can be a great motivator. Especially if badges will have a higher meaning after the site redesign (Or if we will have an achievement announcer in chat, I didn’t give up on that idea yet :P).
“Another point brought by (I think) LFP6 is that we can’t always know for sure the quality of the design- Maybe I submitted a terrible design to learn from it? And what about running # experiments?”
I agree with LFP and there have been good examples in the past where we have learnt from these designs, as not of those turned out to be terrible designs.
“Solutions to those problems might be giving a high amount of points to the first couple of solutions (per @Gerry’s answer above) only to give a motivation boost for entering the lab in the first place”
its not a race.
“maybe letting the devs decide on the top lab players and sorting the others by total lab score or by a formula, and scoring them based on their leaderboard position.”
Which leaderboard ?
For the second quote- what do you mean by a race? Btw the first couple of solutions (at least in my quote, not sure about the original idea) is for the first solutions the player submits, not the first ones the entire playerbase submits, if this is what you meant.
And in the third, I meant creating a leaderboard at the end of the lab, in which the devs will choose the top players based on their contribution, and the other players will be placed base on their scores. Sorry for being so unclear.
Do we all want to start receiving points for labs?
-For me it does not matter .
That is the reason we want points in lab- so more people will come there.
For the second quote.
So for each player first couple of solutions submitted they will receive points for those.
Was that per sublab or the entire lab ?
And in the third
Np ok.
No idea!
However I think it isn’t a question about the framework / idea of lab scoring, but a more practical one; It doesn’t really matter for the decision of whether or not to use that idea.
aligning points or badge as close to what the research wants is key. So rewarding points after labs are synthesized based on however they are scored makes total sense. You get what you measure.
The problem is that the score only shows how good is that RNA is folding, now how much did that design helped us learn. Although [new idea] maybe this could be negated by giving a score to a research as a whole? (and maybe a badge too)
I don’t think most players care about badges.
Then it is probably mainly me caring so much about my badges. However, it should still be a consideration for those like me.
And only now I realized how unclear I was on my reply- by “to a research as a whole” I meant that all of those who participate in a small research project (like Eli’s 20A hairpins, or researching how distant should the aptamer be) will get a score boost for that research.
Points for labs is better than no points for labs (for me I guess? and maybe others?). For these high-volume labs, give a very small number of points (less than five?) for each design that ends up scoring better than a preset threshold. The idea here being to encourage large amounts of usable submissions. So if someone ends up with one or two thousand designs that weren’t junk, they would net the number of points equivalent to solving a few dozen puzzles, which is about what a good set of labs were worth the last time points were awarded for labs iirc
1 Like
Like hoglahoo, I like the idea of points for each submission. (Keep it low, about 5 or so.) Then add points for each part of the design that folds correctly to encourage good designs. As for testing or experimenting , they have a chance of that test working and scoring well. It’s a risk, but there are no guarantees for ANY design.
Better to have no points then giving only 5 points per design.
It’s not really 5 per design, but 5 per mutation, which changes things by a lot.
However that is a great point in general; For example we’ll have to reconsider it for TB…
I assume that labs will come with constraints that should be met to be accepted. As long as a submission is accepted, there should be points awarded without someones judgement on how good or bad they are or why the player submitted that particular design.
Accumulation of points would represent a players level of participation.
Individual lab puzzles are scored by the lab.
When there are 100% scores assigned to some submissions, the lab is the real winner. There should be a puzzle achievement score to recognize those players.
In addition to the achievement score, I suggest a high, low and mean score of a players puzzle scores using the players scores between 30 and 90.
My idea is to have scores that can be calculated easily from the data collected.
1 Like
100 points for completing 100% of the lab (no questions asked). 100 points for completing 10+ - 94% or better lab scores.