Hi @eternacac. Good to have you active again!
It is true that more experienced players will often ask for a personal bump in their quota when slots would otherwise go unused. But in my experience, those same players are the ones who are most vocal about finding ways to get more players more involved, not increasing their own allotments. I think the more experience one gets, the easier it is to appreciate the value of diversity in the submissions.
LFP6 is correct. It’s not so much about increasing the turnaround time, but just running the MS@ switches at an earlier time, given there are currently no Eterna rounds in the experimental pipeline afaik. If the next time opening for the sequencer isn’t until the end of this round, or if the budget or work required doesn’t allow for it, then that’s a different set of circumstances.
@Omei Turnbull Oh well. As somebody with 420 designs submitted should I delete 60 of them now, or wait for the lab to fill? I certainly don’t want them to be randomly chosen. Do you notify relevant players before this cut so they can react? I mean when I started I had no idea that there were other engines available in the lab so my first designs were Vienna only and so I wouldn’t feel that bad deleting these.
Btw is the maximum number of slots 3600 or 7200? The first number is advertised in the lab page and the second one on the home page. Maybe these numbers should be somewhat more unified.
@LFP6, @Omei Turnbull I see your points and mostly agree. But in my experience there usually is a following problem with voting:
If you give players the ability to vote for themselves they will do it. Considering that right now there is less votes available than slots for any given subproject I can’t see many people voting for anything but their designs. Add to that the fact that it also ensures the design to be synthesized and I would bet more than 80% of all votes ever casted were by players for their own designs. Maybe I’m too cynical, but that’s how I feel about it.
If you don’t give players this ability they will be less willing to vote. Incentives could help, but there is still the problem whether these votes determine which designs will be synthesized. If votes help the design then why would people vote for those that are made by their “competitors”. I mean score and achievements are nice, but the most cool thing here is to have your design synthesized. So if not voting helps in any way, people won’t vote. And if voting doesn’t help the design then it loses most of its relevance.
Well, I thought this is something worth mentioning. I’m sure you’ll find a way to balance these things. Just don’t underestimate the players’ ability to always act in their best interest (And I don’t mean that in a bad way).
@Pi I would say that if you want to replace some of your early designs with better ones, go ahead and start doing that now. But otherwise, you might as well wait until things are more settled. For example, @Brourd asked if it would be possible to limit this lab round to just the puzzles that are currently available, thus doubling the number of available slots for these puzzles. I don’t think that’s feasible, but I’m not the one who would make that decision, so I have brought the question to Rhiju’s attention.
As for the numbers, the current plan is to release another set of puzzles (i.e. a second project) that also has an allocation of 3600 slots. Together, the two projects would have 7200 slots in this lab round.
@Pi Agreed. So let’s see if we can design a voting system that aligns an individual’s self-interest with the scientific interest of the Eterna community. Here’s a strawman proposal.
The vote allotment per player for a puzzle is set as some fraction of the design quote, say 40%. So if the submission quota for a puzzle is 30 designs, each player would have 8 votes. A player could use these votes to “protect” certain of their own designs from being cut, if cutting had to take place, but they couldn’t use their votes to increase their slots (unless perhaps other players were voting for their other designs.)
The main incentive for voting would be to gain game points by getting good at predicting which designs would do well in the experiment. To that end, each submission that made it through the experiment to the point of getting results would be awarded game points in proportion to its Eterna score. But these points would not all go to the submitter, they would be be split equally among the submitter and all the players that voted for the design. As an example, suppose a design with an Eterna score of 90 earned 900 points. If 9 (non-designer) players voted for the design, each of those, plus the designer, would get would get one tenth (90) of the 900 points. On the other hand, if no one voted for the design, the designer would get all 900 of the points.
What do you (and others) think of that as a voting system?
I cant see it being a fair thing that 8 voters could get as many points as someone who has designed the puzzle.
@Omei Turnbull That’s not bad. I like the idea of “stealing” points. Though it could incentivise to vote late and only for designs nobody voted before. If there was one vote on a 90ES looking design and zero votes on 60ES looking design I would go for 60 and guaranteed 300 points. Well, it’s never guaranteed but you get the point that any design that already has a vote is not very lucrative.
I’d probably prefer a more nonlinear approach since the difference between 100 and 60 score designs is that the first one is useful and the other useless (if I underestand it correctly).
There also could be a problem of delaying design creation in labs that guarantee synthesys. Or you could just delete and reupload designs right before the end just to ensure all points (if there were enough slots). And that would make a mess deleting all the casted votes as well.
I’ll give you my idea. Lock 10 slots and unlock them only if the player casts 8 votes. That way it is more mandatory and the player doesn’t feel bad for helping 8 competitor designs. I think I would prefer the concept of banning self-voting completely, or separating it from the “Best design vote”. Maybe you could do 8 non-self votes + 8 self votes. Using self votes to ensure those designs for synthesis and the rest could be cut based on votes in general. That way you could award design makers with their full EternaScores and Voters with EternaScores if > 60. Or something along those lines. (Considering you already have “Votes” and “My Votes” in your database I’m pretty sure you could do this even without adding a column)
So this way when you first enter the project page you have only 20 slots and there is some convincing visual saying “Unlock 10 more slots by giving 8 votes”. You can start designing if it is early and there are no other designs, or you can go and vote giving you 30 slots for every subproject you voted on. This way people know about voting and if you give them some incentives to vote for the best designs they might do exactly that.
And btw you could also make some use of that score players are getting (it seems pretty useless to me) and maybe give some additional slots based on the rank. Something like 10 additional slots for the top 100 players, 9 for 101-200, … , 1 for 901-1000. And lower the base slots accordingly. So for example 15 slots guaranteed, +10 more for votes and +10 possible according to the rank. That way people could start caring about that ($) score number.
@Pi Removed the extra posts.
I personally think that designers should not be penalized for supports. Having a mechanic for voters though seems good, to promote voting on original designs and not just going on the bandwagon - I do need to question it a little though, as it could create some negative feelings as new people vote, they’re “hurting” previous voters. Not saying that it’s a huge issue, but I’ll throw it out there.
As far as “self-votes”, I’d like to see what was discussed in dev chat about bundles (which I still want to see) - guarantee a few slots, then the rest go to voting. I would find it interesting if we could adjust the scoring mechanism so that the “$” can be used to “purchase” synthesis slots/votes/etc (with some cap), with rank/“XP” being determined otherwise (ie my suggestion of a more holistic scoring approach, again still need to post that). Not a right now thing, but I think worth investigating at least.
My question is, what is the utility in implementing any of these things in the game? These problems are only relevant for lab rounds with a large number of targets and fewer synthesis slots. Is the eventual goal of Eterna to focus on rounds similar to this one, where there are 32+ design and synthesis objectives, with a limit of 30 slots/player?
I would also assume this discussion is focused on the eventual update to the Eterna game UI and move from the Flash platform, which would occur in the distant future, and the needs of the system may be completely different when that time finally comes.
@Brourd The utility is 1) to encourage more players to look at the designs and help identify what distinguishes good ones from bad ones, and 2) do it in a way that leverages the appeal of game playing.
As for implementation, if there is sufficient interest, we can start the development with this round. We already have the voting mechanism in place. i could write an Eterna Script that pulled and summarized the current voting status, and we could run that on a regular basis and post each one here in the forum as the round progresses. We would probably treat the first round as a dry run, with award points calculated, but not actually rewarded in game.
If this trial round is a success, we can see where it goes from there. But I think this would actually work very well as an Open Source enhancement, because the integration effort with the legacy code could be kept to a minimum. A reasonable goal for the next round would be to tweak the scoring system based on the dry run, and to create the mechanism for awarding real points based on the voting results. In subsequent rounds, we could build a new HTML UI for the new activity, and integrate it into the Eterna UI in a way similar to the new data browser. That would actually be where most of the effort would go.
This is all right in line with the way Rhiju and the dev team see Eterna going. The limiting factor is how much players want, and are able, to contribute their energy.
@Astromon, @Pi, @LFP6 I had seen my proposal as being very favorable to designers (even though they would have to “share” the reward with their supporters) because for all their designs that didn’t get any votes, they would get all the reward. But as your examples comments out, things would be more transparent if we just treated designing awards and voting awards as being distinct.
As for sharing a “pot” for each design equally among those who voted for it, it’s analogous to parimutuel betting. Previously, voting was very subject to the snowball effect because many players would not by looking at the designs, but voting for the same designs that others had, with the presumption that the previous voters actually knew what they were doing. Not surprisingly, there was not a very high correlation between votes and scores. With parimutuel betting, following the crowd is counter-productive for the individual.
As of now and the last 7 or 8 labs I have participated in there is no points for designers or voters so any point system put in is a plus. That said I think designers should get points for good designs . (so many for an 80, 90, 100) the voters can get points for voting for a good design but it shouldn’t effect the points that the designer gets. (and would be much less than the designing players points) Is that not doable ? Thanks!
Thanks for all the comments. We are really excited about the current round since we hope to demonstrate that the great results for the fmn/ms2 switches can be generalized to any combination of aptamer and reporter. That is the reason for the late number of puzzles. In the previous round we noticed that same state switches seemed easier and that placing the reporter between the aptamer arms was better. We want to test this using different puzzles with the same aptamer. I was very impressed by the large number of solutions already. The limiting factor for experiments is both cost and time and separating the round into two experiments doubles both.
We are trying to gauge the excitement among the players so we can better match the experiments with the design. Would more slots fill up, similarly to the fantastic results so far, such that adding more capacity would be welcomed by players?
@Omei Turnbull You could also make the voting private to avoid the snowball effect. If you don’t show casted votes at all during the voting process then you’ll know for sure that they don’t affect other votes. And you can reveal them again when the project closes.
I think the extra sub-labs will be welcomed and filled within the six weeks allowed. If not a couple extra weeks can be added. By all means add them as soon as possible, it will be a good test within itself to see if 26 labs will fly!
Considering the cost and time saving factor I think this can be done. Thanks!
@Omei I don’t understand how your proposal would encourage players to look at designs to distinguish good ones from bad ones. If a player is only given 40% of their total slots as votes, that would mean that for the hundreds of designs submitted for each target, a player would have, given 30 design slots, 12 votes. If there is a hypothetical 300 solutions submitted for a lab puzzle, their vote would only account for 4% of all submitted sequences. Given the average score for most sequences in puzzles like these have typically been between 60 and 75, as @Pi stated above, as a player there is about as much of an incentive to comb over the details of the thermodynamic, kinetic, chemical, and structural characteristics of a sequence to ascertain whether or not it’s a good switch as there would be to randomly place my votes using a random number generator. This would also detract from the appeal of game playing given the serious time investment required to do any of these things. One of the most important aspects of games that require significant time investments is a feeling of progression, whether that progression is an artificial inflation of the player’s avatar using a change to the status or due to a gain in skill from better understanding the mechanics of the game.
Hence, I would agree with @LFP6 that an ELO system for comparing sequences would honestly be a better system, but you run into another problem. What’s a good switch? How many players, when given two sequences, are likely to pick the “better” sequence of the two? If my memory serves me right, the Eternabot for switches hasn’t done particularly well based on the strategies players proposed has had the same success as the original, which may be indicative of a more widespread issue.
I would probably say that there needs to be a place where players can “train” in order to better identify good switches, before any change to the voting system.
It’s sad to think that players creativity might be randomly reduced by eliminating their submissions. The fact that players can supply way more than the lab can consume is a credit to the players to be sure. Maybe letting the current 12 + 16 puzzles play out with a max of 50 submissions for the first 12 and making 30 submissions for the second, closing the lab when 7000 or whatever is the lab max limit would be the way to go. There are about 20 players or less in the puzzles now.
I’m still trying to sort out R104,5 and 6, so I’m not very active in R107. For me, the time between the Rounds is not enough to get and analyze the spreadsheets, fusion tables graphs and 2D structures. I don’t mind playing in the fast lane, but when I come to the fork in the road, I’d like to have some feedback as to which way to turn 