Lab Entry Exam?

Feedback Feature Suggestions
1

From the general quality level of design I have seen in the Lab since the Game went Public, I am afraid it may be a good idea at the least, or possibly even a necessity - to begin to require passing some kind of Lab Entry Exam before allowing entry to the Lab.

To see the top positions in the voting list, and the designs likely to be sent to the lab - be largely composed of G-C only designs, or Super-Low energy designs… well, it is nearly enough to make even a non-scientist weep.

Does anyone have a better idea?

2

D9, I think this is a really good idea.

I personally, however, think that before we add additional requirements to the lab, we see how new players do over 2 rounds. New players made their decisions that “Low energy” is important, and I think we should let players verify what comes out of it. The first round of “The Finger” (in beta testing) was sort of like what we have now.

If after 2 rounds player votes still seem to focus on energy or many GC pairs, then we should seriously design more lab tutorials or lab exams.

3

Yes, thanks Jee… I was just hoping to avoid the senseless waste of precious lab resources, but I suppose you are right; there will be nothing anyone can say in advance that will have the impact of seeing one’s “Masterpiece Christmas Tree, or one’s " -100C Low Energy Baby” come back from the lab with a Big Red “FAILED” stamped on it… or a Big Magenta “UNREADABLE” stamped on it.

Hopefully, I am wrong, and it won’t be so bad, or maybe some of the really great new player designs will overtake these beginner-mistake designs in the voting over the weekend.

4

I think you’re going to continue to see that as new people join the lab, but I think this is the only round that might be skewed in a bad direction. This group of newbies (including myself) will learn what y’all have already learned. It’s only been through additional reading of posts (some of them yours) that I’ve begun to understand how different the lab is from the puzzles. No where in the puzzles does it tell you that all GC is bad. And judging from some of the GU contests, you might decide that tons of GU is good.

Maybe we should break the puzzles up into chapters/lessons to teach something and end the chapter/lesson with what you were supposed to learn.

Full disclosure: One of the xmas tree designs is mine. I’d love to pull it, but someone voted for it. I unvoted from everything else and am now looking at the round 1 One Bulge Cross winner as well as the Cross winners to learn what’s viable.

5

the solution fot that is very simple… in the same you you can vote for someone, you can also UNVOTE for some design… like for example i think that design is not gonna work, i give a negative vote on it…
so you have positive votes and negatives ones,
because be indiferent ( not voted) its different from (not liked…)

6

I agree that it’s too easy to get into the lab with no training to prepare for the difference between the puzzles and the way things actually synthesize.

Maybe entry to the lab could be in stages. Stage 1: Entry to be able to study how it works and the way synthesis results look, Stage 2: The option to submit designs, Stage 3: The option to vote

Progression to the next stage could require more difficult puzzles, more points, etc.

More importantly, I think the voting should be more than just a simple yes/no vote. When looking at a new design, a person should have to color in each nucleotide either blue or yellow to show how they predict it will fold in actual synthesis. They would be creating what they think the synthesis results will look like.

This would force people to think about how each design will actually perform.

The blue/yellow mapping of the design then becomes not just a vote but an actual prediction of a score. All the predicted scores then average to a consensus on how each design will actually perform when synthesized. Then the designs with the highest average predicted scores get chosen.

The reason a player should have to fill in a design with blue/yellow rather than just predict a number score is to force people to actually think about why and where a design will succeed or fail.

In addition, as well as these blue/yellow predictions being used to rate design candidates, they can also be used to rate us all as to how good we are at making predictions and therefore how well we understand the process.

Everyone making these blue/yellow predictions will then have an average accuracy rating and our future predictions can be weighted according to how accurate we have been in the past.

This type of system would protect the voting from popularity votes and educate everyone at the same time. There would be an incentive to become more accurate at predictions and to understand why RNA folds the way it does in the real world.

This could be implemented from the current voting screen by allowing a person to select any design as we do now and while looking at it, just color in the dots either blue or yellow. Then submit that blue/yellow colored design as their “vote”.

A couple of things would help to keep things fair.

  1. When scoring a blue/yellow prediction, the number of accurate nucleotides should be counted as well as how close the predicted errors are to the actual errors. This prevents people from just marking 3 random error points and beating the odds.

For example, if 85% of my nucleotide point predictions were accurate then that number gets reduced by the distance between my predicted error points and the actual error points. That number then becomes my accuracy rating.

  1. Designs that are not chosen for synthesis should not count towards the accuracy rating of anyone making a blue/yellow prediction for those designs. Only designs that are chosen for synthesis should count. This prevents people from just choosing the worst designs and marking them as all failure points, and thus padding their accuracy rating. It also gives people an incentive to make predictions about the most promising designs.

Even if this idea isn’t used for voting, it would still be a nice exercise to do. Sort of a “what’s wrong with this picture” type of puzzle.

8

As just some guy who wandered in here, I wish I understood even half of the issues in your message here, I’m still trying to grok the process of moving from coloring book to synthesis issues. If there’s enough on this site to guide one to doing that intelligently, I’ve missed it.

11

@JRStern The “blue/yellow” that is being discussed in the long post above you is that “blue” bases in the lab are the ones that generally successfully bonded with things, while “yellow” bases are the ones that didn’t successfully bond with anything. You shouldn’t worry too much about the long post above you, and instead you should look at some past labs and try to learn something from them. Just open an old lab submission that got synthesized, see what went wrong, and see if you can learn anything from it.

I’m keeping track of the things I’ve learned, but they are not necessarily correct since sometimes new experiments contradict old things I “learned.” But it might be a good place to start at least to get an idea for what kinds of things you can try to learn? :slight_smile: http://eterna.cmu.edu/htmls/player.ht…

Edit: also here is a little compendium of links too: http://getsatisfaction.com/eternagame…

12

Agreed! We are all thrown into the lab without any real instructions on what to do. There’s a big difference between how RNA synthesizes in the practice challenges and how it does in the real world. The point of all this is for us to figure out how it actually works in a test tube.

From what I’ve gathered, we should:

  1. Look through the synthesis results and find designs that did well.
  2. Write down the sequences that worked. (Not the whole thing)
  3. Search through the synthesis results to find all the designs that used those sequences and make sure they are robust.
  4. Search through all the new designs for those sequences.
  5. ???
  6. Vote for new designs that look promising.

Just to clarify, when I say “search”, I mean to scroll to the right in the synthesis results window, type in the sequence and click “search”.

Seriously, most people won’t do that and end up voting for someone they know or like. Since only 8 designs are synthesized each round, it’s kind of hard to test a theory other than to search through the synthesis results on the chance that someone tested an idea you like.

If you figure out more, let me know :slight_smile: