Discussion on new lab system with 20,000 synthesis per month

I agree with much of this, especially with the immediate integration of a new interface to allow us to go through the designs, so we can see what needs to be changed for that. I also like the analysis reward, however, what would be the best method to do that. Would it just be through simple posting of thoughts and strategies, and then, how do we determine where one idea begins and ends.

As for why we go with the one high scoring design, then move on, I believe many of the old labs will be made into projects. This will give players the chance to revise old designs and test various hypotheses, and will also allow us to find ALL “correct” designs. If they are not made into projects, I propose that they should be, or at least the projects with very few successful designs.

Yes.

Interesting set of scenarios.

Both public and private puzzles have advantages. I would opt that there is an option for either. At times, private would be desirable and at others public would be useful. And one may wish to switch back and forth depending on how the sequence develops.

Also, sometimes collaborative (or co-designs) may be desirable as opposed to individual designs and one might wish an option here as well to go from collaborative to individual as the sequence develops, with the option to go from public to private as well. So, some questionnaire as to what “expertise” people have might be desirable so that designers or others can select co-partners which might be useful for “newbies” anyway?

Then, maybe, there might be selectable designs-which might be generated by “the bots”-from which people might select to modify as a project which may fall into the category of public or private designs as well?

The “puzzle owner” is the most likely to choose the designs to select as he would know the parameters he is trying to maximize, but of course one could come up with an infinite number of possibilities, but the owner is really trying to determine certain features of the sequence which is desirable. So, if one is to select slots for a certain round, maybe those should be a limited number per puzzle per round? Really only the owner or co-owners would know when they finished with the design?

The question which seems somewhat vague, is what is the competition in the new format? The ultimate goal is to know if a particular sequence has desirable synthesis characteristics, binding capacity, or whatever, so then badges, points, or whatever should be available for these matters

I fear that the elaborate schemes construed above will turn players into proxy lab scientists. I feel that the ‘gaming’ aspect of eterna is evaporating swiftly.

My suggestion: Flat even distribution of synthesis slots, period.

Gaming is in part about rewards. If you don’t see additional slots as a reward, keep playing the puzzles instead of labs. Besides, having slot reservations does not mean you have to use them. It’s an opportunity.

I think the main thing that I have not seen contention with is assign slots to users, not labs, and let users choose where to use them. That would probably also mean assigning slots to Ensemble/EteRNA for internal use. Say 40%, or 8000 slots per month. That’s enough for 12 slots each month on over 600 labs or projects.

Allocate a good chunk of the rest, say 40% (8 slots/user/month, or 2/user/week) to the top 1000 ranked lab players. (Not a hard target, currently about 23K points.). Save 4000 slots per month (20%) for bonuses that can be earned somehow.

At the end of each lab round (1 week, 2 weeks or 1 month) throw the unused slot reservations into a pool that is spread uniformly to all lab players that month. Probably slots will be left over to allow even lab qualified players not in the top 1000 to get a few slots. Or make it 4 slots to each of the top 2000 players as the baseline - but keep some slots as bonuses that can be earned!! Please!!

Whatever…

Background documents with chatlogs from the discussion of the ideas mentioned above.

More ideas for badges

Brainstorm session

Okay, what I’d like to know is if I’m supposed to be putting designs into the player projects now, or if I should stop because it will mess up all of these schemes that are being designed. I would hate to have my work discarded because criteria changed.

I was trying to get through the projects because I kind of felt bad because it seemed not very many people were doing them (possibly this was an illusion caused by how many there are) and because there hadn’t been enough designs for the last set at the deadline.

Are we going to do more of the player projects soon or are we going to get more into designing how they work? I would be happy to go back to player puzzles (I like the puzzles, and hope there will continue to be time to work on them as well). Or I can continue to add designs to the projects. What should I do?

First of all, we’ll keep synthesizing 1000 or more designs from player projects to test the new high throughput pipeline. This will happen once in 1-2 months, but the cycle will eventually converge to 20,000 designs / month. That’s when we’ll change to the new system.

If we can get the new system ready in 5-6 month, that means will be able to do 2-3 more rounds of player projects synthesis.

Whenever we switch to the new system , all the player puzzles will be transferred to the new system as well.

We’ll also allocate a big proportion of first 20,000 slots to projects transferred from player puzzles - If we keep running the tests mentioned above there’ll be only 2000-3000 unsynthesized designs left in the player projects when the new system rolls out.

  1. Lab rewards?

I don’t need any rewards. just having the designs synthesized.is a reward. Eterna should get rid of the points and ranks, replace it ith virtual money to buy badges or virtual pets like pogo

I agree with Josh, not everyone is motivated by competition and even then not everyone who is is motivated by it the same way. I think it would do good to decide on competitive vs cooperative game sooner rather than later. History is quite a good example on how competition can propel science, the question is: is it worth it?

Right now the game is competitive, and there are people that care more about the points than learning. There are also other people that don’t give a damn about points and just want their designs synthed. This is two distinct groups of people with minimal overlapping. The obvious fear on part of the devs is that there are not enough people in the second group, so if they let go of the first they’ll be stuck with noone left. I think this is completely unfounded, so if there are any rewards they should be the ones that motivate the peolpe that are here to learn: give them synth slots.

But if you really want to give things a twist, let them buy it with their points. Instead of badges.

Is it really even necessary to have player projects, or even any kind of projects for players to submit their designs? I think a big thing that keeps people away from trying anything is the complexity of it, i can’t just go and say: “i’d like to see how this sequence folds. Can i?”. This would of course mean there have to be ‘private projects’ so to speak.

This is a touchy topic because i feel that i can solve some problems more easily using my own methodology. I suspect at least some of you feel the same way and the only purpose of the public projects is really to show that we can also work together. A noble goal, which i feel should not be put either above or below the work of the people doing it.

The first thing i would do is provide a “Quick Synthesis” option where you can just input your sequence, no puzzle maker, no shape data, no 100 word essay. Sequence, period. Keep the current systems, not everyone lacks creativity like me, just don’t restrict us to them. All submissions, whether project or quicksynth should go into the same database, and players could browse their sequences (along with all other data if its a project sequence). Treat it like your own private lab. It also plays in with being able to script your own energy model: your lab, your rules. Similarly, projects would have their own lab with rules set up by the admins. Although, when i say private lab, i’d probably allow everyone to view other players’ and projects’ labs but not to enter without permission.

When it comes to synth slots, i’d give one to everyone that wants one first. That is, every player and every project. If there is at least that many slots you’re green. In your private lab, you order your own designs. In the public labs, the admins do that (can be based on voting or any other creative idea). If there’s enough left to do another round, keep going until there isn’t. Use the rest for the betterment of mankind. We’ll all happily believe that you do as long as you give us our share if there is something to give :stuck_out_tongue:

This is all good and well as long as you have enough slots to give everyone at least one each round. If there’s a shortage of slots, the public labs could be scored and private slots assigned based on score alone. Public labs should start out as private ones, if there are too many of them restrict them by requiring a set number of other players to go public.

Obviously all synth results should go into a public searchable database accessible from every lab. There should be a warning if you try to synth something that already has been, offering you to see the result first. If you still want to go ahead, the database should be able to keep both results rather than the average of the two (assuming they are even slightly different).

The puzzle maker would probably still be the best tool for Quick Synthesis but your design should not have to solve/fold to be allowed to synth it.

Adrien said we should post here if we had more ideas to improve the database and comming game changes.

Ideas for improving the player projects

I have noticed a few things while I’m trying to get started on player projects. Here is my ideas for how the gameplay can be improved.

  1. I read the description before I start solving. But sometimes I forget the description or I need some of the information included in the project description. So I would like to be able to open the project info from within the lab.

  1. In the project above, Quasispecies have also kindly provided the link to SCOR where one can see structures from nature and use as inspiration for solving. It would be very helpful if these links opens in a seperate window, so I can then switch between the pages when I need to. Now when I click the link I get taken away from the player project.

Big thanks

Also on Adrien’s prompt…

The database could include both lab and puzzle results. Perhaps puzzles would wait until there were at least N solvers or until you had solved that puzzle (or both) before you could see other people’s solutions. :slight_smile:

For lab results, it could sometimes be important to see which bases matched and which ones mismatched. Sometimes one might want also to specify only 100% matches (successful) as part of the query, or only mismatches, or both, or a count of how many times a pattern was successful versus unsuccessful?

In searching for patterns in the database, it might sometimes be useful to search on structure, or on bases, or on success/failure, or some combination.

I’d like to be able to search for say successful(?) 2-2 loop designs with at least 2-stacks, perhaps using a pattern something like this:

findBasesByStructure(pattern,success) and countBasesByStructure(pattern,success) where success is:

-2: cases where the pattern failed in a lab but succeeded in a puzzle
-1: cases where the pattern failed in a lab
0: any cases that whether successful or unsuccessful or used in a puzzle
1: cases where the pattern succeeded in a lab
2: cases where the pattern succeeded in a puzzle
3: cases where the pattern succeeded in a lab or puzzle

findBasesByStructure("((…((#))…))",1) finds successful cases of bases that match the structure pattern and folded successfully in the labs.

Here the “#” in the pattern would stand for any bracket balanced/matched string, so the pattern matches a 2-2 loop with at least a 2-stach on either end.

Or a quad multi-loop with at least 3-stacks:

findBasesWithStructure("((((((#)))(((#)))(((#))))))",0) looks for all examples of a quad multiloop with 3-stacks.

Or for a triloop on a quad-stack tested in the lab:

countWithStructure("((((…))))",0) returns the count of successful, unsuccessful, an puzzle usages that match that pattern for each unique base string.

Constraining for specific bases might be easiest to specify by interleaving the base constraints after the structure (with N for *any* being the default; ). For example:

findBasesByStructureConstrained(pattern,success) and
countBasesByStructureConstrained(pattern,success) where success can be:

-2: cases where the pattern failed in a lab but succeeded in a puzzle
-1: cases where the pattern failed in a lab
0: any cases that whether successful or unsuccessful or used in a puzzle
1: cases where the pattern succeeded in a lab
2: cases where the pattern succeeded in a puzzle
3: cases where the pattern succeeded in a lab or puzzle

countBasesByStructureConstrained("((((G…)C)))",3) counts examples from labs or puzzles with a tri-loop closed by GC on a quad-stack. The results may include a count of how many successes, failures (zero) and puzzles there were for each unique pattern.

findBasesByStructureConstrained("((((G…)C)))",3) instead finds specific examples of the pattern used successfully in a lab or puzzle with some indication or the lab/puzzle id, the bases that matched the pattern, the success/match string (in this case all “+++++++++++” since we only looked for successes, and the starting offsets of the pattern(s) (in this case one pattern) within the whole structure.

findBasesByStructureConstrained("((G…(A(#)U)…)U)",0) looks for examples of a 2-2 loop with one-end closed by GU and the other end closed by AU, which may or may not have worked. The results may include both cases that worked and cases that didn’t, and and an indication of success(1)/failure(-1)/puzzle(2) for each instance.

findBasesByStructureConstrained("(((.G…))))",-1) finds examples of a quad-loop with a G boost point that failed.

countBasesByStructureConstrained("(((.G…))))",-1) returns a collection of unique matching sequences that failed, with a count of the failures for each sequence.

findBasesByStructureConstrained("(((.A…))))",0) finds examples of a quad loop with an A in the first CCW open base; the results may include both cases that worked and cases that didn’t, and a count of how many successes and failures and puzzles there were for each unique pattern.

Alternatively, one could search primarily on bases:

countByBases(pattern,success) where success can be as above.

countByBases(“UGU”,0) counts any labs or puzzles that use that sequence, with counts of success, failure and puzzles.

findByBases(“UGU”,-1) finds examples from labs that contain the sequence where at least one of the bases is mismatched.

findByBases(“UGU”,0) finds labs or puzzles that use that sequence, successfully or not. (Hmm, -3 for labs only, no puzzles?)

Note that for puzzles, there are no “unsuccessful” cases, so;

findByBases(“UGU”,2) finds any examples from puzzles that contains the sequence.

Hmm, Looks like Adrien is directing a number of people here :wink:

This post is a request for the EteRNA “roadmap” to be posted, as soon as the developers have a rough idea as to what they want to do with the new lab system, and what the future holds for us EteRNA player. It was mentioned back in December and January, and a few players are wondering what happened to the plan :slight_smile:

Thanks for the game, and I look forward to the new lab system :slight_smile:

Lab interface improvements

I would be very happy that each time I want to hide the dot and meltplot away, I wouldn’t have to click cancel. It would be much easier if I could just press escape and it disappeared.

It would be even better if we could get a hot key to drag forth and pack it away, without having to do all the mouse clicking.

Further on this vein, when doing labs, I keep exporting the sequence to RNAfold and RNAshapes to get better feedback on the overall design strength and the location of weak spots. This is an area where the lab tools are currently weak.

RNAshapes gives alternative foldings with probabilities.

RNAfold gives per-nucleotide local entropy values as well as fMFE and ED.

Now in *theory* some of this data is present in the dot plot, but in practice it’s hard to interpret it or to see the plot and relate the points in it to the nucleotides in the structure.

Assuming that the EteRNA model can compute some of this data, some thoughts on how to present it:

a) Include fMFE and ED (or near equivalents) on the dot/melt plot page or even on the main lab screen near the FE.

b) Add a positional entropy graph (like that of RNAfold) under (or beside) the dot-plot with the positions aligned with the dot plot columns (or rows). When hovering over a position in the graph, highlight the node in the structure.

c) Allow optional coloring or shading of the rings around the bases to indicate positional entropy values (and/or allow the use of more degrees of colors or shading rather than just red/white in the bonding status icon). Also, when hovering over a spot in the bonding status icon, highlight the same region in the main structure.

d) provide a(n optional?) mode where hovering over a nucleotide with say SHIFT-MOUSE-DOWN would show radiating lines of various colors or shades to indicate relative probabilities of bonding between that nucleotide and others. The ring around the nucleotide would then indicate the probability of remaining unbonded. Hovering over a node may also draw a vertical line on the dot-plot and/or entropy plot for the region that represents that node when both are visible, and/or

(e) provide a mode where hovering over a dot in the dot plot would draw a line between the two related pairs that was colored or shaded to indicate the probability of those two bases bonding.

(f) provide a slider to adjust the *temperature* of the model to see what happens to the bonding as the simulated temperature changes. This might also help to indicate weak spots as they would be the first to break as the temperature rises.

(g) something I’ve not yet thought of which occurs to you based on how to make the data available in the model or parameters that can be reasonably tweaked made more accessible to the user.

Any way to improve the feed back on where the weak spots are in a design would be greatly appreciated!!

(h) When marking a node with CTRL-CLICK, also mark the same node in the dot-plot and bonding status icon.

BTW, I *love* this game - I just want to make it even *better*!

And just to be clear here, I’m *not* talking about integrating RNAfold or RNAshapes into the game. I just want to get a better fix on the information that the EteRNA model already *has* about the design. If it can draw a dot-plot it *must* have some idea of positional entropy and probabilities of pair-bonding and I want to expose that in a better fashion to the end-user. Likewise if it can draw melt-plot, it must be able to vary the temperature…

It’s about making it available in real-time to the user withought having to go through a submit/wait/look cycle. Also the UI of the game makes it possible to flip back and forth from the structure to the graphs in ways that the external resources cannot (but it doesn’t do that yet)