Discussion on new lab system with 20,000 synthesis per month

1

As some of you may already know, EteRNA is expecting a groundbreaking change in the very core of the lab. Until now we could only synthesized 16 sequences every 2 weeks. However, Das lab (biology side creators of EteRNA) at Stanford University is developing a major breakthrough which would allow us to synthesize 20,000 sequences every month. This would mean that basically EVERYONE could get at least 20 RNA sequence synthesized each month if 1000 people participate in the lab.

We are planning 3 major updates along with the change

  1. Massive RNA lab puzzles

With so many synthesis slots available, it wouldn’t be the best idea to use entire 20,000 synthesis for solving 1 lab puzzle at a time as we do now. Instead, we’ll ask research labs around the world and YOU to propose your own RNA lab puzzle and use synthesis slots. Every month 20,000 synthesis slots will be distributed to every active lab puzzle. For example if you have a specific shape of RNA you want to try synthesize, you could propose it as a lab puzzle - you’ll soon be assigned synthesis slots, and you can submit any sequences you want to get them synthesized.

  1. synthesis database

Every synthesis result from every lab puzzle will be publicly available. We’ll deploy a database and appropriate interface in which you can search/review experimental results. You’ll be able to search synthesis results by sequences, secondary structures, and in many other metrics.

  1. Scripting

We’ll provide a scripting interface with which you can design your own computer algorithms for solving puzzles, designing good lab submissions, scoring lab submissions …whatever you want to do. (This is a generalized version of “Strategy Market” in which algorithms were limited to scoring lab submissions.) Scripting itself will be a fairly expert feature since it requires programming skills, but we’ll develop a way for players to share scripts so that everyone can benefit from the expert effort.

As we are planning these massive changes to the game, there remain many important questions that are yet to be answered, and we would love to hear our players’ opinions.

  1. What will a lab puzzle look like?

Now that everyone can submit their own lab puzzle, what will it look like? Should each lab be public so that everyone can participate and submit sequences? or should it be private so that only the puzzle owner can submit a sequence?

If a lab puzzle can be public, how do we pick sequences for synthesis from each lab? Does the owner of the project decide which sequences get synthesized…or do we still want to keep voting interface in each lab?

Can other players review/rate the project? If so, what would be the best way to represent them? Also, can more than 1 players be co-admins of 1 lab project?

How do we know when a lab puzzle is done? Does the puzzle owner decide that?

  1. How do we allocate synthesis slots?

Say we have 2,000 active lab puzzles. How do we assign 20,000 synthesis slots to those lab puzzles? We could just evenly distribute 10 slots per each lab - but is there a better way to do it? Is there a way that we allocate more synthesis slots to those who are good and can make more efficient usage out of the slots? If so, is it possible to still give newbies a chance to get synthesis slots?

One idea we had was we could again entirely leave this to the community. If we have 1000 lab members, we could simply give 20 synthesis slots to each person, and everyone can decide how they will ‘donate’ their synthesis slots to projects. This is as transparent we can get with the resource allocation, but will share same cons with the current voting method.

Another idea is we come up with an resource allocation algorithm that will consider lab puzzle owners’ performances, current community rating of each lab projects, number of submissions in each process, etc and automatically decide allocation. The obvious downside is that this process is not transparent, and we have no idea what kind of things the algorithm needs to consider at this point. However, if we tune the algorithm well, it would be perfectly objective allocator which could maxmize the outcome.

  1. Lab rewards?

What is the best way to reward players for creating/pariticipating in lab projects? Until now, everyone got rewarded for submitting/voting in the lab, even if the designs they submitted/voted for did not get synthesized. This turned the lab rewards to be more like a weekly income that is not as exciting as it should be. Perhaps the lab rewards should be big/hard to earn such that good lab players can easily climb up the rank and the entire rankboard has more dynamic movements. On the other hand, the lab rewards shouldn’t be to hard to earn as new players might get scared away.

These are some of the main issues that the dev team has been actively discussing lately.

Let us know what you think!

EteRNA team

2

This aspect of EteRNA has proven to be an exciting feature, and I cannot wait for it to be implemented. Now, onto the ideas for improvement.

  1. What will a lab puzzle look like?

Whether labs should be private or public.

Answer - Public, however, I understand why some authors may want their design to be private, such as the ability to directly control the designs synthed, because other designs may not fit the criteria of their experiment. This could easily be solved by giving authors more creative control over the project sequences, by allowing locked bases, required pair percentages, not having a certain number of bases in a row, etc. Authors will of course be required to solve their puzzles using whatever criteria they set.

Which designs are chosen from a specific lab?

This one is a bit more difficult to answer, since, hypothetically, players may only be given 20 slots. Now, I would suggest that players be given the ability to prioritize designs for synthesis. Let’s say I want my designs for main labs and my experiments to be higher priority, over the designs submitted for another lab. I would simply mark them higher on my priority list, and they would be chosen for synthesis first. If no priorities are set, designs chosen per player will be based on date published.

The solvability of lab puzzles and designs.

There are many secondary structures impossible in the current energy model, and I believe players should be given the chance to test those. This will require players to submit designs that may not work, and I believe whatever feature that does this should be implemented at the same time. This could also be one of the features authors could set for their lab project, whether or not it needs to be solved using the Vienna energy parameters, and how much players can stray from the path. Like other requirements, authors would be required to solve.

Puzzle Rating/Review

The community should be allowed to rate a puzzle, maybe through the old star rating and comments. Still not decided on this one.

Co-Admins

Yes, I believe that players should be able to codesign a project and oversee it, just as I believe players should be allowed to coauthor micropapers with those same players that were a part of creating the project.

When a Lab puzzle is done?

Here, it will be entirely up to the player in charge to decide when a project is done, unless they preset a synthesis score that they are aiming for. Now, if a project goes cold, with no more submissions, and the author is inactive, the developers, or a community member assigned by the developers, should be able to review the project, to determine if synthesized designs have met the criteria set forth by the author. If not, the design could be moved to the front of the projects, or maybe removed altogether. The “cold” period should be a reasonable amount of time, such as six months, that way the author is given a reasonable chance to create designs for it.

Final suggestion for this, limiting the number of projects one player can have open at any one time. Sure, it may be great data for the ensemble algorithm if all the scoring is implemented, however, even if players are able to script sequences for every project, they still would have to sift through all that data in order to improve their own scripts, unless players are going to be given the chance to build their own self-learning algorithm. So, I would suggest that the feature to create projects be locked until 20,000 points, and this will unlock the ability to post one project. Then, as the number of points you gain increases, you will be given the chance to unlock the ability to make more, until you reach a maximum. I don’t know what this max would be, maybe 50, or maybe 100, but players have to have earned the ability to make that many projects.

I will consider this the end of post 1, which is probably full of errors and typos.

3

Post 2 - How do we allocate synthesis slots?

My first suggestion for this would be a hybrid of all the options: puzzle allocation, player’s choice, and automatic allocation, and it would also be partially be based on the type of lab puzzle. For developer projects and projects specifically chosen by the developers, at least one design from each player should be synthesized, unless they choose for it not to. Then, if the submitted desgns do not exceed or match the number of assigned slots, players will be given the chance to vote on the remaining designs, to see which ones are chosen for synthesis.

All players will also recieve a base amount of slots that they can freely distribute to whatever project they want. For our purposes here, let’s say this base amount is 10. Now, the rest of the synth slots will be automatically allocated at the end of the month, after all designs are submitted, and the number allocated will be determined separately. The algorithm that does this will take into account activity, scores, etc. Then, the designs on the top of the player’s priority list will take precedence over those near the bottom, and will be chosen for synthesis. So, if a player was allocated 20 extra slots, the top 20 designs on his priority list are chosen for synthesis.

So, sound off in the comments below. This will be considered the end of Post 2

4

Post 3 - Lab rewards

The concept of lab rewards is kind of a touchy topic, as some believe they should not be there, etc. Now, One part of lab rewards needs to be badges. Lots and lots of different badges that players can share, post comments about, be envious over, etc. The reason being that anyone can get them, they just have to participate and try their best to get the badges.

Some things that I would like clarified include what makes a lab reward big/hard. If it is the construction of a perfect 100 design, that may prove to be impossible for those who spend time with projects that are not likely to ever have a perfect synth score. So what else is big/hard, as a clarification of this would be much appreciated.

We’ll consider this the end of post 3 for now, although, expect future posts for all these ideas as they pop into my head. Thanks for reading.

5

You could assign an equal number of synthesis slots to the first 1000 people and assign the remaining synthesis slots based on lab results. For example, assign 10 slots to the first 1000 people and assign the remaining 10,000 slots to players based on how well their previous designs have scored in the lab. Newer players would be able to earn extra slots by doing well in the labs. Lab rewards would be based on how well the designs score. Voting should be eliminated altogether. I believe this will help eteRNA retain newer players.

6

Brourd wrote: This aspect of EteRNA has proven to be an exciting feature, and I cannot wait for it to be implemented.

Ditto! I think this may help with retention of experienced players if managed well.

  1. What will a lab puzzle look like?

Whether labs should be private or public.

Free synthesis slots should be Public. (If there is pay to play, that’s another matter.) I also agree with Brourd that authors should have more creative control over the project sequences, by allowing locked bases, required pair percentages, not having a certain number of bases in a row, etc. Authors will of course be required to solve their puzzles using whatever criteria they set.

A loophole/exception may be made for certain projects at the discretion of the project designer. This might be in the nature of maximum number mismatched bases allowed on a candidate/template, normally zero, that would be accepted during submission. This would allow testing of design elements near the limits of the energy model to see how they fold.

Which designs are chosen from a specific lab?

This ties into my answer to part 2 below (How to allocate synthesis slots) and partly depends on how the 20K slots will be spread through the month. More details
later, but basically a system of reserved slots and “like”-influenced lottery slots.

Puzzle Rating/Review

Tied in with Reserved Slots and Likes and Lotteries below. I would suggest that players be limited in the number of active projects that they can have open. The number should be large enough to generate 5000 candidate submissions each week, but not infinite. A project may be deactivated at any time by its owner, or after 4 weeks of no new submissions and no newly-synthesized candidates. (More below.)

I like Brourd’s suggestion to limit the number of active projects for players, but I’m not sure about the exact scheme/count. He suggests “the feature to create projects be locked until 20,000 points, and this will unlock the ability to post one project. Then, as the number of points you gain increases, you will be given the chance to unlock the ability to make more, until you reach a maximum.” Alternatively, there could be a fixed number of projects allowed past some points/rank limit which not everyone will exercise fully. Slots are assigned to candidates, not projects under my scheme, so the desire is simply to avoid project “pollution” of too many active but “silly” projects, while maintaining enough “serious” projects to get submission of at least 5000 synthesis candidates per week.

Co-Admins

As with Brourd, I believe that players should be able to codesign a project and oversee it, just as I believe players should be allowed to coauthor micropapers with those same players that were a part of creating the project. The creator can moderate requests to co-admin a project (accept/reject) but cannot “kick-out” an co-admin, who must resign.

When a project is retired, it should be eligible to be adopted by another player who must make sure that at least one design is synthesized every 4 weeks, either by slot allocation or by lottery, else it will be deactivated.

When a Lab puzzle is done?

A project which (a) has active admins and (b) is getting at least one design synthesized every 4 weeks is considered active. When an admin is finished with a Lab Puzzle, they may resign as (co-)admin and regain their project management “token” to use in administering another lab project.

If the project has no active admins (all have resigned), it is deactivated, even it it has unsynthesized designs left. It remains in an old-project archive however and may be “adopted” but will not show up in the general list of active projects. Candidates will not be selected for synthesis from old projects unless the total synthesis slots assigned to active project candidates is less than 5000. In that case, some old candidates may be used to back-fill, but this probably means the submission/selection parameters need tweaking.

If a lab has active admins, but has not had any synthesis done for four weeks (due to no reserved slots, likes, or lottery wins being assigned to any of its unsynthesized candidates) then it will be retired and the admins released from their roles. It will also be placed in the old-projects archive.

  1. How do we allocate synthesis slots?

I see a multi-level rank-based strategy combined “reserved slot allocation” with “likes” and a slot lottery. Assuming for the moment that here are 5000 slots available each week, I would allocate reserved slots to players based largely on rank.

rank 1-50: 30 slots per week
rank 51-100: 15 slots per week
rank 101-200: 8 slots per week
rank 201-500: 3 slots per week
rank 501-1000: 2 slots per week

Total slots: 4950 slots per week by allocation

“Like” allocation is more democratic: Each Lab Member player (over 10,000 points) would be give a “like” budget of around 200 (TBD) Likes per week. They are free to review lab/project candidates and say “I Like this one”, much like the current votes.

Anyone over 10,000 points is a Lab Member and can submit up to 5 (or 10?) candidates to each active project each week. Lab Member can also review and “like” up to 200 (TBD?) designs per week. Depending on the number of active projects they may be able to submit more candidates than they can “Like” and definitely more than they can assign reserved slots to.

Rank 501 to 1000 players also have the option exercising up to 8 slot reservations every four weeks (2 per week) or about 40% what they would get at a 20-slot per person per month strategy for the top 1000 players. Highest ranked players however, would be allocated 120 slots every four weeks or about 6x what they would get on an even allocation basis. This allows the more experienced players more option for influencing synthesis and the scope to run controlled “projects” where they can use their slot allocations to help select candidates for synthesis. Slot reservations do not “roll-over”; you must use them in the assigned week or else they get assigned by lottery that same week.

Now here’s the trick. Reserved slots are allocated based on your rank at the start of the week. You are free to assign from your allotment of slots to *any* project candidate, whether you designed it or not, whether it is in your project or not. If you assign a reserved slot to a synthesis candidate in any active project it *will* be synthesized. However there are fifty slots unallocated and not everyone will use up their allotted slot reservations, so at the end of the week, there will be synthesis slots left over.

If you are not certain that you want to assign one of your reserved slots to a candidate, you can simply *like* a design. “Likes” work like a lottery ticket. The more “likes” a design has, the more chances it has of getting one of the leftover synthesis slots in a weekly slot lottery. However, you only get to like it once per week; it gains more chances by being liked by more players, or by lasting multiple weeks. (A player may delete old synthesis candidates, even if they have associated “Likes”; this is different from the current model.)

It is still possible that even all the allocated candidates and all of the “Liked” candidates there are still less than 5000 designs for some week. In that case, the remaining “unliked” submissions are put into an even chance lottery for the balance of the slots. If there are *still* less than 5000 candidates for that week, candidates from deactivated project may be used, but we probably need to tune some of the other parameters to get more submissions!

Weekly rollover.

At the end of a week, some candidates will be selected for synthesis, and some may not. All of the candidates assigned a “reserved slot” will be scheduled for synthesis. Some of the “Liked” and other candidates may or may not, depending on the degree of project participation and lottery results.

The admins have the option automatically to “reset” or “retain/roll-over” the “Like” counts for candidates in each project after synthesis selection. If they are retained, then a user may “Like” a synthesis candidate a second (or third…) time in subsequent weeks to boost its chance of eventual synthesis. If they are not retained, then like the current project setup, the counts are reset to zero and new designs compete equally for slots in that week. Both approaches have their merits and some project admins may prefer one approach over the other. Note: it is not that they have a line-item veto: they either clear the Like counts for *all* remaining synthesis candidates, or none of them.

Even if the “Like” counts are not reset, each player has a new opportunity to submit 5 (or 10) new designs to each project each week. The number is higher than the current 3 as we are trying to get at least 5000 candidate submissions each week, and given perhaps 200 active lab players, that means 25 designs per player per week on average!

Slot reservations do not “roll-over”; you must use them in the assigned week or else they get assigned by lottery that same week.

7

Lab Rewards

I’m tempted to say that labs are their own reward?! There may be some point reward based on exercising aspects of lab behavior that we want to encourage:

a) being a project admin during a synthesis cycle (50 points per project with active synthesis)

b) exercising your reserved slots (10 points per slot used?)

c) liking a design (one point per Like?)

d) submitting a lab design (5 points for submission, 10 if it is synthesized)

e) points based on synthesis score on lab puzzles marked by the admins as targeting 100% designs

9

a.2) 10 points for each synthesis of a design in project that you manage - rewards the design of projects that people want to submit candidates to that people are willing to see synthesized

11

c.2) 1 point for each design of yours that someone “Likes”

12

I hear that it may not be possible to start a new round every week. If so, I would change the deactivation criteria for a project to no (synthesis) activity on a project for three rounds (from no activity for 4 weeks)

Also concerns were raised over high scores getting so high no new players could make the top 1000 rank; FoldIT degrades scores over time with a half-life scheme to avoid this and encourage active participation.

If some of the slots need to be reserved for Ensemble use, then reduce my player allocations to allow that.

14

Could work. A lot simpler than my tiered scheme.

In chat discussions, Brourd suggested that lab points be separated from puzzle points. You need 10K puzzle points to get lab status, then start gaining lab points as a separate score from puzzle points, much like Solo versus Evolver points in FoldIT. Perhaps the first 10K slots are allocated evenly to all players who have achieved lab status. But what if players stop exercising their lab rights? It seems a waste to have slots reserved round after round for players who don’t use them? Perhaps you need to have 10K points *and* be among the first 1K/2K/5K players to submit designs in a round? Those who snooze lose?

Also like FoldIT we discussed having a scheme that rewards active players as well as successful ones, either by giving lab points a half-life (like in FoldIT) or basing the extra slots on recent activity/success (past N rounds) rather than lifetime score accrual. The concern was that long term players could raise the bar so high that making it into the top 1000 is difficult.

However with any reserved slot system, there may be people who do not exercise their slot options in a given round, and rather than waste them, they could be allocated to “backup” designs submitted by players - but how should they be ranked/ordered and how many would they be allowed to submit?

Proposal:

a) each of the first N=1K(/2K/5K - TBD) lab qualified players to exercise their lab options in each month/round are guaranteed at least 10K/N slots for the month. If fewer than N players submit, then slots are divided equally by the number of players submitting in the round.

b) players may submit synthesis candidates to any active player or development lab in each round

c) players rank their own designs (across all labs) in order of synthesis preference

d) after the standard slots are used, the remaining synthesis slots are allocated to players based on some function of *current* lab rank (which is biased in favor of recent performance)

e) If slots still remain, additional slot assignment rounds proceed based on the relative ranking of the players with remaining designs

So: players choose which of their designs they think are most worthy of synthesis and based on (recent) performance they may get additional slots

15

Oh. By the way the synthesis database and scripting are *way awesome* ideas too. But I think I’ve said too much today :wink:

16

Ideas for the new lab with player projects

First I like to say that Brourd and JL have lots of great thoughts and ideas for our new lab. I however am not too keen about JL’s and Brourds joint idea of limiting the number of projects to 20 projects open at a time. Though I admit it will reduce complexity.

PLAYER PROJECT IDEAS

The ideas below is dragged out from a brainstorm session Brourd started and steared, some time back between Brourd, Starryjess, Mat and me, about the new lab and which direction we thought Eterna should go in.

QUEST STRUCTURE FOR PLAYER PROJECTS

I would love to have a Khan like system, where one unlock more player projects as one goes.

That will also help break the hugeness of the current player projects down, to a more concievable size. Make things more gamelike and make us forget about how many projects there really are.

THEME GROUPPED LABS

We were talking about grouping the puzzles in themes. That could be done for player projects too.

Back then I was thinking on sorting puzzles after theme, like dragons in one group, and so on. But it could be done after player, elements, like 1-2 loops, 3-3 loops, zigzags and so on.

AWARDS

Generally we love badges, lots of badges.

Scholarships. Brourd came up with the idea of rewarding scholarships. He also said that there should be rewards for players who do micropapers.

EXPLORATION AND IDEAS FOR VISUALS

Brourd mentioned that right now, everything a player can do is known to them, perhaps, blacking out some things with question marks. Or as Starryjess said: people crave novelty.

Brourd: Same thing with puzzles, instead of going down some dull list, you can move some kind object through a cloudy solution. Perhaps a strand, and when you get close to an unlocked set of puzzles, the strand forms in the shape of the secondary structure. then you solve it: your strand of RNA returns to normal, and you move onto the next challenge. I am meaning through a type of adventurous landscape

Brourd: The goal is to get to the next set of puzzles, but we need it to be an epic and grandiose fashion, to get the awe and excitement building in a player
starryjess: when you solve a puzzle, new corridors open up. At first, they’re limited. Something about only having certain puzzles available at first? I wonder if a maze could be involved somehow?
Brourd: not just the same list over and over again, something that makes the move from one puzzle to the next an adventure.

DAILY LAB CHALLENGE

What about lab challenges of the day from the player projects. That would be thirty on a month. Which will be doable. Even three lab challenges a day will somehow be doable.

Brourd: And then, we could even redo labs, but with different req.
Eli Fisker: Yes as next level. One could choose how many one wants pr. day, and with time one will get through a lot
Brourd: It would also increase the daily lab challenge pool as well. So those 30, could potentially be 150
Eli Fisker: Just allow for people to choose how many they would like to do a day.
Brourd: personalize it

17

Not sure If my 20 projects point was understood. That was 20 active projects per administrator, not 20 active projects in total. The intention is to keep some focus in project creation rather than a zillion random projects that no one wants to delve into, so if there are 1000 project administrators with 20 projects each, they would each get one slot per month on average :slight_smile: However I expect most players would not run the full 20 projects and probably most would not run any, so at any time there would probably be a few hundred active projects. It might seem like lots of projects, but at 20K slots, you would need for example 200 projects with an average of 100 submissions each to fill all the slots!!

Another possibility might be to rate restrict project creation, much like you are only allowed 3 new player puzzles per day. One new project per week for new lab members, with more projects per week as you advance?

Having a linked project space that you explore might also work, but do you have to solve or complete a project to advance to the next one? I have not played Khan, so I don’t fully get the reference.

18

I’ve taken some heat for the “Likes” idea. Sounds too much like FaceBook say some. Internet lotteries are illegal in my state say others. I guess I was hoping to use it to give crowd-sourced feedback to designers in part.

In some ways, the rank-your-own submissions strategy may be better, or at least as good. However, does it mean that only the top-ranked 1000 (or 2000) ranked lab players get synthesis permissions? Currently that is about 23K points for top 1000 and 12K points for top 2000. But how do we deal with players who run up their score then stop playing and hog reserved lab slots that go unused? First-come first served? Right now I think about 200 players play the labs each round even though over 2000 are qualified, so perhaps first-come-first served works fine.

I think a scheme that rewards higher ranked lab players with more reserved slots is reasonable; they are more experienced and more knowledgeable and may be able to better use the slots, plus it is incentive to participate to raise your lab rank. Lab-points erosion via half-life would mean players must keep playing labs to maintain their preferred status. The slots bonus could be moderate or large. Moderate might be to give the top 250 ranked lab players an extra 12 slots each, the next 250 players (lab ranks 251…500) an extra 8 slots, the next 250 ranked lab players (lab ranks 501…750) an extra 4 slots, for a total of 6000 additional slots. That leaves 4000 slots left for Ensemble or other uses. More extreme might be to give the top 100 lab-ranked players an extra 30 slots each month, the next 200 players (lab ranks 101…300) an extra 10 slots, and the next 200 players (lab rank 301…500) an extra 5 reserved slots.

No matter the distribution, unused reservation slots could be given out to players “backup” design submissions using their rank ordering. Since we have already given some guaranteed rank-based bonuses to the higher ranked players a suitable way might be to run through the lab ranks from top to bottom assigning one extra slot to each player that has a backup design which has not been scheduled for synthesis. Once all lab members have been given an opportunity to submit a backup design, the process repeats as long as there are remaining slots and remaining backup designs. This process could even include lab members beyond the first 1000 who are late to the game that month, perhaps beginning with them for the first pass.

It still might be good for project admins to be able to assign slot reservations to submissions by other players to their projects, perhaps by adding them to their own synthesis candidates rank-order list?

20

Hi JL!

Sorry for not expressing it clear enough. Ok, I did understand that it were pr. player and I do see your point with having to follow up on the data. I’m not in wild dissagreement. Maybe just want a couple, like 5 or 10 more. :slight_smile:

21

I don’t have much time, so I’ll just say that one thing I’d like to see more of is labs where we’re actually solving a known or predicted problem, such as placing FMN binding sequences in otherwise purely synthetic designs.

22

Paramodic, great idea, that sounds like a fun game to me.

That remind me what Rhiju wrote in the conclusion in his latest science paper Why can’t we predict RNA structure at atomic resolution:

Recent progress, especially in all-atom refinement and design, makes us particularly excited about the future; a solution to RNA structure prediction appears more and more feasible. We propose that the time is ripe for the creation of a community-wide CASP-style experiment, where groups compete to produce blind models of RNAs about to be solved by crystallography or NMR. The prospect of such blind trials bodes well for the maturing and eventual practical impact of the RNA structure prediction field.

It would be fun being a part of that.

23

In worrying about limiting how many of anything is available at a time, maybe we should remember that for the first player project synthesis, at the deadline there weren’t enough entries. Maybe there won’t be a problem in that direction.

I think the first thing that needs to be added if we are going to be able to do the big multiple projects is a good way for players to keep track of what they’ve submitted and where. Where in the pipeline everything is, and a much better way to display multiple designs to compare them, ours and everybody else’s. Before we go off running maybe we need to get this ramped up and debugged. Working on the player projects right now is a major frustration, because I can’t even tell from the front page if I have designs in any given project. A major problem in a number of areas of biological research, including ours, is the ability to generate way more data than analysis.

I also think a clearer statement of what the desired end result of all this would be helpful. Are we maybe getting into designing specific RNAs? Or are we really just trying to develop a way to design RNAs? If the latter, I don’t really see the reason for preventing anyone from getting into design, because I would think a range of designs of varying “correctness” is what we really need, and we don’t need exclusively experts for that. Because in learning to design, we aren’t looking for a single right answer, but a range of answers, and how to find any/all of them. I’ve always thought it was strange that we stopped the labs when we found one “correct” design, because if we are trying to learn design, what we theoretically want to find is ALL the “correct” designs, and how to predict them.

Last thought for now: what we are rewarding players for right now is data. Maybe we should find some way to reward analysis as well.

24

Definitely need to think on UI issues. How do you navigate 100 projects let alone 10000 projects? Or remember which ones you submitted to? Or get results back?

Multiple search/sort interfaces might help:

Show me all projects by Quasispecies that are new in the past 7 days.
Show me all projects where I have synthesis results in the past 14 days…
Show me all projects where I have unslotted designs.
Show me all projects by EteRNA where I have no designs.
Show me all projects with more than 100 designs.
Show me all projects with less than 3 designs, newest first.

Show me all my designs that are unslotted, in ranked order.
Show me all my designs that were synthesized in the past 2 months.
Show me all my designs that have not been synthesized in the last 2 months.
Show me all designs by Brourd with a tri-loop on a 4-stack.

And so on.