Many of us EteRNA addicts are also “spreadsheet nuts.” I know myself and at least a few others would love to be able to download the contents of the complete Lab Submission Data Listing (complete with full sequence data) into a spreadsheet, so that we can crunch away to our heart’s content without having to manually input the data (which would be completely prohibitive with the current number of submissions).
What say Devs? Can a CSV file of the Lab Data be auto-constructed and auto-updated which continually reflects the current state of the data in the online Lab submission window? And can this CSV file then be made available to download?
I think that the “synthesis score” column in that spreadsheet is the score for whichever one of the 8 designs synthesized in that round was most like each non-synthesized design.
In Round One, donald’s submission scored 94, and apparently three other solutions were considered “closest to” that out of the eight synthesized. There’s the two you mention and also anneromaine’s Bulge1.1 (nid 26657).
If I have a theory like “there is a correlation between how often Gs are adjacent to each other and how well the RNA folds,” it is hard to test it without the list of sequences.
Of even more interest is the SHAPE column – how do we interpret that? It looks excellent.
I just had a look on what the flashplayer loads in the background, and apparently it’s a nice CSV file
And I would just “guess”, that the number 26603 corresponds to the ‘myVal’ number which occurs in the URL of the current lab.
If you sort the list of Round One submissions by most-similar to the Christmas Tree, the closest one that was actually synthesized is donald’s.
I’m not saying that I think the two are similar, just that however “similarity” is calculated, the Christmas Tree was “more similar” to donald’s than to any of the others that went to synthesis.
Honestly, I think it might be a bit of a flaw in the lab rewards system. “Similar” doesn’t really mean much if you’re talking about a design that was very dissimilar from all the synthesized designs.
That 94% for that “ChristmasTree,” I’m thinking, must be an error or an anomaly; every single other design with 27 to 33 G-C pairs which has already been scored, received the absolute minimum score …all 73 of them.
However, there are still 72 High G-C designs submitted which have hot yet received a score (as of the date of this spreadsheet download), so we will have to see if there are any other unexpected (and suspicious) exceptions. It could be Extremely Enlightening if we are to discover that there are actually SOME high G-C designs that actually do work!
If that turns out to be the case, it will raise some very interesting questions, like: “What is it that made these 1 or 2 High G-C designs (out of nearly 200) actually work, when all the rest failed miserably?”
If these high scores were not determined in synthesis, but by the Similarity Algorithm, then it is almost certainly time for a serious review of this algorithm.
If an error is possible that erroneously elevates, what by all other accounts is a design that ought to fail or score very low, then perhaps it is also possible (or at least conceivable), - that a very good design could also be evaluated very negatively, and wrongly receive a very low score.
Jee. I apologize, but I cannot see HOW this all-G-C design can possibly be closest to Donald’s 94% winner, when all the other all-G-C designs received the absolute minimum score. It seems something must be wrong somewhere. Please see my last entry above; edits were added since you read it.
My guess from looking at the numbers is that the “similarity” is based very simply on the # of nucleotides two designs have in the same position. So if Christmas Tree had the same orientations of GC pairs at all the positions that donald’s design had GC pairs, that would lead to relatively high “similarity”, despite the fact that other measures like overall % of different kinds of bonds/nucleotides, melting point, and free energy might be very different.
Another thing to note is that I don’t think any actual “christmas trees” were synthesized in that particular round, and I think the comparison only goes within a round.
Ding’s right - comparison only goes within rounds. That Christmas tree was submitted in round 1 and there was no synthesized GC pairs design in round 1.
As you can see it differs by 43 bases from the 94 design, which means about half of bases are different, basically meaning they are completely different RNAs. The tree was completely different from all 8 synthesized designs but it happened that 94 design was relatively closest.
This scheme was designed so that every design (even non-synthesized) in the lab at least gets some score, and people can at least get some rewards by participating. This obviously isn’t an optimal way of evaluating. We are actively looking for better way to evaluate non-synthesized designs along with a better way of selecting candidates.
@Ding: I fear you may be correct & I say “fear” because that would be a serious flaw, if it means that the same design could score 10% (minimum) in one round and 94% in another, just because of a positional similarity of G-C pairs to a very good design, irrespective of the quality of the rest of the design. I am certain the developers would not want this perception of possible inequities in the scoring system, or possible flaws in the similarity algorithm - to proliferate. I hope something can be done.
