@Omei: Thanks, the link is working now. I’ve read the code. If I click start from copy, what comes next? I think that I must save it some way as my own code because we can’t run other players code. Correct?
Once I get to run the code, maybe I will be able to understand what the script does.
I got a full exeriment example set up in this lab. Just search for experiment under title.
I want to experiment with the booster a bit, specifically with SVG design, which can be placed on top of the applet prett easily. I will PM nando tomorrow about creating more hooks for boosters, for example change the base you are stamping, etc, so we could have the svg replace the flash buttons if we see it’s the right way to go.
(EDIT: What I meant to say is that I plan to expiriment with it a bit, and if I get stuff working I will submitthe results, but I can’t promise much tangable progress because school starts really soon)
For most boosters, you can “start from copy”, make sure that the Script Type is still Booster and click on “Submit Script”. The script should now appear when you press the lightning bolt icon in the game.
But because jandersonlee’s booster modifies the UI and needs code around to handle the user interactions, it falls in the category of “asyncronous” boosters, and needs some special treatment. In this case, you need to replace the 7466741 in
the lines
// IMPORTANT: when copying, edit line 6 to reflect your URL script id var blv4Sid = "7466741";
at the top of the script with the ID of your copy of the script, which you can pull out of the browser’s address bar. Make sure to click SubmitScript (again, perhaps) after making the change.
If you care to know the other special voodoo about asynchronous scripts, see http://eternawiki.org/wiki/index.php5/Flash_API_for_Boosters#Synchronous_vs_Asynchronous).
@MasterStormer Do you have an example booster that overlays the flash UI? I’m interested in seeing how robust the positioning is.
@Omei: After “submit script” I saw the url change and I copied down my script ID number. I got out of script and went back in, tried to search on my iD and was left hanging. Saw the search by date and clicked on it to see my script at the top of the page. Saw the edit button, clicked on it to edit the ID number in the code to reflect my ID.
Went to Control ON puzzle in the new labs, clicked on the lightning bolt, clicked on “script executer” and a box asked for my ID. Entered it and the HTML boxes (windows) appeared along the bottom. Pulled the bulk loader data box over to expand it horizontally. The only thing I got to respond was the mutate button. It mutated every base and pasted the full sequence for each mutation. All reported false of course because I did not have a design with constraints met to begin with.
If any of the other buttons are active, I could not tell by just messing around.
I have more questions, but I’ll save them for a better channel perhaps in slack.
The positioning is the problematic part, as zooming makes it uncalculatable (I am pretty sure)… I have a prototype called remove all markers, but I didn’t yet calculate it’s positioning, so it’s just thrown on the left. However I know Andrew had over half of it below the screen in 125% zoom
Thinking aloud…
I think I need to double the set I proposed, so I test both Exclusion and Same state.
What would be best if one could test both puzzle sets at the exact same conditions. So do the same experiments to both lab types. Well knowing that some of the design types will be futile.
The first 4 experiments I proposed are good for exclusion, but not for same state puzzles. While I could make them without big trouble. These next four sets are good for Same state but not Exclusion.
Same State
5a) One static stem at late position, ms2 distanced to aptamer, with GU
5b) One static stem at late position, ms2 distanced to aptamer, without GU
6a) One static stem at early position, ms2 distanced to aptamer, with GU
6b) One static stem at early position, ms2 distanced to aptamer, without GU
7a) Two static stems, MS2 distanced to aptamer, with GU
7b) Two static stems, MS2 distanced to aptamer, without GU
I’m not really able to double the last task for Same state labs. Unless I do one lab where I put the static stems next to the MS2 and another where I put them next to the aptamer gate.
The Same State lab works typically by a double sequence pairing between the MS2 and the aptamer or aptamer gate. Whereas Exclusion work by a nearby turnoff sequence.
Exclusion
5a) One static stem, ms2 distanced to aptamer, turnoff sequence before ms2, with GU
5b) One static stem, ms2 distanced to aptamer, turnoff sequence before ms2, without GU
6a) One static stem, ms2 distanced to aptamer, ms2 before turnoff sequence, with GU
6b) One static stem, ms2 distanced to aptamer, ms2 before turnoff sequence, without GU
7a) Two static stems, ms2 distanced to aptamer, turnoff sequence before ms2, with GU
7b) Two static stems, ms2 distanced to aptamer, turnoff sequence before ms2, without GU
8a) Two static stems, ms2 distanced to aptamer, turnoff sequence before ms2, with GU
8b) Two static stems, ms2 distanced to aptamer, turnoff sequence before ms2, without GU
Conundrum
These two late sets won’t be the same, as I’m not giving the Same state labs a turnoff sequence, and I’m not giving Exclusion a double sequence pairing between ms2 and the aptamer/aptamer gate. While I know both will likely do ugly, the above is not completely comparable tests between two switch type labs.
I have been thinking further.
Same State Experiment Set
I wish for two of each puzzle settings, one of them with GU in the switching area and one of them without. So two set of each 8 puzzles. For same state I’m primarely interested in the last 4 as these are the typical same state types. But I’m kind of want a comparison with exclusion lab
- 1 static stem, ms2 next to second aptamer sequence
- 1 static stem, ms2 next to first aptamer sequence
- 2 static stem, ms2 next to second aptamer sequence
- 2 static stem, ms2 next to first aptamer sequence
- 1 static stem, ms2 distanced to aptamer,
- 1 static stem, ms2 distanced to aptamer,
- 2 static stem, ms2 distanced to aptamer
- 2 static stem, ms2 distanced to aptamer
Generally I like the static stem next to the aptamer gate. But that bit can be varied. I’m imagine a series of each puzzle type where things like static stem position is varied, multiloop loop size is varied and the position of the MS2 is slightly varied.
Exclusion Experiment set
For the exclusion set I’m primarely interested in the first 4 as these are typical in exclusion solves.
- 1 static stem, ms2 next to second aptamer sequence, turnoff sequence before ms2
- 1 static stem, ms2 next to first aptamer sequence, turnoff sequence after ms2
- 2 static stem, ms2 next to second aptamer sequence, turnoff sequence before ms2
- 2 static stem, ms2 next to first aptamer sequence, turnoff sequence after ms2
- 1 static stem, ms2 distanced to aptamer, static stem late, turnoff sequence before ms2
- 1 static stem, ms2 distanced to aptamer, static stem early, turnoff sequence after ms2
- 2 static stem, ms2 distanced to aptamer, turnoff sequence before ms2
- 2 static stem, ms2 distanced to aptamer, turnoff sequence after ms2
What to do next?
Now my problem is that the Same State Experiment Set and the Exclusion Experiment set are not really the same anyway, despite the similar solve set up.
Since the Exclusion type uses a turnoff sequence right after the MS2, something that Same State solves do not.
So to get a full comparison between the labs, I would have to run two set of 16 puzzles each for each lab. So I’m considering if it is worth it or it gets too complicated. I mean rather a double set of 4 puzzles that we get good data on, that a double set of 16 that are comparable across labs, but we get little data on.
Also I wonder if it is a good idea calling the experiments things like 1a and 1b etc, since the labs are already called something similar. Perhaps 1x and 1y would make more sense?
Also I have tried figure how to put up the comming data in a spreadsheet. I have two different settings below. The first one where I have both double sets of 8 grouped together.
The second one where I have the split the sets after if they got GU in the switching area or not. So they are kind of prepared for plotting in the average fold change and max fold change for the sets. Also I would need a way to pull those sets afterwards. I would kind of need to pull more hashtags at the same time and I’m wondering if this is going to give me trouble.
An Experimental Template
I have been thinking further about making an experimental plan. I want to set it up in the simplest way possible.
The plan is to end up with something that will make it easy for you to copy my experiment template or adjust it to what you need, so you can afterwards stick your experimental data into Omei’s data analysis template and see if your hypothesis is confirmed or not.
-
I do 4 different experiments in each lab. (Two will be fine too) These experiments I will repeat across all labs.
-
The experiments will be fitted to if it is an Exclusion lab or a Same State lab. I will use design setup that had shown themselves to work there.
-
Each experiment will have two puzzles series with close to identical designs. One puzzle series without the specific feature you wish to test and the other with. In my case a GU in the switching area. 4 puzzle series with GU and 4 puzzle series without. So this make 8 series of puzzles in each lab.
-
I will as far as possible attempt to make the design with GU and the design without GU identical. (May have to change a base in a less important spot if the design I intended is already taken, or in some cases if I can’t make a stable version of either the one with GU or the one without)
-
I will distribute 120 slots to each lab for these experiments. That will give me 15 designs in each experiment category. And I will still have 30 slots left to do funny but suspected useful mirror, double and triple aptamer experiments with.
The test outcome
The main variable in my entire experiment is GU. If the fold change between the set without the GU and the set with GU in the switching area differs considerably, then the variable I’m testing is having an effect. Since it is the only thing that is really different.
My hypothesis is that GU are helpful in the switching area. But if I instead put it as a question, it is simpler to see the outcome.
Question: Are GU in switching area helpful or harmful to the switching ability of an RNA switch?
Lab conclusion
In reality I can end with 2 or really 3 different answers to the question in my hypothesis.
- Yes, GU is really helping in the switching area
- No, GU in the switching area makes everything worse
- Nah, there really is no fold change difference between designs with GU in the switching area or not. Try a new hypothesis.
How to analyse the data when it comes back
Here is how I plan to use Omei’s analysis template to compare my results.
-
I will pool all the a-series together afterwards to compare all the b-series designs. This part of the work will be easiest doing in a spreadsheet. This I will do for each lab. That way I get to compare 60 designs with GU to compare with 60 designs without GU. Those two sets of 60 designs will be near identical except for the GU content.
-
Then I will likely pool all the Same state designs in two sets and all the Exclusion designs in two sets and see if there are different trends for these two lab types. It may be that GU in the switching area is helping Exclusion designs more than in Same state designs, etc.
My Experiment Plan
There will be an identical b-experiment for each a-experiment, just without GU in the switching area.
Exclusion Set
1a) One static stem, MS2 before second aptamer
2a) One static stem, first aptamer sequence before MS2
3a) Two static stems, MS2 before second aptamer
4a) Two static stem, first aptamer sequence before MS2
Same State
5a) One static stem at late position, ms2 distanced to aptamer
6a) One static stem at early position, ms2 distanced to aptamer
7a) Two static stems, MS2 distanced to aptamer, static stems near MS2
8a) Two static stems, MS2 distanced to aptamer, static stems near aptamer gate
I only make one experiment template but I will repeat it across all the labs. So when you have first decided what to test and how, everything will be far easier from there.
You could use the same basic switch element position as in my puzzles, but test a different thing than GU. Or choose a different puzzle setup but still test for GU. Its fully up to what you can imagine, you wish to test. Also there are plenty of ideas here.
I figured how to call all the A experiments in one go. Instead of calling experiments like 1a, 2a, 3a etc and then trying to figure how to pool them. I can call them Experiment A1, Experiment A2 etc. That way I can search for Experiment A… and I will get them all in one go in a lab or even across labs.
I am assuming “bulkloader” allows you to upload many solution into the lab designs.
There is also talks of mutation scripts. Coding is difficult enough without doing too much in one script.
Any scripts that:
- only do “bulkload”.
2.)only look for valid mutations from a given valid sequence
I am really only interested in #2 at this time since I have submitted enough labs to get an idea of where I want to go. What’s the best existing script that comes closest to #2 for any lab so I can modify it?
I think jandersonlee’s BulkLoader v4 (devops) booster best fits your #2 requirement. Despite its name “bulkloader”, it doesn’t actually submit designs in bulk. One of the things it can do, though, is generate a lot of different kinds of mutations and evaluate each for whether it meets all puzzle criteria (with the currently selected folding engine.) It keeps track of those that don’t as well as those that do, but all you have to do to remove the ones that don’t meet all the criteria is to click on the “Purge” button.
To submit the remaining “valid” designs, you need to use the Next and Previous buttons to load the designs into the game UI one by one, submitting each one from the UI as you would normally.
I have a question/suggestion.
In the “New Progression” there is a tool called Eterna Bot which works to generate a score based on what is thought to be a favorable lab design. I recall that the increase in Eterna Bot score corresponded well with what has worked well in the recent labs and so I was curious as to why Eterna Bot isn’t available as a tool/feature in any of the Lab rounds I have seen.
One thought was that Eterna Bot couldn’t handle switches as a whole, but each switch has a single-state component and I would think that Eterna Bot could handle those.
If Eterna Bot doesn’t have any issue scoring switches, then I would like to see this made into an available tool, similar to Melting Curves and Pairing Probability Plots as yet another aid to improve lab designs.
That version of Eternabot (essentially) attempts to find a sequence that makes the target structure as stable as possible, which implies trying to eliminate the possibility of alternative foldings. That’s not good for making a switch. For a switch, you want to have 2+ foldings that are close enough in energy that the presence or absence of the input(s) controls which structure forms.
1 Like
Since this is kind of our during experimenting forum post, I decided to stick some of my current lab musings in here too.
Making great switches switchier
What switches want
- Most of all symmetry
- Double aptamers (Ok, probably not all labs want them - but the riboswitch designs with best fold changes have them.)
Getting some more symmetry into Exclusion labs
I have been thinking about how to get some more symmetry stuck into our past not so symmetric solves of Exclusion lab puzzles.
Till now our Exclusion labs have been less symmetrical than our same state labs. Due to the MS2 liking to be directly next to the aptamer, it does a skewed pull. Not like the more straight pull between MS2 and aptamer in the Same state designs.
I think I have found a way around this.
I did a solve where I doubled what have worked in a past Exclusion NG 2 winner.
http://www.eternagame.org/game/browse/8047736/?filter1=Id&filter1_arg2=8143991&filter1_arg1=…
Ignore the #alternativems2 bit, both MS2’s are true MS2’s.
Blueprint addition by rotation
I like to talk about a riboswitch blueprint - which is really just a template of how the switch elements prefer to be placed in relation to each other.
There is one for Same state labs (ms2 distanced to aptamer) and one for Exclusion labs. (ms2 next to aptamer)
A thing that has worked earlier for me is to add up blueprints. Like take two different working versions of a design and add the structure of them together to get to a new structure with a shot of working. This time I rotated the blueprint 180 degrees instead.
https://drive.google.com/open?id=0BxJGPGrJ3fkKRlhCZlpYTkFzak0
Ultimate Symmetry
I have made a new Exclusion series called Ultimate symmetry which has:
-
Lots of symmetry
-
Double aptamers
-
Double MS2 (will often be alternative MS2’s.)
https://drive.google.com/open?id=0BxJGPGrJ3fkKcWFRWUNVcnI3Zkk
What may we learn?
Not all designs will benefit equally from having double aptamer or double MS2. I think it will all depend on the balance of strength between aptamer and switch hairpin. (MS2, K4 - kissing loop hairpin etc.)
Basically varying the number of aptamers in relation to MS2’s in labs with different aptamers are a way to probe the strength of the aptamer versus MS2. I suspect that some of our new aptamers will take more work opening compared to the FMN we have worked with so long. These may benefit from having one aptamer against 2 MS2’s. (At least in Same State labs)
In our past FMN/MS2 labs, two FMN countering MS2 worked real sweet. Two FMN versus one MS2 are in are in balance in the Same State NG2 lab.
Future wishes
What I wish for next round. Even bigger switch bubbles. 
Basically a switch bubble big enough to stick 4 ms2’s into and leave room for a small loop sequence between them. I would like to see if I could make an exclusion design work that way with 4 MS2’s against one aptamer. I suspect the MS2’s may help the aptamer do its job, because they may also be able to switch each other off.
The lack of a big enough switch bubble didn’t stop me from trying though.
I went fibonacchi on the MS2 ON control freestyle puzzle.
Which really stems back to a beautiful idea that Zama brought up. She asked me if RNA could be fibonacchi like.
Since I had already planned to try stick in 4 MS2’s in somewhere and static RNA literally hates too much symmetry and too similar elements, I decided to instead of spreading the MS2’s in an even way, to spread them in a growing or decreasing order. Aka fibonacchi like.
So Fibonacchi helps add in asymmetry, yet in a systematic way. Alternative MS2 add in sequence variance in the otherwise identical elements elements.
Hereby Zama’s idea is passed on. Go get creative.
Is there a way to find designs that almost have qualities (such as those Eli has highlighted) and bring them to attention to eterna’s top designers for targeted alteration?
Creating (puzzle) games within a (active lab) game?
Combining design insights with our group’s designing skills?
I want to share one insight that came from another game I play (Quantum Moves - simulates moving atoms with laser tweezers)…the power of dropping bias.
The easiest way I have found to find bias is just to observe my own tendencies and then ask if these are in concert with the project’s objectives. If they are not, then that tendency (which is also highly likely to be held by others) is a bias to drop.
Here is an example. Quantum Moves has 23 game levels - each simulating some aspect of quantum physics quandaries of quickly and stably moving atoms. I noticed that I had a desire to get a top score in at least one of these levels. So I had a tendency to focus on one game or a certain group of games where I thought I had a better chance of getting a top score.
Since each level incorporated different aspects of quantum physics aspects, not learning about the aspects of other game levels was not in concert with the project’s goal. Therefore the tendency to want to win at one level was a bias against cross level learning.
To remove this bias, I simply shifted my goal of wanting to get a highest score in a game to improving my worst score of all games. This put my efforts more in alignment with the project’s goals (of improvement) and also gave me two advantages that other players did not have.
The first advantage was better cross level learning. By always focusing on my worse game, I improved the simulated quantum physics aspect that I was worst at. And then could apply that to other games.
The second and probably more powerful advantage was to create better motivation for sustained engagement. It was both easier to play this way, less frustrating and more fun. So I played much more than other players.
As a result of dropping this one bias, and after a year of playing in spite of the fact that I had never played any computer games before, I now have the top scores in Quantum Moves in 18 of the 23 levels, including all levels from 9 and up and being in the top 5 in the other five lower levels.
In eterna, the bias I have noticed within myself is to want to do the games myself without help to prove that I can. In the lab, this has translated to a feeling wanting to create my own designs.
I assume others feel the same way.
This prevents cross person design collaboration during the most important time (labs).
How do we drop that?
3 Likes
Mutation Tool - improving player-oriented efficiency
I have been using Nupack less in Lab because the engine is both slower to run and process submissions. Awareness of that bias has me looking for what unique Nupack engine values I may be overlooking.
Both @Zama and I have noticed that when running the mutation tool on TEP designs (which we favor Vienna 2) our hit rates for multiple engine designs was low. When looking for what wasn’t working I noticed that it was often a stem within Target mode, State 2 of Nupack.
So instead of just fixing the aptamer and running my TEP designs thru Mutation, I also fixed combination of nt’s are were more successful for this stem within Nupack first. And then my multiple engine hit rate increases significantly…which hopefully improves Lab submissions!?
Other potential ways of improving Mutation tool efficiency is selecting designs within design similarity proximity of attractive characteristics. Such as symmetry (@Eli Fisker), minimal cross-engine nt movement (@Zanna) and/or fixing attractive design elements such as FMN aptamer orientation.
1 Like