Using Hashtags to Improve Lab Collaboration and Experimentation

The project description for the current round of NG puzzles encourages players to submit groups of designs tied together with a hashtag.  But this is not something that has been widely discussed, and so I thought it would be useful to give some background, and invite a discussion among players.

For some time, Rhiju has been looking for a way to improve the lab experience by enhancing the support for collaboration among players. Last summer, he made two general suggestions – teams and design bundles – that were widely discussed in chat, dev meetings, and here in the forums.  But we players are a very diverse group of people, and no consensus really developed about how those ideas should be implemented, or whether the increased structure they implied was even a good thing.

Then, at a dev meeting a month or two ago, Nando suggested an alternative – provide a little support for hashtags (in the style of Twitter) and let the players figure out how they wanted to use them.  So that became the plan.  In the coming months, as part of Eterna Medicine, there will be additional software support for working with hashtags.  But there is really no reason to wait for that to happen.  As it is, the ability to do full-text searching on the design name and description fields in the data browsers is quite adequate for finding and grouping related designs that share a hashtag.

So this conversation is an invitation to interested players, either to discuss the use of hashtags in general, or to propose specific ones for a specific purpose.  Or simply to ask questions, starting with “What is a hashtag?”

4 Likes

Hi Omei!

Big thx for your post and the background on how this idea arose.

I have done some thinking about how to go about using Hashtags and do experiments.

The way I have decided to go about it, is to pick something like 3-5 good original designs in each of the labs that we already have data for. Then I do a series of different experiments on them and let the hashtag signal which.

Then I double or triple the amount of experimental data by rotating the original design onto one or in some cases 2 of the labs that has the aptamer rotated the same way compared to the original design. Then I repeat the same lot of experiments that I did on the non-rotated original. This way I get the most bang for my buck.

I use the original design name as identifier in the title after the Hashtag. So the original design name can be used to pull together all the designs, no matter what experiment is run on them, and the hashtags can be used to pull the designs that have the same kind of experiments done to them.

I have put a bunch of #Hashtag experiments that I have either done or am thinking about doing in a document, together with some links to the background behind this kind of experiment.

Feel free to get inspired and use.

Ideas for Hashtag use

1 Like

When trying to work out a good way to go about the Hashtag hypothesis thing, I decided that I wanted the name of the original design that I was experimented on in the title after the Hashtag. This way I would have every design in the series that is an experiment on that, link back to it.

The original design I in the beginning called something like #Original. But I realized that it made more sense if I called it after its main feature. Like if it was a DoubleMagnetSystem design or its main personality feature was a PyrimidineTurnoffSequence. That way one could ideally also sort the originals after type and this way get a bigger set for comparison, than just the original and the experiments done onto it. Because these similar originals, would each have a similar set of experiments done to them.

Which brings me to something that it turns out I’m not the only one who have been thinking off.

As Omei said:

The next issue that comes to mind is that right now, we can’t edit design titles or descriptions, so we can’t update the hashtags on designs we’ve already submitted, as we improve our ideas about how best to use them.  

He is touching on something that I find very relevant. I have deleted and resubmitted several of my early designs to get either their description and especially the hashtag title updated to my current understanding of what is optimal.

Most important would be to edit the title containing the hashtag. That’s where the need most arise, as I can always add a comment if I have a correction to the description. Also I use my document for updating what I mean with my chosen hashtags. 

1 Like

I would like to be able to add hashtags to another players designs and be able to search for that later. I am going to try and make predictions of what designs that are submitted by other players will do well.

1 Like

I agree; this is a good use for hashtags.  I’ve opened an internal discussion with jnicol about how to get this incorporated into the new browser.

Also have to figure out when #apple and #orange mean the same thing to different designers.
 

Stumbled upon a good conversation about hashtags on the NG labs page and I figured it would be good to add it in here. Also this kinda does go along with what JR siad about figuring out when #apple and #orange mean the same thing to different people. I feel this can be alleviated by having a central place to put hashtag definitions. I think the wiki would be a good use for this… 


JR   Can I put the Hashtag anywhere in the title or does the title have to start out with the hashtag?
16 Dec 2015


Omei   Assuming that these prove useful and software support is added, I think we should stick to the de facto Twitter standard of limiting the # to the beginning of a word. But it wouldn’t have to be the first word. So instead of "JR_#CentralOpenLoop_IENG1Sub001, you could use “JR #CentralOpenLoop_IENG1Sub001” as the title.

In fact, I would go even farther in separating the hash tag from the rest of the title. For example, you might make the title “JR #CentralOpenLoop_IENG1 Sub001” or “JR#CentralOpenLoop IENG1 Sub001” so that the hashtag itself was either CentralOpenLoop_IENG1 or CentralOpenLoop, whichever one seemed most descriptive of you hypothesis. (i.e. is it limited to the ING1 puzzle, or mother puzzles as well.) The idea is to have the hashtag itself group together related designs. That way we could, for example, have a dictionary of hashtags, each on of which might describe some concept.

Also, Nando didn’t intend to limit each design to a single hashtag. In the previous round, I started adding multiple hashtags to my description for each design. Each of these tagged the design as having one certain characteristic that I thought might be useful in the future for finding designs with that characteristic. It was Eli’s idea to encourage the use of a single hashtag in the title to identify the primary “hypothesis testing” group for this round.
16 Dec 2015


JR   Thanks. I use “_” the same way space is used so I will just drop the “_” separator in my titles for now on. You have around 80 of mine with the “_” in the title. If anyone wants to edit them out it is OK with me.
16 Dec 2015


Meechl   Thanks again for the reply, Omei. Also, similar to the discussion with JR, I was thinking it could be very useful to indicate in the title whether the design is expected to be good or bad, perhaps just by adding a #good or #bad tag.
18 Dec 2015


Omei   Yes, I think adding #good and #bad makes sense as a “secondary” tag where it is applicable. There is certainly nothing wrong with assigning multiple tags. And although one could mark designs as “good” and “bad” without using a hashtag, one of the hoped-for benefits of the hashtag concept is to develop a shared vocabulary for players who want to help, and learn from, each other. And the notion of controlled testing of ideas is important.

Building on this idea, perhaps #control would be good hashtag for experiments where there is more than one variation being tested, or where the experiment is simply trying something different, without any clear notion whether it will hurt or improve the switch.
18 Dec 2015


JR   I was thinking of using #filler for any of my puzzles that I crank out for no other reason that it works and then guess with #good or #bad if I think it will actually work or not.
18 Dec 2015


Eli Fisker   Hi Meechl, Omei and JR! I like the ideas with #good, #bad and #control. The #filler one is also a good idea. We need something to mark designs that are #denovo - made over from scratch. Trying out something new. I also miss a hashtag for when I make a minor change to something in a current working design, without following a specific hypothesis, just to see if it works. We need a way to highlight that too. When only doing minor changes that we don’t know if it is beneficial.
19 Dec 2015


Eli Fisker   After I posted I realized that #mod will probably do in that case. :slight_smile:
19 Dec 2015


Eli Fisker   And when thinking about it I think that my #mod is exactly the same as Omei’s #control hashtag is already covering and covering better.
19 Dec 2015

Hashtags as a helping hand to our bot

I heard a lecture on machine learning. And it hit me that we can use the hashtags, for something more than just for keeping track of our experiments

If the hashtag says something about the data, it may be possible to help the bot to take note of specific things. As for teaching it the different tastes of RNA.

We could use different hashtag sets as training sets for the machine learning program. As a kind of primitive version of supervised learning - that tends to need a lot of input.

Typically when I make hashtag experiments, I use them to mention something I find characteristic about a design. Or I make a hashtag experiment where I target and destroy a specific feature that I think important.

Background document: Ideas for hashtag experiments

Often I add an extra hashtag in the description, though not always, where I state if I expect the result to be #good or #bad.

It could also be expectations of a #Better or #Worse score in relation to the original design one were modifying.

I hijacked the mental image of a Linear Regression curve, sigmoidal style and filled it with my own stuff.

This could be done in a lot of different ways. Just throwing the idea out there.

Deep dreaming

If you are up for a good laugh, check out Googles Deep dreaming project.

Deep Dream: Artificial intelligence meets hallucinations

Finally we have an answer to the question: Do androids dream of electric sheep?

The answer may be sheepdog instead. :wink:

Since the google bot has been mainly trained to recognize dogs from cats and especially determine one kind of dog from another kind of dog, it has a tendency to see dogs everywhere.

Charming. :slight_smile:

I put my # tags in an excel graph and it looked like someone leveled a shot gun at the chart so I decided to try again later.

Just playing so far:

But here is why hashtags help:

L1                        Eterna_Score

 ##Bound Average       77

 #2Pairs Average        34

 #Bound Average        54

 #CentralOpenLoop Average 30

 #kiss Average            70

 #NotBound Average   60

 #Static Average         32

CBMulMS2 Average   47

Grand Average          56

Neat!

Key

; #CentralOpenLoop - molecule NTs in a central loop
#Filler - a valid design
#Good - might have a chance - clean design
#Bad - Doubt it - too much going on

#Static - portion of switch is same in both states
#Hairpin - Molecule NTs are in hairpins of state 1
#DistanceFar - sides of molecule set far apart in state 1
#DistanceClose - sides of molecule set close together in state 1
#StaticP - Prime ends are attached and similar in both states
#Noisy - Too much going in each state
#NP - NuPack
 #NotBound - MS2s not connecting in state 1
#Noisy - Too much going on in one or both states
#Bound - MS2s matching in state 1
#Static - portion of switch is same in both states
#StaticP - Switch bound at prime ends
 #NP - NuPack

Results

Type-Hashtag Eterna_Score

Short#Bound#Static Average 76
Short#NotBound#NP Average 72
Short#NotBound#StaticP#NP Average 71
Short#Bound#Static#StaticP#NP Average 68
Long#NotBound#StaticP#Noisy#NP Average 68
Short#Kiss#boarder Average 67
Long#NotBound#StaticP#NP Average 66
Long#Bound#StaticP#Noisy#NP Average 66
Short#Kiss Average 66
Short#Bound#StaticP#NP Average 64
Short#Bound#Static#NP Average 61
Long#Bound#Static#Noisy#NP Average 61
Long#Bound#Static#StaticP#Noisy Average 61
Short#Bound#NP Average 61
Long#NotBound#NP Average 60
Long#Bound#StaticP#NP Average 59
Short#NotBound#Noisy Average 59
Long#NotBound Average 57
Long#Bound#Noisy#NP Average 57
Long#Kiss Average 55
Long#Bound#Static#NP Average 52
Short#Bound#StaticP#Noisy#NP Average 52
Long#Bound#Static2#NP Average 50
Long#Bound#NP Average 47
Short#NotBound Average 45
Short#Bound#Static2#NP Average 37
Long#Bound#Static2#StaticP#NP Average 36
Long#2Pairs Average 34
Short#Bound#Static2#StaticP#NP Average 34
Long#Bound#Static#StaticP#NP Average 32
Short#Static Average 32
Short#CentralOpenLoop Average 30