Eternacon 2018 Followup: Finally figuring out the points issue

So as the first follow-up conversation to Eternacon, I believe that the issue of point allocations should be finally put to rest. Mostly because it took a little bit too much of the future of Eterna discussion :stuck_out_tongue:

Anyway, my suggestion is this dynamic scoring function to determine the number of points that a puzzle should be worth:

With a function like this, the possible scores (subject to change) would look like this

The older and fewer solvers a puzzle has will increase its score, with a maximum of 500 points, and newer puzzles with more solvers will be worth the fewest points, at 30. And the maximum number of points that a puzzle with many solvers can have is 50 points.

The rate of change for which puzzles gain points can be altered, as well as the fraction lost based on the number of solvers by altering various values here. However, this is all based on the idea of dynamic point allocation and definitions for puzzles. I"m honestly not too sure if thatā€™s a viable option for the site given the number of puzzles and continuing growth of the game, so letā€™s just say this post opens feedback, and for those who post ideas, make them *concrete models* for scoring puzzles.

Also, I think something we need to think about before you post ideas for using a bot to solve a puzzle is to realize that the difficulty of many puzzles originates from the constraints. So unless a bot is trying to solve on sequence constraints, then itā€™s not a useful metric for determining puzzle difficulty.

2 Likes

Oh, two other things.

1st - players who solve their published puzzles should probably receive the minimum number of points for solving, which is 30 points, or 30% of the current 100 points. However, this will still increase the number of solvers by 1.

2nd - This is dynamic scoring, which requires the number of points that players gain/lose to be dynamic. Whenever the database is updated (probably monthly, unless it can be done in real time), players will gain and lose points based on the puzzle value at that point in time.

Dynamic scoring would make the game very interesting and I welcome whatever changes the more established and involved players believe should be done in this regardā€¦ the only thing for which I am adamant is that I think players should receive ZERO points for solving their own puzzles and that should be retroactiveā€¦ such is my 13th post in 6 years.

Zero points is fine with me :slight_smile:

Ideally all of the above would result in a serious retroactive allotment of points, and then dynamic point awards from this point onward.

I think it will take too long for dynamic points based on # users. so:

I would have 4 independent rankings plus one grand total:

  1. puzzles solving==30 points if you solve your own puzzle, 100 points otherwise.
  2. Lab solutions==whatever the calculation is now. 
  3. puzzles solved last 30 days== (same calculation as #1 (for newer players).
  4. Top ten ranking (new)==  puzzles selected with 10 or less solvers, 100 points for solving and no points for solving your own puzzle.
  5. Add 1-5 together for overall ranking.
     
    Re-calc all for new rankings redo.

  

Providing some kind of weighted ranking for solving puzzles with less than 10 solvers is a rather fascinating idea. However, it runs into similar issues like dynamic scoring, in that there is going to be a constantly updated value based on the number of solvers the puzzle has. For example, when a puzzle is first published, the first 10 players will receive 100 points, and then none after that will receive any. It mostly rewards those who are on all the time to solve puzzles, and doesnā€™t alter the reward for more difficult/older puzzles.

Besides that, pretty much everything is identical to the current method of allocating points, which doesnā€™t seem to fix the issue players have with correlating a tangible solving reward with puzzle difficulty.

Item 4 above is a slow dynamic, the scoring would be to select player puzzles with 10 or less solvers ( or a lower number)  then sum the number of those puzzles each player has solved ( should be fast because not many players have solved those puzzles) then replace the previous ranking with the new ranking ( you can move up in points as well as down)
 
Calculating the ranking Sunday evening weekly should work.
The selected puzzles will change each week.

 You get nothing for solving first. I have no interest hanging around waiting for a puzzle to show up so I can be first solver

And the 30 points for solving your own puzzles does address what I think is the main issue.

So I disagree with your reply.

If you are worried about who solves what first then you may want to forget thoughts about dynamic scoring and replace it it with a ā€œhow long did It take to solve this puzzleā€ type of scoring.
You would need to create a timer that logs first started and solved puzzle time. Then rank by time. I would think you get one shot which would be when you first open the puzzle.  

I hope you were joking about timing puzzles. I work them for a few minutes at a time over the course of hours or maybe days. How about we just make it slot machine scoring- you solve a puzzle- a wheel spins and wherever it stopsā€¦ ;) You could even add dynamic scoring to the wheelā€™s scoring range.

If you limit the puzzle selection to say ā€œpuzzles with 6 or less playersā€ and update the
rank weekly you will be able to rank those players anyway you want, the calculation time will be minimal. I would want that ranking both separate and added to a grand total. 

So your request is just to have a ranking for players who have solved some select population of puzzles? Iā€™m sure thatā€™s a reasonable request.

Not quite, not a selection of puzzles, exactly: puzzles with 6 or less solvers older than 10 days. Those players can then be given points for who solved first, etc and/or  just for solving the puzzle.  Re-calculated weekly. 
I think of this as bonus points for taking the time to solve tough puzzles. 

 

What exactly is dynamic scoring anyway. All I know is that it sounds sexy.
Scripting will take too long as Brourd explained above. A database system
should solve that problem. 
I am talking here to collect ideas.

Creator is to be excluded from following calcs.
Say you have a file keyed by puzzle # that collects first   puzzle#,creator,  first
solver, second solver, etc up to 6th solver. Point allocation is 100 points for solving 
puzzle. ( reply with your prefered point distribution). First through 6th gets extra points based on ranking. Once a 7th player solves the puzzles the extra points are removed. 

I would like to know (1) if this is a direction interest players want to head and (2) how can this be ā€œgamedā€. - in other words - better try to break it now rather than later.
 
  

Dynamic scoring, with respect to puzzles, would be the idea of changing puzzle scores based on some changing variable. In the case of my proposal above, those variables would be time and number of solvers. Itā€™s my opinion that finding an algorithm to determine puzzle score/difficulty based on the ability for an algorithm to solve it is quite unlikely. Most algorithms/bots donā€™t solve to the constraints of a puzzle, and there are some structures that algorithms may not be able to solve based on internal inconsistencies or constraints.

It was mentioned by Cynwulf28 that the reason he (and others) solve their own puzzles is to save their solutionsā€¦ I actually think this is very reasonableā€¦ What I suggest is that when a puzzle is submitted the solution is automatically saved for the person submitting the puzzleā€¦ I still think zero points should be allocated for self-solving, although I also understand the argument that intensive puzzle makers are then being put at a disadvantage by offering to other players points that they cannot themselves obtainā€¦ As yā€™all know Iā€™m in it for the bubbles, not for the points, but my primary kā€™vetches regard the multistate puzzles (especially the illegal dark side ones) that take minutes to make and are virtually impossible to solveā€¦ Iā€™m also not a fan of the Bobs and the spread energies and the like as I feel these are all gaming the systemā€¦ Again whatever the more established and involved players believe should be done can carry the day, but needed to correct that the previous post was my 14th not my 13th and this will be my 15th. Enough Already!

The issue I am having with time is what I call the ā€œnew player conundrumā€. We want
new players not be penalized for showing up to the party late, years late in some cases. Nor do we want to penalize players for taking their time solving puzzles ( the count issue). So I avoid these problems by creating the database of solvers by puzzle# up to a specific number. As an example: Solvers up to 6 receive 700,600,500,400,300,200 respectively ( or whatever). Once solver 7 kicks in those points are subtracted down to 100 for those players on the list and the puzzle is dropped from the database.  A quick glance gave 1500 single player puzzles and 800 switches below the 6 threshold.  
Not cutting off point competition at some point (at solver 7 in my example) and assigning 100 points for a solution will be a bit insulting for us slowpokes so I wish to avoid.
This proposal should give enough leeway for top player competition without dragging everyone else (who may not want to be competitive) into the fray.
Your thoughts please.
  

    

Yeah, I understand the ā€˜new player conundrumā€™, but I feel like the main conversation besides that was finding a way to reward players for solving old, difficult puzzles, which I donā€™t see any developer input or comments here for anything. Kind of discouraging, considering that nothing is ever agreed upon if nobody official provides input. :stuck_out_tongue:

The idea of rewarding players based on solving order is fine, but I think it disproportionately rewards people for solving fast, not to mention that the reward isnā€™t something that anybody can benefit from.

Iā€™m going to post a second model for puzzle difficulty which could be useful, and doesnā€™t use a dynamic scoring model.

Take my proposal and award an even amount of points for the 6 slots. The age of a ā€œOld puzzlesā€ is going to be to much of a moving target to nail down with players coming and going.
Unless I can think of something - awarding points on age of puzzle is way too easy to gameā€¦ 
But I will start thinking specifically about " reward players for solving old, difficult puzzles". 
If a developer does pipe in they might answer why ā€œreward players for solving old, difficult puzzlesā€ is important to them. I hate those puzzles.
Looking forward to your proposal. 

Puzzle Scoring Based on Sequence Constraints

Instead of using a constraint for time and number of solvers, an additional scoring method could be based on the sequence constraints of the puzzle.

So this function takes the fraction of G-C and G-U base pairs, and uses them to determine the points awarded for solving a puzzle.

For a brief explanation, C1 and C2 are the max points awarded for solving a puzzle with a certain number of G-C base pairs and G-U base pairs. F_GC and F_GU are the fraction out of 1 for that particular base-pair in a design (For example, for a design with 10 base pairs, maximum number of G-C base pairs equal to 2, the F_GC would be 0.2). Here is a matrix with example scores based on C1=300 and C2=100. C_F_GU is a constant fraction of G-U base pair which acts to shift the center of the parabolic function. The reasoning behind this is that puzzles may be *more* difficult with a high A-U base pair requirement, so you could shift the center towards a higher G-U fraction to account for this.

This method isnā€™t dynamic, given G-C and G-U constraints are fixed, but unlike the previous dynamic scoring, it doesnā€™t capture the true difficulty of some puzzles, where difficulty may exist due to the structure.

Also, by multiplying -F_GC by some constant, you can increase or decrease the rate of the function. There is quite a bit of leeway in how points could be awarded here.