How is the score calculated in SRS?

[kirkyonyx]

Hi,

I'm wondering how the SRS system by Pleco works. So I've tried to read the documentation but I couldn't find the math behind the scoring system in learning my flashcards in the Pleco Built-in SRS system with the default settings. What happens with the score when I tap "didn't know" for example?
Would be glad if someone here could help me out!

 

[Shun]

Hi kirkyonyx,

The basic principle you need to understand is that the score increases if you gave a correct answer, and decreases if you've given an incorrect answer, and that the amount of this increase/decrease is determined by the easiness of a flashcard, which is a number saved with the flashcard along with the score. You get an idea of how easiness works on this page in the manual:



As you can read in other places in the Flashcards reference in the manual, you can fine-tune a lot of things around scoring. If you tap "didn't know" or "incorrect answer", the score will probably go down if you haven't already been tested on the same card that day. Pleco 4's system will be radically different, but it will come with a full Pleco 3 compatibility mode, as Mike has stated.

Hope this satisfies your curiosity to some extent,

Shun

 

[kirkyonyx]

Hi Shun,

thank you for your answer! That describes the general approach pretty good, but I'm interested in how the score value is changed when I tap "didn't know", you know? I know that I can modify the perccentage increase/decrease depending on the classification, but how is this solved by default?

 

[Shun]

Hi kirkyonyx,

you're welcome. The default values for the change of easiness are just a reasonable number. How it happens is always more or less the same, i.e. through the same formula. I created a new profile which has the default settings, and it says there that the easiness will change by -10 if you give an incorrect answer, and by +4 if your answer was correct.

It seems to me that there are two main paths: If you have Prompt for quality enabled, the easiness changes in one of six ways you can set under Tweak Parameters (which depend on your answer in a test, how well you knew the card from 1-6). If you have Prompt for quality disabled, which is the default way, there are only the two values above you can set under Tweak Parameters for the easiness to change, one of which would be for the "didn't know" you mentioned. How the easiness affects the score change is influenced by the easiness divisor, which works like this:

“Easiness divisor is used in the formula for calculating a new card score after a correct answer. The card’s easiness factor (which defaults to 100) is divided by this amount, then multiplied by the card’s score to get the new card score. This is an easy way to quickly make the algorithm more aggressive (by lowering this value) or less aggressive (by raising it) in how quickly it increases cards’ scores.”

Pretty simple, isn't it?

(I know it's tricky to get one's point across over the forums without the ability to ask back and forth easily.)

Hope this helps, otherwise someone else can add to this.

Shun


PS: To be sure, I don't fully understand the current Pleco 3.2 scoring system. I prefer manual control, and I think Pleco 4's system is the one I will look at more fully. I just thought you might be happy with my level of knowledge regarding Pleco 3.2's system.

 

[mikelove]

I know that I can modify the perccentage increase/decrease depending on the classification, but how is this solved by default?

The default behavior in this case is that the score goes back to its minimum value of 100. (= 1 day SRS interval)

 

[kirkyonyx]

The default behavior in this case is that the score goes back to its minimum value of 100. (= 1 day SRS interval)

And what's the factor when I mark the flashcard as 'correct'?

 

[mikelove]

And what's the factor when I mark the flashcard as 'correct'?

If it's the first time correct, 400, otherwise it multiplies it by the "easiness" divided by 40 (which by default would mean 250%).

 

[bryanwithay]

I would also like to better understand the formula, so that I'm more confident before changing the score of a card, etc. Maybe I could experiment, but if the information is already out there, I would like to know. Is the algorithm known?

It seems to me that the main factors are score and easiness. Perhaps the overdue number would also factor if a card is overdue.

 

[Shun]

Hi bryanwithay,

we can try to reproduce the formula from the manual. It isn't complete, but we can combine it with what we found out earlier in this thread to get a fuller picture. A summary from the Pleco Reference:

Flashcards

One part of the formula calculates how much time has passed since the last time a card was tested (the number of days, either as 24 hours/day or calendar days), and compares that number to the result of the following division:

score of card
-----------------------------------
points per day (default: 100 points)​

So if a card has a score of 400, and you left "points per day" at the default of 100 points, it will appear in the pool of cards to be studied if 4 or more days have passed since the last time you were tested on it. If you were a little lazy in the last week, then of course a much larger backlog of cards will come due. Or, as you suggest, you could increase the cards' scores by a large amount, or reduce the "points per day" number as a means of catching up. You just need to make sure the total number of semi-learned cards isn't too large for your short- and medium-term memory, and everything is able to trickle down into long-term memory.

There is also a card priority setting which overrides the above mechanism, but we can ignore it for now. There is also the Learned threshold. I'm unsure if this one is used mostly for the Random card selection, or also for Repetition-spaced.

Combined with the information on easiness further up in this thread, I think you should now have enough information to experiment with scores. You can always back up the flashcards database first, then restore it to its previous state.

Cheers,

Shun

 

[HW60]

In 2014 I made an Excel sheet with the formulas valid at that time. You can enter figures in yellow cells, e.g. tweak parameters and test results for 10 successive tests. Maybe some of the formulas have not changed since then ...

It is not allowed to upload Excel files, so I added .txt to the file name.

 

[bryanwithay]

Great, thanks!

 

[Cameroon]

Hi,
I calculated my scores based on manual and didn't get the scores Pleco actually gives me (partially):

The card with score 315 / easiness 92 after Answer 1 became: score 267 / easiness 80 (the score 84.7% while 85% in settings due to approximation that's ok, the easiness changed exactly as in settings)
The card with score 315 / easiness 92 after Answer 2 became: score 283 / easiness 82 (the score 89.8% while 90% in settings due to approximation that's ok, the easiness changed exactly as in settings)
The card with score 315 / easiness 92 after Answer 3 became: score 299 / easiness 84 (the score 94.9% while 95% in settings due to approximation that's ok, the easiness changed exactly as in settings)

The above part raised no questions.

But with the right answers and increasing formula I didn't get any understanding:

The card with score 314 / easiness 92 after Answer 4 became: score 531 / easiness 88 (again easiness changed as it should have, but as I expected the final score should have been 457 (easiness 92 divided by easiness divisor 60 multiplied by score 314 and then again multiplied by correct score increase % 95).
The card with score 315 / easiness 92 after Answer 5 became: score 544 / easiness 92 (score as per formula expected 483)
The card with score 315 / easiness 92 after Answer 6 became: score 556 / easiness 96 (score expected 507)

Or maybe the formula I used is incorrect?
I used:
New score = Old score * (easiness / easiness divisor) * correct score increase.

 

[mikelove]

Any chance that this is implicating the 'if review early' / 'if review late' options?

 

[Cameroon]

Any chance that this is implicating the 'if review early' / 'if review late' options?

Exact, I totally forgot this factor weighing in.

So the formula for correct answers should look like:
? New score = Old score * (easiness / easiness divisor) * correct score increase * review late/early modifier ?
Where review late/early modifier equals to (days actually passed / total days till due date initially) squared, square rooted, unchanged, changed linearly or normally depending on settings.

For incorrect answers the formula is:
? New score = Old score * incorrect score decrease ?
That's it, no easiness factors/divisors/review late/early modifiers?

Also, when we set '[number] in row' filter, does this mean 'last in row', or 'anywhere in row'?

And if more than one filtering rule is set, is a logical 'OR' or 'AND' implied?

Plus a suggestion, maybe qualified answers shall also be taken into account, i.e. make it possible (in filters) to require not just a number of correct or incorrect answers in different sequences but also differ how correct or incorrect the answers were (6 available (in)correctness flavors). It could be useful for example when 3 'Remembered perfectly' answers are enough to get to the next level of testing but if the answers were not so perfect (4 or 5) then you shall collect at least 5 of them to prove you are ready for more.

 

[mikelove]

For incorrect answers the formula is:
? New score = Old score * incorrect score decrease ?
That's it, no easiness factors/divisors/review late/early modifiers?

Correct.

Also, when we set '[number] in row' filter, does this mean 'last in row', or 'anywhere in row'?

Last in row.

And if more than one filtering rule is set, is a logical 'OR' or 'AND' implied?

Logical AND.

Plus a suggestion, maybe qualified answers shall also be taken into account, i.e. make it possible (in filters) to require not just a number of correct or incorrect answers in different sequences but also differ how correct or incorrect the answers were (6 available (in)correctness flavors). It could be useful for example when 3 'Remembered perfectly' answers are enough to get to the next level of testing but if the answers were not so perfect (4 or 5) then you shall collect at least 5 of them to prove you are ready for more.

Likely possible in future updates, yes.

 

[Cameroon]

Thanks Mike for your answers,
Does your 'Correct' also apply to the formula I supposed to be used in case of right answers?
I.e. is the below correct as well:

? New score = Old score * (easiness / easiness divisor) * correct score increase * review late/early modifier ?
Where review late/early modifier equals to (days actually passed divided by total days till due date initially) squared, square rooted, unchanged, changed linearly or normally - depending on settings

And this,

Logical AND.

Sorry for reasking (was sure it ought to be OR), is it really so?
So with every additional filter we're actually narrowing the scope of cards for testing, not expanding it with additional arrays?

 

[mikelove]

Does your 'Correct' also apply to the formula I supposed to be used in case of right answers?
I.e. is the below correct as well:

Yes, I think so.

Sorry for reasking (was sure it ought to be OR), is it really so?
So with every additional filter we're actually narrowing the scope of cards for testing, not expanding it with additional arrays?

That's correct, yes. If we wanted to do OR's we'd probably have to allow you to have the same filter appear in multiple OR'ed blocks.

 

[Cameroon]

Hello Mike, forum members,
I am still in doubt on how this formula works.

? New score = Old score * (easiness / easiness divisor) * correct score increase * review late/early modifier ?
Where review late/early modifier equals to (days actually passed divided by total days till due date initially) squared, square rooted, unchanged, changed linearly or normally - depending on settings

In my case the intervals are still too close to each other for all three variants of correct answers (while I'd like to port Anki/Supermemo SRS logic into Pleco to disperse the mound of cards amassed in my flashcard module).
I expect, based on the formula, a 1.2 / 2.5 / 3.25 proportion (answer #4 vs. #5 thus must be different more than twice, and less than 1.5 - #5 vs. #6). In fact, the difference is no more than mere 1.3!
1.2 / 2.5 / 3.25 is set for this parameter (correct score increase) in my Pleco settings and is actually the only variable here, because all the other parameters are taken (for all flavors of answers) from the previious score equally (so are percepted as constants).
The late/early modifier shan't make difference here as I understand, because it's a multiplier and is the same for all answers, too.
And I set this parameter to 'change normally'.
How the math is actually performed inside Pleco's algorithm?

 

[mikelove]

How the math is actually performed inside Pleco's algorithm?

As described, as far as I know. However, some of the relevant code is very old and does some workarounds for math overflows (a big problem back in the 16 bit Palm era) that may lose precision or cause it to gate / limit extreme values, so it's possible that it's not quite diverging as much as you'd expect it to thanks to that.

 

[Cameroon]

Thanks, I see...
Below is my settings and a sample flashcard.
As you see, the difference between answers 4, 5 and 6 is disappearingly insignificant, while based on the settnigs I expected 4 and 5 difference to be more than 2 times, and almost 1.5 between answers 5 and 6.
How should I alter my settings so that choosing between different flavors of 'correct' has sense?
Or be like default Anki SRS settings: 120% for 'Barely remembered' ('Hard' in Anki), 250% for 'Remembered' ('Good') and 325% for 'Remembered perfectly' ('Easy')?