Maximum Score

[jiacheng]

Since I've been using pleco's SRS for my flashcards for quite some time now, I'm accumulating cards with scores in the range of 102400-204800. As 204800 is the maximum score, many of these cards will hit the ceiling on the next viewing, which could be around two years from now. So what I would like to know is, what is the plan for upping the maximum score? two years might seem like a while, but it will be here before you know it.

If there is no plan to change it within that time, do you see any issue with scaling down the algorithm to use 10 points per day instead of 100 and start the minimum score at 10, and divide all the current scores by 10?

 

[mikelove]

Since I've been using pleco's SRS for my flashcards for quite some time now, I'm accumulating cards with scores in the range of 102400-204800. As 204800 is the maximum score, many of these cards will hit the ceiling on the next viewing, which could be around two years from now. So what I would like to know is, what is the plan for upping the maximum score? two years might seem like a while, but it will be here before you know it.

If there is no plan to change it within that time, do you see any issue with scaling down the algorithm to use 10 points per day instead of 100 and start the minimum score at 10, and divide all the current scores by 10?

I don't see any issue with that. But we have no current plans to change the maximum score; I tend to think that SRS principles break down at those durations - do you really think that you'd want to wait more than 6 years between repetitions? At some point a card ought to hit "max level" and not have its interval advance further - SRS may be great for shifting cards from short- to medium-to long-term memory, but facts retained for half a decade probably don't need to be reviewed any more at all; if you're not encountering a given word anytime in a given 6-year interval (and hence having your memory of it refreshed by that encounter) it's probably not worth knowing.

So I'd recommend either regularly culling cards with scores that high or setting a "score filter" in card filters to do the same thing.

 

[jiacheng]

At some point a card ought to hit "max level" and not have its interval advance further - SRS may be great for shifting cards from short- to medium-to long-term memory, but facts retained for half a decade probably don't need to be reviewed any more at all;

You might have a point that facts retained for such a long interval may not need to be reiviewed anymore, and culling cards with a maximum score would be reasonable default behavior. However, by setting a maximum score, you are actually creating the opposiite effect by default. A card which reaches 204800 would continue to be reviewed once every 6 years. But if the score were not limited, then reviewing a card once every ~12 or ~24 years seems much preferable to reviewing it once every 6.

By limiting the score, you are forcing the user's hand to either add a filter, or continuing to review those cards once every 6 years unnecessarily by default. I think that the simplest and most intuitive way to achieve a cut off and take the card out of the pool completley, would be to have pleco set a default score filter. That way, users could raise, lower or remove it as they see fit. And if there is a need to set a maximum score, that can still be done using the "maximum score" prameter in the automatic algorithm.

By the way, Is there some sort of technical reason that the maximum score in the db is 204800? seems odd that it's not an integer power of 2.

 

[HW60]

If you change points/day in the Tweaks section of advanced settings to 10 instead of the default value of 100, you should be able to extend the time until a card is presented again (at maximum score) from 6 to 60 years - if that does not help, maybe a value of 5 points/ day does.

 

[mikelove]

You might have a point that facts retained for such a long interval may not need to be reiviewed anymore, and culling cards with a maximum score would be reasonable default behavior. However, by setting a maximum score, you are actually creating the opposiite effect by default. A card which reaches 204800 would continue to be reviewed once every 6 years. But if the score were not limited, then reviewing a card once every ~12 or ~24 years seems much preferable to reviewing it once every 6.

By limiting the score, you are forcing the user's hand to either add a filter, or continuing to review those cards once every 6 years unnecessarily by default. I think that the simplest and most intuitive way to achieve a cut off and take the card out of the pool completley, would be to have pleco set a default score filter. That way, users could raise, lower or remove it as they see fit. And if there is a need to set a maximum score, that can still be done using the "maximum score" prameter in the automatic algorithm.

The number of complaints we've gotten about "limit unlearned" being on by default (people don't understand why they're only seeing 200 cards) leaves me extremely wary of adding any more default filters - extending the intervals to a ridiculous duration probably would make more sense than that.

By the way, Is there some sort of technical reason that the maximum score in the db is 204800? seems odd that it's not an integer power of 2.

No hard technical reason, just some fudging to deal with integer overflow - there are a couple of places in our code (mostly in Frequency-adjusted card selection) where we have to add up all of the scores of all of the cards eligible to be included in a test, and since we're storing that value in an unsigned 32-bit integer, a maximum value of 204800 means that we can safely handle 20971 cards without any overflow. 20,000 is about the largest deck we've ever seen, and even if a few people's decks get bigger it's very unlikely that all of their cards would have maxed-out scores.

The choice of 204800 specifically was because it's a power-of-2 multiple of the default score of 100, but a flat 200,000 would work just as well - we just wanted something of around that order of magnitude. There was no particular reason other than tidiness to pick something that was an even power of 2, and 262144 felt a little high, 131072 a little low.

We would have just designed the system with 64-bit integers originally, but runtime support for those on Palm OS was extremely buggy; that's no longer an issue on iOS and Android, but it's still some very dicey code rewriting so it doesn't really make sense to do it until we're already rewriting that part of our flashcard system for some other reason.

If you change points/day in the Tweaks section of advanced settings to 10 instead of the default value of 100, you should be able to extend the time until a card is presented again (at maximum score) from 6 to 60 years - if that does not help, maybe a value of 5 points/ day does.

That's what I'd recommend for now - I'm pretty sure that by 2072 we'll have switched everything over to 64-bit

 

[jiacheng]

In trying to scale down the algorithm, I have stumbled onto a bug. I am using the automatic algorithm with parameters equivilant to the old automatic with aggressiveness of 3. I changed points per day to 10, first increase to 40, and reduced all card scores by 90%. When I start a session, I am coming up with ~5000 cards due. The overdue intervals are showing up as negative numbers, being reviewed 100 days or more before they are due. Any clue why this might be happening?

 

[HW60]

Some time ago I made an Excel sheet to calculate Pleco's Scores. Maybe you can test your tweak parameters first using the spreadsheet. There are actually 2 sheets: the second is for the tweak parameters, the first for looking at the results.

 

[mikelove]

In trying to scale down the algorithm, I have stumbled onto a bug. I am using the automatic algorithm with parameters equivilant to the old automatic with aggressiveness of 3. I changed points per day to 10, first increase to 40, and reduced all card scores by 90%. When I start a session, I am coming up with ~5000 cards due. The overdue intervals are showing up as negative numbers, being reviewed 100 days or more before they are due. Any clue why this might be happening?

Not sure... honestly not a scenario that we've tested that much, but I don't know any reason why the math wouldn't work. If you PM or email me a copy of your flashcard database we can take a look at it here and try to figure out what's going on, though.

 

[mikelove]

It turns out that there's a minimum points per day of 20 - added so long ago on Palm that I completely forgot about it, but if you set your points per day to 20 and scale down everything by 80% instead of 90%, that should work correctly. Sorry about that - we'll make sure that the 32-bit overflow issue is fixed long before the new effective maximum imposed by 204800 becomes a problem.

 

[jiacheng]

Awesome! Thanks a bunch