Recover from Falling behind

[HW60]

Falling behind (Backlog) means not being able to do Pleco reviews for some time, thereby considerably increasing the number of due cards of a session (as described in "Mapping my Understanding of SRS to Pleco flashcard settings"). The question is how to recover from this situation.​

​

http://ankisrs.net/docs/manual.html#falling-behind​

Falling Behind​

If you fall behind in your reviews, Anki will prioritize cards that have been waiting the longest. It does this by taking the 200 cards that have been waiting the longest, and showing them to you in a random order up until your daily review limit.​

​

​

Anki uses a daily review limit, e.g. limits the daily number of cards. Mike usually does not like a daily limit, but also​

​

it should be possible for somebody to create some flashcards, tell our system how many cards they feel like reviewing right now, and have that review happen in an intelligent / optimized way without having to think about any of the other settings​

​

​

That probably means that after the user decided on the number of cards to review, the only problem is to find out what is an "intelligent / optimized way without having to think about any of the other settings". Actually I would not select from the cards that have been waiting the longest as Anki does: If you have 2 cards, one with score 10,000, one with score 100, and both have been waiting for 20 days, the probability to remember the card with the score of 10,000 is rather high, but the card with a score of 100 will more likely be forgotten. Therefore - as mentioned several times before - I would let Pleco select as many cards with the lowest scores that are needed to reach the decided number of cards. From my point of view that is probably all that is to do; of course if the number of cards selected by the user is too small, the user will never recover from falling behind. But Pleco could tell him.​

 

[mikelove]

Up to a point, maybe, but the other thing about a card with a score of 100 is that you've probably invested a lot less time in learning it, and so can afford to have it drop out and come back again later when you have more time. But this also shows off a weakness in the way we think about scoring, since a card could have a score of 100 because you just answered it incorrectly after answering it correctly a bunch of times or because you just added it to your review pool, and we don't necessarily want to handle those situations in the same way.

Right now our SRS prioritizes overdue cards not by score but by how far past due they are; one of the many advantages to this approach is that ensures roughly the same mix of old and new cards that you'd have any other day, and does that regardless of how many cards you get through before you stop. So you're working both your short- and long-term memory with each review. I don't see prioritizing cards with low scores as an improvement over that, since then you're only studying a) new cards (which you shouldn't be adding now anyway since you're so far behind schedule) or b) cards that you just remembered incorrectly. It's true that you may then get some score-10,000 cards early on in a test this way, but they're score-10,000 cards that you should have reviewed two weeks ago, and forgetting them now after taking all the time to learn and review them in the past probably isn't a good thing either.

But your basic complaint is with the fact that you're confronted with all of your overdue cards at once, and I simply don't see that as a significant enough problem to justify adding a new option for it in advance of this larger redesign, particularly not given its potential for misuse - we'd have to either put it in an obvious place and have people misuse it, or put it in an out-of-the-way place and have even most of those who would use it correctly not know it exists. Personally I find it helpful to be confronted with the full scope of the problem when I come back to studying after a long absence - motivates me to get through my backlog.

But in our big redesign this is another case where it may be worth sacrificing transparency / customizability for something that works better; if a card had 3 or 4 different types of "score" associated with it (perhaps even including external factors like how recently you'd viewed the dictionary entry for that word), it would be next to impossible for even an advanced user to customize or even really to understand how Pleco dealt with all of those, but we could probably deliver more optimized testing that way.

 

[HW60]

Up to a point, maybe, but the other thing about a card with a score of 100 is that you've probably invested a lot less time in learning it, and so can afford to have it drop out and come back again later when you have more time. But this also shows off a weakness in the way we think about scoring, since a card could have a score of 100 because you just answered it incorrectly after answering it correctly a bunch of times or because you just added it to your review pool, and we don't necessarily want to handle those situations in the same way.

That is a little bit contradictory - maybe I invested a lot of time for repeating a card which now has a score of 100 because I answered it incorrectly. And actually while I have an increasing number of overdue cards which I review with the "number of cards option" (by selecting the appropriate score filter), I have found time (!) to learn new cards in a different profile I use for new cards.

I don't see prioritizing cards with low scores as an improvement over that, since then you're only studying a) new cards (which you shouldn't be adding now anyway since you're so far behind schedule) or b) cards that you just remembered incorrectly. It's true that you may then get some score-10,000 cards early on in a test this way, but they're score-10,000 cards that you should have reviewed two weeks ago, and forgetting them now after taking all the time to learn and review them in the past probably isn't a good thing either.

The score-10,000 cards are cards that count as "learned cards" since a long time ago. I certainly will not forget them so quickly as a card with a low score. And reading books and newspapers also helps to keep them in memory. My upper score filter today (after 5 weeks recovering from falling behind 2 weeks) is close to 1,000, therefore the overdue cards are mostly the score-1,000 to score-10,000 cards. And when I answer these cards incorrectly due to the delay caused by falling behind, they had been learned and quickly climb up the score again.

But your basic complaint is with the fact that you're confronted with all of your overdue cards at once, and I simply don't see that as a significant enough problem to justify adding a new option for it in advance of this larger redesign, particularly not given its potential for misuse

I think this whole discussion about the best way for SRS - this is not a Pleco problem, but a SRS problem - should just help to find the best solution for a problem which has not been solved in most SRS systems and definitely was not meant as a complaint. Personally I found a very good way with changing the score filters and therefore have plenty of time to wait for your new super-easy system (or keeping the last version of Pleco before the big update ...). I don't think that the "number of cards" is an option - that is normal input to a flashcard system like the category. It is not necessary to enter "number of cards" as long as the SRS reviews do not fall behind, but afterwards "number of cards" is much better input for any kind of user than "upper limit of score filter".

Personally I find it helpful to be confronted with the full scope of the problem when I come back to studying after a long absence - motivates me to get through my backlog.

I think that a daily Pleco flashcard session of 1 hour is all I want to invest, and therefore I reduced (!) the number of cards I review per day since my falling behind. The additional free time I use reading books, listening mp3 and TV, and learning new cards (!).

But in our big redesign this is another case where it may be worth sacrificing transparency / customizability for something that works better; if a card had 3 or 4 different types of "score" associated with it (perhaps even including external factors like how recently you'd viewed the dictionary entry for that word), it would be next to impossible for even an advanced user to customize or even really to understand how Pleco dealt with all of those, but we could probably deliver more optimized testing that way.

Sacrificing customizability could make the program more easy to use for Pleco beginners, but I do not understand sacrificing transparency. If people do not know what is going on, the number of questions you will have to answer like "where are my flashcards to review?" will probably increase. And 3 or 4 different types of scores does not sound too super-easy. But let's see ...

 

[gato]

Backlog Problem

The main problem I was concerned with was how to go through a large list of new (not yet learned) cards ("New Cards") -- say I have a list of 2000 New Words to learn. If the SRS program requires following a rigid schedule and forces you to study all cards up for study on the same day, it won't be long before one gives up. This is similar to the backlog problem involving learned cards that one is just reviewing ("Review Cards"). In both cases, the number of cards scheduled by the SRS program for review on a single day is too much.

In effect, there are two backlog problems: (1) New Cards, and (2) Review Cards.

But I see that both Pleco and Anki have tackled the New Card backlog problem with a limit on the New Cards presented.

http://pleco.com/ipmanual/flash.html#cardselsettings
Limit # of unlearned is a very useful option that lets you limit the pool of non-"learned" cards to a finite number; this keeps you from being overwhelmed by more new words than you can manage at a time, so if you add 8000 new flashcards to your database you won't suddenly be expected to start memorizing all 8000 at the same time. You simply set a threshold at which a card is considered "learned" and then Pleco will try to keep the number of un-"learned" cards that you're studying at any given time below a certain number.

http://ankisrs.net/docs/manual.html#new-cards
New cards/day tells Anki how many new cards you’d like introduced on each day you open the program. Missed days will not cause the cards to pile up.

This appears to be adequate to address the New Card backlog problem.

I still think more about the backlog problem with Review Cards before commenting further.

UI Issue

On the UI issue, it may be useful to thinking about what are the common usage cases, and then make it as easy to use the system for those cases as possible.

I can see a few usage cases are common but not necessarily easy to configure for under the current UI:
(1) Starting with a large list of New Cards
(2) Backlog of Review Cards due sporadic study schedule (e.g. two or three-day lay offs between study days)
(3) Backlog of Review Cards due to vacation
(4) Cram session over two or three days involving a large number of both New Cards and Review Cards before a test

Even a beginner to Pleco (someone not familiar with Pleco's settings) may have one of these usage cases -- even beginner users in terms of the UI may have somewhat complicated usage needs -- so one shouldn't just simplify the UI for the simplest usage cases and refer people to a big manual for somewhat more complicated but still common usages.

 

[HW60]

In effect, there are two backlog problems: (1) New Cards, and (2) Review Cards.

There is one important difference between these two: With New Cards there is usually no time pressure, with Review Cards the longer it takes to recover from falling behind the greater the danger to forget the cards.

But I see that both Pleco and Anki have tackled the New Card backlog problem with a limit on the New Cards presented.

I think a per day review limit should override the New Card backlog problem as it should override the Falling behind problem.

 

[mikelove]

I don't think that the "number of cards" is an option - that is normal input to a flashcard system like the category. It is not necessary to enter "number of cards" as long as the SRS reviews do not fall behind, but afterwards "number of cards" is much better input for any kind of user than "upper limit of score filter".

I'm not saying "number of cards" isn't an input we should have, only that a "number of cards" option for SRS that makes it possible for users to fall ever-farther behind on their reviews by not studying enough cards per day is a bad idea. And until we have a good / user-friendly way of doing "number of cards" for SRS it's not worth doing it at all - we'd either have to bury it so nobody would use it or put it out front and have lots of people mis-use it.

Sacrificing customizability could make the program more easy to use for Pleco beginners, but I do not understand sacrificing transparency. If people do not know what is going on, the number of questions you will have to answer like "where are my flashcards to review?" will probably increase. And 3 or 4 different types of scores does not sound too super-easy. But let's see ...

I meant transparency at the level of the scoring algorithm - we don't necessarily need to reveal / explain all of the numbers that we keep track of and how they go together, at least not within the app's UI.

I can see a few usage cases are common but not necessarily easy to configure for under the current UI:
(1) Starting with a large list of New Cards
(2) Backlog of Review Cards due sporadic study schedule (e.g. two or three-day lay offs between study days)
(3) Backlog of Review Cards due to vacation
(4) Cram session over two or three days involving a large number of both New Cards and Review Cards before a test

I think 1/2/3 at least are within what a well-designed automated system ought to be able to take care of without any extra configuration; we have enough data to figure out for ourselves whether you're studying sporadically or just on vacation. 4 would make sense to keep as a separate option, though.

There is one important difference between these two: With New Cards there is usually no time pressure, with Review Cards the longer it takes to recover from falling behind the greater the danger to forget the cards.

But if it's a Review Card that you reviewed exactly once and remembered incorrectly (thus keeping its score at 100), is it really such a great sacrifice to not review it again right away?

I think a per day review limit should override the New Card backlog problem as it should override the Falling behind problem.

Without a) removing some cards from active review or b) requiring the user to do a lot of custom configuring / schedule some extra time to catch up, I don't think it would work for falling behind.

 

[HW60]

But if it's a Review Card that you reviewed exactly once and remembered incorrectly (thus keeping its score at 100), is it really such a great sacrifice to not review it again right away?

What is the final result of the recovery from falling behind? Same number of flashcards, no more due cards?

 

[mikelove]

What is the final result of the recovery from falling behind? Same number of flashcards, no more due cards?

The final result is that you're back to having enough time to a) review all of your old cards as often as they need to be reviewed and b) introduce some new cards every day as well. But cards may go from "old" back to "new" if you can't keep up.