Tentative 2.0.4 Flashcard Change List

[mikelove]

Though we may now end up calling 2.0.4 2.0.5 to sync up with the version numbers on iPhone (the first, flashcard-less release of which we're already calling 2.0.4).

Anyway, here's what we're tentatively planning to change about the flashcard system for the next minor update:

• Add a button in "Card Info" to immediately add a card to the active study pool, and an indicator that it's in that pool. Probably also a button to remove it from the pool if you want to get rid of it.
• Add a shuffle option for Review Incorrect.
• Add an undo button for the previously-tested flashcard at least.
• Add support for multi-char words in stroke order tests, and possibly also an option to draw lines with the stroke outline hidden and check the order / direction of those instead of just tapping on strokes.
• Break rep-spacing into "strict" and "flexible" options - "strict" works as rep-spacing does now, while "flexible" prioritizes cards by roughly how far you are past the time you should have reviewed them, and lets you keep going after you've finished every card that's due (with a prompt coming up before crossing into that group)
• Add an option to weight automated score changes by the interval that's passed since you last reviewed a card (so if the card was due to review again 6 days later but you brought it up again 3 days later, the score increase on a correct answer would only be half as big) - how this works with increases versus decreases is TBD, there may be some code added to keep a bunch of wrong answers close together from dragging a card's interval down too much too, and with intervals longer than the target one we may not weight the changes linearly but taper them off instead
• Possibly add a priority setting for cards (probably only factoring into flexible rep-spacing), and the ability to change that setting on a bunch of cards at once in the "batch" screen
• Switch score / difficulty numbers to floating-point values (i.e. they can now have decimal digits), to better accommodate the above changes

As you can see, most of these are features that don't require much work on the UI side of things, the notable exception being improved stroke order tests.

We're aiming to have 2.0.4 out in beta form in late January, though that could change if there's some high-priority iPhone task (fixing some massive and nasty bug / dealing with some new incompatibility in iPhone OS 3.2 / etc).

 

[ben_gb]

Hi Mike,

Something I've noticed on rep-spacing is that (for me) the ramping up of the gap between views seems to frequently end up with the card taking too long to come round again (eg 2 or 3 months in some cases), so I have trouble remembering it (even though earlier I was able to instantly recall it, so was able to click on the "6" score several times). On the other hand, earlier on in the testing of a new card, I sometimes feel a card is coming up a bit too often.

I'm using agressiveness 3 and Difficulty scaling as "First Increase".

Is this likely to be affected by your changes?

One thing I think would be handy is an overall maximum delay between showing a card, so regardless of how often you've got a card correct, there will only ever be a maximum of X days between showings (eg I might set this to 60 days).

Ben

 

[mikelove]

We're planning to make the various constant values that go into the algorithm user-configurable (as opposed to being tied to a particular "aggressiveness" level), so yes, you'd be able to tweak those to increase the initial spacing interval but decrease the speed with which intervals increase. An overall maximum delay is an interesting idea... we'll see if we can work that in too. Thanks.

 

[mongrel]

An overall maximum delay is an interesting idea...

Over several years I changed the mnemosyne/anki algorithm in various ways, eventually conceding that my changes just weren't essential. The adjustments I've kept are merely concessions to my weakness, not fundamental. But one change I still believe in is limiting the maximum interval. Here is my argument:

If the study routine includes seeing the flashcarded items outside of the flashcard program, as it usually will, then this can sabotage the algorithm: If I see an item daily, and today its *card* comes up after an interval of x >> 1 day, I mark it right *without having proven that I can remember it for x*, and the algorithm promotes it further to 2x not knowing, and not practically able to know, that I'm in effect cheating it.
Under these circumstances, of seeing the items outside the flashcard program, this algorithm that works so well at shorter intervals gradually turns to logical mush, continuing to promote items without justification. Some flashcarding theorist-marketeer should embrace this as a principle, like this: "The standard memory-stretching theory breaks down when you read your lessons outside of the program, and you probably *should* do that. Therefore, this feature we've provided..."

The practical solution I suggest is a user-configurable limit on the max interval.
Does anyone else, acknowledging the problem, see a better solution?

Now I want to continue a little but the above is my main point.
If a limit is set, what to do then with the cards that are "too easy"? Either pass them easily, accepting this necessity, or boldly suspend them. Further justify suspension if desired by scheduling the reading/listening/dictation/etc. of the whole poems/dialogs/stories from which the suspended cards were extracted (assuming you did break down something you wanted to be able to read later). For example, if 我, 人, 是 and twenty others are in lesson 1, and you know you can read, listen, even dictate lesson 1, then you could schedule this whole lesson if desired, and suspend those cards. This kind of scheduling can be handled very well by the flashcard program, by the way. In my case, about 1700 cards are condensed into 130 cards, in a separate deck, each having a whole poem/story that I can now read aloud easily, about four per day, every month (with no desire to stretch that, actually). I feel safe counting these as "known", so I could remove this much of the load from the main flashcard routine. (That is the idea, though I haven't actually forced myself to suspend those 1700 yet.) But these are just some random suggestions; please remember the problem described above was my main point.

...we'll see if we can work that in too.

Thank you for thinking about it.
(P.S. I own the iPhone version, so I haven't actually seen your flashcard module yet.)

 

[jiacheng]

mongrel:

I kind of feel like the card difficulty rating takes into account you encountering the word outside of the flashcard session. If it's a word that you see often, you are unlikely to miss it and thus the score will progress faster. If you didn't see it, then the chances of you missing it are greater, thus resetting the score and the score progresses more slowly the next cycle.

There actually is a score limit of 204800, which is about a 5.6 year interval. I personally would be very careful mucking with this even if we had the ability to do so. To set it lower could really increase your daily load. For me, having a deck of over 7000, and 3 different tests, the daily load is kind of hefty and limits how quickly I can pull in new vocabulary and do other kinds of studying outside of flashcards.

The act of viewing a card outside of a flashcard session may skew the accuracy of the algorithm somewhat, but I don't see how limiting the maximum score fixes the problem. If anything, encountering the word outside of a session is going to boost your memory of it, thus the score on that particular card would be even higher if the score is to accurately represent how well an item is ingrained in your memory.


If you're afraid you'll forget the card, you could always just mark it wrong when you see it again in the session.

 

[ben_gb]

I think the problem that mongrel was describing is possibly that if you see the word outside the flashcard sessions, then maybe it's your short(er)-term memory that is remembering it (when it next comes up in the pleco session), whereas in fact it hasn't sunk into your longer term memory. But because you mark it as correct, Pleco is assuming that it is in fact embedding in your long term memory and so lengthens the time before the next showing.

The thing is that you probably don't know whether an answer for a card comes from your long or short term memory, so you might not realise that you don't know it as well as you think you do. I've had this problem many times, when I have instant recall of some words/characters so keep marking them as '6', but then they disappear for a while and next time they come back I haven't a clue what the correct answer is.

I do think the 'maximum time limit' setting should be optional, so you don't have to use it if you don't want to.

I guess the problem is that if you consider that 5+ years is the current maxium, then that's a long time to wait to discover you don't know a card.

However, I too would use the max time limit with caution, as I often find that the process of 'just' forgetting a word (ie so it's ' on the tip of my tongue', but I cant quite recall it so I have to 'fail' that card), actually seems to help me to remember it better in future...in fact this often seems more effective than seeing a word very frequently.

Sigh....this memory stuff is complicated :?

Ben

 

[HW60]

After some time using Pleco flashcards, every single flashcard has a history of its own, because there is a steady income of new cards and more and more cards have to be repeated. Sometimes new cards get a very high score at the beginning, some lessons would deserve more time to repeat, but the time is needed to learn new cards. Using flashcards I always select categories and sometimes use Card filter, so Pleco's algorithm only changes the score of these cards, and the history of the flashcards gets more and more different. Therefore a year ago I started with a new scoring file and had to repeat all cards again (I would not recommend doing that). I use the default values of Pleco's algorithm, because I want to spend more time learning new words than find out which features I should change. Still I have 2 suggestions to make, which in my opinion would improve the usability of this really helpful tool:

1. I would appreciate (again, I know) a card filter based on "Cards learned", because I think cards learned is an information that really helps to concentrate on difficult to remember cards. Because of the different card history the score of the flashcards cannot be of similar value.

2. Actually every flashcard user knows his own weaknesses best. I would appreciate an Anki-like possibility to determine the next appearance of a flashcard after some days in the future. This should work on top of Pleco's algorithm, and would only be used for those cards I either really know (+200 days) or for those I always forget (+1 day).

Cards learned and the time to the next appearance of a card are very easy to understand.

 

[mikelove]

mongrel - I actually believed in / cited the reviewing-outside-of-the-software problem as a reason for not adopting difficulty-tracking spaced repetition in PlecoDict 1.0, though I've ultimately come to believe that that sort of basic algorithm still does more good than harm - at the very least, it's pretty much essential nowadays for marketing reasons alone. But I do think it's a problem, and a maximum interval seems like a sensible enough solution - then again, one could argue that if you're still remembering a card well after, say, 90 days (with or without a few extra reviews in between), you probably are about at the point where you can suspend reviewing it anyway, so it may not matter if the interval stretches farther at that point.

jiacheng - yes, you could mark it wrong, or use the manual score setting button to knock it back to a shorter interval.

ben_gb - the maximum would certainly still be optional; even the 204800 limit now is for purely mathematical reasons, some business with decimal precision and performance optimizations on Palm that we'll soon be able to stop worrying about. Very complicated stuff, yes... probably the most brain-straining aspect of all the Pleco design work.

HW60 - so #1 would be a feature specifically based on your "learned" setting? What benefit would this offer over the current system of adjusting the ratio of learned to unlearned cards?

#2 is actually already an option - go to Advanced Settings / Controls, check the "Score" box, and set its default behavior to "inc" by "20000" "pts" - that way, tapping on that button will instantly add 200 days to the card's repetition interval. You can tap-hold on the same button for alternate score change options - "Ignore Answer" puts the card right back so it'll come up again the next time you review cards.

 

[HW60]

HW60 - so #1 would be a feature specifically based on your "learned" setting? What benefit would this offer over the current system of adjusting the ratio of learned to unlearned cards?

#2 is actually already an option - go to Advanced Settings / Controls, check the "Score" box, and set its default behavior to "inc" by "20000" "pts" - that way, tapping on that button will instantly add 200 days to the card's repetition interval. You can tap-hold on the same button for alternate score change options - "Ignore Answer" puts the card right back so it'll come up again the next time you review cards.

#1 The line "unlearned card ratio" was missing in my "Cards" screen, but became visible after I changed Card Selection from repetition-spaced to random. I did not understand the result, however: I selected a category with 139 words and Cards learned 117 (3 in a row) or 103 (score>1000). With "Random" and "Unlearned card ratio"=90% and Number of Cards = all in Test Setup for that category I would expect 90% of (139-117)= 20 cards or 90% of (139-103)= 32 cards, but in both cases I got 137 from the 139 cards. Probably I must have misunderstood something? If 90% of something is 137, than something is 152, which is more than all cards of the category. What would be the input to a test of all or almost all cards unlearned of a category? I would like to test all unlearned cards of a category, in the above example 22 or 36, but 90% would help for the beginning. --- I just tried Unlearned Card ratio = 10% and again got 137 cards to test ...

#2 I had not found out that 20000 points is equal to 200 days (and 100 points equal to 1 day?), but I will try that option. The additional possibilities of tap and hold of the score button seem to be promising - thank you. One more question: Has "Skip Card" in Advanced / Controls the same meaning as tap and hold of the Score button + Ignore Answer? I could deactivate the Skip Card then. Probably I will spend part of the time learning new words for learning this powerful tool!

 

[mikelove]

Per the manual:

Unlearned card ratio lets you choose what percentage of cards in the session will be "unlearned" (see below for what determines if a card is "learned"). This lets you ensure that you're getting a good mix of new and old cards, and that cards you've only started studying will come up more often than cards you've been studying for a while (a less complicated version of the "Frequency-adjusted" option below). If there aren't enough learned / unlearned cards available to maintain the desired ratio, cards will be mixed at the beginning of the session in the correct ratio, then once the system runs out of cards of one type it'll continue displaying only cards of the other type.

So 90% would mean that about 90% of the cards that you see in your next session should be "unlearned." However, it sounds like in this case you don't have enough "learned" cards to make up even 100-90 = 10% of the cards in the session. In general this option works better when applied over a larger list - the idea behind the learned/unlearned management system really is to let you dump a huge pile of new cards into Pleco (say an entire HSK level) and then not be overwhelmed by thousands of new cards to study; "limit unlearned' makes sure these cards are doled out slowly, and "unlearned card ratio" lets you make sure that if you have some already-learned cards in your pool, those are still coming up occasionally even if based on their number / frequency the system would normally give you almost nothing but unlearned cards.

And yes, 100 points equals 1 day by default in "repetition spaced" study at least. "Skip Card" has almost the same meaning as "Ignore Answer," but with the latter, the card is still eligible to come up again later in the session (mainly important if you're doing an endless / frequency-based session rather than a repetition-spaced one).

 

[HW60]

I think the problem Pleco cannot solve is to make a selection with the desired unknown card ratio, when "all" cards of a category are selected. In that case Plece can select cards from "all" according to the ratio, but there comes a point when Pleco has to add a card which is not according to the ratio - and does not stop to limit the total number of the test (and keep the ratio!), but takes almost "all" cards. Unknown card ratio only works, when a fixed (depending on the amount of known and unknown cards eventually small) number of cards is selected from a category instead of "all". I made a successful test with ten cards: exactly 9 cards had a score below 1000 (= learned if), one was above.

Therefore my proposal for a change of the flashcards is: let Pleco stop selecting cards from a category (and offer a smaller number of cards for the test than selected), when there are no more cards to select without disturbing the ratio (that can also be a problem, when by Card Filter selections the ratio cannot be maintained, or when I know 99% of all cards and select 90% unknown card ratio for a comparatively high number of cards).

 

[mikelove]

That's a reasonable enough change, I guess, at least as an option - no reason you shouldn't be able to make that a strict ratio if you want to. We could integrate this with the new code / UI we're adding for more flexible repetition-spaced study, letting you stop after you've reviewed all the cards that are due or past due or letting you continue and review cards that aren't ready yet if you want to put in some more study time - this would be a similar prompt for non-rep-spaced tests, letting you stop after you've reviewed the max # of cards you can review at your desired ratio or letting you keep going. Though I don't know if we can get this worked in in time for the next update...

 

[HW60]

If you then include unlearned card ratio of 100% (and maybe (or not) 0% for symmetric reasons), than I could finally test unlearned cards only ...

 

[mikelove]

HW60 - We could do that, though at that point it might make more sense to just add another item to Card Filter for learned / unlearned cards.