Flashcards being skipped

[smiss]

Card Selection: repetition-spaced
Day type: Hours
Points per day: 100
Limit is unchecked
Scoring type: Automatic
Aggressiveness: 3

Since a few days Pleco seems to skip a lot of flashcards while doing repetition-spaced sessions. Starting the session the counter in the upper left corner says for example 40 cards will be tested. But then it doesn't count 40, 39, 38, ... but skips cards randomly instead, and it doesn't matter if I answer correctly or not. The session is over much earlier then before. I'm quite sure that this did not happen before. It obviously does not make a lot of sense to include cards in the session and then don't test them.

I use this flashcard deck for quite some time now and some cards have a pretty high score. Maybe it has something to do with that, but I can't figure out what's the problem here. :?

 

[mikelove]

That's probably happening because those cards are invalid or incomplete in some way; in order to keep it from taking an incredibly long time for sessions to start, Pleco doesn't double-check that every card is valid before adding it to the session, so it occasionally has to skip over invalid cards during the session. The easiest way to find invalid cards is to go into Manage Flashcards, choose "Advanced" from the top-left menu, then choose "incomplete" as the first and only search type.

If you're doing a "self-scored" session, you can include those cards in it by going into Advanced Settings / Tests and checking the "allow missing sections" box (if it isn't already checked). For multi-choice and free-answer tests, though, there's no way for the software to test you on a card if any part of it is missing, so incomplete cards are always skipped in those.

Also, do you use the user-created dictionary feature at all? Do any of your flashcards link to user dictionary entries? If you recently deleted your user dictionary, or if some sort of error prevented Pleco from loading it, that might make any cards that link to it "incomplete." Likewise, if you recently deleted any other dictionary database files, cards that link to those might be rendered "incomplete." You can go into Manage Flashcards / Batch and use the "Remap to dict" option to easily switch those flashcards over to link to a different dictionary.

 

[smiss]

Thanks a lot. The problem is solved now and it had to do with links to non-existing cards.

Thanks to your advice I've checked if there are any "incomplete" cards, but Pleco did not show any. So I had a look at some of my cards and I noticed that there were some which did not show any translation. That would be one of my definitions for "incomplete"... whatever, Pleco's might be different. All of these cards have been in the additional Chinese-German dictionary HanDeDict. Two or three weeks ago I've updated this dictionary which seems to have been scaled down a bit now. Some of the words I have added before were missing in the updated dictionary.

Fortunately, I still got an old backup with the earlier HanDeDict. Now, everything's fine again. But it's still odd, that Pleco did not show any incomplete cards, although there have been a lot, obviously.

 

[johnh113]

That's probably happening because those cards are invalid or incomplete in some way; in order to keep it from taking an incredibly long time for sessions to start, Pleco doesn't double-check that every card is valid before adding it to the session, so it occasionally has to skip over invalid cards during the session. The easiest way to find invalid cards is to go into Manage Flashcards, choose "Advanced" from the top-left menu, then choose "incomplete" as the first and only search type.

If you're doing a "self-scored" session, you can include those cards in it by going into Advanced Settings / Tests and checking the "allow missing sections" box (if it isn't already checked). For multi-choice and free-answer tests, though, there's no way for the software to test you on a card if any part of it is missing, so incomplete cards are always skipped in those.

Also, do you use the user-created dictionary feature at all? Do any of your flashcards link to user dictionary entries? If you recently deleted your user dictionary, or if some sort of error prevented Pleco from loading it, that might make any cards that link to it "incomplete." Likewise, if you recently deleted any other dictionary database files, cards that link to those might be rendered "incomplete." You can go into Manage Flashcards / Batch and use the "Remap to dict" option to easily switch those flashcards over to link to a different dictionary.

Dear Mike,

I've had this problem in the past when using a newer version of a dictionary. The flashcard still comes up but it doesn't update the hanzi. This just happened to me when I finally last week, a couple of months late, updated the adso and cc dicts. At that time, I searched for "remap" but couldn't find an explanation of where to find it or how to use it. But now you have explained where it is and how to use it. Thank you.

But I don't think it works. When I went to advanced and searched for "incomplete" the search returned nothing after about a second. Now you either have discovered some miraculous new indexing/search algorithm which is going to make you a gazillionaire, or you didn't really search all 30,000+ cards that I have. I suspect that latter, because I know there are broken links because I've already deleted two such cards when they came up after I installed the newer versions of the dictionaries.

John

 

[mikelove]

smiss / johnh113 - that's odd; maybe there's some sort of bug that causes it to fail to flag deleted entries in still-installed dictionaries as invalid.

johnh113 - As far as the hanzi not being updated, normally they aren't updated in the dictionary either - if an entry's hanzi change then it's assigned a new ID. What dictionary have you witnessed this happening in? You can fix this with a hidden option in Manage Flashcards / Batch: in the "Remap to dict" menu, scroll own to the very bottom and you'll see a "Rebuild" option. Select that option, then tap "Remap to dict" and your flashcards will have their headwords / Pinyin updated with the latest definitions in their linked dictionaries.

 

[johnh113]

smiss / johnh113 - that's odd; maybe there's some sort of bug that causes it to fail to flag deleted entries in still-installed dictionaries as invalid.

johnh113 - As far as the hanzi not being updated, normally they aren't updated in the dictionary either - if an entry's hanzi change then it's assigned a new ID. What dictionary have you witnessed this happening in? You can fix this with a hidden option in Manage Flashcards / Batch: in the "Remap to dict" menu, scroll own to the very bottom and you'll see a "Rebuild" option. Select that option, then tap "Remap to dict" and your flashcards will have their headwords / Pinyin updated with the latest definitions in their linked dictionaries.

Dear Mike,

I described it as hanzi not being updated but maybe I should just describe what I observe when the problem occurs. This has occurred on 3 flashcards since I updated adso and cc about a week ago, so probably related to adso update. What happens is I complete a flashcard by clicking on the check or the x signifying right or wrong. Normally the next card with its hanzi comes up. But in this case, the hanzi doesn't change. But it is the next card. I can use the back button to go back and of course nothing changes because it just went back to the last valid card. I use the back button again and it goes back to the previous card. I use the forward button and it goes forward to the next card, forward again and it goes to the problem card but doesn't update the hanzi. I still see the hanzi from the last good card. I can use the info button to see the statistics and I can use the delete button to delete the bad card which is what I normally do in this situation. The same thing happened to me a couple of years ago when I updated another dictionary. After deleting the bad card, everything is back to normal until I come across another bad card.

I tried the rebuild option. I set the search for dictionary and searched for adso and cc-edict. I then ran the remap to dict rebuild command. It returned instantly. That always scares me because it was so fast, and there was no visual indication of anything happening. I did it twice to make sure. I'll see if I get any more bad flashcards.

John

 

[mikelove]

Have you enabled "Force defns from dicts" in Advanced Settings / Display? There was a bug that could cause the definition update issue you describe with that, but it's been fixed in the latest 2.0.8 beta (see the beta forum to download that).

With the rebuild option, are you sure the entries you wanted to rebuild were all actually listed in the results (i.e. that you'd tapped on the Search button after setting up your search)? The "incomplete" search issue turns out to have been another bug, so that should be fixed in the next 2.0.8 beta.

 

[johnh113]

Have you enabled "Force defns from dicts" in Advanced Settings / Display? There was a bug that could cause the definition update issue you describe with that, but it's been fixed in the latest 2.0.8 beta (see the beta forum to download that).

With the rebuild option, are you sure the entries you wanted to rebuild were all actually listed in the results (i.e. that you'd tapped on the Search button after setting up your search)? The "incomplete" search issue turns out to have been another bug, so that should be fixed in the next 2.0.8 beta.

Dear Mike,

I'm running 2.08b3. Yes, I always use "Force Defns from dicts".

My search command was wrong. I did an advanced search for adso AND cc-cedict. I should have done adso OR cc-cedict. When I did the OR search it found 317 cards and I then did the "Batch", "Remap to dict", "Rebuild" command and it processed the words one by one. We'll see if I get any more bad cards.

On another point, since 2.0.8b2 (I think), while using flashcards, the program suddenly stops for 5-15 seconds after I press the "check" or the "x" to go to the next card. Nothing happens. If I press again, or press back or audio, the commands are stored up and run after Pleco comes back to life. It is as if it is very busy doing something else, like maybe writing to disk or something.

And with 2.0.8b3, after starting up the flashcards (seems longer than it used to), I get a blank screen for 5 seconds before the first hanzi appears, and I have to wait another 5 seconds or so before the program is ready to respond.

No big deal.

I've purchased my iPhone and have played around a little with the free dictionary. Just waiting for flashcards to transfer my license.

John

 

[mikelove]

If you turn off "Force defns from dicts," does that get rid of the background-working, definitions-not-updating, and/or blank screen problems? And have you updated your flashcard database to the new format? (via Preferences / Flashcards) That should make starting up sessions a lot faster.

 

[johnh113]

If you turn off "Force defns from dicts," does that get rid of the background-working, definitions-not-updating, and/or blank screen problems? And have you updated your flashcard database to the new format? (via Preferences / Flashcards) That should make starting up sessions a lot faster.

Dear Mike,

I just now updated the flashcard database to the new format and turned off "Force defns from dicts" and it did eliminate the 5 second blank screen after starting a session. But I like "Force defns from dicts" so I turned it back on and the blank screen was still not there, so probably not related to "Force defns from dicts". Also the definitions not updating is related to changing a dictionary, not to the bug in "Force defns from dicts." I did still have a bad card immediately after updating the database and turning off "Force defns from dicts," so I suspect I will still have some of those because of the dictionary change even though I did the rebuild. I will run for a while without "Force defns from dicts" to see if the background-working delays go away.

John

 

[mikelove]

Hmm... could you possibly email me your flashcard database file? And let me know which profile is likely to turn up some of these invalid / not-updating-correctly cards? By "changing a dictionary" you mean that you remap cards to a new dictionary and the definitions then stop updating correctly, right?

 

[johnh113]

Hmm... could you possibly email me your flashcard database file? And let me know which profile is likely to turn up some of these invalid / not-updating-correctly cards? By "changing a dictionary" you mean that you remap cards to a new dictionary and the definitions then stop updating correctly, right?

Dear Mike,

I usually delete the bad cards when they come up. Next time one comes up I'll save it and send you my flashcard database file.

This is happens when you release a new version of a dictionary. I download and install the new version of an existing dictionary. This overwrites the old version. Then I get a few cards that don't display correctly, typically no hanzi, just the hanzi from the previous card. But the flashcard is not the previous card, it is the "next" card with its own statistics, etc. When this happened about a year ago, the bad cards even had audio, just no hanzi. This time, no hanzi and no audio.

Not a big problem. Just an annoyance.

By the way, I did about half an hour with "Force defns from dicts" turned on and a half hour with it turned off and in neither case did I get the hang that seemed to indicate some background process. Maybe changing to the new database format fixed it. If it recurs I'll let you know. But also not a big problem, just an annoyance, and I will be abandoning my Palm and changing to iPhone anyway as soon as flashcards are available.

John

 

[mikelove]

OK, thanks! Sounds like this isn't a "force defns" issue as much as an invalid entry one - we'll see if we can reproduce that here. (likely to affect the iPhone version too, so we definitely need to check it out)

 

[johnh113]

OK, thanks! Sounds like this isn't a "force defns" issue as much as an invalid entry one - we'll see if we can reproduce that here. (likely to affect the iPhone version too, so we definitely need to check it out)

Dear Mike,

I've now got two bad flashcards that I can send you. Same as described before. When I go to the next card, everything is from the next card except no hanzi (the hanzi just remains unchanged from the previous card) and of course no definition. These two cards both play audio and have all of their statistics. To send you the database I used Plecomover to backup the PlecoFlash from today to my SD Card. I'll email you the PlecoFlash.pqb-backup file. I don't use profiles so you can just use the default profile. Set your date for 13 March 2010 and there should be 11 cards that come up for repetition spacing. Two of those cards will be bad. I left the others in there so you get a flavor of what happens when the next card is a bad card. One of the cards only appears in the adso dictionary, so that strongly suggests that this is a problem created by updating to a newer version of a dictionary (improperly, I'm sure, as I only copied the newer version over the top of the older one; although after you suggested "rebuild" I did a "rebuild" for adso and cc-cedict but I still get bad cards).

On a second note, I am still getting these hangs when Pleco just stops as if it is doing something in the background. I did flashcards for about an hour with "force defns from dict" and got a couple of short hangs (shorter duration and less frequently than before I converted to the new database), I turned off "force defns from dict" and ran for an hour with no problems. I turned "force defns from dict" on again and got another hang. I'm not sure that this is significant though. It probably has nothing to do with "force defns from dict." The hangs are much less frequent and much shorter duration now that I have changed to the new database.

Two other data points that are probably unique to my usage of flashcards are that 1) I have over 30,000 flashcards with a file size of 9.7MB and 2) Against you better wishes, I store my flashcards in Internal (PDB) on my Palm Treo 680.

John

 

[johnh113]

Dear Mike,

The weirdest thing. The two bad flashcards li4ji2 痢疾 (my last email was wrong when I identified it as 立即) and she4tao4 设套 have come back to life! When I moved the date to tomorrow, 14 Mar 2010 (as I had finished all my cards for today), they were the first two cards that came up (they were the two remaining cards from "yesterday" that I hadn't answered correctly because I wanted to keep them coming up every day until you have a fix for my bad flashcards). They are both from adso which confirms the updated dictionary connection.

But although they are alive again, they are incredibly slow. It takes almost 5 seconds to move from one card to the other and almost another 9 seconds before audio works. But then audio works instantly for as many times as I want to repeat it.

The fact that the two cards from yesterday that I hadn't answered correctly came up first is another issue. Since the recent 2.0.8 betas, cards seem to come up in some kind of order rather than randomly. I definitely prefer random and the degree of randomness in repetition spacing has declined dramatically.

By the way, I tried to send you my 9.7MB flashcard file but I think your server rejected it as too big. Odd because I've sent you this file before when it was only a little bit smaller.

John

 

[mikelove]

It looks like the problem was actually your use of the "head / pron from dicts" option - hadn't thought to test that, but with it enabled the invalid cards immediately started coming up incorrectly. A "rebuild" won't help in these invalid cases - there's no valid dictionary entry to regenerate the card header from - only "remap" will, but the newly muscled-up "incomplete" checker in Beta 4 should be able to identify those cards so that you can remap them without affecting any other cards.

The "hang" I think is just a combination of Palm OS and the size / location of your flashcard database - there's no way your Palm can load the entire thing into its working memory at once, so it could easily take a few seconds in some of these cases to load a particularly far-flung database record with all of the necessary cache-flushing / reloading involved.

As for the order of cards in repetition-spaced tests, there was actually supposed to be a checkbox on that screen to sort them randomly - absent that, they show up in the order of what time they're due, with just a few hours' worth of random variation to catch cards with identical (or nearly-identical) histories. But with that box missing, the option will always be turned off. So we'll try to get that enabled for the next beta too.

 

[johnh113]

Dear Mike,

Thanks for finding the problems.

I work for a small company that has about 1000 employees worldwide and uses in-house software and a lot of off-the-shelf software for image creation and animation so I have lots of opportunities to judge software developers. You are the best I have ever seen in terms of finding and fixing bugs, customer support, maintaining good customer relations, etc., and of course you have a great, great product.

Thank you for all that you do.

John

 

[mikelove]

You're very welcome! Thank you very much for the bug report (and the kind words), and congratulations on (almost) hitting the 1 million cards reviewed mark - not sure if that's a Pleco-wide record but it's got to be pretty close.

 

[mfcb]

congratulations on (almost) hitting the 1 million cards reviewed mark - not sure if that's a Pleco-wide record but it's got to be pretty close.

no danger from my side, hehe, just 0.4 million cards reviewed (in the last 2 years), so i will need another 3 years at the current rate, :lol:

 

[johnh113]

It looks like the problem was actually your use of the "head / pron from dicts" option - hadn't thought to test that, but with it enabled the invalid cards immediately started coming up incorrectly. A "rebuild" won't help in these invalid cases - there's no valid dictionary entry to regenerate the card header from - only "remap" will, but the newly muscled-up "incomplete" checker in Beta 4 should be able to identify those cards so that you can remap them without affecting any other cards.

The "hang" I think is just a combination of Palm OS and the size / location of your flashcard database - there's no way your Palm can load the entire thing into its working memory at once, so it could easily take a few seconds in some of these cases to load a particularly far-flung database record with all of the necessary cache-flushing / reloading involved.

As for the order of cards in repetition-spaced tests, there was actually supposed to be a checkbox on that screen to sort them randomly - absent that, they show up in the order of what time they're due, with just a few hours' worth of random variation to catch cards with identical (or nearly-identical) histories. But with that box missing, the option will always be turned off. So we'll try to get that enabled for the next beta too.

Dear Mike,

1. With 2.0.4b4, I used your newly muscled-up "incomplete" checker. It found over 200 cards that were incomplete in some way. I then remapped them to ABC and ran the "incomplete" checker again. Still a lot of cards that were incomplete (but fewer than before). Remapped again to a different dictionary. Ran the "incomplete" checker again. Still some incomplete. Remapped to a different dictionary. I did this for every dictionary I have until there were only 3 cards left that couldn't match any dictionary. These were previously in adso but must have been removed from the latest adso dictionary. My point in explaining this is two-fold. First, the new incomplete checker works. Second, there should be a better way to get everything remapped without having to iteratively run the incomplete checker and the remapper so many times. Perhaps a list of dictionaries that determines the order for the remapping. That way I would only have to remap once. And maybe, after the remap, replacing the search results with a new list of cards that couldn't be remapped. Also, after the remap, there have been no problem cards where the hanzi doesn't show up.

2. I no longer think the "hang" is related to big database and slow Palm Treo 680. I now realize that every card that hangs is an adso card. Every time a flashcard that is mapped to adso comes up, there is a 5 second delay before the card appears, and then an 8 second delay before the audio plays. If I just look at dictionary entries in adso, there is no delay at all. The delays are just with the flashcards.

3. I still think that I'm not getting random cards, but I put that information in the 2.0.4b4 feedback thread.

I've got my iPhone and will be moving Pleco over as soon as you release flashcards.

John