Suddenly can't link flashcards to user dictionary

[Captain Planet]

Hello,

I'm using flashcards linked to my user dictionary. I prepare them lesson-by-lesson, and import with the "store imported definitions in user dict" option on. Suddenly this has stopped working and new entries are not appearing in the dictionary anymore. I am able to import the flashcards, but they appear as "custom". Also, when I try to convert them using the "custom -> user dict" option (either in batch or one-by-one), this doesn't work. In both situation there is no error message or any sign of problem, it's just that no new entries are appearing in the dictionary, and when I close the the flashcard properties and open them again, they're still custom.

I'm using Pleco 2.0.4 on WM. The dictionary is converted to new format and not locked. There are about 2300 entries in it.

I tried locking and unlocking the dictionary and compacting the databases but it doesn't make the problem disappear. Any ideas what to do?

 

[mikelove]

That's a new one... is your flashcard database converted to the new format too? (under Preferences / Flashcards) Any chance the user dictionary might be on a locked storage medium, like an SD card? User dictionaries and flashcard files should really be kept in internal memory on WM - even on unlocked SD cards they can be somewhat dicey, and there's not much we can do about it since WM's SD card drivers are so hopelessly awful.

 

[Captain Planet]

The flashcard database is converted to the new format too. Seems the problem is between the keyboard and the chair. I'm thinking the cause could be that some time ago I accidentally connected the phone through USB in disk mode while Pleco was still running (normally I use ActiveSync). At that time Pleco was in flashcards mode with an open session, which then terminated prematurely (of course). After that I closed the program, but then ran it again (initially didn't realize the USB disk mode unmounts the whole SD card filesystem). Pleco didn't start without access to the SD files (my setup is that basically everything is on the card) but still created another flashcard database file in the internal memory, which I later deleted. I was thinking everything was fine since then but now I've also noticed there are six orphaned entries in the flashcards, which disappeared from the dictionary (only pinyin left).

So it seems it's the PQB files that somehow got corrupted and the question now is how to get everything back to work. I guess I should export the dictionary and flashcards, delete everything, and import them back. Is that right? Can you suggest the best import/export strategy? Only export flashcards and recreate the dictionary from them?

I can post my PQB files if it's useful in any way.

 

[mikelove]

Well if the user dictionary was corrupted that could certainly explain why you can't add entries to it - bit odd that you wouldn't see any kind of an error message, though. It's also possible this could just be a bug in Pleco 2.0.4, but you'd be the first person to report it if that's the case.

The first thing I'd recommend is that you exit Pleco, remove the user dictionary file from your system (back it up on your computer, then delete it), create a brand new user dictionary, and see if you can successfully add flashcard entries to it - if that works then you can be reasonably certain the problem was related to user dictionary file corruption, and you can then export / re-import your old user dictionary into that new database file.

 

[Captain Planet]

OK, I've been sitting on this all day today. First I recreated the dictionary by exporting and re-importing definitions to a new dictionary. This worked.

Then I tried remapping the flashcards to the new dictionary. Not only did this not work, but also after attempting to do it, all the flashcard information is lost ("The dictionary or entry that this card links to appears to be missing or is unavailable").

So, I restarted from backup and tried exporting all the flashcards - this worked. I have them both as XML and text files. The files appear well-formed. Then deleted all the flashcards and tried to re-import them from the XML file, with definition source as "dict only" and my new rebuilt dictionary as the only one on the list. This is when it gets interesting:

1) Generally it doesn't link the flashcards to the new dictionary; instead, they appear with the message "The dictionary or entry...".
2) During the import there is a lot of messages that more than one definition was found (which shouldn't be the case), asking me to choose the proper definition; however, the available definitions displayed from the dictionary are all empty.
3) When I interrupted the import (as it was obviously going the wrong way), it terminated with an "Unknown error".

So basically the today's question seems to be: how can I get it back in order without losing my test profiles and flashcard scores (that's why I didn't try (a) deleting Pleco Flashcards.pqb - I understand the profiles are kept there and cannot be backed up - or (b) importing from the text file - as this would mean parting with all the scores (months of hard work ).

 

[Captain Planet]

I think it's a bug with Pleco after all. I created a simple test suite. The dictionary and flashcard files only have two entries each.

Before doing all these steps I removed all the pre-existing PQB files.

What I am doing:

1. Create a new dictionary with entries from test-dict.txt.

Dict / Manage Dicts... / Create New... / Create / (move to top) / (select) / Configure... / Import / Text Encoding: UTF-8 / Tick Unsafe mode (faster) / Import... / test-dict.txt / Yes...

2. Import flashcards from test-flash.xml.

2a. Import flashcards trying to remap them to the dictionary:

Modules / Open Flashcards... / Import / Choose file...: test-flash.xml / Skip/Prompt/Prompt / Tick Unsafe... / Dicts... / Definition source: Dict only / Dictionary search order: New-... (only) / Start

Result: failure (see test-result-a.png)

2b. Import as custom flashcards and batch-remap later (with a freshly created Pleco Flashcards.pqb, just to be sure):

Modules / Open flashcards... / Import / Choose file...: test-flash.xml / Skip/Prompt/Prompt / Tick Unsafe... / Dicts... / Definition source: File only / Start

Custom flashcards imported OK so far.

Batch / Remap to dict: New-... / Yes

Result: failure (see test-result-b.png, slightly different)

2c. Import as custom flashcards and remap individually later:

Remapping the cards one by one seems to still work as it should (see test-result-c.png). Only it's not feasible with 2300+ cards, which is my situation.

Mike, can you help me with that, please?

I'm sorry I don't know how to describe it more clearly.

 

[Captain Planet]

Result screenshots:

 

[mikelove]

Reproduced and fixed, thanks - looks like a performance optimization we made in the new user dictionary database format was preventing the flashcard database from correctly linking to user dictionary entries that it found through a search in the flashcard importer / remapper. We'll correct this in the next update, but in the meantime, here are a couple of possible workarounds:

1) Go back to the old user dictionary format; it shouldn't exhibit this same problem.
2) Import the entries as flashcards, then dump them to new user dictionary entries with the batch "Custom -> User Dict" command - the links should be preserved correctly in that case.
3) Exit Pleco completely (Quit command in the Dict menu), copy your flashcard database to your desktop, open it up in a SQLite database browser utility (like the aptly-named SQlite Database Browser or the popular SQlite Manager Firefox add-on), and execute the following SQL command:

UPDATE pleco_flash_cards SET dictentry = (dictentry & 16777215) WHERE (dictcreator > 0) AND (dictentry > 16777215);

You'll have to run this again when you add more entries, though running it on entries that have already been fixed shouldn't cause any problems; make sure to back up your database before you do this, though.

 

[mikelove]

You'll have to run this again when you add more entries

Sorry, I meant to say "when you add more cards that you want to link to user dictionary entries" - simply adding new user dictionary entries or flashcards won't screw up the ones that are already there, you'll just want to run that SQL query again after you've imported some new user-dictionary-linked cards or remapped some cards to the user dictionary.

 

[Captain Planet]

Wow, that was quick! Thank you for the response. It's great that it's been identified and will be solved with the next release. The temporary workaround with SQL query doesn't seem to solve the problem, the situation is still the same. How do I downgrade the dictionary to the old format?

 

[mikelove]

There's no way to downgrade, unfortunately, you'd need to go back to a backup of the old dictionary. But I'm not sure why the SQL query wouldn't work - did you upload the finished database to your handheld after you were done? And are you sure Pleco wasn't running in the background? If you email me your flashcard database file I can try applying the fix for you - remember, though, this fixes cards that were already imported / remapped, it doesn't do anything for cards you import / remap after you apply the fix (until you apply it again).

 

[Captain Planet]

Thanks, your support is great. I really appreciate it. I've just sent you an e-mail. I think I was applying the fix correctly, I tried three times to be sure. Perhaps I'm not familiar with the software (SQLlite Manager). The file I sent should already have the fix applied if I did it properly.