custom dictionaries not found

[haraldalbrecht]

First, I would like to make explicit that I like and use the OXCEC dictionary quite often. It has become kind of indispensable to me. It is fun to watch people staring at me when I read Chinese learning texts and lookup terms with the Palm when on the train.

One bug that still persists even in the newest 2.0.4 release is that custom dictionaries are not found at every startup. I've installed CEDICT and UniChin, both to internal RAM. However, sometimes the dictionary software finds them on startup, sometimes not (around 50:50 chance). I'm using the dictionary on a TT with PalmOS 5.0 and the latest Palm patches installed.

Have other users experienced the same problems? Has someone managed to even find a solution or workaround to this problem? Especially CEDICT has become handy since it often contains definitions and explanations more suitable to beginners, also many terms not found in the Oxford but other paper dictionaries.

-- Harald

 

[herve]

Hi,
I experienced some problems as well, but was using Oxford Dict on an external card.
I add some problems with the IDX files, which I solved.
I recall that checking the "search" options did help me as well, but I now have performance issues sometimes ...
I also noticed that a reset of my Clie did solve some errors (this comment is not necessarily related to Oxford Dict ;-), but it might help you...).
Herve

 

[haraldalbrecht]

Hi,
I also noticed that a reset of my Clie did solve some errors (this comment is not necessarily related to Oxford Dict ;-), but it might help you...).
Herve

Well, since I regulary back up my TT with BBVFS I did even a hard reset and first installed the TT patch before restoring everthing else. But unfortunately this did not solve the problem. I even checked on a clean TT without any applications but only the TT patch and the Oxford CE dictionary and it showed the same behavior of finding the custom dictionaries only for a 50:50 chance. The single install was to make sure that not another application was going "boing" and overwriting parts of the Oxford CE dictionary. But unfortunately, no luck.

 

[mikelove]

That's an odd one, custom dictionaries opening only half the time - is there any kind of a consistent pattern to this or does it happen completely randomly? And you're aware that they won't show up if the dictionary's in English-to-Chinese mode, right? (you have to switch it to C-E mode first)

Some basic diagnostic questions: how do you have the Oxford configured? Which set of files / installer options did you select? When you cleaned out your TT, how did you restore the Oxford files? By a HotSync? If so, did you disable the Backup conduit or rename the "Backup" folder in your Palm Desktop user folder first? (if you don't do that, it'll restore all of the files from your old setup automatically the first time you HotSync)

I'm not initially inclined to think this is a bug, since version 2.0 was developed mostly on a TT and I use custom dictionaries every day now on my TT3, but this is certainly very odd behavior...

 

[mikelove]

I recall that checking the "search" options did help me as well, but I now have performance issues sometimes ...

herve, is there any way that you could move the "IDX_" files back into your device's RAM? That should help somewhat with the performance problems. Really, though, if you have a choice (and don't have to copy both) it's much better to move the Oxford database files to VFS than the CEDICT/UniChin ones - the compression/encoding scheme used for the Oxford is very efficient for random access, so it does very well on an external card, whereas the custom dict encoding format is based on the PalmDOC format which was never designed to be randomly accessed off of a memory card and is therefore comparatively slow. (a third format which is faster and offers better compression than either of the first two is in the works for the ABC)

 

[haraldalbrecht]

And you're aware that they won't show up if the dictionary's in English-to-Chinese mode, right? (you have to switch it to C-E mode first)

:wink: Yes, I'm aware of this -- most of my time I'm using your software in C-E mode. But I don't mind that you are asking this question, since most often users just tap into the simplest traps (me included).

Some basic diagnostic questions: how do you have the Oxford configured? Which set of files / installer options did you select? When you cleaned out your TT, how did you restore the Oxford files? By a HotSync? If so, did you disable the Backup conduit or rename the "Backup" folder in your Palm Desktop user folder first? (if you don't do that, it'll restore all of the files from your old setup automatically the first time you HotSync)

For the clean install: I did a hard reset, even switched to English locale (instead of my native locale being Germany) and then only installed a full set of the dictionary including handwriting recognition and hires fonts to internal RAM. No other applications installed. However I installed the Tungsten T patch (FAT patch, as far as I remember). The problem persisted. I admit that this is strange and the nightmare of every programmer; I have no idea as to how to get nearer to the root cause of the problem. BTW -- everything installed to RAM, no SD card.

The full restore was done using BackupBuddy VFS without any hotsync involved. BBVFS restores from VFS files, so it basically also does some kind of garbage collection. Same problem persisted.

In RAM: Lexicus_CHWR_GENERAL, Lexicus_CHWR_GENERAL2, Lexicus_Detail_Template2, Lexicus_GB_segment, Lexicus_GB_templatePtr1, Lexicus_GB_templatePtr2, Lexicus_Gen_template_GB, Lexicus_HK_General, Lexicus_HK_template1, Lexicus_Stroke_Segment_Info, Lexicus_templatePtr2, Oxford Dict, Oxford Flashcard Database

On SD card: in /Palm/Programs/Oxford: CEDICT.PDB, IDX_CEDICT.pdb, IDX_UniChin.pdb, Oxford Chars.PDB, Oxford Database C-E.PDB, Oxford Database E-C.PDB, Oxford Index C-E.PDB, Oxford PY Accelerator C-E.PDB, Oxford Radical SC.PDB, Oxford Radical TC.PDB, Oxford SC 12.PDB, Oxford SC 24.PDB, Oxford Table C-E.PDB, Oxford Table E-C.PDB, Oxford TC 12.PDB, Oxford TC 24.PDB, UniChin.pdb

 

[herve]

is there any way that you could move the "IDX_" files back into your device's RAM? That should help somewhat with the performance problems.

Hi Mike,
In fact, I moved the CEDICT and UniChin dict to RAM some time ago, and it seems that it is better now.
What is new for me is the ability to move the IDX_ index files to RAM, and leave the databases on the external memory card. I just did it a few minutes ago, and it seems to work as if both index and dict were on RAM.
This maintains performances, and saves *bytes* in RAM!
Could you add some note about this in the manual?
Is there any way you could add this possibility during the install process?
Herve.

 

[mikelove]

For the clean install: I did a hard reset, even switched to English locale (instead of my native locale being Germany) and then only installed a full set of the dictionary including handwriting recognition and hires fonts to internal RAM. No other applications installed. .....

OK, between all of that info I ....'ed through you've given me enough info to try to reproduce that here, so I shall do so and report on my findings later on; thanks! (and I should have predicated that C-E/E-C remark with "this is probably a stupid question, but...")

herve, I'm afraid there's no way we can add this option during the install process (at least not until we've got the new InstallShield-based installer up and running), but it doesn't feature very prominently in the manual and we should definitely add it to the installation section somewhere. Glad to hear it's running better now.

 

[mikelove]

I haven't been able to reproduce this problem yet; have you tried moving the two index files IDX_CEDICT.pdb and IDX_UniChin.pdb to your device's RAM? That might help matters; the dictionary has a much easier time finding custom dictionaries if the index files are in RAM.

It still sounds like a bug, but until we manage to find it and fix it this will hopefully keep the custom dicts running reliably.

 

[RonLPleco]

Same problem with custom dicts

Indeed I have always had to put the IDX in RAM even though the dictionary is on the card.

 

[mikelove]

Your custom indexes can't be put into VFS because you haven't checked the "Search VFS for custom dict indexes" option in Options - we leave it disabled by default because it can significantly slow down the startup sequence on some Palms. The indexes themselves also work very slowly from VFS, and in general we recommend that you put everything custom dict related into RAM if at all possible - we're planning to introduce a revamped database system this summer to remedy this (you can put the one and only dictionary file in VFS and the software will copy everything it needs into RAM) but for now custom dicts are slow and awkward and there isn't much we can do to fix that.

 

[RonLPleco]

Custom Dicts on VFS

Even with 'Search VFS' chceked I have never been able to use a custom dict without the .IDX files being in RAM. Oh well.

 

[mikelove]

Do the files in the card have names beginning in "IDX_"? If you changed the names that could be causing some problems... you might want to try copying them over again with a utility like the freeware program Filez (http://www.nosleep.net) which does a good job of preserving filenames.

Aside from that, I can't think of much of anything to suggest on this... if we can find out the source and there's an easy fix we'd certainly like to fix this, but otherwise the problem's going to disappear with the summer release anyway (and hopefully not be replaced by an even larger one