Pleco 2.04 on Palm OS

Atheist

Member
I'm using Pleco 2.04 with full features (SD depended keyfile) on Palm OS / Centro

Found several issues:
  • Pleco doesn't have "Stroke order" (0/0) for 辵 [chuo4], so to find it via HWR is too complex.
  • "Back" often works tooo slow (about 10-20 sec) with checked "Integrate full-text in" ("Preferences\Result")
  • Creating / Importing new dicts:
    • importing on Centro is the several times slower than on Treo 680.
      so I have to insert microSD via SD adaptor into Treo 680 and import, for example, 27000 entries takes about 7-8 hours
    • The worst thing is errors during importing - always don't know where and why the count of imported entries is several times smaller than original. Log file is appreciated! So I need spend many nights for import testing on Treo 680 with microSD reinserting.
      It would be great if Pleco DEMO could import dictionaries. But on other SD card (not authorised) there is no way to create new dictionary so I copy previously created empty dictionary on the legal microSD. On importing txt file Pleco DEMO stops whenever it wants :(
 

mikelove

皇帝
Staff member
Atheist said:
Pleco doesn't have "Stroke order" (0/0) for 辵 [chuo4], so to find it via HWR is too complex.
That's odd, the stroke order diagrams on Palm/WM are based on the same database we use for handwriting. However, 辵 by itself is a pretty rare character so perhaps it's not supported by handwriting either...

Atheist said:
"Back" often works tooo slow (about 10-20 sec) with checked "Integrate full-text in" ("Preferences\Result")
Sounds like an issue with file access speed... does it help matters any if you load a few of your Chinese-English dictionaries into internal memory instead of accessing them from an SD card?

Atheist said:
Creating / Importing new dicts:
importing on Centro is the several times slower than on Treo 680.
so I have to insert microSD via SD adaptor into Treo 680 and import, for example, 27000 entries takes about 7-8 hours
The worst thing is errors during importing - always don't know where and why the count of imported entries is several times smaller than original. Log file is appreciated! So I need spend many nights for import testing on Treo 680 with microSD reinserting.
It would be great if Pleco DEMO could import dictionaries. But on other SD card (not authorised) there is no way to create new dictionary so I copy previously created empty dictionary on the legal microSD. On importing txt file Pleco DEMO stops whenever it wants
Honestly, I wouldn't recommend trying to import a dictionary that large on any variety of Palm - we spent a LOT of time optimizing SQLite to squeeze as much performance as we could out of it, but there was only so much we could on a system that old. We're planning to offer a desktop user dictionary converter, but we've been waiting until we update our user dictionary database format to support full-text search (a heavily-requested feature that keeps getting pushed back due to people insisting that we support every new-fangled smartphone platform that comes along :) ).

Do you know anybody with a copy of Pleco on iPhone/iPod/iPad? The user dictionary database format is the same, so someone could import a database for you on that and then back it up / transfer it over to your Palm.

The most common reason that user dictionary imports fail is because one of the entries is too long, so I'd recommend double-checking your database to make sure that none of them are more than a few thousand characters.
 

Atheist

Member
mikelove said:
Do you know anybody with a copy of Pleco on iPhone/iPod/iPad? The user dictionary database format is the same, so someone could import a database for you on that and then back it up / transfer it over to your Palm.
I have a friend with Pleco on iPhone but he doesn't know how and where to find imported txt-dictionaries
 

mikelove

皇帝
Staff member
Atheist said:
Xi1 覀 doesn't have Stroke order but could be recognized via HWR
Is it the same character or the smaller component version (top half) that you posted in this message? Stroke order might not link that one to the full-size one.

Atheist said:
mikelove said:
Do you know anybody with a copy of Pleco on iPhone/iPod/iPad? The user dictionary database format is the same, so someone could import a database for you on that and then back it up / transfer it over to your Palm.
I have a friend with Pleco on iPhone but he doesn't know how and where to find imported txt-dictionaries
Settings / Manage Dicts - the Add New common creates one, then tap on it to see options to Import it and to Backup Database to transfer it to another device.
 

Atheist

Member
Here is my experience in user dictionary creation/importing on iPhone for Palm OS needs.

I took friend's iPhone 3G with Pleco for a week. Why so long? I found out that iOS 4 operates as Palm OS - no multi-tasking for standart applications, so you cannot use your smartphone during importing, moreover iPhone should be in airplane mode in order to prevent incoming calls that stop application.

First of all iPhone should be jailbroken in order save backup of the dictionary! (this was the main issue for my friend - he hesitated, so I left him with iPad :))

The second it is well structured (via tabs) txt-dictionary. The most problem is too long line!
As I realised Pleco for iPhone could accept lines about 4000 chars but in some cases it stops with error. The most safe length is less than 3800 chars (I splited all lines longer than 3800 chars so the most longest lines take about 3600+ <3700 chars).
The good question is how to calculate lengths of lines? I found the simplest way:
  • Excel is needed, for long dictionaries (longer than 65000 lines) Excel 2007/2010 is needed.
    Open Excel and make active the second cell in the first row, than from Data menu choose import from txt file. In the import dialog do not specify separators.
    After importing the second row will be filled with lines from txt-dictionary.
    Go to the first cell and create formula =LEN(B1) or =LEN(R1C2), than copy this cell and paste for on the first column.
    Now you have lengths of lines. But you should found ones longer than 3800. So select and copy the first column, than go to the another sheet and use Special paste with checked "values". In this sheet descending sort the column.
    Find each number greater than 3800 in the first column of the first sheet looking not in formulas but in values.
    In Word you should split found long lines

Now you may import txt-dictionary with Pleco. But the more entries in the dictionary the more time (geometric progression) for importing is needed. It depends on number of entries were imported and is not affected by whether txt-dictionary was splited in several files or it is one huge file.
So approximated time for the first number of entries to be imported
  • 30000 entries - about 1 hour
    60000 entries - about 2.5 hours
    75000 entries - about 3.5 hours
    90000 entries - about 6-7 hours
    120000 entries - about 13-15 hours
    200000 entries - about 2 days

Backed dictionary (pqb file) could be copied out from iPhone via SSH, iFunBox or T-PoT plugin for Total Commander.
All dicts works fine on Palm OS.

As you may know Palm OS doesn't support Unicode so Pleco for Palm OS could show only latin chars and chinese. In my case I had to transliterate all cyrillic chars in the source txt before importing.

Obviously source txt-dictionary should be in Unicode format. There are several types of Unicode formats:
  • UTF-8
    UTF-16LE
    UTF-16BE
Very often "UTF-16LE" is mentioned as "Unicode" (in Word, Notepad etc). I prefer this format. So in Pleco you have to choose UTF-16LE format on importing Unicode txt-dictionary.
 

mikelove

皇帝
Staff member
Sorry for the slow reply to this.

Atheist said:
I took friend's iPhone 3G with Pleco for a week. Why so long? I found out that iOS 4 operates as Palm OS - no multi-tasking for standart applications, so you cannot use your smartphone during importing, moreover iPhone should be in airplane mode in order to prevent incoming calls that stop application.
That's true on the 3G but not on newer iPhones - not much we can do about it on the 3G, though if you turn on "Unsafe Mode" that should at least make the import a lot faster. (it's only unsafe if your system crashes - once the import finishes successfully there's no lingering danger)

Atheist said:
The second it is well structured (via tabs) txt-dictionary. The most problem is too long line!
As I realised Pleco for iPhone could accept lines about 4000 chars but in some cases it stops with error. The most safe length is less than 3800 chars (I splited all lines longer than 3800 chars so the most longest lines take about 3600+ <3700 chars).
That's actually proven surprisingly tricky to fix, for some very obscure technical reasons - FWIW, though, we think we've got a solution for the next major update at least.

Atheist said:
So approximated time for the first number of entries to be imported
30000 entries - about 1 hour
60000 entries - about 2.5 hours
75000 entries - about 3.5 hours
90000 entries - about 6-7 hours
120000 entries - about 13-15 hours
200000 entries - about 2 days
Yikes, that's way worse than the numbers we usually see - you should definitely turn on Unsafe Mode. (bottom option in the import screen)

BTW, a fix for the long line problem that nobody has actually tried yet but that might theoretically work; requires a bit of SQLite hacking, though. In Excel, assign every user dictionary definition a unique number, and then import into Pleco a file containing those numbers in place of definitions. Create another file with SQL commands to replace the numbers with entry texts, like this:

Code:
UPDATE pleco_dict_entries SET 'defn' = "(correct definition)" WHERE 'defn' = "(number)"
One line like that for every line in your spreadsheet. Save that file with a .sql extension, then open up your imported Pleco user dictionary database on your desktop in a SQLite utility (SQLite Database Browser / SQLite Manager for Firefox / etc) and execute that file on it; that should replace all of the numbered definitions with the full versions.
 

Atheist

Member
It was in Unsafe Mode

I've tried importing 201K entries twice:
(i) transliterated txt-dictionary splited in 2 files (140K and the rest)
(ii) cyrillic txt-dictionary in 1 file
the total elapsed time is similar, but for (ii) attempt it took 1-2 hours more (i think it depends on internal chars, in (ii) case it was cyrillic and hanzi, in (i) case it was latin and hanzi)

As for long lines:
in case of error (too long line) importing process is stoped, number of enries in the sql-dictionary is grown by correctly imported lines. So it is possible to determine in the source txt-dictionary the incorrect (too long) line (its number is greater by 1 then number of imported enties). So if to erase all lines prior incorrect one in the source file and make shorter incorrect line it is possible to continue import process instead of to reimport all again.
How to shorter too long line (for example, dui definition)? Such long entries always contain several translations and definitions, so this line should be splited in the middle prior to next definition. The new line should be filled at the begining before definitions with <word><tab><transcription><tab>
 
Top