nyancrimew/LatinIME
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
3665 commits 1 branch 0 tags 363 MiB
Commit graph

136 commits

Author SHA1 Message Date
Jean Chalard
cba9e23bb1 Activate the implementation for the new dictionary format. 
Bug: 4392433
Change-Id: I20b05d012b1fb193002312792d0309d36c89d16a
 2011-07-14 06:55:20 +09:00
satok
3e41c071e6 Merge "Add a flag for a profiling" 2011-07-12 23:27:51 -07:00
satok
20d9fdae3a Add a flag for a profiling 
Change-Id: Iae509a24fd0f0f416376c3f8051aa2eb92d48659
 2011-07-13 15:21:10 +09:00
Jean Chalard
0adf7ae299 Merge "New dict format, step 7" 2011-07-12 22:48:45 -07:00
Jean Chalard
1059f27364 New dict format, step 7 
This actually implements the new dictionary format, but does not
activate the implementation through #defines.

Bug: 4392433
Change-Id: I9b26b9bcb4b823a36e0984799b69730acfc6f7f3
 2011-07-13 14:33:48 +09:00
Doug Kwan
ce9efbff53 Compile code used in logging conditionally so that gcc does not complain 
about unused-but-set variables.

Change-Id: I141f438694a1854d54d08cb5a74c23222dd9d85e
 2011-07-08 00:29:11 -07:00
Jean Chalard
bb15e77511 Move a function to make next commit more readable 
Change-Id: Ieaa935ff4d68ce88137dcc5c672a4149a4c9c64f
 2011-06-30 20:14:38 +09:00
Jean Chalard
e6715e32d5 Move a function out of a #endif to reduce a future commit 
Change-Id: Ic8f3160a96b6d79ba19ff9c8eda1692e94a38e98
 2011-06-30 19:47:25 +09:00
Jean Chalard
0584f02ee1 Rename parameters for future change 
Change-Id: Id15a17340fb26f91c72687f30bef24b2d8b94940
 2011-06-30 19:23:16 +09:00
Jean Chalard
432789ac93 Internal cleanup 
Moving functions around, renaming parameters

Change-Id: I3ab480f483d7d9700b9328cb07b16b51005098e5
 2011-06-30 17:50:48 +09:00
Jean Chalard
ffefdb6c1a Cleanup. 
Function renaming, moving around for future patch readability

Change-Id: Id33b961cf2e899b5a3c9189951d2199aba801666
 2011-06-30 17:22:19 +09:00
Jean Chalard
980d6b6fef Internal cleanup. 
Function renaming, useless function supressing, fix comments

Change-Id: I148acbaf367cd556a85b89016676b46cc971af81
 2011-06-30 17:02:23 +09:00
Jean Chalard
594a9a1963 Internal cleanup. 
Removed unused function prototypes.

Change-Id: Ia56ea8e285deed17ce8377df855b045b7850d58d
 2011-06-30 16:51:17 +09:00
Jean Chalard
85a1d1ea74 New dict format, step 6 
Copy the modified functions to be able to see the diff

Bug: 4392433
Change-Id: Ic9b83b4b4b7b89cc922eed1825507d7d516aff24
 2011-06-21 22:24:54 +09:00
Jean Chalard
bc90c72faf New dict format, step 5 
Move functions that will be modified and enclose those that will
be replaced into #ifdefs.
This change does not modify any code, only move some code around.

Bug: 4392433
Change-Id: Ibefbda1eb8bdc8a0c72de47ad9c67a08d0aca960
 2011-06-21 12:15:00 +09:00
Ken Wakasa
ce9e52a12a Clean up in LatinIME native code 
Change-Id: I0062200a0181a491690115ac0fab8d11358e2f14
 2011-06-18 23:52:09 +09:00
Jean Chalard
23eb0fa0b5 Merge "New dict format, step 4" 2011-06-17 05:30:26 -07:00
Jean Chalard
ca5ef2890e New dict format, step 4 
Consolidate terminal cases, streamline the word adding process
and create the entrances for adding alternate spellings with an
empty implementation.

Bug: 4392433
Change-Id: I781c93ec49945d71c7c20624c86596aa49add4c8
 2011-06-17 20:59:21 +09:00
Jean Chalard
4fd9650f0b New dict format, step 3 - followup 
Make the passing of an argument clearer
Bug: 4392433

Change-Id: Id82662ff4dc25282f70a08bee77378fee2b4b590
 2011-06-17 17:08:09 +09:00
Jean Chalard
581335c3fb Fix a bug where bigram search would never return 
Bug: 4690487
Change-Id: Ie8f3f651508cc48bbb043a0b308f7e0d1524371c
 2011-06-17 12:45:17 +09:00
Jean Chalard
17e44a72e8 New dict format, step 3 
Some refactoring and add of a parameter that will be necessary.

Bug: 4392433
Change-Id: I17f001a7efd4f69f4c35f94ee1ca8e97391b81d5
 2011-06-16 23:28:09 +09:00
Jean Chalard
8124e64dcc New dict format, step 2 
Move some methods around and make static some methods

Bug: 4392433
Change-Id: I2bbe98aec118a416d21d1e293638e1d324505b9b
 2011-06-16 22:33:41 +09:00
Jean Chalard
293ece0f34 New dict format, step 1 
This renames some variables and removes dependancies to values that
will disappear

Bug: 4392433
Change-Id: I79a44462d6bf25248cc2de0d63d7918fc6925d68
 2011-06-16 22:18:10 +09:00
Jean Chalard
e93b1f2209 Allow reading a binary dictionary even without proximity info. 
This prepares the way for spell checking, which is to be done
without context so without proximity info.

Bug: 4176026
Change-Id: I1b4bfaefe2611e1b484acdf3c33598cb80f81ff4
 2011-06-02 12:10:13 +09:00
satok
99c908a595 Tweak the demotion rate for the mistyped space correction 
Bug: 4402942

Change-Id: I7f5412b9fd2f1506f529cff0c3399d748c6ece92
 2011-05-24 14:31:06 +09:00
satok
bb68d80119 Tweak the demotion rate of mistyped space correction 
Bug: 4402942

Change-Id: I6e0421dfa99e261c72a901c9699fec864ab4b3c5
 2011-05-23 18:35:29 +09:00
satok
d8db9f86d0 Fix a bug on the calculation of the freq on the mistyped space error correction 
Bug: 4402942

Change-Id: I0b611e3d0e8c25ca528ef7408c3949200e5cad85
 2011-05-18 18:36:54 +09:00
satok
0b6b0a5a98 Enable fast power 
Change-Id: I00a91381f63cde62d9e7cf7e17f75869294cf2df
 2011-04-27 16:29:27 +09:00
satok
b2e5e5937c Handle overflow properly in multiplyRate 
Bug: 3401513
Change-Id: I8dd2523caa58bb51c378a01e160a58f9106ce9b8
 2011-04-26 22:03:26 +09:00
satok
9674f654a7 Fix a bug that 2 length words were demoted. 
Change-Id: I4a3558d0f1f1b0a9d6a36c3f75db3089b0566d7f
 2011-04-20 17:15:27 +09:00
satok
63546344b3 Merge "Promote a word with a missing space because the formula was changed by Ifa4338c5f4" 2011-04-19 07:54:02 -07:00
satok
cbc66e0711 Promote a word with a missing space because the formula was changed by Ifa4338c5f4 
Change-Id: Id4bc965aef387800facb64164d8c36a3bdd2fa07
 2011-04-19 23:48:36 +09:00
satok
4c981d3a40 Demote a word with mistyped space and missing space according to the length of each word 
Change-Id: Ifa4338c5f43b37e6bcd0700767ef2178189de3af
 2011-04-19 23:14:27 +09:00
satok
a4374d2eb7 Promote the correction of words with a missing space character 
Change-Id: I37ba618b54f7115163a3e9c6c555485e7024dc92
 2011-04-18 12:36:11 +09:00
satok
9d2a3020ba Promote a word with a proximity character 
Bug: 4293295

Change-Id: Ib0ec8aff087c71c4fbe983f3f5bc78e9c7868fd8
 2011-04-14 20:30:25 +09:00
satok
72bc17ec9f Promote a word with only one proximity character. 
Bug: 4271049

Change-Id: I755986f582f43417fda6b117207530c519233baf
 2011-04-13 19:11:13 +09:00
satok
dc5301e590 Change the formula of the missing character. 
- Bug: 4271049
- Due to the result of the recent user study, a word with a missing character needs to be promoted a bit.

so I changed the formula from:
- freq * 70 * (n - 2) / (n - 1)
to:
- freq * 90 * (10n - 12) / (10n - 2)

Change-Id: Ibff72cbdb0f2d7b91460a06a0fd39a9f5749aa46
 2011-04-13 10:44:18 +09:00
Ken Wakasa
de3070a71b Add -Werror flag to catch more warnings and errors 
Change-Id: I9c39ba24578931944aae8182918ed48a2e82eb39
 2011-03-19 10:12:15 +09:00
satok
e07f93d3ab Merge "Tweak the demotion rate for a word with missing letter" into honeycomb-mr1 2011-03-07 22:44:16 -08:00
satok
0bddb2f4d6 Tweak the demotion rate for a word with missing letter 
Bug: 4027223

Change-Id: Ie9a5552d2f41d60f433573fde52efc097f5143bf
 2011-03-07 19:44:52 -08:00
satok
1df8c82d71 Fix a bug that a word with only one missing word is not promoted 
Bug: 4027223

Change-Id: Icf7c5b917c18b565dca95b98b96c1c8e2963f540
 2011-03-07 18:01:09 -08:00
satok
3c4bb7747d A bug fix for the mistyped space algorithm 
Bug: 3311719

-- also fixed compiler warnings

Change-Id: I6941c0d02f10d67af88bc943748dde8d8783fabb
 2011-03-04 23:25:48 -08:00
Jean Chalard
eaecb56f94 Merge "Demote skipped characters matched words with respect to length." into honeycomb-mr1 2011-03-04 22:43:16 -08:00
satok
817e517e46 Add the suggestion algorithm of words with space proximity 
Bug: 3311719

Change-Id: Ide12a4a6280103c092fa0f563dd5b9e3f7f5c89b
 2011-03-04 20:37:18 -08:00
Jean Chalard
07a8406bc1 Demote skipped characters matched words with respect to length. 
Words that matched user input with skipped characters used to be demoted
in BinaryDictionary by a constant factor and not at all in those dictionaries
implemented in java code. To represent the fact that the impact of a skipped
character gets larger as the word is shorter, this change will implement a
demotion that gets larger as the typed word is shorter. The demotion rate
is (n - 2) / (n - 1) where n is the length of the typed word for n >= 2.
It implements it for both BinaryDictionary and java dictionaries.

Bug: 3340731
Change-Id: I3a18be80a9708981d56a950dc25fe08f018b5b89
 2011-03-05 13:20:19 +09:00
Jean Chalard
a787dba83b Fix a bug with umlaut processing. 
Issue: 3275926
Change-Id: Ibcb00aaea3ff05ad59ad4e8e54dd3caab5ab9bca
 2011-03-04 13:07:07 +09:00
Jean Chalard
c2bbc6a449 Use translation of fallback umlauts digraphs for German. 
For German : handle "ae", "oe" and "ue" to be alternate forms for
umlaut-bearing versions of "a", "o" and "u".

Issue: 3275926

Change-Id: I056c707cdacc464ceab63be56c016c7f8439196c
 2011-03-03 11:52:23 +09:00
satok
8fbd552292 Add proximity info to native 
Bug: 3311719

Change-Id: Ie596304070e321ad23fb67a13bf05e2b6af1b54b
 2011-02-23 23:04:00 +09:00
Jean Chalard
f5f834afcd Rename variables with obscure names. 
The `snr' variable has a very obscure name. Rename it to `matchWeight'.
Also, the `toLowerCase' function is error-prone, since it actually returns
a lower case version of the BASE char, that is without diacritics. Hence,
rename it to `toBaseLowerCase' and update variables with similar names.

Change-Id: Ibdbe73018a33ee864db59a51d664c3b104d5fb3f
 2011-02-22 16:43:19 +09:00
Jean Chalard
a5d5849701 Force autocorrection of matching words with different accents. 
When entering a word without accents the user expects the system to
add accents automatically if there is no other matching word. This
patch ensures the accented version is promoted accordingly and
autocorrection really takes place.

Issue: 3400015

Change-Id: I8cd3db5bf131ec6844b26abecc1ecbd1d6269df4
 2011-02-22 15:27:06 +09:00
Tadashi G. Takaoka
887f11ee43 Remove next letters frequency handling 
Bug: 3428942
Change-Id: Id62f467ce4e50c60a56d59bf96770e799a4659e2
 2011-02-17 13:59:41 +09:00
Jean Chalard
8dc754a411 Promote full matches with differing accents. 
Stop considering accented characters as different from their base
character for proximity scoring.
Also give a huge boost (basically overriding frequency) to a word
fully matched with only differing accents.

Bug: 2550587

Change-Id: I2da7a71229fb3868d9e4a53703ccf8caeb6fcf10
 2011-01-27 17:29:24 +09:00
satok
fd16f1d2a3 Handle the last char correctly in excessive char correction algortihm. 
bug: 3278422

Change-Id: I651d3cb0130ab9834ed9d7a97f41360c6eaa9de1
 2011-01-27 16:44:54 +09:00
satok
58c49b9132 Fix auto-correction threshold and promote full matched words 
Bug: 3374359
Bug: 3278422

"zbe" will be auto corrected to "be" by fixing s-line
"teh" will be auto corrected to "the" by promotion of full matched words

Change-Id: I314c632820e4e0b1501edeca60ada205d291451f
 2011-01-27 12:53:13 +09:00
Ken Wakasa
da50e1e98d Fixes in close() in BinaryDictionary. 
Avoid using 'synchronized' in finalizer as well.

bug: 3340837
Change-Id: I9b28f54e4490ecb844ba33a379f71b625e4246a2
 2011-01-17 15:58:07 +09:00
Ken Wakasa
90d96615bc Clean up: Update variable names to comply with spec of ApplicationInfo. 
ApplicationInfo.sourceDir may or may not be apk file name.  It can be a directory as well.
The spec just says it's "Full path to the location of this package".

Also, added error handling in loadDictionary().

Change-Id: I5e64d0aba4b1ec7634f4b3ac5537e7a774433ece
 2011-01-09 16:53:33 +09:00
Ken Wakasa
e90b333017 Load main dic in native 
Follow up to Id57dce51

bug: 3219819
Change-Id: I00e11ef21d0252ffa88c12dffb9c55b0f2e19a66
 2011-01-07 19:51:45 +09:00
satok
f7425bb15b Supress overflow at mulitplying demotion rate 
Change-Id: I2003c5f88a5062b11e2f21522095bb94b1eb4efd
 2011-01-05 16:43:17 +09:00
satok
61e2f85e3f Add profiler for native dictionary code 
Change-Id: I2569756c9ef4fa677ae52f2ccfcb90d2115d129f
 2011-01-05 15:47:29 +09:00
Ken Wakasa
458249e703 Consolidate main dictionary files. 
This change is a preparation for upcoming optimizations on dictionary file loading.
* We can consolidate dictionary files because we are no longer relying on Asset Manager.
* Stopping compressing dictionary files as planning to use mmap() on the region in the apk file.
* Probably we won't rely on Asset Manager.  Instead we'll probably use offset and size obtained from AssetFileDescriptor.

Change-Id: Id57dce512fd3d2397a58628f8264bd824194da76
 2011-01-05 11:31:58 +09:00
Jean-Baptiste Queru
2912c654b4 Manual merge from gingerbread-plus-aosp 
Change-Id: If048354f9b80bcaa1e97df5c31c3df80231eb612
 2010-12-29 14:55:20 -08:00
Bruce Beare
1dea9bdc1a Don't use stale NDK libraries for IA 
IA builds will break (due to the bionic _dso_handle bug) if stale libraries are
used. For now, just guard the defns against IA builds.

Change-Id: Ic9df6e0de78a0e221b95370ba6f01ce07714edde
Signed-off-by: Bruce Beare <bruce.j.beare@intel.com>
 2010-12-27 12:26:16 -08:00
satok
54fe9e0e20 Suggest words with excessive chars out of proximity chars 
Bug: 3273807

Change-Id: Ib8f48e562bcf4c2aac0ad5cb46809fd5f539a322
 2010-12-13 17:44:14 +09:00
satok
a3d78f606e Suggest words with transposed chars 
Bug: 3193883

Change-Id: I884b669258bfc522bc04e14f22a7646164a4cac5
 2010-12-10 18:34:23 +09:00
satok
e07baa6fab Limit the suggestions with an excessive character by filtering proximity characters 
Change-Id: Iad26dad545f1a431aa0fa53f99198b27defd03a3
ug: 3269482
 2010-12-10 00:47:37 +09:00
satok
aee09dc5fa Fix a bug that We can't suggest words with missing space if one of the words starts with a capitalized character. 
Bug: 3268825
Change-Id: I0634a243ad1e45dd096b30824b463c366a2e7f0f
 2010-12-09 21:41:26 +09:00
satok
662fe69ba2 Suggest words with missing space 
Bug:  3193883

Change-Id: I8d25f3e1d4db10be733d85edfa4f55a094feef80
 2010-12-09 14:26:27 +09:00
satok
cdbbea735f Suggest excessive characters 
bug: 3193883

Change-Id: Iea7a0fce7ce62d8779a7c7e4613d50db30d82b07
 2010-12-08 16:56:06 +09:00
satok
d299792368 Make no-recursive getWordRec 
Change-Id: Id90f3ca86ef490834cefa92f0d6958b1289fc633
 2010-12-07 16:45:32 +09:00
satok
f5cded1c6c Fix a crash when MAX_WORD_LENGTH is too short. 
Change-Id: Idcb5aa2685321b8d0ac7d846caecbd1c79e4dd77
 2010-12-06 22:58:56 +09:00
satok
48e432ceb8 Breakdown getWordRec 
Change-Id: I4fef02c227fb858334dbe2eabf2762d5b6e1d919
 2010-12-06 18:45:48 +09:00
satok
683192684c Trim the flow of getWordRec 
Change-Id: Ic0cfa64ee1e55682ca73681c585db6a5cb510900
 2010-12-06 14:56:11 +09:00
satok
cfb24365f9 Merge "Breakdown getWordRec" 2010-12-03 02:29:52 -08:00
satok
28bd03b9f5 Breakdown getWordRec 
Change-Id: I8556efb1dd053eff9a9681971cbe1014abf0333f
 2010-12-03 19:25:42 +09:00
Ken Wakasa
a46683a47b Just code cleanup. 
Change-Id: I796b47aa168b3b2cc29de65fc7dc5dab5a7f4b17
 2010-12-03 17:13:04 +09:00
satok
715514d7dd Breakdown getWordRec and add comments 
Change-Id: I88bad8a4a8177e3540b995b664c47b86d6904027
 2010-12-03 10:01:09 +09:00
satok
18c28f431e Detach bigram functionarities from unigram_dictionary 
Change-Id: Ie35164a5f293e5370885a1ba13d6ed7caf6000ec
 2010-12-02 18:24:53 +09:00
satok
e808e436cb Refactor: Move utility functions and no suggestion functions from unigram_dictionary.cpp to dictionary.cpp 
Change-Id: I6f695e4f5852547d2c00de5ee54a650fef9accbe
 2010-12-02 16:11:35 +09:00
satok
3008825948 Fix parameters of native functions and refactor Dictionary 
- created bigram/unigram dictionary classes

Change-Id: I233a28ed8d611870db3f4cf8f25fc45b5d41529b
 2010-12-02 01:16:44 +09:00
satok
d4952c8fe9 Move a logic for finding words with a missing character to the native code. 
Change-Id: I58338643830ff4f9708f78a9c26f75c8bf2ebf45
 2010-12-01 19:26:36 +09:00
satok
15dc33d9f6 Add an easy way to output native debug logs 
Change-Id: Ieff2b8e60c5e7dedb7f86e17f7c37b349a912ab4
 2010-12-01 15:56:17 +09:00
Ken Wakasa
9b195dda60 Revert https://android-git.corp.google.com/g/#change,63791 as library loading issue seems to have been fixed now. 
bug: 2963855

Change-Id: Iafdfda801083f85126c443508755e3cc711fbf52
 2010-09-03 22:44:52 +09:00
Amith Yamasani
ca4e42785f Fix UnsatisfiedLinkError for Latin IME. Don't use old NDK version in a platform build. 
Also added a clean step, since it's only a Makefile change.

Change-Id: Ide1684a5f34e99496e969edb8443dc4545b57306
 2010-08-31 12:04:30 -07:00
satok
979f869096 DO NOT MERGE. Backport LatinIME from master to Gingerbread 
TODO: Cleanup will follow.

Change-Id: I4a68ba9f2f55760aa24187f1f13fdfa8a0b70963
 2010-08-20 14:37:03 +09:00
Dan Bornstein
bf45d3619f For issue #2651381, allow library prelinking. 
Change-Id: If7015f2fa5932f69807797ffcc496550d5cb1630
 2010-05-05 09:38:52 -07:00
Amith Yamasani
07b1603a3f Don't let the native code target be included twice when unbundling. 
Move java code to a different directory so that the unbundled
version doesn't try to compile the native code again.

Change-Id: I05cf9e643824ddc448821f69805ccb0240c5b986
 2010-03-09 15:01:09 -08:00