nyancrimew/LatinIME
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
27729 commits 1 branch 0 tags 363 MiB
Commit graph

736 commits

Author SHA1 Message Date
Keisuke Kuroyanagi
fdcd67ba15 Implement split and create node methods. 
Bug: 6669677
Change-Id: Iac4afb637a8ba3d55f7a88f4a532831c1731f93e
 2013-09-10 15:27:19 +09:00
Keisuke Kuroyanagi
c5f35a9c86 Merge "Calculate parent offset from the head of moved node." 2013-09-10 06:08:56 +00:00
Keisuke Kuroyanagi
9c820de7a1 Calculate parent offset from the head of moved node. 
Bug: 6669677
Change-Id: I710453f1c37ec14e5b6830595c1afb2caff83cf2
 2013-09-10 15:03:44 +09:00
Keisuke Kuroyanagi
1d4a07f6c7 Merge "Re-call getBuffer after writing in bigram/shortcut policy." 2013-09-10 03:27:30 +00:00
Keisuke Kuroyanagi
c987120cbc Re-call getBuffer after writing in bigram/shortcut policy. 
Bug: 6669677
Change-Id: I35dc290aa9582e019681b8aae49db3f8d37c90d4
 2013-09-10 12:12:42 +09:00
Keisuke Kuroyanagi
a159ad473c Implement create children array and add child method. 
Bug: 6669677
Change-Id: Ic89c3dfe01937e771db8fc59d5259496a9464a47
 2013-09-09 16:44:17 +09:00
Keisuke Kuroyanagi
7bd7dc5d0d Implement update node probability method. 
Bug: 6669677
Change-Id: I61ac0d05e362fc7d8a967ddd8286580998c70487
 2013-09-09 15:29:11 +09:00
Keisuke Kuroyanagi
3fbc5ef196 Implement inserting new node into PtNode array. 
Bug: 6669677

Change-Id: I0171476231181e41234dde76ac9061febb2e8c35
 2013-09-06 20:40:08 +09:00
Keisuke Kuroyanagi
40182c737d Merge "Compute probability using structure policy." 2013-09-06 08:10:07 +00:00
Keisuke Kuroyanagi
65d19946be Compute probability using structure policy. 
Bug: 6669677
Change-Id: Icd50fc30274cce65ebeb2e7cc8368b59e5cda05a
 2013-09-06 16:50:09 +09:00
Keisuke Kuroyanagi
e4dcebe0c3 Add node flags creating methods. 
Bug: 6669677
Change-Id: Ieb4178da61fa38f6b32baefa099cc0724f9fb6be
 2013-09-06 14:56:59 +09:00
Keisuke Kuroyanagi
96a0631664 DynamicPatriciaTrieWritingUtils to write PtNodes. 
Bug: 6669677
Change-Id: I67cce3c1d338cf747ed2204c96768907d990d269
 2013-09-05 20:12:56 +09:00
Keisuke Kuroyanagi
2a41a35635 Add code point writing methods for updatable dictionaries. 
Bug: 6669677
Change-Id: If7ab6fefdb9a7f0d21c397edbb7d9e7fddc0e0db
 2013-09-05 14:24:07 +09:00
Keisuke Kuroyanagi
5901b5e1bd Check usesForgettingCurve in HeaderPolicy. 
Bug: 6669677
Change-Id: I47ebfc50f477b2a6514fba6fad421dd90f29ecb1
 2013-09-05 12:28:38 +09:00
Keisuke Kuroyanagi
91486bdb3d Merge "Employ a header attribute map in headerPolicy." 2013-09-04 10:06:05 +00:00
Keisuke Kuroyanagi
5754817a5e Employ a header attribute map in headerPolicy. 
Bug: 6669677
Change-Id: I4a084cdd7fcd7a8ed3a70fb7e365031eedd981e5
 2013-09-04 19:00:53 +09:00
Keisuke Kuroyanagi
5d8fd5457d Change name of bigram_list_reading_utils.* . 
Bug: 6669677
Change-Id: Ifd2baa11fb4ee083be1d23e8235f6f903c1b5747
 2013-09-03 20:44:58 +09:00
Keisuke Kuroyanagi
202e416b51 Add bigram writing methods. 
Bug: 6669677

Change-Id: Ib62f6c31c4bcbb41ac200dccccb13e366b3c2173
 2013-09-03 20:44:51 +09:00
Keisuke Kuroyanagi
9e3082e016 Add DynamicShortcutListPolicy::copyAllShortcuts(). 
Bug: 6669677
Change-Id: I74959fe3ca59d77ba0279c75f4d72db13735322f
 2013-09-03 18:36:53 +09:00
Keisuke Kuroyanagi
f1cd7a081c Implement reading side of updating methods of dpt. 
Reading methods are used to determine the position to
be updated in dynamic patricia trie.

Bug: 6669677

Change-Id: Ie9acdcdbc73aa3fd4294c97aa39451a96b165ed9
 2013-09-03 14:47:30 +09:00
Keisuke Kuroyanagi
75d1cb667d Add buffer writing methods. 
Bug: 6669677

Change-Id: Ic3345e36a7842b77598c21de9b54f7c2863df38f
 2013-09-02 19:27:01 +09:00
Keisuke Kuroyanagi
2660f83a12 Change name of extendable_buffer.h to buffer_with_... . 
Bug: 6669677

Change-Id: Id2bd79d8f86c690d5c16d1ab726c5f6fa28ba5e3
 2013-09-02 17:18:41 +09:00
Keisuke Kuroyanagi
3e76487c6c Consolidating dict buffers into BufferWithExtendableBuffer. 
Bug: 6669677

Change-Id: Ied58c123b68199bddb91293ff5cf5d96691e1f6b
 2013-09-02 17:18:41 +09:00
Keisuke Kuroyanagi
4d814bfcb7 Introduce DynamicPatriciaTrieReadingHelper. 
It supports iterating nodes and dealing with additional
buffer. It counts nodes and node arrays to avoid infinite
loop.

Bug: 6669677

Change-Id: I322e7263c0535e098635a1e5de098838de09467d
 2013-08-30 19:41:58 +09:00
Keisuke Kuroyanagi
69ebca0643 Fix children position reading for dynamic patricia trie. 
Bug: 6669677
Change-Id: I68ad68361676f952bd25903c253c22617d3ce361
 2013-08-29 18:31:15 +09:00
Keisuke Kuroyanagi
6c4d09e9e1 Use extendable buffer for reading node info. 
Bug: 6669677
Change-Id: I78ba80100e3a38f2b49e43db1e6aef4e56ed062c
 2013-08-27 18:06:42 +09:00
Keisuke Kuroyanagi
2f0c1253e2 Use extendable buffer for reading bigram and shortcut. 
Bug: 6669677

Change-Id: I4017c0ecc5d8baff1b16ff370d3eec85de9a0f9f
 2013-08-27 17:48:04 +09:00
Keisuke Kuroyanagi
27b12933cd Rename CharGroup to PtNode in native code. 
Bug: 10233675
Change-Id: I5a4215cc79b37bfd317e60101e06b2ab9c332653
 2013-08-26 14:51:43 +09:00
Keisuke Kuroyanagi
ad4e0108a9 Fix: reading uninitialized area. 
Bug: 10402083
Change-Id: I083beea29fe563b1e7739653d756b77820753e3f
 2013-08-23 19:55:55 +09:00
Satoshi Kataoka
02833d11c3 Fix the autocorrection normalized score calculation 
Bug: 10441240
Change-Id: I256021dc55481960d6a605046daa17b1a2d55b95
 2013-08-23 16:22:02 +09:00
Keisuke Kuroyanagi
941811cbd6 Introduce ExtendableBuffer for dynamic update. 
Bug: 6669677

Change-Id: I6343c348334c0dace8b12a71eb992d6f040e8c95
 2013-08-22 18:05:15 +09:00
Keisuke Kuroyanagi
a72a33388f Fix native crash by caused wrong munmap. 
Bug: 10402083
Change-Id: I5aa2cecd20cd3202c2326b1cbb6758257afd6d5d
 2013-08-21 18:57:58 +09:00
Yuichiro Hanada
112257e40f Rename BinaryDictDecoder to Ver3DictDecoder. 
Change-Id: Ibf9b95b658df6e2c2218bdb62e2380f326a03832
 2013-08-20 17:11:51 +09:00
Keisuke Kuroyanagi
4a65258bc7 Fix possible SIGSEGV. 
Change-Id: I6008c0457a8da9b90434a3b96a51da1487948f1b
 2013-08-19 18:58:10 +09:00
Keisuke Kuroyanagi
381c12df20 Move methods in BinaryFormat to PatriciaTriePolicy. 
Bug: 6669677

Change-Id: Ic9bc03a9d8ec789281d83d4b9e58042a083c3ba1
 2013-08-16 16:42:17 +09:00
Keisuke Kuroyanagi
0624cc6cf3 Remove BinaryDictionaryInfo and introduce MmapedBuffer. 
Bug: 6669677
Change-Id: I70e47fa20109060b6ea7a04bcc42ab8f7c543f77
 2013-08-16 12:52:04 +09:00
Ken Wakasa
0238e1eb01 Merge "Revert "Remove BinaryDictionaryInfo and introduce MmapedBuffer."" 2013-08-15 11:39:08 +00:00
Ken Wakasa
d4fcf1dbc5 Revert "Remove BinaryDictionaryInfo and introduce MmapedBuffer." 
This reverts commit d14ffcfcbc.

This broke a bunch of unit tests -- https://sponge.corp.google.com/target?id=a8a62bd0-3aab-4520-ae64-53f9652ef64b&target=com.google.android.inputmethod.latin.tests&searchFor=&show=FAILED&sortBy=STATUS

Change-Id: I1eae460e09808513eacefd523609330819d73d16
 2013-08-15 11:37:48 +00:00
Yuichiro Hanada
94460eba11 [Refactor] Divide BinaryDictInputOutput into BinaryDictEncoder and BinaryDictDecoder. 
Change-Id: I7c3269d77e3e3b567e459dcaa1bc029903941744
 2013-08-15 20:23:07 +09:00
Keisuke Kuroyanagi
0663bc047b Merge "Remove BinaryDictionaryInfo and introduce MmapedBuffer." 2013-08-15 09:34:28 +00:00
Keisuke Kuroyanagi
d14ffcfcbc Remove BinaryDictionaryInfo and introduce MmapedBuffer. 
Bug: 6669677

Change-Id: I1459bb53a9761d920427e4b5d925a2844955c88c
 2013-08-15 18:17:44 +09:00
Jean Chalard
e7870a2c0d Add an initial JNI interface to dicttool. 
Bug: 10100269
Change-Id: I883992c2033e7d9e7c754c0bf653767728b221b6
 2013-08-15 17:58:55 +09:00
Ken Wakasa
dc83fe37ba Merge "Revert "[Refactor] Divide BinaryDictInputOutput into BinaryDictInputUtils and BinaryDictOutputUtils."" 2013-08-15 08:56:28 +00:00
Ken Wakasa
117f18e844 Revert "[Refactor] Divide BinaryDictInputOutput into BinaryDictInputUtils and BinaryDictOutputUtils." 
This reverts commit 4c63d0614e.

Change-Id: I1fa277d720bab4d895259df7d6d82eebfa5eb6c5
 2013-08-15 08:54:29 +00:00
Ken Wakasa
f74ddfbb06 Merge "[Refactor] Divide BinaryDictInputOutput into BinaryDictInputUtils and BinaryDictOutputUtils." 2013-08-15 06:53:23 +00:00
Yuichiro Hanada
4c63d0614e [Refactor] Divide BinaryDictInputOutput into BinaryDictInputUtils and BinaryDictOutputUtils. 
Change-Id: I0d476abe763c11ba9005152f928e8dccf15ac9de
 2013-08-15 15:46:58 +09:00
Keisuke Kuroyanagi
08ce337d3f Fix: BinaryDictionaryInfo.isDynamicallyUpdatable(). 
Change-Id: Ifb3acded5e19f1302c7fcc8881b74addd20c1571
 2013-08-15 13:39:32 +09:00
Keisuke Kuroyanagi
6ec3f63d59 Merge "Make dictionary structure policy have updating methods." 2013-08-15 03:15:43 +00:00
Yuichiro Hanada
46ebaa49e9 Fix a comment. 
Change-Id: I5d09fd2474c7a0e6f42b6c7c6e717692636fbc6d
 2013-08-15 11:57:58 +09:00
Keisuke Kuroyanagi
7fd9667d76 Remove NodeFilter. 
Change-Id: I27d905dd3581cbb0e67d071dba04066aaa2ba22e
 2013-08-14 18:20:15 +09:00
Keisuke Kuroyanagi
dfecc86f25 Fix ShortcutListPolicy::getStartPos(). 
Bug: 6669677
Change-Id: I1fa9967fc6aa0956986954c8ea1f990a47c15764
 2013-08-14 17:23:57 +09:00
Keisuke Kuroyanagi
90619ae02b Move utils for reading dict to policyimpl. 
Bug: 6669677
Change-Id: Ia1da3cb84b457665cf26172d3810e1d28329bca4
 2013-08-14 15:50:42 +09:00
Keisuke Kuroyanagi
66facd37dd Make dictionary structure policy have updating methods. 
Bug: 6669677

Change-Id: I6aa8e377c6347e0db4b7a82293b316ec64d87bec
 2013-08-14 11:34:42 +09:00
Keisuke Kuroyanagi
23d3ed962f Move files for headerPolicy to policyimpl. 
Bug: 6669677
Change-Id: Ic90feb9c3a9a7b698e588c8ad25c765a246706a8
 2013-08-13 20:35:16 +09:00
Keisuke Kuroyanagi
76e579c7ca Move header reading methods to policyimpl. 
Bug: 6669677
Change-Id: I24cb32f28cf355485b431cc56d877000e3f946d4
 2013-08-13 20:22:40 +09:00
Keisuke Kuroyanagi
d81654cd61 Use DictionaryStructureWithBufferPolicy for traversing. 
Bug: 6669677
Change-Id: If7bc038278b7217ed5ba4cb77b1238b2dad0001b
 2013-08-13 19:38:20 +09:00
Keisuke Kuroyanagi
e607e0722a Merge "Remove TerminalAttributes." 2013-08-13 05:58:35 +00:00
Keisuke Kuroyanagi
b40d14226b Merge "Quit using BinaryDictionaryInfo in header reading methods." 2013-08-13 05:49:49 +00:00
Keisuke Kuroyanagi
99007de979 Quit using BinaryDictionaryInfo in header reading methods. 
Bug: 6669677
Change-Id: I5198ae28fcc271dd6ab3c33df940a2d15513a8c8
 2013-08-13 12:26:01 +09:00
Keisuke Kuroyanagi
6abdafc671 Remove TerminalAttributes. 
Use BinaryDictionaryShortcutIterator instead of
TerminalAttributes.

Bug: 6669677
Change-Id: Ib7176e3b302ba383344cc6fcc037e23568c702a8
 2013-08-12 17:46:13 +09:00
Keisuke Kuroyanagi
f4688f8df0 Cleanup ShortcutListReadingUtils. 
Bug: 6669677
Change-Id: Ifd61022665c89f492933dde9811ec644f7e1f5c4
 2013-08-12 17:46:13 +09:00
Keisuke Kuroyanagi
f1e383ed45 Change name of shortcut reading util class. 
Bug: 6669677
Change-Id: If93916f4c3d568119ff6b83e22e95c7086423d2f
 2013-08-12 17:37:59 +09:00
Keisuke Kuroyanagi
7c48352af8 Move methods for Shortcut reading to policyimpl. 
Bug: 6669677
Change-Id: If0a565de015564f803429f4fbf21b05687f80553
 2013-08-12 17:18:28 +09:00
Keisuke Kuroyanagi
fd10db04e0 Move shortcut iteration methods to poilcy. 
Bug: 6669677
Change-Id: I6e535ce7962de8e153a52fd1085fe961649ffe16
 2013-08-12 15:53:09 +09:00
Keisuke Kuroyanagi
277053af7c Move bigram reading methods to BigramListReadingUtils. 
Bug: 6669677
Change-Id: Icb024a8af577323b4dcf74f4c4054cfb8b57a943
 2013-08-12 11:54:31 +09:00
Keisuke Kuroyanagi
668870be43 Move bigrams iteration methods to policy. 
To support various bigram format.

Bug: 6669677
Change-Id: Ifc8c1a855b03cd5a39d97a6e10872ef8ef76475b
 2013-08-09 20:20:34 +09:00
Keisuke Kuroyanagi
e1ebef6124 Make DictionaryStructurePolicy have buffer info. 
To support various format dictionary, extendable
buffers or multiple files.

Bug: 6669677
Change-Id: I203a5a4adc049a6322cfba4eacb4cb6715f8dfc2
 2013-08-09 05:27:22 +00:00
Keisuke Kuroyanagi
b2408c7f71 Fix parent position reading for ver3 dict. 
Bug: 6669677
Change-Id: Ia55d4ecb43c7d37b9bc466b848d5142bc13cd823
 2013-08-08 16:31:17 +09:00
Keisuke Kuroyanagi
60eb250194 Support ver3 dict in native code. 
Bug: 6669677
Change-Id: Iafc3287f1f87588a6296593d6b43ea4464c8f553
 2013-08-08 14:53:05 +09:00
Keisuke Kuroynagi
744084defb Implement getTerminalNodePositionOfWord for ver3 dict. 
Bug: 6669677

Change-Id: Ib9f88d23e6502b8281921ae2f1e47f930b16972f
 2013-08-07 17:59:52 +09:00
Keisuke Kuroynagi
9601df5aac Implement getCodePointsAndProbabilityAnd... for ver3 dicts. 
Bug: 6669677
Change-Id: I5c3b53d015496e11cb6c1f19938b8ee8a4c0f983
 2013-08-07 11:53:34 +09:00
Keisuke Kuroynagi
2b1dd6e253 Implement a part of ver 3 dictionary reading methods. 
Bug: 6669677
Change-Id: I83c159838ad99b2093907b9de7df6cb7a97b4165
 2013-08-06 10:18:20 +09:00
Keisuke Kuroynagi
22f951c881 Merge "Remove checkFirstCharacter from BigramDictionary." 2013-08-05 04:56:23 +00:00
Keisuke Kuroynagi
cf1f7675ea DynamicPatriciaTrieReadingUtils for version 3 dictionary. 
Bug: 6669677
Change-Id: I7105258a00159a5ddda7ff520c490c4ac9fa7902
 2013-08-01 16:15:49 +09:00
Keisuke Kuroynagi
647c000707 Give PatriciaTrieReadingUtils methods for reading nodes. 
The remaining methods in BinaryFormat will be removed.

Bug: 6669677
Change-Id: I866f828d69a600c0ac03b68d71b1f6ee2ce4dd36
 2013-08-01 16:15:35 +09:00
Keisuke Kuroynagi
f0a7f70cf6 Fix: Include wrong file in dynamic_patricia_trie_policy.cpp. 
Change-Id: If994f17e41c7d07e979fa6b0aa4661da0523eb0a
 2013-08-01 14:48:20 +09:00
Keisuke Kuroynagi
8045b7cd24 Merge "Introduce DynamicPatriciaTriePolicy for ver 3 dictionary." 2013-08-01 05:39:07 +00:00
Keisuke Kuroynagi
2401f7c62d Move binary_format.h to policyimpl. 
Bug: 6669677
Change-Id: I3712f60528c7fcf364746f792cec6e180ab2dbef
 2013-07-31 21:07:32 +09:00
Keisuke Kuroynagi
11a431fca6 Fix: appendTwoWords cannot handle MAX_WORD_LENGTH-chars word. 
Bug: 9892258
Change-Id: Ib4ef7850c8fc8b1d99d594bb7ef68ce645f0f1f3
 2013-07-31 16:58:09 +09:00
Keisuke Kuroynagi
2a2aac2568 Remove checkFirstCharacter from BigramDictionary. 
Bug: 10028452
Change-Id: I27b147e83b312d73e975a0b2bc8074b33906e56e
 2013-07-31 13:47:23 +09:00
Keisuke Kuroynagi
26de7079b6 Introduce DynamicPatriciaTriePolicy for ver 3 dictionary. 
Bug: 6669677
Change-Id: Idd3189558aa6fbbd6b2fd007c5721bf39dde9f48
 2013-07-30 17:41:13 +09:00
Keisuke Kuroynagi
1fb11da36a Move creating children methods to structurePolicy. 
Bug: 6669677
Change-Id: I20760f8ab95b18d21383960f6466db0d79429200
 2013-07-29 19:31:27 +09:00
Keisuke Kuroynagi
9d618d1431 Move files only used in dicNode to "internal". 
Change-Id: Ib31ddeff99b480d9b1f33a5b5b207a8acce22858
 2013-07-29 17:03:21 +09:00
Keisuke Kuroynagi
4447b14b78 "much" in suggestions of "m" after "very" 
Make cache size larger for single point input.

Bug: 9823678
Change-Id: Ic6bbcebd18eccaf9f9ac07d9c0d456a62f6c5287
 2013-07-26 18:35:42 +09:00
Satoshi Kataoka
75322cecb9 Support terminal insertion error correction 
Bug: 9421356

Change-Id: I19685763ca487b5636019d62e150708c63ce6fc2
 2013-07-25 22:07:38 +09:00
Keisuke Kuroynagi
cb4a7c1d87 Fix: "saomething" -> "something". 
Fixed by making MAX_CACHE_DIC_NODE_SIZE larger.

Bug: 9682742

Change-Id: I0312952c90b0820bed37df99c223921c68450570
 2013-07-24 17:55:25 +09:00
Keisuke Kuroynagi
62f3b4e9c9 Merge "Fix: huge bigram costs for blacklisted words." 2013-07-23 05:32:09 +00:00
Keisuke Kuroynagi
5b7688bbb5 Fix: native crash while iterating bigramslist. 
Bug: 9964153
Change-Id: Ia2eb922b61300a7c0e34122cd6c18358f1ee7ba9
 2013-07-23 13:13:12 +09:00
Keisuke Kuroynagi
b179199830 Fix: huge bigram costs for blacklisted words. 
Bug: 8844931
Change-Id: I523005c5ed9a3d401a67b0e4e1c3ff2e4574e6df
 2013-07-22 22:05:16 +09:00
Keisuke Kuroynagi
c38ec47596 Get BigramsPos and ShortcutPos via structure policy. 
Bug: 6669677
Change-Id: I924d5512377cfcfe72d3470006b81a36db3865b6
 2013-07-17 20:13:04 +09:00
Keisuke Kuroynagi
8a7129530b Replace NOT_VALID_WORD with NOT_A_VALID_WORD_POS 
Bug: 6669677
Change-Id: If5fb58e8a98b04b767eaaf624cb6a40841b64f54
 2013-07-16 15:53:03 +09:00
Keisuke Kuroynagi
b6f286bfa5 Merge "Make bigram dictionary and traverse session use structure policy." 2013-07-16 02:45:36 +00:00
Keisuke Kuroynagi
34047d8905 Quit ignoring language cost of exact matches. 
Handle exact matches in DicNode::compare() and calculateFinalScore().

Bug: 8844931
Change-Id: I17c78d4e352b0a4308727229b02a8004b38468bf
 2013-07-11 08:25:17 +00:00
Keisuke Kuroynagi
1311cdcb62 Make bigram dictionary and traverse session use structure policy. 
Bug: 6669677
Change-Id: I5a13290366afbb788da997b98d1c593963e92f3e
 2013-07-08 14:25:50 +09:00
Keisuke Kuroynagi
5b03213db1 Remove flags from dicNodeProperties. 
Bug: 6669677
Change-Id: I6f4aef8c4e4376f078ed8375394823788aa35002
 2013-07-05 16:35:07 +09:00
Keisuke Kuroynagi
0933353c0c Merge "Read version 3 dictionary header." 2013-07-04 14:19:35 +00:00
Keisuke Kuroynagi
cac66f29eb Merge "Return NOT_A_PROBABILITY in getProbability for inalid dictionary." 2013-07-04 09:59:40 +00:00
Keisuke Kuroynagi
a829188f54 Allow force auto commit when the top is long multiword. 
Bug: 9688196
Change-Id: I73ff2cd77b932bf7307b3541ea9da3fa4afa11b4
 2013-07-04 17:39:01 +09:00
Keisuke Kuroynagi
72f69fd621 Return NOT_A_PROBABILITY in getProbability for inalid dictionary. 
Bug: 9682742
Change-Id: Ifad4b9811bb5e1a94aed59eba7aedcdb6da5cf9c
 2013-07-03 20:12:38 +09:00
Keisuke Kuroynagi
9a4f7a3761 Disable shortcut output for multi-word suggestions. 
Bug: 9682742
Change-Id: I9e0a5acd9053d141e018c4653e4baf0f77aacb45
 2013-07-03 17:22:36 +09:00
Keisuke Kuroynagi
5ae8722bd5 Read version 3 dictionary header. 
The header structure of version 3 dictionary is same as version 2.

Bug: 6669677
Change-Id: I26c8723333938fbf102de1a40f11d97444da9075
 2013-07-03 17:10:48 +09:00
Keisuke Kuroynagi
07718a5e7d Merge "Fix: Reading invalid dictionary position." 2013-07-02 09:42:53 +00:00
Keisuke Kuroynagi
f6aa9e9a1d Fix: Reading invalid dictionary position. 
Bug: 6669677

Change-Id: I3f4078aaa8cf72f3b366e7851457e9c753c0102f
 2013-07-02 18:39:56 +09:00
Keisuke Kuroynagi
2164856205 Merge "Move children filtering methods to DicNodeChildrenFilter." 2013-07-02 09:35:06 +00:00
Jean Chalard
b6f5d3e39d Fix offdevice compilation 
Change-Id: I8bccccbc51b308fddfbbc990f7bfe1c490259372
 2013-07-02 15:19:16 +09:00
Jean Chalard
7eba0198c0 Dump binary dictionary information when opening 
Bug: 9459517
Change-Id: I122583e734936ae0284e1c7500c6c9242bc7973b
 2013-07-01 18:41:00 +09:00
Keisuke Kuroynagi
28ade0938e Merge "Remove format version 1 related logic." 2013-07-01 07:31:45 +00:00
Keisuke Kuroynagi
92db5e10b1 Remove format version 1 related logic. 
Removed from DictionaryStructurePolicyFactory

Bug: 9617181
Change-Id: Ice4b4a24e019c88b0ec129dc69176c7541f8530c
 2013-07-01 16:28:06 +09:00
Keisuke Kuroynagi
109ba3ace3 Merge "Regenerate old version dictionaries using new format." 2013-07-01 07:16:15 +00:00
Keisuke Kuroynagi
a62b5b22ef Regenerate old version dictionaries using new format. 
Bug: 9617181
Change-Id: I228a3eac959f81b38e4092015e8e3c429d46fa60
 2013-07-01 15:25:33 +09:00
Keisuke Kuroynagi
7a06a79287 Move children filtering methods to DicNodeChildrenFilter. 
Bug: 6669677
Change-Id: If18055e2da8dd5d783942ba384d3ea1d97030e3a
 2013-07-01 15:07:15 +09:00
Keisuke Kuroynagi
c5e6efafff Introduce patriciaTrie to abstract traversing version 2 dictionary. 
Bug: 6669677
Change-Id: Ifef72f3d7a7ba67c5232b98c7835485d72d7322d
 2013-06-28 19:47:00 +09:00
Keisuke Kuroynagi
d9e08beebf Merge "Add DictionaryStructure as a policy." 2013-06-27 03:58:03 +00:00
Keisuke Kuroynagi
68e7edfd99 Use NOT_A_DICT_POS instead of 0 to express not valid ditctionary position. 
Bug: 6669677
Change-Id: I4bf2284f8221a0a2ae4534b4a06b0e59c420a5f9
 2013-06-27 12:53:31 +09:00
Keisuke Kuroynagi
e394abab60 Add DictionaryStructure as a policy. 
Bug: 6669677
Change-Id: I4cc53e623824e57c1e9a351eb0befe9ee988706f
 2013-06-27 12:33:25 +09:00
Keisuke Kuroynagi
c96b56a5ec Merge "Reduce the number of arguments required to initialize dic_node." 2013-06-27 03:30:48 +00:00
Ken Wakasa
c445369732 Fix class names 
Change-Id: Ib3e15bdf46585976aa5b42132805754fc2e2908b
 2013-06-26 23:06:14 +09:00
Keisuke Kuroynagi
6379a4de29 Reduce the number of arguments required to initialize dic_node. 
Bug: 6669677

Change-Id: I52249b83f72560d8f5ab028da5cfb5c50f6e40b8
 2013-06-26 20:55:14 +09:00
Keisuke Kuroynagi
c0d7a376e2 Merge "Move shortcut reading methods." 2013-06-26 05:44:56 +00:00
Keisuke Kuroynagi
5ac44bdc2a Move shortcut reading methods. 
Moved form BinaryFormat to BinaryDictionaryTerminalAttributesReadingUtils.

Bug: 6669677

Change-Id: Ia1ab25854effbf61df37837fe26755ac7dc4d020
 2013-06-26 14:41:22 +09:00
Keisuke Kuroynagi
5bf1be7162 Add jni methods for dynamically handling a dictionary. 
Bug: 6669677
Change-Id: I8a26623adbb41a78e3c023c652be635c635e3b47
 2013-06-26 12:51:33 +09:00
Keisuke Kuroynagi
3b33656ce3 Add canBeDynamicallyUpdated() in BinaryDictionaryInfo. 
Bug: 6669677
Change-Id: I7391641253b99bfcb1f9380ebee2037015e7dc14
 2013-06-25 16:03:33 +09:00
Keisuke Kuroynagi
f3b62900c7 Allow mmapping dictionary file in writable mode in openNative. 
Bug: 6669677
Change-Id: I447668932de118b158fa029ddc977b3f3cc06cb8
 2013-06-25 16:03:33 +09:00
Keisuke Kuroynagi
46159d5422 Move file and mmap information form Dictionary to BinaryDictionaryInfo. 
Bug: 6669677
Change-Id: I05d435e0884430f8abbbed6ea2df775005ee8622
 2013-06-25 16:03:30 +09:00
Keisuke Kuroynagi
80a4b7c92e Merge "Extend center point vertically for bottom keys on a keyboard." 2013-06-24 03:26:55 +00:00
Keisuke Kuroynagi
77ec8713ce Extend center point vertically for bottom keys on a keyboard. 
Bug: 9505397
Change-Id: Ic415ac4065d23cb832dd3bb85876e2a4dc9e9fc3
 2013-06-24 11:58:56 +09:00
Keisuke Kuroynagi
940cca4fa9 Merge "Check adjacent proximity chars for insertion for typing" 2013-06-24 00:52:02 +00:00
Keisuke Kuroynagi
af9d92f590 Rename BinaryDictionaryBigramsReadingUtils to BinaryDictionaryTerminalAttributesReadingUtils and remanme the class methods 
Bug: 6669677
Change-Id: I960dcf51a950ead2c3d7b9dedcde0a450569cbe9
 2013-06-24 09:42:37 +09:00
Keisuke Kuroynagi
be2a172d4e Merge "Give the best score to the most probable exact match." 2013-06-24 00:28:02 +00:00
Keisuke Kuroynagi
24706f8f5c Check adjacent proximity chars for insertion for typing 
Change-Id: I6ce1b065bba055516fca0c8af6f14853a111aa68
 2013-06-24 09:25:46 +09:00
Keisuke Kuroynagi
f0dc0b1b74 Give the best score to the most probable exact match. 
Bug: 9490364
Change-Id: If7fe5da190516d4b5f7a41b752ef5960205a5899
 2013-06-21 19:11:39 +09:00
Keisuke Kuroynagi
21dfdcdba8 Merge "Add ByteArrayUtils::readSignedInt24andAdvancePosition to read dynamic dictionaries." 2013-06-21 08:15:05 +00:00
Keisuke Kuroynagi
8a914a66e8 Add ByteArrayUtils::readSignedInt24andAdvancePosition to read dynamic dictionaries. 
Bug: 6669677
Change-Id: I75eeedbb50868432f6a7ff5fa63624d03e788356
 2013-06-21 12:58:18 +09:00
Keisuke Kuroynagi
09858cbde4 Change method name of getDepth and getTotalLength. 
Change-Id: I4b3aa54e17d7de7c31f956dc9d59941d79856fd7
 2013-06-20 20:55:23 +09:00
Keisuke Kuroynagi
ec7457eb7f Initialize inputStatusG at the constructor. 
Bug: 9490519

Change-Id: I1806f765f7a0b5335df253b2618b40cfd17aab14
 2013-06-19 22:20:25 +09:00
Keisuke Kuroynagi
2d3f2daf12 Check compound distance of node and prune useless node after creating new word. 
Before:
136309.89 ms
After:
73133.95 ms

Bug: 8591918

Change-Id: If029b768e2b459c0e566188bcf425f71b2df866d
 2013-06-18 17:07:23 +09:00
Keisuke Kuroynagi
6bfeeb8694 Support multi words input for gesture. 
Bug: 8591918

Change-Id: I344f936907a385cc6c3c8a2215a6603938483401
 2013-06-17 19:04:29 +09:00
Keisuke Kuroynagi
0052dbda76 Support keys that have uncommon width. 
Bug: 8591918

Change-Id: I1e01e1560200333f9e35993af0aa7e5a17e6944f
 2013-06-14 20:41:39 +09:00
Keisuke Kuroynagi
1ff81e8890 Use bloom filter in multi bigram map. 
Evaluated with previous word "this".
without bloom filter (use only hash_map):
Total 147792.34 (sum of others 147771.57)
with bloom filter:
Total 145900.64 (sum of others 145874.30)
always read binary dictionary:
Total 148603.14 (sum of others 148579.90)

Bug: 8592527
Change-Id: I821dc39454543826adb73b9eeeef6408fad8ae28
 2013-06-14 20:35:41 +09:00
Keisuke Kuroynagi
a71ed8caa2 Introduce BinaryDictionaryBigramsIterator to access bigrams attributes in binary dictionaries. 
Bug: 6669677

Change-Id: Ifb1adebc5305a930c80396f6b4ec31d84400a9dc
 2013-06-14 18:12:56 +09:00
Ken Wakasa
f473f4b1eb Remove unused macros in defines.h 
bug: 8550444
Change-Id: I77ae6bd607032def05044e3045aa7cc3746b0a5b
 2013-06-11 20:09:07 +09:00
Keisuke Kuroynagi
0b7cd6a0b2 Fix: Android.mk for autocorrection_threshold_utils. 
Change-Id: I3ea0811ca7c2c1a88351ee2ba5cc13d7096865ba
 2013-06-11 18:05:04 +09:00
Keisuke Kuroynagi
527c128309 Remove unused methods. 
Bug: 8550444

Change-Id: Ie627084143846407ca36df82c2a7f904a0e6f49c
 2013-06-11 17:04:40 +09:00
Keisuke Kuroynagi
bd0d1afdb2 Introduce BinaryDictionaryHeader to access binary dictionary header. 
BinaryDictionaryHeader abstracts header structure and access header information via only its instance.

Bug: 6669677

Change-Id: Ib5ab0e8fef12265ddabc1d0165548b69953bba6a
 2013-06-11 12:01:49 +09:00
Ken Wakasa
ad0c6d7b36 Cleanups in JNI related code 
Removed the malloc version in binary dictionary support -- this has not
really been tested well so far, and the mmap version has been working pretty
well after all.

Several cosmetic fixes etc.

Change-Id: Iad0da58b300b769fb5946a3e73fc96f56215980e
 2013-06-04 19:16:47 +09:00
Ken Wakasa
11dc3a371d Clean up macro variables 
Change-Id: I0d9983c1a766b97182953328fa45f488e110bc01
 2013-06-04 15:40:30 +09:00
Ken Wakasa
156d39303e Remove a bunch macros that are no longer used 
bug: 8550444
Change-Id: Ib775c0bed27c499dc131c54dfaff34c5a8169276
 2013-06-03 20:00:02 +09:00
Satoshi Kataoka
db0aa747be Merge "Purge WordsPriorityQueue" 2013-06-03 10:21:29 +00:00
Ken Wakasa
4caf594198 Follow up to I907150be994c4f9ffc030c063de521eeb177d3e1 
Change-Id: I8779d397b0ebe2abbf7c3fbc1a79a1a0939c04f0
 2013-06-03 19:14:02 +09:00
Satoshi Kataoka
04e37fb925 Purge WordsPriorityQueue 
Change-Id: If14034c09d0dd335c84dd3a6ea2775655eefd38f
 2013-06-03 19:04:37 +09:00
Satoshi Kataoka
cbb1ee485c Fix build 
Change-Id: I907150be994c4f9ffc030c063de521eeb177d3e1
 2013-06-03 18:59:45 +09:00
Satoshi Kataoka
dceebee4b8 Purge UnigramDictionary 
Bug: 8550444
Change-Id: I899f3bdce977c521515e548ca0c4ee6d29d6450b
 2013-06-03 18:54:03 +09:00
Satoshi Kataoka
6c4ad527dc Migrate used functions in UnigramDicitonary 
Bug: 8550444
Change-Id: Ic1ed2d85537cd47cf18a8373203f331abb7508f6
 2013-06-03 18:28:50 +09:00
Satoshi Kataoka
d858b5815f Move obsolete files to obsolete dir 2 
Bug: 8550444
Change-Id: Ia83796d696e04f51adb39867dfd21d1ac993b63c
 2013-06-03 17:42:27 +09:00
Satoshi Kataoka
e0bbb98d60 Fix build 
Change-Id: I2a876df097d59324aeb58eb83698ec622ac6191b
 2013-06-03 17:23:25 +09:00
Satoshi Kataoka
01cae939a9 Move obsolete files to obsolete dir 
Bug: 8550444
Change-Id: Ie65eea25da4c41e1128d1d8d8643fb6afe8f41e4
 2013-06-03 17:07:59 +09:00
Satoshi Kataoka
7c92b421ee Purge DicTraverseWrapper 
bug: 8550444

Change-Id: Iad017e66ac579c6727b9f60ad9cda64e478200e5
 2013-06-03 16:45:51 +09:00
Satoshi Kataoka
d18112046a Move bigram_dictionary 
Change-Id: I2c243ac75bc02b912f6b0fabedd372148adf8076
 2013-06-03 14:13:39 +09:00
Ken Wakasa
addea83bad Create "src/utils" directory 
Change-Id: I0b7c51d181edc4786e343f7a953367b040ee6b66
 2013-06-03 12:29:34 +09:00
Ken Wakasa
f497e1e5a0 Move suggest_options.h to suggest/core 
Change-Id: Ie95872a4c90ee3089183c1ebafb67243bd2688a4
 2013-06-03 11:45:06 +09:00
Keisuke Kuroyanagi
712fefd85f Merge "Use BinaryDictonaryInfo instead of raw pointers." 2013-06-03 00:51:44 +00:00
Keisuke Kuroyanagi
0ecfb94247 Use BinaryDictonaryInfo instead of raw pointers. 
Bug: 6669677

Change-Id: I9792a872f1609de7c1ba0fc08d916047d6724c0b
 2013-06-02 16:34:42 -07:00
Keisuke Kuroyanagi
88ad30f40b Move dictionary format independent probability calculation methods to ProbabilityUtils. 
Bug: 6669677
Change-Id: Idc09a2fbb04e4d843e11313011178022177616aa
 2013-06-01 14:08:20 -07:00
Ken Wakasa
464d3ba432 Move char_utils to the dictionary directory 
Change-Id: Id397485407fe63600c9bb5c80d223042942b4d8a
 2013-05-31 20:39:34 +09:00
Keisuke Kuroyanagi
f88f9dbbde Move "#define LOG_TAG" before include defines.h. 
Change-Id: Iacd2c2ea3878ad2cde5df36b15e1359d10244f60
 2013-05-30 22:26:21 -07:00
Ken Wakasa
a65c267b1f Move some dictionary related implementations to the dictionary directory 
Change-Id: I6fe71a540ad91ca46755be234abc8894242e2413
 2013-05-31 00:02:57 +09:00
Ken Wakasa
94da44e43e Move additional_proximity_chars to the layout directory 
Change-Id: Ie892ea9f3199a66e9d62a92a0e1ab2dff62826c3
 2013-05-30 20:12:47 +09:00
Ken Wakasa
29432f843a Create a new directory for layout-related implementations 
Change-Id: Ia227bb296f20dab0dfc03dfcde3a06c624eca76c
 2013-05-30 19:45:53 +09:00
Keisuke Kuroyanagi
338ef65077 Make DicTraverseSession having suggest options. 
Change-Id: I2b802c06634e5226c763fd838f3fc0918fc4a161
 2013-05-29 17:18:03 -07:00
Satoshi Kataoka
599d8778f4 Specify OOV for the OOV word correction 
Bug: 9127749
Change-Id: I9ff0590428888afe57a009a1cbf8f008011b1544
 2013-05-27 17:51:35 -07:00
Keisuke Kuroyanagi
50db5f0fff Passing options to native suggestion method. 
Change-Id: I6da180cfa1b9e5aa39600e78e3bc41209a40f583
Signed-off-by: Keisuke Kuroyanagi <ksk@google.com>
 2013-05-24 10:21:04 -07:00
Jean Chalard
03f8c6aed3 Be careful about the dictionary size in detection methods 
Bug: 8857618
Change-Id: I29345ec96d53da601571ba73197a6485643a10a7
 2013-05-08 18:55:18 +09:00
Keisuke Kuroynagi
a09f9709e0 Checking key edit distance of candidate words for gesture scoring. 
bug: 8616704
Change-Id: I98c3248877b1e3e722a6c00a2851e543e5fcbe11
 2013-05-07 19:47:20 +09:00
Satoshi Kataoka
5a346afab9 Fix exact match profanity 
Bug: 8661070

This is a revival version of I421bed14cc71af771510142365

No diff in the test

Change-Id: I96bc9cec531682cce78bcf83223dcd2395b2be77
 2013-05-07 15:52:38 +09:00
Ken Wakasa
c5a29fa864 Revert "Fix exact match profanity" 
This reverts commit df2eae6e69

Change-Id: I42635cc09ad3ea4c177d02e0b7607fb7aa78506c
 2013-05-02 14:21:29 +00:00
Satoshi Kataoka
df2eae6e69 Fix exact match profanity 
Bug: 8661070
Change-Id: I421bed14cc71af7715101423652d706e33958445
 2013-05-01 13:34:08 +09:00
Tom Ouyang
8da0c93292 Reduce MAX_NORM_DISTANCE_FOR_EDIT in typing model after recent logic improvements 
After recent logic improvements we can now safely reduce MAX_NORM_DISTANCE_FOR_EDIT.
This leads to better regression test performance and faster runtime speed.

Bug: 8633962
Change-Id: Ie0db73680b32687ff4d8ea5dfae47dee5c7c66a3
 2013-04-29 21:21:26 -07:00
Keisuke Kuroynagi
a107dcaeb6 Fix: Demote first char capitalized new words. 
Bug: 8680578
Change-Id: I84bcdf3a20428c7561f600332bfef4685c72b67f
Signed-off-by: Keisuke Kuroynagi <ksk@google.com>
 2013-04-30 11:28:18 +09:00
Tom Ouyang
9559dd2e30 Improve bigram frequency lookup 
Bug: 8592527

Change-Id: I1908bcb552279b9acb140fe4d8d26b10ed9eda72
 2013-04-26 12:22:23 -07:00
Ken Wakasa
dad23dda9d A small follow-up to I8d03bae3264974eff7b790e27c073b0a8758d17a 
Change-Id: Id3727f075e74c0102edcb51eabcfdbef745b94b7
 2013-04-27 00:24:34 +09:00
Ken Wakasa
866a6ced57 Just cosmetic cleanups 
Change-Id: I3dc8a4bb3f89f4a18cfffd5f13a71503738452e6
 2013-04-26 19:58:14 +09:00
Tom Ouyang
fd02ec10f0 Always consider corrections of intentional omissions (e.g., apostrophe) 
Example: "thts" -> "that's"

Bug: 8721815
Change-Id: I8c1efbf651d2569ba3600516cc7447d46bf30fa4
 2013-04-25 20:30:15 -07:00
Jean Chalard
99e998286d [ZF1] Check profanity in Java rather than in native 
Bug: 7226877
Change-Id: Ib6c32bfee9977c27dbf7e1e590b2b00d9ceb7301
 2013-04-25 20:29:11 +09:00
Keisuke Kuroynagi
60a169f0c3 Fix exact match checking for words with digraph. 
Bug: 8624690
Change-Id: I8946d6618609c8e0cb638aa96992b51b435232c7
 2013-04-25 16:47:52 +09:00
Keisuke Kuroynagi
911f326857 Fix proximity/edit correction detection that affects the terminal cost. 
Bug: 8624690
Change-Id: If59305674a781e0fcc284d236e3d8ba9cdf9ed08
 2013-04-24 20:21:18 +09:00
Tom Ouyang
f99f1a757b Merge "Check bigrams for lowercased previous word if original is not found" 2013-04-24 05:12:02 +00:00
Tom Ouyang
14aa8f1d32 Fix autocompletion lookahead handling 
Bug: 8689370
Change-Id: I056b1788764bd5c4d8ad9da86e790562d2781f06
 2013-04-22 19:36:14 -07:00
Tom Ouyang
5fa33a701d Check bigrams for lowercased previous word if original is not found 
Bug: 8592527
Change-Id: I70c4b81263d8b2cb7050364c124589f66900cfce
 2013-04-22 18:16:36 -07:00
Jean Chalard
c9688ef267 Fix a small bug 
Tests results in Ibcd6c110f1d5582425f9592c42e31152131ef80c

Bug: 7226877
Change-Id: I8d03bae3264974eff7b790e27c073b0a8758d17a
 2013-04-22 18:30:46 +09:00
Tom Ouyang
90cb956c4f Remove duplicate new-word bigram cost (part 1) 
Removes a duplicate application of the new word bigram cost and updates only
the related parameters (those associated with multi-word suggestions).

Note: test results will improve after full optimization.

[Category diff]
+1     357
-1     485
+2      16
-2      20
+3      20
-3      16
+4     198
-4     226
+5     510
-5     443
+6     518
-6     368
+7     394
-7     455

[Weighted category diff]
+1     482
-1     532
+2      22
-2      22
+3      22
-3      22
+4     233
-4     381
+5     578
-5     500
+6     617
-6     498
+7     522
-7     521

Bug: 8633962
Change-Id: I3c3ecc9460e8e03e44925e11b2d4b037a6c3b99e
 2013-04-17 19:25:48 -07:00
Tom Ouyang
3970352ea9 Make digraph lookups case insensitive 
Example: Be able to gesture dictionary words that start with an upper-case
German umlaut (e.g., Übermensch).

Bug: 8493920
Change-Id: I3253bcd236d34c0af60116e5b41e359f2715bf97
 2013-04-16 20:08:42 +09:00
Ken Wakasa
2dfaf7a4ac Update SORTED_CHAR_MAP 
bug: 7873416
bug: 8493920

Change-Id: Ifddb04cad90da71df5d34dd7076850dbcf51fba4
 2013-04-16 19:17:41 +09:00
Satoshi Kataoka
252412d7eb Use additional multi-word cost per language (for Russian) 
Bug: 7540133
Change-Id: I7eb7b8399746c15452ed2ed5069955e88fb546d3
 2013-04-16 16:42:36 +09:00
Satoshi Kataoka
837f46dcb3 Enable touch coordinate correction for new algorithm 
Bug: 8505668

Change-Id: I07eb785c74c446777524104a3d2b61f0f591a498
 2013-04-15 19:10:27 +09:00
Satoshi Kataoka
a94bb198cf fix assertion 
Change-Id: I34ff470122aa0f34c66425d2eeae7590187d3074
 2013-04-15 12:03:21 +09:00
Ken Wakasa
2daea19709 am 47bb8d7d: am 4fffa27d: Fix a native crash in AOSP LatinIME 
* commit '47bb8d7d9cd2dcc8e80f90283e4d9b7d4f68e70b':
  Fix a native crash in AOSP LatinIME
 2013-04-11 22:21:10 -07:00
Ken Wakasa
4fffa27d3f Fix a native crash in AOSP LatinIME 
bug: 8597610

Change-Id: I585c629636a52a1ac45b8cc686bc94427bb601df
 2013-04-12 12:53:28 +09:00
Satoshi Kataoka
8b3009ef48 add a debug flag to evaluate the most probable string 
Change-Id: I34dc08c14feb9cbc0304c1d0ff06922e3b65819d
 2013-04-11 11:08:12 +09:00
Tom Ouyang
fd9f3d97ae Add ability to handle digraphs in gesture and tap typing 
Example: Gesture the German word "ueber" and the system will now replace
the transliteration "ue" with the correct u-umlaut.

Bug: 8493920
Change-Id: I03e1d6311b4187b6c7f802d8fef43a50eee12721
 2013-04-10 22:52:49 +00:00
Tom Ouyang
25e8eda9af Add mechanism to handle digraphs in DicNode 
This is needed to support for German umlaut handling for both gesture and typing input.

Bug: 8493920

Change-Id: I292e2a4911c54aa5566c38411016a220bb35a1f4
 2013-04-10 14:35:49 -07:00