nyancrimew/LatinIME
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
31834 commits 1 branch 0 tags 363 MiB
Commit graph

955 commits

Author SHA1 Message Date
Keisuke Kuroyanagi
927aff72bc Add jni interface to get header information from native. 
Bug: 12810574
Change-Id: I2508483bcb3157f04cc1fe13f24f864d301f1f28
 2014-02-04 21:42:09 +09:00
Keisuke Kuroyanagi
1d3058daa1 Merge "Don't boost exact matches for personalized dicts." 2014-02-04 09:13:48 +00:00
Keisuke Kuroyanagi
54622d38c2 Don't boost exact matches for personalized dicts. 
Bug: 12800726
Change-Id: Ia3a2f82db6cdc9645cad0e329c0b79328d7452cc
 2014-02-04 17:47:14 +09:00
Keisuke Kuroyanagi
941734695b Implement Ver4PatriciaTriePolicy::getNextWordAndNextToken. 
Bug: 12810574
Change-Id: Idea44f03c477964f58e65fbf2b55e3fcd77a2934
 2014-02-04 12:42:14 +09:00
Keisuke Kuroyanagi
38f341a2a5 Add a jni method to iterate words in a dictionary. 
Bug: 12810574
Change-Id: I87b13559765f0262dfd06c2ba40d43af6437ea7f
 2014-02-03 14:51:58 +09:00
Keisuke Kuroyanagi
9221772ab7 Get bigram information via getWordProperty(). 
Bug: 12810574
Change-Id: I2750a5659ccbc3e31307c28e35dd9a1dbdffbeee
 2014-02-03 11:47:26 +09:00
Keisuke Kuroyanagi
94d9a2309f Extend jni interface to get bigrams in WordProperty. 
Bug: 12810574
Change-Id: Ia4b88d02ea8790a5c47d32376cc0b84c3e071ddd
 2014-01-31 20:32:44 +09:00
Keisuke Kuroyanagi
17e7768a3e Fetch bigram information in getWordProperty(). 
Bug: 12810574
Change-Id: Ic67e964b01c2753f2eec82b079dc0e5650f340bb
 2014-01-31 20:24:23 +09:00
Keisuke Kuroyanagi
20b7786ece Add BigramProperty and ShortcutProperty in WordProperty. 
Bug: 12810574
Change-Id: Ia9e497c49bb2cf10897bae3a79317113325db819
 2014-01-31 15:26:43 +09:00
Jean Chalard
b0df28f4cc Merge "[HD02] Make the date header attribute mandatory." 2014-01-31 03:58:55 +00:00
Keisuke Kuroyanagi
748f32feba Merge "Rename UnigramProperty to WordProperty." 2014-01-31 03:06:44 +00:00
Keisuke Kuroyanagi
e9085da8cb Merge "Simplify UnigramProperty." 2014-01-31 03:06:33 +00:00
Keisuke Kuroyanagi
080a35e959 Rename UnigramProperty to WordProperty. 
Bug: 12810574
Change-Id: If5ddd803948aaf6e491ddcbaa5436fb3af3f7257
 2014-01-31 11:06:42 +09:00
Jean Chalard
a137c21c9c [HD02] Make the date header attribute mandatory. 
Bug: 11281748
Change-Id: I48a6f3d95ca89ced0c51335527201ecabfb7998e
 2014-01-30 18:19:21 +09:00
Keisuke Kuroyanagi
5f7f6a1615 Simplify UnigramProperty. 
Bug: 12810574
Change-Id: I606f559b5dd340b7525280ecead2c2c7ec920a78
 2014-01-30 17:10:23 +09:00
Jean Chalard
324adefca3 [HD01] Small initial refactoring 
Bug: 11281748
Change-Id: I3c64c75ecd17729bdacf03048311d76ca9f37ae2
 2014-01-30 17:08:13 +09:00
Keisuke Kuroyanagi
21e2380234 Improve the double letter word promotion logic. 
The old logic only checks the top 3 suggestions before the
final adjustment. It leads to instability in scores.

Bug: 10700674
Change-Id: I986aed2aefd66c1fba6196a8f100fcb5bc838a38
 2014-01-28 16:33:20 +09:00
Keisuke Kuroyanagi
51c5ec10f9 Add length check for dict update operations. 
Bug: 12602903
Change-Id: I6835dee8bf8b356f0f1cf6c0531bee5b3415a13f
 2014-01-24 16:32:42 -08:00
Keisuke Kuroyanagi
3e0777e726 Use memmove for all place src and dest can be same. 
Change-Id: I03e5c8f95cb22c7096ff545f1181f914b80a6c58
 2014-01-20 11:27:01 -08:00
Keisuke Kuroyanagi
d78a447d10 Add a dedicated method to control time in native code. 
Bug: 12443085
Change-Id: I41a5cf6c895cb59e54af98b40efded36afcd3247
 2014-01-15 14:55:57 -08:00
Tadashi G. Takaoka
20a8a4a3d9 Fix native method signature 
Change-Id: Ia2d566f162f746c6b2b2b874f7278d86abd9e839
 2014-01-15 18:56:40 +09:00
Keisuke Kuroyanagi
9638693479 Fix native crash in BigramDictionary::addWordBigram() 
Bug: 12479653
Change-Id: I7d263f744f711202000524a47769dc2d3d0e0de6
 2014-01-14 18:51:49 -08:00
Keisuke Kuroyanagi
84c52a3c81 Boundary check for ProximityInfoState::getProximityTypeG(). 
Bug: 12481052
Change-Id: Ic0b64d2fa17eb8f65f3ad1a7b468cdf6dddf7e70
 2014-01-10 20:15:36 +09:00
Keisuke Kuroyanagi
05113c1847 Check whether the word is OOV or not for amanatto update. 
Bug: 11736529
Change-Id: I27973212da7e922b2185dbb4febfb5bb4dfbdcbb
 2014-01-08 21:02:35 +09:00
Keisuke Kuroyanagi
b68dd6cd0b Merge "Differentiate exact matches' minor errors." 2013-12-19 04:14:11 +00:00
Keisuke Kuroyanagi
afa6c07d30 Differentiate exact matches' minor errors. 
Bug: 8844931
Change-Id: I06754643188ccd28ba8c873878f7761787d91ff6
 2013-12-19 12:37:56 +09:00
Ken Wakasa
abe3a35abf Add C++ compiler warning options 
Change-Id: I0d79d450c5cca887d02a1087128c1d3eb0bf0b7f
 2013-12-19 00:12:01 +09:00
Keisuke Kuroyanagi
d8f35f7b4c Move methods for outputting from Suggest. 
Bug: 8844931
Change-Id: I83dde6c37b75ed1e1ae4e0972e169d30ada8faf5
 2013-12-18 16:47:23 +09:00
Keisuke Kuroyanagi
9ee9095528 Move methods for scoring from Traversal to Scoring. 
Bug: 8844931

Change-Id: I6137d06e597f4e7b81aaf29555199e18984f2c39
 2013-12-18 07:13:44 +00:00
Keisuke Kuroyanagi
d5f5dd74c2 Quit suggest words that do not include proximity chars. 
Bug: 10416722

Change-Id: I74d27c6a22b4ca47d02eb238b2fbcc49f5dee99c
 2013-12-18 14:31:10 +09:00
Keisuke Kuroyanagi
b5215d456a Increment version number of ver4 format to 401. 
Bug: 11073222
Change-Id: I3878aef02040b1ef73165645fc70ede586289c51
 2013-12-16 23:02:54 +09:00
Keisuke Kuroyanagi
9b08a9e611 Fix: incompatibility of sparse table. 
Bug: 11073222
Change-Id: Id17178efd8f9f3828d821cc718d35606e8bd4b1a
 2013-12-16 22:46:29 +09:00
Keisuke Kuroyanagi
c1163c8518 Align block sizes for ver4 format to java side. 
Bug: 11073222
Change-Id: I09646cbc81529576d5e34dcfa7bc29083a64076f
 2013-12-16 20:42:32 +09:00
Jean Chalard
a245d15da5 Have dicttool use the native library to generate v4 dicts. 
Yay !

Change-Id: Iea8ced9e81031b9ab7eff05ad9ef7215be248de9
 2013-12-13 18:18:20 +09:00
Ken Wakasa
2fa3693c26 Reset to 9bd6dac470 
The bulk merge from -bayo to klp-dev should not have been merged to master.

Change-Id: I527a03a76f5247e4939a672f27c314dc11cbb854
 2013-12-13 17:13:32 +09:00
Keisuke Kuroyanagi
6bc5acaa79 Fix: Suggested words from user history are invalid. 
- Suggestions form user history can contain invalid words.
- isValidWord always returns false.

Bug: 11139426
Change-Id: I6075b275603332ddb00f4a9284afcaa82d824270
 2013-10-22 05:23:48 -07:00
Keisuke Kuroyanagi
6d91e4ce74 Fix offdevice test build. 
Change-Id: I42143bdc4f6f5aa4f55530de84819ab722a22f72
 2013-10-08 16:05:42 +09:00
Keisuke Kuroyanagi
6d3d53955e Merge "Fix a bug: wrong parent position is written when moving." 2013-10-08 04:48:53 +00:00
Keisuke Kuroyanagi
802e6383ea Fix a bug: wrong parent position is written when moving. 
Change-Id: I93d33ba9105eb85327108cca5706d8db3399de6a
 2013-10-08 12:44:00 +09:00
Jean Chalard
86f7711407 Merge "Don't auto-correct to non-whitelist shortcuts" 2013-10-07 10:04:18 +00:00
Jean Chalard
f309f2288b Don't auto-correct to non-whitelist shortcuts 
Bug: 11076722
Change-Id: I0de10ab24d33ec31f45664fe92d80d5a7bec9c20
 2013-10-07 19:01:57 +09:00
Keisuke Kuroyanagi
cfb018ba6d Fix bug and Add large test for decaying dictionary. 
- GC gets failure when the dictionary become empty.
- Useless unigrams are sometimes not removed.

Bug: 10197478
Change-Id: I8d1479c01efba61a81f03bc077da6bcb4797a940
 2013-10-07 17:31:01 +09:00
Keisuke Kuroyanagi
15b1a0fd14 Parameter optimization. 
[Category diff]
+1     194
-1      21
+2      15
-2       1
+3       1
-3      15
+4     152
-4      90
+5     108
-5     197
+6      94
-6     136
+7      28
-7     132

[Weighted category diff]
+1     230
-1      23
+2      15
-2       1
+3       1
-3      15
+4     180
-4     106
+5     122
-5     224
+6     107
-6     158
+7      29
-7     157

show diff for ./en_user_log_phones_2011_08.csv
+1     171
-1       9
+2       7
-2       1
+3       1
-3       7
+4     123
-4      76
+5      84
-5     157
+6      66
-6     119
+7      20
-7     103

Before
---- count ----
Count(1):75.64%, 177913
Count(2):1.06%, 2510
Count(3):0.56%, 1319
Count(4):6.63%, 15612
Count(5):9.87%, 23223
Count(6):2.79%, 6580
Count(7):3.41%, 8038
---- weight ----
Weight(1):96.88%, 2009907
Weight(2):0.24%, 5157
Weight(3):0.09%, 2025
Weight(4):0.77%, 16095
Weight(5):1.11%, 23077
Weight(6):0.39%, 8172
Weight(7):0.48%, 10146
---- score ----
Conservative score = 66.406
Normal score = 81.512
Agressive score = 89.064
Insane score = 92.841

After
---- count ----
Count(1):75.71%, 178087
Count(2):1.07%, 2524
Count(3):0.55%, 1305
Count(4):6.66%, 15677
Count(5):9.83%, 23133
Count(6):2.78%, 6539
Count(7):3.37%, 7930
---- weight ----
Weight(1):96.89%, 2010118
Weight(2):0.24%, 5171
Weight(3):0.09%, 2011
Weight(4):0.78%, 16184
Weight(5):1.1%, 22972
Weight(6):0.39%, 8109
Weight(7):0.48%, 10014
---- score ----
Conservative score = 66.431
Normal score = 81.531
Agressive score = 89.08
Insane score = 92.855

No diffs for 10+ freq user log.

Change-Id: Ica4a7604507a2c91aaa7daf5bacf4fce4039f547
 2013-10-07 12:39:51 +09:00
Keisuke Kuroyanagi
3e5f4b53b0 Merge "Check the elapsed time for decaying." 2013-10-04 07:16:17 +00:00
Keisuke Kuroyanagi
67c855ea6f Check the elapsed time for decaying. 
Bug: 6669677
Change-Id: I0401a4c0908c702ce65abfa5e017dd16cb1296f9
 2013-10-04 16:11:11 +09:00
Xiaojun Bi
0b62621446 Merge "Optimize the parameters for the typing algorithm" 2013-10-04 03:25:43 +00:00
Xiaojun Bi
2ed09c214a Optimize the parameters for the typing algorithm 
The new set of parameters are optimized for both
keystroke savings and regression test scores.

This change is corresponding to the patch 6 of
Change-Id: Ib8439a85de26cbbba58837344717f0acbfcf0ab0
in the regression test repository.

Here is the link to the doc analyzing the
“nnow → know” case:
https://docs.google.com/a/google.com/document/d/1XIHfAHLlP1f3F8R-u7yxPGWcP58tokygzN_lQKXFFgM/edit

Here is the link to the spreadsheet showing
the diff of the words with frequencies >= 10
in en_user_log_phones_2011_08.csv.
https://docs.google.com/a/google.com/spreadsheet/ccc?key=0Am_RQlV4zC6SdFFVTlhJT0RLUENCWWZOZ3JQTUpMQ2c&usp=drive_web#gid=0

Updated TRT Results for this change:
[Category diff]
+1     783
-1     717
+2      20
-2      34
+3      34
-3      20
+4     575
-4     777
+5     765
-5     755
+6     785
-6     666
+7     479
-7     472

[Weighted category diff]
+1     999
-1     965
+2      21
-2      35
+3      35
-3      21
+4     804
-4     980
+5     970
-5     971
+6     954
-6     831
+7     645
-7     625

Bug: 10613502

Change-Id: I5cd447673a196ad5af641a2118602bec879bdefb
 2013-10-03 19:30:26 -07:00
Keisuke Kuroyanagi
864ee881bd Merge "Fix UserHistoryDictionaryTests." 2013-10-03 12:02:34 +00:00
Keisuke Kuroyanagi
11f7cae094 Fix UserHistoryDictionaryTests. 
Bug: 6669677
Bug: 10667710

Change-Id: I6cdc6a6c9cacc7f276fda3a26ec31e3eb928471c
 2013-10-03 20:55:34 +09:00
Keisuke Kuroyanagi
9cb1e78a44 Fix: Native crash when an empty attribute is written. 
Bug: 10964805
Change-Id: Ia3cd891d20aee2226092f0f646d8f280901786fc
 2013-10-03 17:30:54 +09:00
Keisuke Kuroyanagi
3d92954120 Fix build. 
Change-Id: Ib33b27214e67378646e44f5b45bda298f960524f
 2013-10-02 23:02:42 +09:00
Keisuke Kuroyanagi
7c4dcf1e91 Use probability table for decaying dictionaries. 
Bug: 6669677
Change-Id: Ib5994a2e343dfcdaf23755e523f52846709b1c6c
 2013-10-02 18:01:47 +09:00
Keisuke Kuroyanagi
c76bbceedc Stochastic decay. 
Bug: 6669677
Change-Id: Ib2d9228b951c77dab7a8675ce9db60677e87e771
 2013-10-02 17:51:12 +09:00
Keisuke Kuroyanagi
13d5dc914a Change name of DecayingUtils. 
Bug: 6669677
Change-Id: I97839b2184b95ae1659c55a9b31d22e2d601cba1
 2013-10-02 16:23:44 +09:00
Jean Chalard
74577bedb3 Merge "Implement the heuristic for auto-commit." 2013-10-01 12:10:57 +00:00
Jean Chalard
ff9476f10a Merge "Fix typos" 2013-10-01 12:10:37 +00:00
Jean Chalard
3e954347e3 Fix typos 
I think I may pass out from the sheer ugliness

Change-Id: I1e709f03ef47ec713552d41fd9e37e01b810ae88
 2013-10-01 21:08:14 +09:00
Jean Chalard
459cd6f8ef Implement the heuristic for auto-commit. 
Bug: 9059617
Change-Id: I066abf018df5aaeabf415425dd822ebe233e6008
 2013-10-01 21:00:40 +09:00
Keisuke Kuroyanagi
6995310996 Separate "GC" and "GC with decay". 
Bug: 6669677
Change-Id: I9d6aba76cef2616f0549e612db9701e1d6a19467
 2013-10-01 18:09:31 +09:00
Jean Chalard
bb57090f1d Create the wiring for auto-commit confidence computation. 
Bug: 9059617
Change-Id: I7e4e05ffa8c68cfebbe362a977fd176555cad12a
 2013-10-01 15:04:15 +09:00
Jean Chalard
bf1beefd08 Merge "Surface the distance after first word for autocommit." 2013-09-30 14:06:02 +00:00
Jean Chalard
da06e385f5 Surface the distance after first word for autocommit. 
Bug: 9059617
Change-Id: Ie9b4cc8148ae8e0ff437b3337ab6c1bde95500f5
 2013-09-30 23:02:04 +09:00
Keisuke Kuroyanagi
fd02b2d6ee Implement simple dictionary decay. 
Groundwork and implement simple decay.
Increment probability when typed and decrement probability
at GC.

Bug: 6669677

Change-Id: Ib12caead0cbeef4ce7808fe8ac0b00ee331523fe
 2013-09-30 20:53:35 +09:00
Keisuke Kuroyanagi
c18510049a Prepare dictionary decay. 
Bug: 6669677
Change-Id: I8fbae190dd44a6bdbee7e9b6d3a16208322727f7
 2013-09-30 13:57:54 +09:00
Keisuke Kuroyanagi
1cf4789ba6 Merge "Set header attributes for ExpandableBinaryDictionary." 2013-09-29 05:14:22 +00:00
Keisuke Kuroyanagi
5ed30a7660 Set header attributes for ExpandableBinaryDictionary. 
Bug: 6669677

Change-Id: I2462777eaa07600f1c774226adfc4f351882f523
 2013-09-28 12:50:09 +09:00
Keisuke Kuroyanagi
68bc4ba5c2 Merge "Add boundary checking for PtNode Array reading." 2013-09-27 14:25:24 +00:00
Keisuke Kuroyanagi
8b580819a5 Merge "Add unigram/bigram counting." 2013-09-27 14:14:49 +00:00
Keisuke Kuroyanagi
31097a57cc Add unigram/bigram counting. 
Bug: 6669677
Change-Id: I05ea2201d822dddf062b08c8467daa336760e16c
 2013-09-27 23:12:12 +09:00
Keisuke Kuroyanagi
67cb282b64 Add boundary checking for PtNode Array reading. 
Bug: 6669677
Change-Id: Iea5e19d98d2fc26f137046dd5f8e339239672351
 2013-09-27 22:28:54 +09:00
Keisuke Kuroyanagi
85463c7f40 Merge "Use base lower character to get point to key length." 2013-09-26 08:28:41 +00:00
Keisuke Kuroyanagi
e639d7346c Use base lower character to get point to key length. 
Bug: 9490364
Change-Id: I3755f00ab22f926f087fc917b95a2db7d722fa00
 2013-09-26 16:36:05 +09:00
Keisuke Kuroyanagi
5ef6209656 Create empty dictionary file in native code. 
Bug: 6669677
Change-Id: I46d23deee1fd091678f4593561cb8687eb815212
 2013-09-26 12:59:02 +09:00
Keisuke Kuroyanagi
10b56df486 Merge "Prepare dictionary file creating in native code." 2013-09-26 02:38:19 +00:00
Keisuke Kuroyanagi
c6015327fb Merge "Add empty PtNode checking for PatriciaTriePolicy." 2013-09-26 02:32:03 +00:00
Keisuke Kuroyanagi
9155eec0d9 Add empty PtNode checking for PatriciaTriePolicy. 
Bug: 9503570
Change-Id: I784d29c307c1040699fa0aadfc6a47d97f03ffe4
 2013-09-26 11:19:04 +09:00
Keisuke Kuroyanagi
36f4592128 Merge "Refactoring header attribute reading." 2013-09-26 01:57:22 +00:00
Keisuke Kuroyanagi
1592eb8a3a Prepare dictionary file creating in native code. 
Bug: 6669677
Change-Id: I7c476a6e99ec7ac883f05d84ef306265255154da
 2013-09-26 10:48:43 +09:00
Keisuke Kuroyanagi
989596844e Refactoring header attribute reading. 
Bug: 6669677
Change-Id: Ifc11da614d5c331ac61019a324e3a0ff187329cd
 2013-09-26 10:45:03 +09:00
Keisuke Kuroyanagi
009dcac33f Boundary check for children creating of PatriciaTriePolicy. 
Bug: 10537529
Change-Id: I0cd8b6de230795498395ca08789ccc1c74780f2c
 2013-09-25 19:54:31 +09:00
Keisuke Kuroyanagi
3d2802ef2e Allow large memory space for GC. 
Bug: 6669677
Change-Id: I021c92a1ca5cd28f745cbfe729c99e346c58074d
 2013-09-24 19:21:17 +09:00
Keisuke Kuroyanagi
f3cd38f2c7 GC step 5. Add large tests and fix bugs. 
Bug: 6669677
Change-Id: I41e44a64730f9c66645995095a7ecff51ffd5351
 2013-09-24 16:32:25 +09:00
Keisuke Kuroyanagi
2cfe7f9e3b GC step 4. Update all positions in new dict and add a test. 
Bug: 6669677
Change-Id: I035d848a210653dcd3c1a1f21781968adced9225
 2013-09-24 15:29:56 +09:00
Keisuke Kuroyanagi
0355bf2d4c GC step 3. Place and write valid PtNodes. 
Bug: 6669677

Change-Id: Id2b7359bafb34acf12dc3bf4c595a03073fda805
 2013-09-20 21:11:49 +09:00
Keisuke Kuroyanagi
77ef75cbe6 GC step 2. Finding garbage bigram entries. 
Bug: 6669677
Change-Id: Ie3f406776ab56bf589ac10e15e2534ff8cb1f275
 2013-09-20 20:04:57 +09:00
Keisuke Kuroyanagi
2a64726a16 Step 1 to implement GC. Finding garbage PtNodes. 
Bug: 6669677
Change-Id: I3551fe2f16a09d2bf7761f4e1d73ebd4a03380e7
 2013-09-20 15:37:14 +09:00
Keisuke Kuroyanagi
b7e8a9abf5 Groundwork for implementing GC. 
Bug: 6669677

Change-Id: Ia2b2f338795ab62cc9eef73731285b2a5a0ac5e4
 2013-09-19 14:48:59 +09:00
Keisuke Kuroyanagi
e28927dfb6 Change name of header_reading_utlis.* 
Bug: 6669677
Change-Id: I91389050148237580ef640d3a58b935ea752954e
 2013-09-18 18:27:01 +09:00
Keisuke Kuroyanagi
484fa7b59c Add dictionary header writing methods. 
Bug: 6669677
Change-Id: I392ac4776b55779903cbaa17e683005d80017a41
 2013-09-18 18:08:33 +09:00
Keisuke Kuroyanagi
72e53a1e38 Merge "Implement dictionary flush." 2013-09-18 05:20:05 +00:00
Keisuke Kuroyanagi
1560543754 Implement dictionary flush. 
Bug: 6669677
Change-Id: I1d70ac6319fb6d8700074de382568c2108887960
 2013-09-18 13:19:14 +09:00
Keisuke Kuroyanagi
9465819cf6 Merge "Add BinaryDictionary.getBigramProbabilityNative()." 2013-09-18 04:10:38 +00:00
Keisuke Kuroyanagi
4d02a2d44d Add BinaryDictionary.getBigramProbabilityNative(). 
Bug: 10197478
Change-Id: Id16e3ee22a17679c46cc7a0199a9e28065177aa3
 2013-09-18 11:18:28 +09:00
Keisuke Kuroyanagi
4e17b42d0f Output input index of second word's first char. 
This is for partial commit.

Bug: 9059617

Change-Id: I77a0245b63c29b7234fd9b06d362e05aebf30051
 2013-09-17 20:31:34 +09:00
Keisuke Kuroyanagi
ac7e005ef0 Merge "Prepare dictionary saving in native code." 2013-09-17 09:18:32 +00:00
Keisuke Kuroyanagi
d0246277fd Prepare dictionary saving in native code. 
Bug: 6669677

Change-Id: I0787f66409e2b183c6328fa05805945520e1e93a
 2013-09-17 18:07:16 +09:00
Keisuke Kuroyanagi
d9f450ef00 Implement removeBigramWords() for DynamicPatriciaTrie. 
Bug: 6669677
Change-Id: I99cb517511b4c968b8bf937baab5d7f42b0f534e
 2013-09-17 18:05:32 +09:00
Keisuke Kuroyanagi
78b55a31cb Fix handling multi-bytes characters and add a test. 
Bug: 6669677

Change-Id: Id2154db47adea2929559a4187a726f9dfa83363e
 2013-09-17 15:11:24 +09:00
Keisuke Kuroyanagi
6d1a29eaab Fix build. 
Change-Id: I1c3c2996ca11bfe8864df774e9280893cf17c0d4
 2013-09-17 13:03:12 +09:00
Keisuke Kuroyanagi
cd6a0430c7 Implement addBigramWords() for DynamicPatriciaTrie. 
Bug: 6669677
Change-Id: Ia6aa76c212165690191c65fee9dacdc44be5acd5
 2013-09-17 12:52:21 +09:00
Keisuke Kuroyanagi
cb816e5e16 Merge NOT_A_VALID_WORD_POS into NOT_A_DICT_POS. 
Change-Id: I2ac3aa2a19d14e8c4159a59334e7816bd1409688
 2013-09-16 18:37:23 +09:00
Keisuke Kuroyanagi
9ea9c61c99 Move methods using ByteArrayUtils from .h to .cpp. 
Bug: 10402083
Change-Id: Ic5f1aa11eb403f5bdc83de8e38a6457200519cae
 2013-09-16 18:37:08 +09:00
Keisuke Kuroyanagi
f874728496 Add boundary check for PtNode reading position. 
Bug: 10743878
Change-Id: I6727b12b0ef98bb9cfb3ca95aeab2c79749f47a2
 2013-09-16 18:34:34 +09:00
Jean Chalard
41aaff6cdf Merge "[AC4] Add native methods necessary for auto-commit" 2013-09-13 11:41:55 +00:00
Jean Chalard
24aad5a4d5 [AC4] Add native methods necessary for auto-commit 
Bug: 9059617
Change-Id: I7a47b0675446fc4f39628c60d16de9aea90d1b4d
 2013-09-13 20:15:15 +09:00
Keisuke Kuroyanagi
04bf3cd4e0 Merge "Employ "bigram link" for handling moved bigram target." 2013-09-13 09:46:27 +00:00
Keisuke Kuroyanagi
8c69ddb53e Employ "bigram link" for handling moved bigram target. 
Bug: 6669677

Change-Id: I9a6c0cdb28265e3215ced88f38344df12c25a76d
 2013-09-13 18:36:29 +09:00
Keisuke Kuroyanagi
ebe3b3e8fd Merge "Fix bug related to dynamic patricia trie bigram adding." 2013-09-13 05:49:26 +00:00
Keisuke Kuroyanagi
0243c9ae3d Update children's parent position when the node get moved. 
Bug: 6669677
Change-Id: Ifcaf784afd218c7afd70167045cd2aed8b7909c2
 2013-09-13 14:26:00 +09:00
Keisuke Kuroyanagi
1c0fc852f6 Fix bug related to dynamic patricia trie bigram adding. 
Bug: 6669677

Change-Id: Ia1216a1860a7c5c12a0d8a0816dda8d6a3bea123
 2013-09-13 11:38:53 +09:00
Keisuke Kuroyanagi
4c2767857a Change cache capacity depending on the dictionary size. 
Bug: 10699291

Change-Id: I7042d4c1307da2d991d4dd10d637f18026acb996
 2013-09-12 18:47:56 +09:00
Keisuke Kuroyanagi
1b80a45e79 Merge "Make terminal cache small." 2013-09-12 05:21:49 +00:00
Keisuke Kuroyanagi
80ca7abea3 Make terminal cache small. 
Bug: 10699291

Before:
(0)  2997.63 (0.98%)
(1)  303760.77 (98.92%)
(2)  274.94 (0.09%)
(66)  0.38 (0.00%)
Total 307062.27 (sum of others 307033.73)

After:
(0)  2953.22 (0.96%)
(1)  304633.86 (98.93%)
(2)  293.76 (0.10%)
(66)  0.38 (0.00%)
Total 307915.65 (sum of others 307881.22)

Change-Id: I6905abeb590ef50d48ddfcd8ef8b5351af399150
 2013-09-12 14:18:48 +09:00
Keisuke Kuroyanagi
f1233b58c2 Use same session id for typing and gesture. 
Bug: 10699291
Change-Id: I7c8a3a57e8d189b72a17e4b9c6831becee6ec146
 2013-09-12 12:08:22 +09:00
Keisuke Kuroyanagi
f38969f3f8 Fix bug of dictionary dynamic updating methods. 
Bug: 6669677
Change-Id: I5483adb03e1ac0c27bbfd99d5f4d7cc29809d70a
 2013-09-10 22:32:30 +09:00
Keisuke Kuroyanagi
e531c2241e Move a flag about switching dynamic update to java. 
Bug: 6669677

Change-Id: I6aa99cae4a227f9202179c2873d13473a773e024
 2013-09-10 18:45:53 +09:00
Keisuke Kuroyanagi
210669605a Merge "Fix: ExtendableBuffer" 2013-09-10 06:45:41 +00:00
Keisuke Kuroyanagi
4e79fb24ab Fix: ExtendableBuffer 
Bug: 6669677
Change-Id: I73142fdabac1cbd4128b83c3b26b642b4b4f4da7
 2013-09-10 15:40:11 +09:00
Keisuke Kuroyanagi
216617062a Merge "Implement split and create node methods." 2013-09-10 06:38:05 +00:00
Keisuke Kuroyanagi
fdcd67ba15 Implement split and create node methods. 
Bug: 6669677
Change-Id: Iac4afb637a8ba3d55f7a88f4a532831c1731f93e
 2013-09-10 15:27:19 +09:00
Keisuke Kuroyanagi
c5f35a9c86 Merge "Calculate parent offset from the head of moved node." 2013-09-10 06:08:56 +00:00
Keisuke Kuroyanagi
9c820de7a1 Calculate parent offset from the head of moved node. 
Bug: 6669677
Change-Id: I710453f1c37ec14e5b6830595c1afb2caff83cf2
 2013-09-10 15:03:44 +09:00
Keisuke Kuroyanagi
1d4a07f6c7 Merge "Re-call getBuffer after writing in bigram/shortcut policy." 2013-09-10 03:27:30 +00:00
Keisuke Kuroyanagi
c987120cbc Re-call getBuffer after writing in bigram/shortcut policy. 
Bug: 6669677
Change-Id: I35dc290aa9582e019681b8aae49db3f8d37c90d4
 2013-09-10 12:12:42 +09:00
Keisuke Kuroyanagi
a159ad473c Implement create children array and add child method. 
Bug: 6669677
Change-Id: Ic89c3dfe01937e771db8fc59d5259496a9464a47
 2013-09-09 16:44:17 +09:00
Keisuke Kuroyanagi
7bd7dc5d0d Implement update node probability method. 
Bug: 6669677
Change-Id: I61ac0d05e362fc7d8a967ddd8286580998c70487
 2013-09-09 15:29:11 +09:00
Keisuke Kuroyanagi
3fbc5ef196 Implement inserting new node into PtNode array. 
Bug: 6669677

Change-Id: I0171476231181e41234dde76ac9061febb2e8c35
 2013-09-06 20:40:08 +09:00
Keisuke Kuroyanagi
40182c737d Merge "Compute probability using structure policy." 2013-09-06 08:10:07 +00:00
Keisuke Kuroyanagi
65d19946be Compute probability using structure policy. 
Bug: 6669677
Change-Id: Icd50fc30274cce65ebeb2e7cc8368b59e5cda05a
 2013-09-06 16:50:09 +09:00
Keisuke Kuroyanagi
e4dcebe0c3 Add node flags creating methods. 
Bug: 6669677
Change-Id: Ieb4178da61fa38f6b32baefa099cc0724f9fb6be
 2013-09-06 14:56:59 +09:00
Keisuke Kuroyanagi
96a0631664 DynamicPatriciaTrieWritingUtils to write PtNodes. 
Bug: 6669677
Change-Id: I67cce3c1d338cf747ed2204c96768907d990d269
 2013-09-05 20:12:56 +09:00
Keisuke Kuroyanagi
2a41a35635 Add code point writing methods for updatable dictionaries. 
Bug: 6669677
Change-Id: If7ab6fefdb9a7f0d21c397edbb7d9e7fddc0e0db
 2013-09-05 14:24:07 +09:00
Keisuke Kuroyanagi
5901b5e1bd Check usesForgettingCurve in HeaderPolicy. 
Bug: 6669677
Change-Id: I47ebfc50f477b2a6514fba6fad421dd90f29ecb1
 2013-09-05 12:28:38 +09:00
Keisuke Kuroyanagi
91486bdb3d Merge "Employ a header attribute map in headerPolicy." 2013-09-04 10:06:05 +00:00
Keisuke Kuroyanagi
5754817a5e Employ a header attribute map in headerPolicy. 
Bug: 6669677
Change-Id: I4a084cdd7fcd7a8ed3a70fb7e365031eedd981e5
 2013-09-04 19:00:53 +09:00
Keisuke Kuroyanagi
5d8fd5457d Change name of bigram_list_reading_utils.* . 
Bug: 6669677
Change-Id: Ifd2baa11fb4ee083be1d23e8235f6f903c1b5747
 2013-09-03 20:44:58 +09:00
Keisuke Kuroyanagi
202e416b51 Add bigram writing methods. 
Bug: 6669677

Change-Id: Ib62f6c31c4bcbb41ac200dccccb13e366b3c2173
 2013-09-03 20:44:51 +09:00
Keisuke Kuroyanagi
9e3082e016 Add DynamicShortcutListPolicy::copyAllShortcuts(). 
Bug: 6669677
Change-Id: I74959fe3ca59d77ba0279c75f4d72db13735322f
 2013-09-03 18:36:53 +09:00
Keisuke Kuroyanagi
f1cd7a081c Implement reading side of updating methods of dpt. 
Reading methods are used to determine the position to
be updated in dynamic patricia trie.

Bug: 6669677

Change-Id: Ie9acdcdbc73aa3fd4294c97aa39451a96b165ed9
 2013-09-03 14:47:30 +09:00
Keisuke Kuroyanagi
75d1cb667d Add buffer writing methods. 
Bug: 6669677

Change-Id: Ic3345e36a7842b77598c21de9b54f7c2863df38f
 2013-09-02 19:27:01 +09:00
Keisuke Kuroyanagi
2660f83a12 Change name of extendable_buffer.h to buffer_with_... . 
Bug: 6669677

Change-Id: Id2bd79d8f86c690d5c16d1ab726c5f6fa28ba5e3
 2013-09-02 17:18:41 +09:00
Keisuke Kuroyanagi
3e76487c6c Consolidating dict buffers into BufferWithExtendableBuffer. 
Bug: 6669677

Change-Id: Ied58c123b68199bddb91293ff5cf5d96691e1f6b
 2013-09-02 17:18:41 +09:00
Keisuke Kuroyanagi
4d814bfcb7 Introduce DynamicPatriciaTrieReadingHelper. 
It supports iterating nodes and dealing with additional
buffer. It counts nodes and node arrays to avoid infinite
loop.

Bug: 6669677

Change-Id: I322e7263c0535e098635a1e5de098838de09467d
 2013-08-30 19:41:58 +09:00
Keisuke Kuroyanagi
69ebca0643 Fix children position reading for dynamic patricia trie. 
Bug: 6669677
Change-Id: I68ad68361676f952bd25903c253c22617d3ce361
 2013-08-29 18:31:15 +09:00
Keisuke Kuroyanagi
6c4d09e9e1 Use extendable buffer for reading node info. 
Bug: 6669677
Change-Id: I78ba80100e3a38f2b49e43db1e6aef4e56ed062c
 2013-08-27 18:06:42 +09:00
Keisuke Kuroyanagi
2f0c1253e2 Use extendable buffer for reading bigram and shortcut. 
Bug: 6669677

Change-Id: I4017c0ecc5d8baff1b16ff370d3eec85de9a0f9f
 2013-08-27 17:48:04 +09:00
Keisuke Kuroyanagi
27b12933cd Rename CharGroup to PtNode in native code. 
Bug: 10233675
Change-Id: I5a4215cc79b37bfd317e60101e06b2ab9c332653
 2013-08-26 14:51:43 +09:00
Keisuke Kuroyanagi
ad4e0108a9 Fix: reading uninitialized area. 
Bug: 10402083
Change-Id: I083beea29fe563b1e7739653d756b77820753e3f
 2013-08-23 19:55:55 +09:00
Satoshi Kataoka
02833d11c3 Fix the autocorrection normalized score calculation 
Bug: 10441240
Change-Id: I256021dc55481960d6a605046daa17b1a2d55b95
 2013-08-23 16:22:02 +09:00
Keisuke Kuroyanagi
941811cbd6 Introduce ExtendableBuffer for dynamic update. 
Bug: 6669677

Change-Id: I6343c348334c0dace8b12a71eb992d6f040e8c95
 2013-08-22 18:05:15 +09:00
Keisuke Kuroyanagi
a72a33388f Fix native crash by caused wrong munmap. 
Bug: 10402083
Change-Id: I5aa2cecd20cd3202c2326b1cbb6758257afd6d5d
 2013-08-21 18:57:58 +09:00
Yuichiro Hanada
112257e40f Rename BinaryDictDecoder to Ver3DictDecoder. 
Change-Id: Ibf9b95b658df6e2c2218bdb62e2380f326a03832
 2013-08-20 17:11:51 +09:00
Keisuke Kuroyanagi
4a65258bc7 Fix possible SIGSEGV. 
Change-Id: I6008c0457a8da9b90434a3b96a51da1487948f1b
 2013-08-19 18:58:10 +09:00
Keisuke Kuroyanagi
381c12df20 Move methods in BinaryFormat to PatriciaTriePolicy. 
Bug: 6669677

Change-Id: Ic9bc03a9d8ec789281d83d4b9e58042a083c3ba1
 2013-08-16 16:42:17 +09:00
Keisuke Kuroyanagi
0624cc6cf3 Remove BinaryDictionaryInfo and introduce MmapedBuffer. 
Bug: 6669677
Change-Id: I70e47fa20109060b6ea7a04bcc42ab8f7c543f77
 2013-08-16 12:52:04 +09:00
Ken Wakasa
0238e1eb01 Merge "Revert "Remove BinaryDictionaryInfo and introduce MmapedBuffer."" 2013-08-15 11:39:08 +00:00
Ken Wakasa
d4fcf1dbc5 Revert "Remove BinaryDictionaryInfo and introduce MmapedBuffer." 
This reverts commit d14ffcfcbc.

This broke a bunch of unit tests -- https://sponge.corp.google.com/target?id=a8a62bd0-3aab-4520-ae64-53f9652ef64b&target=com.google.android.inputmethod.latin.tests&searchFor=&show=FAILED&sortBy=STATUS

Change-Id: I1eae460e09808513eacefd523609330819d73d16
 2013-08-15 11:37:48 +00:00
Yuichiro Hanada
94460eba11 [Refactor] Divide BinaryDictInputOutput into BinaryDictEncoder and BinaryDictDecoder. 
Change-Id: I7c3269d77e3e3b567e459dcaa1bc029903941744
 2013-08-15 20:23:07 +09:00
Keisuke Kuroyanagi
0663bc047b Merge "Remove BinaryDictionaryInfo and introduce MmapedBuffer." 2013-08-15 09:34:28 +00:00
Keisuke Kuroyanagi
d14ffcfcbc Remove BinaryDictionaryInfo and introduce MmapedBuffer. 
Bug: 6669677

Change-Id: I1459bb53a9761d920427e4b5d925a2844955c88c
 2013-08-15 18:17:44 +09:00
Jean Chalard
e7870a2c0d Add an initial JNI interface to dicttool. 
Bug: 10100269
Change-Id: I883992c2033e7d9e7c754c0bf653767728b221b6
 2013-08-15 17:58:55 +09:00
Ken Wakasa
dc83fe37ba Merge "Revert "[Refactor] Divide BinaryDictInputOutput into BinaryDictInputUtils and BinaryDictOutputUtils."" 2013-08-15 08:56:28 +00:00
Ken Wakasa
117f18e844 Revert "[Refactor] Divide BinaryDictInputOutput into BinaryDictInputUtils and BinaryDictOutputUtils." 
This reverts commit 4c63d0614e.

Change-Id: I1fa277d720bab4d895259df7d6d82eebfa5eb6c5
 2013-08-15 08:54:29 +00:00
Ken Wakasa
f74ddfbb06 Merge "[Refactor] Divide BinaryDictInputOutput into BinaryDictInputUtils and BinaryDictOutputUtils." 2013-08-15 06:53:23 +00:00
Yuichiro Hanada
4c63d0614e [Refactor] Divide BinaryDictInputOutput into BinaryDictInputUtils and BinaryDictOutputUtils. 
Change-Id: I0d476abe763c11ba9005152f928e8dccf15ac9de
 2013-08-15 15:46:58 +09:00
Keisuke Kuroyanagi
08ce337d3f Fix: BinaryDictionaryInfo.isDynamicallyUpdatable(). 
Change-Id: Ifb3acded5e19f1302c7fcc8881b74addd20c1571
 2013-08-15 13:39:32 +09:00
Keisuke Kuroyanagi
6ec3f63d59 Merge "Make dictionary structure policy have updating methods." 2013-08-15 03:15:43 +00:00
Yuichiro Hanada
46ebaa49e9 Fix a comment. 
Change-Id: I5d09fd2474c7a0e6f42b6c7c6e717692636fbc6d
 2013-08-15 11:57:58 +09:00
Keisuke Kuroyanagi
7fd9667d76 Remove NodeFilter. 
Change-Id: I27d905dd3581cbb0e67d071dba04066aaa2ba22e
 2013-08-14 18:20:15 +09:00
Keisuke Kuroyanagi
dfecc86f25 Fix ShortcutListPolicy::getStartPos(). 
Bug: 6669677
Change-Id: I1fa9967fc6aa0956986954c8ea1f990a47c15764
 2013-08-14 17:23:57 +09:00
Keisuke Kuroyanagi
90619ae02b Move utils for reading dict to policyimpl. 
Bug: 6669677
Change-Id: Ia1da3cb84b457665cf26172d3810e1d28329bca4
 2013-08-14 15:50:42 +09:00
Keisuke Kuroyanagi
66facd37dd Make dictionary structure policy have updating methods. 
Bug: 6669677

Change-Id: I6aa8e377c6347e0db4b7a82293b316ec64d87bec
 2013-08-14 11:34:42 +09:00
Keisuke Kuroyanagi
23d3ed962f Move files for headerPolicy to policyimpl. 
Bug: 6669677
Change-Id: Ic90feb9c3a9a7b698e588c8ad25c765a246706a8
 2013-08-13 20:35:16 +09:00
Keisuke Kuroyanagi
76e579c7ca Move header reading methods to policyimpl. 
Bug: 6669677
Change-Id: I24cb32f28cf355485b431cc56d877000e3f946d4
 2013-08-13 20:22:40 +09:00
Keisuke Kuroyanagi
d81654cd61 Use DictionaryStructureWithBufferPolicy for traversing. 
Bug: 6669677
Change-Id: If7bc038278b7217ed5ba4cb77b1238b2dad0001b
 2013-08-13 19:38:20 +09:00
Keisuke Kuroyanagi
e607e0722a Merge "Remove TerminalAttributes." 2013-08-13 05:58:35 +00:00
Keisuke Kuroyanagi
b40d14226b Merge "Quit using BinaryDictionaryInfo in header reading methods." 2013-08-13 05:49:49 +00:00
Keisuke Kuroyanagi
99007de979 Quit using BinaryDictionaryInfo in header reading methods. 
Bug: 6669677
Change-Id: I5198ae28fcc271dd6ab3c33df940a2d15513a8c8
 2013-08-13 12:26:01 +09:00
Keisuke Kuroyanagi
6abdafc671 Remove TerminalAttributes. 
Use BinaryDictionaryShortcutIterator instead of
TerminalAttributes.

Bug: 6669677
Change-Id: Ib7176e3b302ba383344cc6fcc037e23568c702a8
 2013-08-12 17:46:13 +09:00
Keisuke Kuroyanagi
f4688f8df0 Cleanup ShortcutListReadingUtils. 
Bug: 6669677
Change-Id: Ifd61022665c89f492933dde9811ec644f7e1f5c4
 2013-08-12 17:46:13 +09:00
Keisuke Kuroyanagi
f1e383ed45 Change name of shortcut reading util class. 
Bug: 6669677
Change-Id: If93916f4c3d568119ff6b83e22e95c7086423d2f
 2013-08-12 17:37:59 +09:00
Keisuke Kuroyanagi
7c48352af8 Move methods for Shortcut reading to policyimpl. 
Bug: 6669677
Change-Id: If0a565de015564f803429f4fbf21b05687f80553
 2013-08-12 17:18:28 +09:00
Keisuke Kuroyanagi
fd10db04e0 Move shortcut iteration methods to poilcy. 
Bug: 6669677
Change-Id: I6e535ce7962de8e153a52fd1085fe961649ffe16
 2013-08-12 15:53:09 +09:00
Keisuke Kuroyanagi
277053af7c Move bigram reading methods to BigramListReadingUtils. 
Bug: 6669677
Change-Id: Icb024a8af577323b4dcf74f4c4054cfb8b57a943
 2013-08-12 11:54:31 +09:00
Keisuke Kuroyanagi
668870be43 Move bigrams iteration methods to policy. 
To support various bigram format.

Bug: 6669677
Change-Id: Ifc8c1a855b03cd5a39d97a6e10872ef8ef76475b
 2013-08-09 20:20:34 +09:00
Keisuke Kuroyanagi
e1ebef6124 Make DictionaryStructurePolicy have buffer info. 
To support various format dictionary, extendable
buffers or multiple files.

Bug: 6669677
Change-Id: I203a5a4adc049a6322cfba4eacb4cb6715f8dfc2
 2013-08-09 05:27:22 +00:00
Keisuke Kuroyanagi
b2408c7f71 Fix parent position reading for ver3 dict. 
Bug: 6669677
Change-Id: Ia55d4ecb43c7d37b9bc466b848d5142bc13cd823
 2013-08-08 16:31:17 +09:00
Keisuke Kuroyanagi
60eb250194 Support ver3 dict in native code. 
Bug: 6669677
Change-Id: Iafc3287f1f87588a6296593d6b43ea4464c8f553
 2013-08-08 14:53:05 +09:00
Keisuke Kuroynagi
744084defb Implement getTerminalNodePositionOfWord for ver3 dict. 
Bug: 6669677

Change-Id: Ib9f88d23e6502b8281921ae2f1e47f930b16972f
 2013-08-07 17:59:52 +09:00
Keisuke Kuroynagi
9601df5aac Implement getCodePointsAndProbabilityAnd... for ver3 dicts. 
Bug: 6669677
Change-Id: I5c3b53d015496e11cb6c1f19938b8ee8a4c0f983
 2013-08-07 11:53:34 +09:00
Keisuke Kuroynagi
2b1dd6e253 Implement a part of ver 3 dictionary reading methods. 
Bug: 6669677
Change-Id: I83c159838ad99b2093907b9de7df6cb7a97b4165
 2013-08-06 10:18:20 +09:00
Keisuke Kuroynagi
22f951c881 Merge "Remove checkFirstCharacter from BigramDictionary." 2013-08-05 04:56:23 +00:00
Keisuke Kuroynagi
cf1f7675ea DynamicPatriciaTrieReadingUtils for version 3 dictionary. 
Bug: 6669677
Change-Id: I7105258a00159a5ddda7ff520c490c4ac9fa7902
 2013-08-01 16:15:49 +09:00
Keisuke Kuroynagi
647c000707 Give PatriciaTrieReadingUtils methods for reading nodes. 
The remaining methods in BinaryFormat will be removed.

Bug: 6669677
Change-Id: I866f828d69a600c0ac03b68d71b1f6ee2ce4dd36
 2013-08-01 16:15:35 +09:00
Keisuke Kuroynagi
f0a7f70cf6 Fix: Include wrong file in dynamic_patricia_trie_policy.cpp. 
Change-Id: If994f17e41c7d07e979fa6b0aa4661da0523eb0a
 2013-08-01 14:48:20 +09:00
Keisuke Kuroynagi
8045b7cd24 Merge "Introduce DynamicPatriciaTriePolicy for ver 3 dictionary." 2013-08-01 05:39:07 +00:00
Keisuke Kuroynagi
2401f7c62d Move binary_format.h to policyimpl. 
Bug: 6669677
Change-Id: I3712f60528c7fcf364746f792cec6e180ab2dbef
 2013-07-31 21:07:32 +09:00
Keisuke Kuroynagi
11a431fca6 Fix: appendTwoWords cannot handle MAX_WORD_LENGTH-chars word. 
Bug: 9892258
Change-Id: Ib4ef7850c8fc8b1d99d594bb7ef68ce645f0f1f3
 2013-07-31 16:58:09 +09:00
Keisuke Kuroynagi
2a2aac2568 Remove checkFirstCharacter from BigramDictionary. 
Bug: 10028452
Change-Id: I27b147e83b312d73e975a0b2bc8074b33906e56e
 2013-07-31 13:47:23 +09:00
Keisuke Kuroynagi
26de7079b6 Introduce DynamicPatriciaTriePolicy for ver 3 dictionary. 
Bug: 6669677
Change-Id: Idd3189558aa6fbbd6b2fd007c5721bf39dde9f48
 2013-07-30 17:41:13 +09:00
Keisuke Kuroynagi
1fb11da36a Move creating children methods to structurePolicy. 
Bug: 6669677
Change-Id: I20760f8ab95b18d21383960f6466db0d79429200
 2013-07-29 19:31:27 +09:00
Keisuke Kuroynagi
9d618d1431 Move files only used in dicNode to "internal". 
Change-Id: Ib31ddeff99b480d9b1f33a5b5b207a8acce22858
 2013-07-29 17:03:21 +09:00
Keisuke Kuroynagi
4447b14b78 "much" in suggestions of "m" after "very" 
Make cache size larger for single point input.

Bug: 9823678
Change-Id: Ic6bbcebd18eccaf9f9ac07d9c0d456a62f6c5287
 2013-07-26 18:35:42 +09:00
Satoshi Kataoka
75322cecb9 Support terminal insertion error correction 
Bug: 9421356

Change-Id: I19685763ca487b5636019d62e150708c63ce6fc2
 2013-07-25 22:07:38 +09:00
Keisuke Kuroynagi
cb4a7c1d87 Fix: "saomething" -> "something". 
Fixed by making MAX_CACHE_DIC_NODE_SIZE larger.

Bug: 9682742

Change-Id: I0312952c90b0820bed37df99c223921c68450570
 2013-07-24 17:55:25 +09:00
Keisuke Kuroynagi
62f3b4e9c9 Merge "Fix: huge bigram costs for blacklisted words." 2013-07-23 05:32:09 +00:00
Keisuke Kuroynagi
5b7688bbb5 Fix: native crash while iterating bigramslist. 
Bug: 9964153
Change-Id: Ia2eb922b61300a7c0e34122cd6c18358f1ee7ba9
 2013-07-23 13:13:12 +09:00
Keisuke Kuroynagi
b179199830 Fix: huge bigram costs for blacklisted words. 
Bug: 8844931
Change-Id: I523005c5ed9a3d401a67b0e4e1c3ff2e4574e6df
 2013-07-22 22:05:16 +09:00
Keisuke Kuroynagi
c38ec47596 Get BigramsPos and ShortcutPos via structure policy. 
Bug: 6669677
Change-Id: I924d5512377cfcfe72d3470006b81a36db3865b6
 2013-07-17 20:13:04 +09:00
Keisuke Kuroynagi
8a7129530b Replace NOT_VALID_WORD with NOT_A_VALID_WORD_POS 
Bug: 6669677
Change-Id: If5fb58e8a98b04b767eaaf624cb6a40841b64f54
 2013-07-16 15:53:03 +09:00
Keisuke Kuroynagi
b6f286bfa5 Merge "Make bigram dictionary and traverse session use structure policy." 2013-07-16 02:45:36 +00:00
Keisuke Kuroynagi
34047d8905 Quit ignoring language cost of exact matches. 
Handle exact matches in DicNode::compare() and calculateFinalScore().

Bug: 8844931
Change-Id: I17c78d4e352b0a4308727229b02a8004b38468bf
 2013-07-11 08:25:17 +00:00
Keisuke Kuroynagi
1311cdcb62 Make bigram dictionary and traverse session use structure policy. 
Bug: 6669677
Change-Id: I5a13290366afbb788da997b98d1c593963e92f3e
 2013-07-08 14:25:50 +09:00
Keisuke Kuroynagi
5b03213db1 Remove flags from dicNodeProperties. 
Bug: 6669677
Change-Id: I6f4aef8c4e4376f078ed8375394823788aa35002
 2013-07-05 16:35:07 +09:00
Keisuke Kuroynagi
0933353c0c Merge "Read version 3 dictionary header." 2013-07-04 14:19:35 +00:00
Keisuke Kuroynagi
cac66f29eb Merge "Return NOT_A_PROBABILITY in getProbability for inalid dictionary." 2013-07-04 09:59:40 +00:00
Keisuke Kuroynagi
a829188f54 Allow force auto commit when the top is long multiword. 
Bug: 9688196
Change-Id: I73ff2cd77b932bf7307b3541ea9da3fa4afa11b4
 2013-07-04 17:39:01 +09:00
Keisuke Kuroynagi
72f69fd621 Return NOT_A_PROBABILITY in getProbability for inalid dictionary. 
Bug: 9682742
Change-Id: Ifad4b9811bb5e1a94aed59eba7aedcdb6da5cf9c
 2013-07-03 20:12:38 +09:00
Keisuke Kuroynagi
9a4f7a3761 Disable shortcut output for multi-word suggestions. 
Bug: 9682742
Change-Id: I9e0a5acd9053d141e018c4653e4baf0f77aacb45
 2013-07-03 17:22:36 +09:00
Keisuke Kuroynagi
5ae8722bd5 Read version 3 dictionary header. 
The header structure of version 3 dictionary is same as version 2.

Bug: 6669677
Change-Id: I26c8723333938fbf102de1a40f11d97444da9075
 2013-07-03 17:10:48 +09:00
Keisuke Kuroynagi
07718a5e7d Merge "Fix: Reading invalid dictionary position." 2013-07-02 09:42:53 +00:00
Keisuke Kuroynagi
f6aa9e9a1d Fix: Reading invalid dictionary position. 
Bug: 6669677

Change-Id: I3f4078aaa8cf72f3b366e7851457e9c753c0102f
 2013-07-02 18:39:56 +09:00
Keisuke Kuroynagi
2164856205 Merge "Move children filtering methods to DicNodeChildrenFilter." 2013-07-02 09:35:06 +00:00
Jean Chalard
b6f5d3e39d Fix offdevice compilation 
Change-Id: I8bccccbc51b308fddfbbc990f7bfe1c490259372
 2013-07-02 15:19:16 +09:00
Jean Chalard
7eba0198c0 Dump binary dictionary information when opening 
Bug: 9459517
Change-Id: I122583e734936ae0284e1c7500c6c9242bc7973b
 2013-07-01 18:41:00 +09:00
Keisuke Kuroynagi
28ade0938e Merge "Remove format version 1 related logic." 2013-07-01 07:31:45 +00:00
Keisuke Kuroynagi
92db5e10b1 Remove format version 1 related logic. 
Removed from DictionaryStructurePolicyFactory

Bug: 9617181
Change-Id: Ice4b4a24e019c88b0ec129dc69176c7541f8530c
 2013-07-01 16:28:06 +09:00
Keisuke Kuroynagi
109ba3ace3 Merge "Regenerate old version dictionaries using new format." 2013-07-01 07:16:15 +00:00
Keisuke Kuroynagi
a62b5b22ef Regenerate old version dictionaries using new format. 
Bug: 9617181
Change-Id: I228a3eac959f81b38e4092015e8e3c429d46fa60
 2013-07-01 15:25:33 +09:00
Keisuke Kuroynagi
7a06a79287 Move children filtering methods to DicNodeChildrenFilter. 
Bug: 6669677
Change-Id: If18055e2da8dd5d783942ba384d3ea1d97030e3a
 2013-07-01 15:07:15 +09:00
Keisuke Kuroynagi
c5e6efafff Introduce patriciaTrie to abstract traversing version 2 dictionary. 
Bug: 6669677
Change-Id: Ifef72f3d7a7ba67c5232b98c7835485d72d7322d
 2013-06-28 19:47:00 +09:00
Keisuke Kuroynagi
d9e08beebf Merge "Add DictionaryStructure as a policy." 2013-06-27 03:58:03 +00:00
Keisuke Kuroynagi
68e7edfd99 Use NOT_A_DICT_POS instead of 0 to express not valid ditctionary position. 
Bug: 6669677
Change-Id: I4bf2284f8221a0a2ae4534b4a06b0e59c420a5f9
 2013-06-27 12:53:31 +09:00
Keisuke Kuroynagi
e394abab60 Add DictionaryStructure as a policy. 
Bug: 6669677
Change-Id: I4cc53e623824e57c1e9a351eb0befe9ee988706f
 2013-06-27 12:33:25 +09:00
Keisuke Kuroynagi
c96b56a5ec Merge "Reduce the number of arguments required to initialize dic_node." 2013-06-27 03:30:48 +00:00
Ken Wakasa
c445369732 Fix class names 
Change-Id: Ib3e15bdf46585976aa5b42132805754fc2e2908b
 2013-06-26 23:06:14 +09:00
Keisuke Kuroynagi
6379a4de29 Reduce the number of arguments required to initialize dic_node. 
Bug: 6669677

Change-Id: I52249b83f72560d8f5ab028da5cfb5c50f6e40b8
 2013-06-26 20:55:14 +09:00
Keisuke Kuroynagi
c0d7a376e2 Merge "Move shortcut reading methods." 2013-06-26 05:44:56 +00:00
Keisuke Kuroynagi
5ac44bdc2a Move shortcut reading methods. 
Moved form BinaryFormat to BinaryDictionaryTerminalAttributesReadingUtils.

Bug: 6669677

Change-Id: Ia1ab25854effbf61df37837fe26755ac7dc4d020
 2013-06-26 14:41:22 +09:00
Keisuke Kuroynagi
5bf1be7162 Add jni methods for dynamically handling a dictionary. 
Bug: 6669677
Change-Id: I8a26623adbb41a78e3c023c652be635c635e3b47
 2013-06-26 12:51:33 +09:00
Keisuke Kuroynagi
3b33656ce3 Add canBeDynamicallyUpdated() in BinaryDictionaryInfo. 
Bug: 6669677
Change-Id: I7391641253b99bfcb1f9380ebee2037015e7dc14
 2013-06-25 16:03:33 +09:00
Keisuke Kuroynagi
f3b62900c7 Allow mmapping dictionary file in writable mode in openNative. 
Bug: 6669677
Change-Id: I447668932de118b158fa029ddc977b3f3cc06cb8
 2013-06-25 16:03:33 +09:00
Keisuke Kuroynagi
46159d5422 Move file and mmap information form Dictionary to BinaryDictionaryInfo. 
Bug: 6669677
Change-Id: I05d435e0884430f8abbbed6ea2df775005ee8622
 2013-06-25 16:03:30 +09:00
Keisuke Kuroynagi
80a4b7c92e Merge "Extend center point vertically for bottom keys on a keyboard." 2013-06-24 03:26:55 +00:00
Keisuke Kuroynagi
77ec8713ce Extend center point vertically for bottom keys on a keyboard. 
Bug: 9505397
Change-Id: Ic415ac4065d23cb832dd3bb85876e2a4dc9e9fc3
 2013-06-24 11:58:56 +09:00
Keisuke Kuroynagi
940cca4fa9 Merge "Check adjacent proximity chars for insertion for typing" 2013-06-24 00:52:02 +00:00
Keisuke Kuroynagi
af9d92f590 Rename BinaryDictionaryBigramsReadingUtils to BinaryDictionaryTerminalAttributesReadingUtils and remanme the class methods 
Bug: 6669677
Change-Id: I960dcf51a950ead2c3d7b9dedcde0a450569cbe9
 2013-06-24 09:42:37 +09:00
Keisuke Kuroynagi
be2a172d4e Merge "Give the best score to the most probable exact match." 2013-06-24 00:28:02 +00:00
Keisuke Kuroynagi
24706f8f5c Check adjacent proximity chars for insertion for typing 
Change-Id: I6ce1b065bba055516fca0c8af6f14853a111aa68
 2013-06-24 09:25:46 +09:00
Keisuke Kuroynagi
f0dc0b1b74 Give the best score to the most probable exact match. 
Bug: 9490364
Change-Id: If7fe5da190516d4b5f7a41b752ef5960205a5899
 2013-06-21 19:11:39 +09:00
Keisuke Kuroynagi
21dfdcdba8 Merge "Add ByteArrayUtils::readSignedInt24andAdvancePosition to read dynamic dictionaries." 2013-06-21 08:15:05 +00:00
Keisuke Kuroynagi
8a914a66e8 Add ByteArrayUtils::readSignedInt24andAdvancePosition to read dynamic dictionaries. 
Bug: 6669677
Change-Id: I75eeedbb50868432f6a7ff5fa63624d03e788356
 2013-06-21 12:58:18 +09:00
Keisuke Kuroynagi
09858cbde4 Change method name of getDepth and getTotalLength. 
Change-Id: I4b3aa54e17d7de7c31f956dc9d59941d79856fd7
 2013-06-20 20:55:23 +09:00
Keisuke Kuroynagi
ec7457eb7f Initialize inputStatusG at the constructor. 
Bug: 9490519

Change-Id: I1806f765f7a0b5335df253b2618b40cfd17aab14
 2013-06-19 22:20:25 +09:00
Keisuke Kuroynagi
2d3f2daf12 Check compound distance of node and prune useless node after creating new word. 
Before:
136309.89 ms
After:
73133.95 ms

Bug: 8591918

Change-Id: If029b768e2b459c0e566188bcf425f71b2df866d
 2013-06-18 17:07:23 +09:00
Keisuke Kuroynagi
6bfeeb8694 Support multi words input for gesture. 
Bug: 8591918

Change-Id: I344f936907a385cc6c3c8a2215a6603938483401
 2013-06-17 19:04:29 +09:00
Keisuke Kuroynagi
0052dbda76 Support keys that have uncommon width. 
Bug: 8591918

Change-Id: I1e01e1560200333f9e35993af0aa7e5a17e6944f
 2013-06-14 20:41:39 +09:00
Keisuke Kuroynagi
1ff81e8890 Use bloom filter in multi bigram map. 
Evaluated with previous word "this".
without bloom filter (use only hash_map):
Total 147792.34 (sum of others 147771.57)
with bloom filter:
Total 145900.64 (sum of others 145874.30)
always read binary dictionary:
Total 148603.14 (sum of others 148579.90)

Bug: 8592527
Change-Id: I821dc39454543826adb73b9eeeef6408fad8ae28
 2013-06-14 20:35:41 +09:00
Keisuke Kuroynagi
a71ed8caa2 Introduce BinaryDictionaryBigramsIterator to access bigrams attributes in binary dictionaries. 
Bug: 6669677

Change-Id: Ifb1adebc5305a930c80396f6b4ec31d84400a9dc
 2013-06-14 18:12:56 +09:00
Ken Wakasa
f473f4b1eb Remove unused macros in defines.h 
bug: 8550444
Change-Id: I77ae6bd607032def05044e3045aa7cc3746b0a5b
 2013-06-11 20:09:07 +09:00
Keisuke Kuroynagi
0b7cd6a0b2 Fix: Android.mk for autocorrection_threshold_utils. 
Change-Id: I3ea0811ca7c2c1a88351ee2ba5cc13d7096865ba
 2013-06-11 18:05:04 +09:00
Keisuke Kuroynagi
527c128309 Remove unused methods. 
Bug: 8550444

Change-Id: Ie627084143846407ca36df82c2a7f904a0e6f49c
 2013-06-11 17:04:40 +09:00
Keisuke Kuroynagi
bd0d1afdb2 Introduce BinaryDictionaryHeader to access binary dictionary header. 
BinaryDictionaryHeader abstracts header structure and access header information via only its instance.

Bug: 6669677

Change-Id: Ib5ab0e8fef12265ddabc1d0165548b69953bba6a
 2013-06-11 12:01:49 +09:00
Ken Wakasa
ad0c6d7b36 Cleanups in JNI related code 
Removed the malloc version in binary dictionary support -- this has not
really been tested well so far, and the mmap version has been working pretty
well after all.

Several cosmetic fixes etc.

Change-Id: Iad0da58b300b769fb5946a3e73fc96f56215980e
 2013-06-04 19:16:47 +09:00
Ken Wakasa
11dc3a371d Clean up macro variables 
Change-Id: I0d9983c1a766b97182953328fa45f488e110bc01
 2013-06-04 15:40:30 +09:00
Ken Wakasa
156d39303e Remove a bunch macros that are no longer used 
bug: 8550444
Change-Id: Ib775c0bed27c499dc131c54dfaff34c5a8169276
 2013-06-03 20:00:02 +09:00
Satoshi Kataoka
db0aa747be Merge "Purge WordsPriorityQueue" 2013-06-03 10:21:29 +00:00
Ken Wakasa
4caf594198 Follow up to I907150be994c4f9ffc030c063de521eeb177d3e1 
Change-Id: I8779d397b0ebe2abbf7c3fbc1a79a1a0939c04f0
 2013-06-03 19:14:02 +09:00
Satoshi Kataoka
04e37fb925 Purge WordsPriorityQueue 
Change-Id: If14034c09d0dd335c84dd3a6ea2775655eefd38f
 2013-06-03 19:04:37 +09:00
Satoshi Kataoka
cbb1ee485c Fix build 
Change-Id: I907150be994c4f9ffc030c063de521eeb177d3e1
 2013-06-03 18:59:45 +09:00
Satoshi Kataoka
dceebee4b8 Purge UnigramDictionary 
Bug: 8550444
Change-Id: I899f3bdce977c521515e548ca0c4ee6d29d6450b
 2013-06-03 18:54:03 +09:00
Satoshi Kataoka
6c4ad527dc Migrate used functions in UnigramDicitonary 
Bug: 8550444
Change-Id: Ic1ed2d85537cd47cf18a8373203f331abb7508f6
 2013-06-03 18:28:50 +09:00
Satoshi Kataoka
d858b5815f Move obsolete files to obsolete dir 2 
Bug: 8550444
Change-Id: Ia83796d696e04f51adb39867dfd21d1ac993b63c
 2013-06-03 17:42:27 +09:00
Satoshi Kataoka
e0bbb98d60 Fix build 
Change-Id: I2a876df097d59324aeb58eb83698ec622ac6191b
 2013-06-03 17:23:25 +09:00
Satoshi Kataoka
01cae939a9 Move obsolete files to obsolete dir 
Bug: 8550444
Change-Id: Ie65eea25da4c41e1128d1d8d8643fb6afe8f41e4
 2013-06-03 17:07:59 +09:00
Satoshi Kataoka
7c92b421ee Purge DicTraverseWrapper 
bug: 8550444

Change-Id: Iad017e66ac579c6727b9f60ad9cda64e478200e5
 2013-06-03 16:45:51 +09:00
Satoshi Kataoka
d18112046a Move bigram_dictionary 
Change-Id: I2c243ac75bc02b912f6b0fabedd372148adf8076
 2013-06-03 14:13:39 +09:00
Ken Wakasa
addea83bad Create "src/utils" directory 
Change-Id: I0b7c51d181edc4786e343f7a953367b040ee6b66
 2013-06-03 12:29:34 +09:00
Ken Wakasa
f497e1e5a0 Move suggest_options.h to suggest/core 
Change-Id: Ie95872a4c90ee3089183c1ebafb67243bd2688a4
 2013-06-03 11:45:06 +09:00
Keisuke Kuroyanagi
712fefd85f Merge "Use BinaryDictonaryInfo instead of raw pointers." 2013-06-03 00:51:44 +00:00
Keisuke Kuroyanagi
0ecfb94247 Use BinaryDictonaryInfo instead of raw pointers. 
Bug: 6669677

Change-Id: I9792a872f1609de7c1ba0fc08d916047d6724c0b
 2013-06-02 16:34:42 -07:00
Keisuke Kuroyanagi
88ad30f40b Move dictionary format independent probability calculation methods to ProbabilityUtils. 
Bug: 6669677
Change-Id: Idc09a2fbb04e4d843e11313011178022177616aa
 2013-06-01 14:08:20 -07:00
Ken Wakasa
464d3ba432 Move char_utils to the dictionary directory 
Change-Id: Id397485407fe63600c9bb5c80d223042942b4d8a
 2013-05-31 20:39:34 +09:00
Keisuke Kuroyanagi
f88f9dbbde Move "#define LOG_TAG" before include defines.h. 
Change-Id: Iacd2c2ea3878ad2cde5df36b15e1359d10244f60
 2013-05-30 22:26:21 -07:00
Ken Wakasa
a65c267b1f Move some dictionary related implementations to the dictionary directory 
Change-Id: I6fe71a540ad91ca46755be234abc8894242e2413
 2013-05-31 00:02:57 +09:00
Ken Wakasa
94da44e43e Move additional_proximity_chars to the layout directory 
Change-Id: Ie892ea9f3199a66e9d62a92a0e1ab2dff62826c3
 2013-05-30 20:12:47 +09:00
Ken Wakasa
29432f843a Create a new directory for layout-related implementations 
Change-Id: Ia227bb296f20dab0dfc03dfcde3a06c624eca76c
 2013-05-30 19:45:53 +09:00
Keisuke Kuroyanagi
338ef65077 Make DicTraverseSession having suggest options. 
Change-Id: I2b802c06634e5226c763fd838f3fc0918fc4a161
 2013-05-29 17:18:03 -07:00
Satoshi Kataoka
599d8778f4 Specify OOV for the OOV word correction 
Bug: 9127749
Change-Id: I9ff0590428888afe57a009a1cbf8f008011b1544
 2013-05-27 17:51:35 -07:00
Keisuke Kuroyanagi
50db5f0fff Passing options to native suggestion method. 
Change-Id: I6da180cfa1b9e5aa39600e78e3bc41209a40f583
Signed-off-by: Keisuke Kuroyanagi <ksk@google.com>
 2013-05-24 10:21:04 -07:00
Jean Chalard
03f8c6aed3 Be careful about the dictionary size in detection methods 
Bug: 8857618
Change-Id: I29345ec96d53da601571ba73197a6485643a10a7
 2013-05-08 18:55:18 +09:00
Keisuke Kuroynagi
a09f9709e0 Checking key edit distance of candidate words for gesture scoring. 
bug: 8616704
Change-Id: I98c3248877b1e3e722a6c00a2851e543e5fcbe11
 2013-05-07 19:47:20 +09:00
Satoshi Kataoka
5a346afab9 Fix exact match profanity 
Bug: 8661070

This is a revival version of I421bed14cc71af771510142365

No diff in the test

Change-Id: I96bc9cec531682cce78bcf83223dcd2395b2be77
 2013-05-07 15:52:38 +09:00
Ken Wakasa
c5a29fa864 Revert "Fix exact match profanity" 
This reverts commit df2eae6e69

Change-Id: I42635cc09ad3ea4c177d02e0b7607fb7aa78506c
 2013-05-02 14:21:29 +00:00
Satoshi Kataoka
df2eae6e69 Fix exact match profanity 
Bug: 8661070
Change-Id: I421bed14cc71af7715101423652d706e33958445
 2013-05-01 13:34:08 +09:00
Tom Ouyang
8da0c93292 Reduce MAX_NORM_DISTANCE_FOR_EDIT in typing model after recent logic improvements 
After recent logic improvements we can now safely reduce MAX_NORM_DISTANCE_FOR_EDIT.
This leads to better regression test performance and faster runtime speed.

Bug: 8633962
Change-Id: Ie0db73680b32687ff4d8ea5dfae47dee5c7c66a3
 2013-04-29 21:21:26 -07:00
Keisuke Kuroynagi
a107dcaeb6 Fix: Demote first char capitalized new words. 
Bug: 8680578
Change-Id: I84bcdf3a20428c7561f600332bfef4685c72b67f
Signed-off-by: Keisuke Kuroynagi <ksk@google.com>
 2013-04-30 11:28:18 +09:00
Tom Ouyang
9559dd2e30 Improve bigram frequency lookup 
Bug: 8592527

Change-Id: I1908bcb552279b9acb140fe4d8d26b10ed9eda72
 2013-04-26 12:22:23 -07:00
Ken Wakasa
dad23dda9d A small follow-up to I8d03bae3264974eff7b790e27c073b0a8758d17a 
Change-Id: Id3727f075e74c0102edcb51eabcfdbef745b94b7
 2013-04-27 00:24:34 +09:00
Ken Wakasa
866a6ced57 Just cosmetic cleanups 
Change-Id: I3dc8a4bb3f89f4a18cfffd5f13a71503738452e6
 2013-04-26 19:58:14 +09:00