nyancrimew/LatinIME
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
29926 commits 1 branch 0 tags 363 MiB
Commit graph

622 commits

Author SHA1 Message Date
Keisuke Kuroyanagi
6890246259 Fix reading uninitialized memory in BloomFilter. 
Bug: 12967899

Change-Id: I180b9d728c887c480b6d956bbc4599079c1c359f
 2014-02-14 12:37:03 +09:00
Keisuke Kuroyanagi
a000a32c80 Fix reading uninitialized memory. 
Bug: 12967899

Change-Id: Ia17e4ca9dd8c1e0b24b0fb7e73d07b97c5d81c0c
 2014-02-13 21:18:02 +09:00
Keisuke Kuroyanagi
be61170588 Create Ver2PtNodeArrayReader. 
Bug: 12810574
Change-Id: I7708d24d735680b2fe9e6700316076018e88c98d
 2014-02-10 21:09:55 +09:00
Keisuke Kuroyanagi
cf2b075146 Stop passing buffer directly to DynamicPtReadingHelper. 
Bug: 12810574
Change-Id: I656d99f504d8cf4e057080713b174943afd9dd4c
 2014-02-10 21:06:09 +09:00
Keisuke Kuroyanagi
1e2752924d Create Ver2ParticiaTrieNodeReader. 
Bug: 12810574

Change-Id: I7d3298b5f419d557755ae433c8b8cc0d145f4cc3
 2014-02-10 21:06:09 +09:00
Keisuke Kuroyanagi
1d6afa179c Refactoring: extract PtNode array reading logic form helper. 
Bug: 12810574
Change-Id: I2d2660871862f11630c3ad7bf04bb49ade57c1e7
 2014-02-10 21:06:07 +09:00
Keisuke Kuroyanagi
fc9ca59cc1 Implement getHeaderInfoNative. 
Bug: 11281877
Bug: 12810574
Change-Id: Ia3d85ae2cfdb486e74b8636a62431eae883c85da
 2014-02-06 18:19:42 +09:00
Jean Chalard
43cf9076b2 [HD04] Make the locale mandatory. 
Bug: 11281748
Change-Id: I69281b0053bec404c3e3c713ade3f65a140f51b1
 2014-02-05 17:35:25 +09:00
Keisuke Kuroyanagi
c488048634 Merge "Check isEligibleForAutoCommit when batch input is ended." 2014-02-05 03:34:46 +00:00
Keisuke Kuroyanagi
8c5c01e981 Check isEligibleForAutoCommit when batch input is ended. 
Bug: 11951198
Change-Id: Ibbe09fa38eaa3ebf8f9b69f744a29db7ac598975
 2014-02-05 11:40:17 +09:00
Keisuke Kuroyanagi
1d3058daa1 Merge "Don't boost exact matches for personalized dicts." 2014-02-04 09:13:48 +00:00
Keisuke Kuroyanagi
54622d38c2 Don't boost exact matches for personalized dicts. 
Bug: 12800726
Change-Id: Ia3a2f82db6cdc9645cad0e329c0b79328d7452cc
 2014-02-04 17:47:14 +09:00
Keisuke Kuroyanagi
941734695b Implement Ver4PatriciaTriePolicy::getNextWordAndNextToken. 
Bug: 12810574
Change-Id: Idea44f03c477964f58e65fbf2b55e3fcd77a2934
 2014-02-04 12:42:14 +09:00
Keisuke Kuroyanagi
38f341a2a5 Add a jni method to iterate words in a dictionary. 
Bug: 12810574
Change-Id: I87b13559765f0262dfd06c2ba40d43af6437ea7f
 2014-02-03 14:51:58 +09:00
Keisuke Kuroyanagi
9221772ab7 Get bigram information via getWordProperty(). 
Bug: 12810574
Change-Id: I2750a5659ccbc3e31307c28e35dd9a1dbdffbeee
 2014-02-03 11:47:26 +09:00
Keisuke Kuroyanagi
94d9a2309f Extend jni interface to get bigrams in WordProperty. 
Bug: 12810574
Change-Id: Ia4b88d02ea8790a5c47d32376cc0b84c3e071ddd
 2014-01-31 20:32:44 +09:00
Keisuke Kuroyanagi
17e7768a3e Fetch bigram information in getWordProperty(). 
Bug: 12810574
Change-Id: Ic67e964b01c2753f2eec82b079dc0e5650f340bb
 2014-01-31 20:24:23 +09:00
Keisuke Kuroyanagi
20b7786ece Add BigramProperty and ShortcutProperty in WordProperty. 
Bug: 12810574
Change-Id: Ia9e497c49bb2cf10897bae3a79317113325db819
 2014-01-31 15:26:43 +09:00
Jean Chalard
b0df28f4cc Merge "[HD02] Make the date header attribute mandatory." 2014-01-31 03:58:55 +00:00
Keisuke Kuroyanagi
748f32feba Merge "Rename UnigramProperty to WordProperty." 2014-01-31 03:06:44 +00:00
Keisuke Kuroyanagi
e9085da8cb Merge "Simplify UnigramProperty." 2014-01-31 03:06:33 +00:00
Keisuke Kuroyanagi
080a35e959 Rename UnigramProperty to WordProperty. 
Bug: 12810574
Change-Id: If5ddd803948aaf6e491ddcbaa5436fb3af3f7257
 2014-01-31 11:06:42 +09:00
Jean Chalard
a137c21c9c [HD02] Make the date header attribute mandatory. 
Bug: 11281748
Change-Id: I48a6f3d95ca89ced0c51335527201ecabfb7998e
 2014-01-30 18:19:21 +09:00
Keisuke Kuroyanagi
5f7f6a1615 Simplify UnigramProperty. 
Bug: 12810574
Change-Id: I606f559b5dd340b7525280ecead2c2c7ec920a78
 2014-01-30 17:10:23 +09:00
Jean Chalard
324adefca3 [HD01] Small initial refactoring 
Bug: 11281748
Change-Id: I3c64c75ecd17729bdacf03048311d76ca9f37ae2
 2014-01-30 17:08:13 +09:00
Keisuke Kuroyanagi
21e2380234 Improve the double letter word promotion logic. 
The old logic only checks the top 3 suggestions before the
final adjustment. It leads to instability in scores.

Bug: 10700674
Change-Id: I986aed2aefd66c1fba6196a8f100fcb5bc838a38
 2014-01-28 16:33:20 +09:00
Keisuke Kuroyanagi
51c5ec10f9 Add length check for dict update operations. 
Bug: 12602903
Change-Id: I6835dee8bf8b356f0f1cf6c0531bee5b3415a13f
 2014-01-24 16:32:42 -08:00
Keisuke Kuroyanagi
3e0777e726 Use memmove for all place src and dest can be same. 
Change-Id: I03e5c8f95cb22c7096ff545f1181f914b80a6c58
 2014-01-20 11:27:01 -08:00
Keisuke Kuroyanagi
d78a447d10 Add a dedicated method to control time in native code. 
Bug: 12443085
Change-Id: I41a5cf6c895cb59e54af98b40efded36afcd3247
 2014-01-15 14:55:57 -08:00
Keisuke Kuroyanagi
9638693479 Fix native crash in BigramDictionary::addWordBigram() 
Bug: 12479653
Change-Id: I7d263f744f711202000524a47769dc2d3d0e0de6
 2014-01-14 18:51:49 -08:00
Keisuke Kuroyanagi
84c52a3c81 Boundary check for ProximityInfoState::getProximityTypeG(). 
Bug: 12481052
Change-Id: Ic0b64d2fa17eb8f65f3ad1a7b468cdf6dddf7e70
 2014-01-10 20:15:36 +09:00
Keisuke Kuroyanagi
05113c1847 Check whether the word is OOV or not for amanatto update. 
Bug: 11736529
Change-Id: I27973212da7e922b2185dbb4febfb5bb4dfbdcbb
 2014-01-08 21:02:35 +09:00
Keisuke Kuroyanagi
afa6c07d30 Differentiate exact matches' minor errors. 
Bug: 8844931
Change-Id: I06754643188ccd28ba8c873878f7761787d91ff6
 2013-12-19 12:37:56 +09:00
Keisuke Kuroyanagi
d8f35f7b4c Move methods for outputting from Suggest. 
Bug: 8844931
Change-Id: I83dde6c37b75ed1e1ae4e0972e169d30ada8faf5
 2013-12-18 16:47:23 +09:00
Keisuke Kuroyanagi
9ee9095528 Move methods for scoring from Traversal to Scoring. 
Bug: 8844931

Change-Id: I6137d06e597f4e7b81aaf29555199e18984f2c39
 2013-12-18 07:13:44 +00:00
Keisuke Kuroyanagi
d5f5dd74c2 Quit suggest words that do not include proximity chars. 
Bug: 10416722

Change-Id: I74d27c6a22b4ca47d02eb238b2fbcc49f5dee99c
 2013-12-18 14:31:10 +09:00
Keisuke Kuroyanagi
b5215d456a Increment version number of ver4 format to 401. 
Bug: 11073222
Change-Id: I3878aef02040b1ef73165645fc70ede586289c51
 2013-12-16 23:02:54 +09:00
Keisuke Kuroyanagi
9b08a9e611 Fix: incompatibility of sparse table. 
Bug: 11073222
Change-Id: Id17178efd8f9f3828d821cc718d35606e8bd4b1a
 2013-12-16 22:46:29 +09:00
Keisuke Kuroyanagi
c1163c8518 Align block sizes for ver4 format to java side. 
Bug: 11073222
Change-Id: I09646cbc81529576d5e34dcfa7bc29083a64076f
 2013-12-16 20:42:32 +09:00
Jean Chalard
a245d15da5 Have dicttool use the native library to generate v4 dicts. 
Yay !

Change-Id: Iea8ced9e81031b9ab7eff05ad9ef7215be248de9
 2013-12-13 18:18:20 +09:00
Ken Wakasa
2fa3693c26 Reset to 9bd6dac470 
The bulk merge from -bayo to klp-dev should not have been merged to master.

Change-Id: I527a03a76f5247e4939a672f27c314dc11cbb854
 2013-12-13 17:13:32 +09:00
Keisuke Kuroyanagi
6bc5acaa79 Fix: Suggested words from user history are invalid. 
- Suggestions form user history can contain invalid words.
- isValidWord always returns false.

Bug: 11139426
Change-Id: I6075b275603332ddb00f4a9284afcaa82d824270
 2013-10-22 05:23:48 -07:00
Keisuke Kuroyanagi
6d91e4ce74 Fix offdevice test build. 
Change-Id: I42143bdc4f6f5aa4f55530de84819ab722a22f72
 2013-10-08 16:05:42 +09:00
Keisuke Kuroyanagi
6d3d53955e Merge "Fix a bug: wrong parent position is written when moving." 2013-10-08 04:48:53 +00:00
Keisuke Kuroyanagi
802e6383ea Fix a bug: wrong parent position is written when moving. 
Change-Id: I93d33ba9105eb85327108cca5706d8db3399de6a
 2013-10-08 12:44:00 +09:00
Jean Chalard
86f7711407 Merge "Don't auto-correct to non-whitelist shortcuts" 2013-10-07 10:04:18 +00:00
Jean Chalard
f309f2288b Don't auto-correct to non-whitelist shortcuts 
Bug: 11076722
Change-Id: I0de10ab24d33ec31f45664fe92d80d5a7bec9c20
 2013-10-07 19:01:57 +09:00
Keisuke Kuroyanagi
cfb018ba6d Fix bug and Add large test for decaying dictionary. 
- GC gets failure when the dictionary become empty.
- Useless unigrams are sometimes not removed.

Bug: 10197478
Change-Id: I8d1479c01efba61a81f03bc077da6bcb4797a940
 2013-10-07 17:31:01 +09:00
Keisuke Kuroyanagi
15b1a0fd14 Parameter optimization. 
[Category diff]
+1     194
-1      21
+2      15
-2       1
+3       1
-3      15
+4     152
-4      90
+5     108
-5     197
+6      94
-6     136
+7      28
-7     132

[Weighted category diff]
+1     230
-1      23
+2      15
-2       1
+3       1
-3      15
+4     180
-4     106
+5     122
-5     224
+6     107
-6     158
+7      29
-7     157

show diff for ./en_user_log_phones_2011_08.csv
+1     171
-1       9
+2       7
-2       1
+3       1
-3       7
+4     123
-4      76
+5      84
-5     157
+6      66
-6     119
+7      20
-7     103

Before
---- count ----
Count(1):75.64%, 177913
Count(2):1.06%, 2510
Count(3):0.56%, 1319
Count(4):6.63%, 15612
Count(5):9.87%, 23223
Count(6):2.79%, 6580
Count(7):3.41%, 8038
---- weight ----
Weight(1):96.88%, 2009907
Weight(2):0.24%, 5157
Weight(3):0.09%, 2025
Weight(4):0.77%, 16095
Weight(5):1.11%, 23077
Weight(6):0.39%, 8172
Weight(7):0.48%, 10146
---- score ----
Conservative score = 66.406
Normal score = 81.512
Agressive score = 89.064
Insane score = 92.841

After
---- count ----
Count(1):75.71%, 178087
Count(2):1.07%, 2524
Count(3):0.55%, 1305
Count(4):6.66%, 15677
Count(5):9.83%, 23133
Count(6):2.78%, 6539
Count(7):3.37%, 7930
---- weight ----
Weight(1):96.89%, 2010118
Weight(2):0.24%, 5171
Weight(3):0.09%, 2011
Weight(4):0.78%, 16184
Weight(5):1.1%, 22972
Weight(6):0.39%, 8109
Weight(7):0.48%, 10014
---- score ----
Conservative score = 66.431
Normal score = 81.531
Agressive score = 89.08
Insane score = 92.855

No diffs for 10+ freq user log.

Change-Id: Ica4a7604507a2c91aaa7daf5bacf4fce4039f547
 2013-10-07 12:39:51 +09:00
Keisuke Kuroyanagi
3e5f4b53b0 Merge "Check the elapsed time for decaying." 2013-10-04 07:16:17 +00:00