nyancrimew
/
LatinIME
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
33,885 Commits
1 Branch
0 Tags
363 MiB
Commit Graph

473 Commits (78212a6d3de2c1fdaa394c58a16cbdee3ad5d046)

Author SHA1 Message Date
Keisuke Kuroyanagi 91d7538225 Merge "Add VERSION_4_DEV(402) in native code." 2014-05-14 13:40:10 +00:00
Keisuke Kuroyanagi 7116ea98f4 Add VERSION_4_DEV(402) in native code. 
Bug: 13406708
Change-Id: I96cfacf524d670a6a5637a96a63bcd47aaf09ca0
 2014-05-14 22:37:05 +09:00
Keisuke Kuroyanagi cf700695ea Always keep PtNodes that represent non-word info during GC. 
Bug: 14119293
Change-Id: Ie50b341a512be6a6efe3e05b60e1d09560b01d1f
 2014-05-14 22:29:21 +09:00
Keisuke Kuroyanagi 2dc8e36f8e Use nullptr instead of MmappedBufferPtr(nullptr). 
Change-Id: I9bc35f723f1c105ef6155a49630b8cd492900bc8
 2014-05-14 22:05:00 +09:00
Keisuke Kuroyanagi 728141d55a Merge "Generate dict code for version 401." 2014-05-14 10:51:12 +00:00
Keisuke Kuroyanagi 6bf268132d Generate dict code for version 401. 
Bug: 13406708
Change-Id: I769ac84d54d997e7aefab0c9c16727455a132e0b
 2014-05-14 19:09:01 +09:00
Keisuke Kuroyanagi ed16af72e3 Restrict the permission of dict files created on device. 
Bug: 13497304
Change-Id: I7bf67e2e81bc63a67bd91cbc0f753ecac88349f6
 2014-05-14 17:48:28 +09:00
Keisuke Kuroyanagi c1e86d3f49 Merge "Skip PtNodes with non-Unicode code points for suggestion." 2014-05-14 06:22:22 +00:00
Keisuke Kuroyanagi 14dd663fe5 Reduce the cost for handling intentional omission. 
Bug: 14602770

Change-Id: I3a678a3e9b227fc4c3b8d23012aba18ca2e52da5
 2014-05-14 13:54:32 +09:00
Keisuke Kuroyanagi 0c8ce96bec Add "// namespace latinime" 
Bug: 13406708
Change-Id: I0fa85875fb5b98b4e40a149e3e33e5f56bee9a97
 2014-05-13 15:16:43 +09:00
Keisuke Kuroyanagi 6e4b674f83 Move files under bigram/shortcut to under structure. 
Bug: 13406708
Change-Id: I48e9d4ff30a8b50d4f1694ebbefb02cf189f7713
 2014-05-13 15:16:43 +09:00
Keisuke Kuroyanagi d0d0113983 Code cleanup. 
Change-Id: Ic31b66cccfd77348f2ff7961959a3eaf9f7ad5d8
 2014-05-13 01:32:38 +09:00
Keisuke Kuroyanagi 620a05ae59 Support bigram historical information migration. 
Bug: 13406708
Change-Id: I4bae53e43cb7653eac3b5bd13da2d2bc8aaf88a9
 2014-05-13 01:28:30 +09:00
Keisuke Kuroyanagi 79ba633402 Skip PtNodes with non-Unicode code points for suggestion. 
Bug: 14119293
Change-Id: Id1d3b789b5f18757070878dba35a7980bfb44591
 2014-05-12 19:38:26 +09:00
Keisuke Kuroyanagi 9d7e8c717f Support unigram historical information migration. 
Bug: 13406708
Change-Id: Ibed15b3bc5d5ae68faefa379028dbe10d32b0c0f
 2014-05-12 19:21:06 +09:00
Keisuke Kuroyanagi 6b74f516dc Merge "Refactoring: Use UnigramProperty to add/update unigram." 2014-05-09 11:39:17 +00:00
Keisuke Kuroyanagi b7f71cdf8c Merge "Avoid copying bigram list if possible." 2014-05-09 10:09:54 +00:00
Keisuke Kuroyanagi b636e25e95 Refactoring: Use UnigramProperty to add/update unigram. 
Bug: 13406708
Change-Id: I26fd541fb465d3543faa5f155becc455ddbb6c9c
 2014-05-09 17:22:17 +09:00
Keisuke Kuroyanagi eaa347bc1a Merge "Add VERSION4_TEST for testing." 2014-05-09 07:06:06 +00:00
Keisuke Kuroyanagi 455dc84cf2 Add VERSION4_TEST for testing. 
Bug: 13406708
Change-Id: I0b434717ea3edbb131650af634c53cfbfe113c3e
 2014-05-09 15:17:01 +09:00
Keisuke Kuroyanagi 08ea74eb2c Merge "Implement migrateNative()." 2014-05-08 05:21:06 +00:00
Keisuke Kuroyanagi ad518d9a5b Avoid copying bigram list if possible. 
Constructing en_US main dict using dicttool:
Before:
real    1m8.699s
user    1m10.600s
sys     0m2.390s
After:
real    0m17.204s
user    0m20.560s
sys     0m0.720s


Bug: 13406708
Change-Id: I3b0476be57e5cb93c6497025b3ffa7064ac326c6
 2014-05-08 14:19:33 +09:00
Keisuke Kuroyanagi 8a9d9de72a Implement migrateNative(). 
Bug: 13406708
Change-Id: Ie786a68e780baa298fedf82d21390ba45d871a52
 2014-05-08 12:31:32 +09:00
Keisuke Kuroyanagi ad8f2201e2 Make initial additional buffer size 0. 
Bug: 14113369
Change-Id: I3b7b3a452544725577101dc3b9bcb85b3e08afe9
 2014-05-07 23:44:32 +09:00
Keisuke Kuroyanagi 8e8a5a58c4 Merge "Add new bigram entry at the tail of existing list." 2014-05-01 11:19:36 +00:00
Keisuke Kuroyanagi 8d8fb396a0 Add new bigram entry at the tail of existing list. 
Bug: 13406708
Change-Id: If3162e65fc9aa2c47f046aee528276cb51fad9f4
 2014-05-01 19:29:43 +09:00
Ken Wakasa fc2858c55b Small clean up for LARGEST_INT_DIGIT_COUNT 
Change-Id: I4a61c037370bf57b396d80ba7212d94328eaf7b9
 2014-04-30 19:41:42 +09:00
Keisuke Kuroyanagi 8dac7ce2e2 Move PatriciaTrieReadingUtils under pt_common. 
Change-Id: Ib8ec62b97318563785a2aafb854aabdc3f46f187
 2014-04-30 14:28:08 +09:00
Keisuke Kuroyanagi 3b7984752c Support creating BinaryDictionary without creating dict file. 
Bug: 14166482
Change-Id: Ic8c78ec2c8f39358a1f085a041b608972a380eef
 2014-04-22 13:18:34 -07:00
Keisuke Kuroyanagi ce76821dce Revert "Support creating BinaryDictionary without creating dict file." 
This reverts commit 89939b8708.

Change-Id: I8ca1b55609ad305a93178063bd88bd1cede872af
 2014-04-22 19:34:44 +00:00
Keisuke Kuroyanagi 89939b8708 Support creating BinaryDictionary without creating dict file. 
Bug: 14166482
Change-Id: Ib065279f96e227ab0fee7c8141560c4ada744d3b
 2014-04-22 12:01:33 -07:00
Keisuke Kuroyanagi 903be5bbd3 Implement latinime_BinaryDictionary_createOnMemory(). 
Bug: 14166482
Change-Id: If7ec3345ab34edcd6bc5cef9e72580ced894a0e3
 2014-04-22 12:01:22 -07:00
Keisuke Kuroyanagi 7b43d987fa Pass max trie structure size to on memory Ver4DictBuffers. 
Bug: 14166482
Bug: 14161647

Change-Id: I3b0dd192d7a81c3809f1873bb66ef8fe637f8af0
 2014-04-18 17:57:28 -07:00
Keisuke Kuroyanagi d4e54af0ba Merge "Make addUnigramWord use UnigramProperty." 2014-04-14 11:03:30 +00:00
Keisuke Kuroyanagi 793124855d Make addUnigramWord use UnigramProperty. 
Bug: 13406708
Change-Id: I10c108dd7047fe230c60495f13800f32fdc46d0d
 2014-04-14 18:49:19 +09:00
Ken Wakasa 8ca9be17db s/hash_map_compat/unordered_map/ 
Change-Id: Icce5f9a12b04bdd7540c52750d303a585d71f28a
 2014-04-11 18:07:59 +09:00
Keisuke Kuroyanagi 63d33f42c8 Increase terminal cache size for gesture. 
Before:
(0)  670.00 (10.52%)
(1)  5530.00 (86.81%)
(2)  120.00 (1.88%)
Total 6370.00 (sum of others 6320.00)

After:
(0)  730.00 (11.06%)
(1)  5750.00 (87.12%)
(2)  100.00 (1.52%)
Total 6600.00 (sum of others 6580.00)

Bug: 13773693
Bug: 10701902
Bug: 9505397
Change-Id: Ie53640994180a7acabe05da544ab2166403e2394
 2014-04-09 16:50:31 +09:00
Keisuke Kuroyanagi e41b2ed8d3 Separate unigram/bigram property from WordProperty. 
Bug: 13406708

Change-Id: I48e9fccedd9dcdc1a35ffe027745b58966a83315
 2014-04-08 18:24:06 +09:00
Ken Wakasa cafab169cd s/stdint.h/cstdint/ 
Somehow, we were not able to use cstdint gcc target build. That's no
longer the case with Clang.

Removed unnecessary header inclusions too.

Change-Id: Ic83a4adf696f1d5ec7a9809253f3c95804e622e1
 2014-04-07 12:39:07 +09:00
Keisuke Kuroyanagi 77de3a4b65 Change wrongly used "lookahead" to "completion". 
Change-Id: I111cf59d0d24fa869217788c314fffa94ee9f9f9
 2014-04-02 12:44:41 +09:00
Keisuke Kuroyanagi 7dc7a9a398 Parameter tuning for typing. 
accidently -> accidentally (top suggest)
convienent -> convenient (top suggest)
dissapear -> disappear (top suggest)

Execution time: +31%

Before:
(0)  610.00 (1.66%)
(1)  35820.00 (97.66%)
(2)  130.00 (0.35%)
Total 36680.00 (sum of others 36560.00)

After:
(0)  820.00 (1.69%)
(1)  47200.00 (97.50%)
(2)  210.00 (0.43%)
Total 48410.00 (sum of others 48230.00)

Bug: 13588483
Bug: 13456384
Bug: 9816611

Change-Id: Ia17491e135a63aaca6e980c3a59bd829441dac40
 2014-04-01 17:25:43 +09:00
Keisuke Kuroyanagi d73edf23ac Use SuggestionResults to get suggestion. 
Bug: 8187060
Bug: 13333066
Change-Id: I435096ecf8422453f9b167adb0ca3b9c8a840018
 2014-03-27 20:05:33 +09:00
Keisuke Kuroyanagi cf88cf6593 Add a check for ForgettingCurveUtils::decodeProbability. 
Bug: 13607383
Change-Id: I1a56c52173d0e458864e65fb1c1f4603c0c81c6f
 2014-03-24 11:11:07 +00:00
Keisuke Kuroyanagi ef665816d0 Add boundary check for SparseTable. 
Bug: 13592288
Change-Id: I8bc4ff778c6073d22f86b6050a880b6fb69de31e
 2014-03-24 09:17:53 +00:00
Keisuke Kuroyanagi 4c93770430 Fix: wrong size checking in SparseTable. 
Bug: 13592288
Change-Id: I52f03081c5abf25e3c160e2b8fff5e98f8448499
 2014-03-24 14:28:36 +09:00
Ken Wakasa 622589ab43 Several cleanups 
Remove obsolete comment and unused macros. Cosmetic fixes as well.

Change-Id: I9843331c8ce90d9b10b54f425ea1c7416b57c541
 2014-03-17 16:33:37 +09:00
Keisuke Kuroyanagi 73a2426d45 Change default decaying parameters. 
Bug: 13458617
Change-Id: I4942af4a48881fb6ca866c74140b262e6d323f4a
 2014-03-14 12:23:24 +09:00
Keisuke Kuroyanagi 4bc35daaa2 Check "." and ".." to delete all files in a dir. 
Bug: 13421999
Change-Id: I9d0f65cfc9a0f18c7affc152a5b54f9970048e9c
 2014-03-12 15:31:49 +09:00
Keisuke Kuroyanagi e7ddf49d50 Merge "Use emplace_back in getWordProperty()." 2014-03-11 09:38:00 +00:00
Keisuke Kuroyanagi ed72a768d6 Merge "Make getWordProperty return reconstructed bigram probability." 2014-03-11 09:37:49 +00:00
Keisuke Kuroyanagi ca42ec0f44 Use emplace_back in getWordProperty(). 
Change-Id: If164d2b897bd6ad9c89baf0e9a90356100c5087b
 2014-03-11 15:21:55 +09:00
Keisuke Kuroyanagi a34bdc395b Make getWordProperty return reconstructed bigram probability. 
Bug: 13403530
Change-Id: I7ed95a54344d8b77b88922093cf3ef0e530fcae6
 2014-03-11 15:20:12 +09:00
Keisuke Kuroyanagi b43ea5c335 Use isFirstCharUppercase instead of get first code point. 
Change-Id: I0a28305397bfc1e7218e16df00839781a74f5599
 2014-03-10 18:39:45 +09:00
Keisuke Kuroyanagi c36c97b4f7 Merge "Use unique_ptr." 2014-03-07 09:06:58 +00:00
Ken Wakasa 6767f88e3c Address warnings pointed out by clang 
Change-Id: I7943f6c7f9ba08f283a3116adc2af29495f04a21
 2014-03-07 17:05:09 +09:00
Keisuke Kuroyanagi 4ce480d5ce Use unique_ptr. 
Change-Id: Id92a5b07da4f7f95e2cd293ce8dc1a5f979b7853
 2014-03-07 14:31:54 +09:00
Keisuke Kuroyanagi 865e6cf497 Revert "Revert "Use std::min() and std::max()"" 
This reverts commit f63000abea.

Change-Id: I6056d176a13c6ecad38e658ab8778bd898291d02
 2014-03-07 14:19:56 +09:00
Keisuke Kuroyanagi 6c7ebc4023 Use nullptr instead of 0 for pointers. 
Change-Id: Ic9425de07b896f940ca11c51968c637ebb897514
 2014-03-07 10:04:44 +09:00
Keisuke Kuroyanagi 5128935ac4 Add header attributes for evaluation. 
Bug: 13197276
Change-Id: Ib5247da691ff24a73e13485288237ccc51bb54f0
 2014-02-28 21:06:03 +09:00
Keisuke Kuroyanagi 3d70932857 Use MAX_PROBABILITY for the limit of probabilities. 
Bug: 13197276
Change-Id: Ic3f855e566603b395395dff1f860f3d76ecb019c
 2014-02-26 20:48:53 +09:00
Keisuke Kuroyanagi 7919a594fc Merge "Create more aggressive probability tables." 2014-02-26 11:18:44 +00:00
Keisuke Kuroyanagi b368089dbf Stop dividing unigram probability by 2 for backoff. 
Bug: 13197276
Change-Id: I8eaa24b6f710383a8d257e3ec28c37a1a1da8e31
 2014-02-26 19:40:18 +09:00
Keisuke Kuroyanagi 70566266be Create more aggressive probability tables. 
Bug: 13197276

Change-Id: I1451dcbe61088d7301bbef5ceedc72f5649e6764
 2014-02-26 19:36:57 +09:00
Keisuke Kuroyanagi 2b27e06733 Merge "Read and write forgetting curve header attributes." 2014-02-26 09:06:56 +00:00
Keisuke Kuroyanagi 6112cc26a7 Read and write forgetting curve header attributes. 
Bug: 13197276
Change-Id: I6c38a127e1745903594b7bf0ec80ef179e794feb
 2014-02-26 18:04:12 +09:00
Keisuke Kuroyanagi 94080a37e8 Merge "Check header attributes for forgetting curve." 2014-02-26 08:58:48 +00:00
Keisuke Kuroyanagi 57816c7a8b Check header attributes for forgetting curve. 
Bug: 13197276
Change-Id: Ib360010f309fd1ed3e81cf23aa10c1cfe82781a6
 2014-02-26 17:41:31 +09:00
Keisuke Kuroyanagi 7b496a5861 Add boundary check for ver4 bigram/shortcut reading. 
Bug: 13185316
Change-Id: I5236222e2a5acda69821062a140c9e36a3024bd2
 2014-02-26 14:03:24 +09:00
Keisuke Kuroyanagi be81b75dec Add boundary check for ver2 dict reading. 
Bug: 12916055
Change-Id: I78ad1f98a5401f920dcfc3379aa431eb2311ae02
 2014-02-21 13:26:01 +09:00
Keisuke Kuroyanagi b96012acef Record reading error during traversing dictionaries. 
Bug: 12916055
Change-Id: I439f21aa87cfa1f71e0354370a1514af03d68f6c
 2014-02-20 19:12:32 +09:00
Keisuke Kuroyanagi 303e767728 Fix: crash when opening a broken dictionary. 
Bug: 13085169
Change-Id: Icfb6184dfefc6a336432203c071d9e30ae8bf990
 2014-02-19 22:11:09 +09:00
Keisuke Kuroyanagi 0fc93fe445 Implement PatriciaTriePolicy::getNextWordAndNextToken(). 
Bug: 12810574
Change-Id: Id1d44f90de9455d9cbe7b6e0a161cae91d6d422c
 2014-02-15 17:39:20 +09:00
Keisuke Kuroyanagi 8fa7a09f1e Merge "Implement PatriciaTriePolicy::getWordProperty()." 2014-02-14 09:08:09 +00:00
Keisuke Kuroyanagi c63d183473 Implement PatriciaTriePolicy::getWordProperty(). 
Bug: 12810574
Change-Id: I7bcccfd3641ebbcf2b8d857d33bb4734c42af5eb
 2014-02-14 17:56:45 +09:00
Keisuke Kuroyanagi a000a32c80 Fix reading uninitialized memory. 
Bug: 12967899

Change-Id: Ia17e4ca9dd8c1e0b24b0fb7e73d07b97c5d81c0c
 2014-02-13 21:18:02 +09:00
Keisuke Kuroyanagi be61170588 Create Ver2PtNodeArrayReader. 
Bug: 12810574
Change-Id: I7708d24d735680b2fe9e6700316076018e88c98d
 2014-02-10 21:09:55 +09:00
Keisuke Kuroyanagi cf2b075146 Stop passing buffer directly to DynamicPtReadingHelper. 
Bug: 12810574
Change-Id: I656d99f504d8cf4e057080713b174943afd9dd4c
 2014-02-10 21:06:09 +09:00
Keisuke Kuroyanagi 1e2752924d Create Ver2ParticiaTrieNodeReader. 
Bug: 12810574

Change-Id: I7d3298b5f419d557755ae433c8b8cc0d145f4cc3
 2014-02-10 21:06:09 +09:00
Keisuke Kuroyanagi 1d6afa179c Refactoring: extract PtNode array reading logic form helper. 
Bug: 12810574
Change-Id: I2d2660871862f11630c3ad7bf04bb49ade57c1e7
 2014-02-10 21:06:07 +09:00
Keisuke Kuroyanagi fc9ca59cc1 Implement getHeaderInfoNative. 
Bug: 11281877
Bug: 12810574
Change-Id: Ia3d85ae2cfdb486e74b8636a62431eae883c85da
 2014-02-06 18:19:42 +09:00
Jean Chalard 43cf9076b2 [HD04] Make the locale mandatory. 
Bug: 11281748
Change-Id: I69281b0053bec404c3e3c713ade3f65a140f51b1
 2014-02-05 17:35:25 +09:00
Keisuke Kuroyanagi 1d3058daa1 Merge "Don't boost exact matches for personalized dicts." 2014-02-04 09:13:48 +00:00
Keisuke Kuroyanagi 54622d38c2 Don't boost exact matches for personalized dicts. 
Bug: 12800726
Change-Id: Ia3a2f82db6cdc9645cad0e329c0b79328d7452cc
 2014-02-04 17:47:14 +09:00
Keisuke Kuroyanagi 941734695b Implement Ver4PatriciaTriePolicy::getNextWordAndNextToken. 
Bug: 12810574
Change-Id: Idea44f03c477964f58e65fbf2b55e3fcd77a2934
 2014-02-04 12:42:14 +09:00
Keisuke Kuroyanagi 38f341a2a5 Add a jni method to iterate words in a dictionary. 
Bug: 12810574
Change-Id: I87b13559765f0262dfd06c2ba40d43af6437ea7f
 2014-02-03 14:51:58 +09:00
Keisuke Kuroyanagi 17e7768a3e Fetch bigram information in getWordProperty(). 
Bug: 12810574
Change-Id: Ic67e964b01c2753f2eec82b079dc0e5650f340bb
 2014-01-31 20:24:23 +09:00
Keisuke Kuroyanagi 20b7786ece Add BigramProperty and ShortcutProperty in WordProperty. 
Bug: 12810574
Change-Id: Ia9e497c49bb2cf10897bae3a79317113325db819
 2014-01-31 15:26:43 +09:00
Jean Chalard b0df28f4cc Merge "[HD02] Make the date header attribute mandatory." 2014-01-31 03:58:55 +00:00
Keisuke Kuroyanagi 748f32feba Merge "Rename UnigramProperty to WordProperty." 2014-01-31 03:06:44 +00:00
Keisuke Kuroyanagi e9085da8cb Merge "Simplify UnigramProperty." 2014-01-31 03:06:33 +00:00
Keisuke Kuroyanagi 080a35e959 Rename UnigramProperty to WordProperty. 
Bug: 12810574
Change-Id: If5ddd803948aaf6e491ddcbaa5436fb3af3f7257
 2014-01-31 11:06:42 +09:00
Jean Chalard a137c21c9c [HD02] Make the date header attribute mandatory. 
Bug: 11281748
Change-Id: I48a6f3d95ca89ced0c51335527201ecabfb7998e
 2014-01-30 18:19:21 +09:00
Keisuke Kuroyanagi 5f7f6a1615 Simplify UnigramProperty. 
Bug: 12810574
Change-Id: I606f559b5dd340b7525280ecead2c2c7ec920a78
 2014-01-30 17:10:23 +09:00
Jean Chalard 324adefca3 [HD01] Small initial refactoring 
Bug: 11281748
Change-Id: I3c64c75ecd17729bdacf03048311d76ca9f37ae2
 2014-01-30 17:08:13 +09:00
Keisuke Kuroyanagi 21e2380234 Improve the double letter word promotion logic. 
The old logic only checks the top 3 suggestions before the
final adjustment. It leads to instability in scores.

Bug: 10700674
Change-Id: I986aed2aefd66c1fba6196a8f100fcb5bc838a38
 2014-01-28 16:33:20 +09:00
Keisuke Kuroyanagi 51c5ec10f9 Add length check for dict update operations. 
Bug: 12602903
Change-Id: I6835dee8bf8b356f0f1cf6c0531bee5b3415a13f
 2014-01-24 16:32:42 -08:00
Keisuke Kuroyanagi d78a447d10 Add a dedicated method to control time in native code. 
Bug: 12443085
Change-Id: I41a5cf6c895cb59e54af98b40efded36afcd3247
 2014-01-15 14:55:57 -08:00
Keisuke Kuroyanagi 05113c1847 Check whether the word is OOV or not for amanatto update. 
Bug: 11736529
Change-Id: I27973212da7e922b2185dbb4febfb5bb4dfbdcbb
 2014-01-08 21:02:35 +09:00
Keisuke Kuroyanagi afa6c07d30 Differentiate exact matches' minor errors. 
Bug: 8844931
Change-Id: I06754643188ccd28ba8c873878f7761787d91ff6
 2013-12-19 12:37:56 +09:00
Keisuke Kuroyanagi 9ee9095528 Move methods for scoring from Traversal to Scoring. 
Bug: 8844931

Change-Id: I6137d06e597f4e7b81aaf29555199e18984f2c39
 2013-12-18 07:13:44 +00:00
Keisuke Kuroyanagi b5215d456a Increment version number of ver4 format to 401. 
Bug: 11073222
Change-Id: I3878aef02040b1ef73165645fc70ede586289c51
 2013-12-16 23:02:54 +09:00
Keisuke Kuroyanagi 9b08a9e611 Fix: incompatibility of sparse table. 
Bug: 11073222
Change-Id: Id17178efd8f9f3828d821cc718d35606e8bd4b1a
 2013-12-16 22:46:29 +09:00
Keisuke Kuroyanagi c1163c8518 Align block sizes for ver4 format to java side. 
Bug: 11073222
Change-Id: I09646cbc81529576d5e34dcfa7bc29083a64076f
 2013-12-16 20:42:32 +09:00
Jean Chalard a245d15da5 Have dicttool use the native library to generate v4 dicts. 
Yay !

Change-Id: Iea8ced9e81031b9ab7eff05ad9ef7215be248de9
 2013-12-13 18:18:20 +09:00
Ken Wakasa 2fa3693c26 Reset to 9bd6dac470 
The bulk merge from -bayo to klp-dev should not have been merged to master.

Change-Id: I527a03a76f5247e4939a672f27c314dc11cbb854
 2013-12-13 17:13:32 +09:00
Keisuke Kuroyanagi 6bc5acaa79 Fix: Suggested words from user history are invalid. 
- Suggestions form user history can contain invalid words.
- isValidWord always returns false.

Bug: 11139426
Change-Id: I6075b275603332ddb00f4a9284afcaa82d824270
 2013-10-22 05:23:48 -07:00
Keisuke Kuroyanagi 6d91e4ce74 Fix offdevice test build. 
Change-Id: I42143bdc4f6f5aa4f55530de84819ab722a22f72
 2013-10-08 16:05:42 +09:00
Keisuke Kuroyanagi 802e6383ea Fix a bug: wrong parent position is written when moving. 
Change-Id: I93d33ba9105eb85327108cca5706d8db3399de6a
 2013-10-08 12:44:00 +09:00
Keisuke Kuroyanagi cfb018ba6d Fix bug and Add large test for decaying dictionary. 
- GC gets failure when the dictionary become empty.
- Useless unigrams are sometimes not removed.

Bug: 10197478
Change-Id: I8d1479c01efba61a81f03bc077da6bcb4797a940
 2013-10-07 17:31:01 +09:00
Keisuke Kuroyanagi 15b1a0fd14 Parameter optimization. 
[Category diff]
+1     194
-1      21
+2      15
-2       1
+3       1
-3      15
+4     152
-4      90
+5     108
-5     197
+6      94
-6     136
+7      28
-7     132

[Weighted category diff]
+1     230
-1      23
+2      15
-2       1
+3       1
-3      15
+4     180
-4     106
+5     122
-5     224
+6     107
-6     158
+7      29
-7     157

show diff for ./en_user_log_phones_2011_08.csv
+1     171
-1       9
+2       7
-2       1
+3       1
-3       7
+4     123
-4      76
+5      84
-5     157
+6      66
-6     119
+7      20
-7     103

Before
---- count ----
Count(1):75.64%, 177913
Count(2):1.06%, 2510
Count(3):0.56%, 1319
Count(4):6.63%, 15612
Count(5):9.87%, 23223
Count(6):2.79%, 6580
Count(7):3.41%, 8038
---- weight ----
Weight(1):96.88%, 2009907
Weight(2):0.24%, 5157
Weight(3):0.09%, 2025
Weight(4):0.77%, 16095
Weight(5):1.11%, 23077
Weight(6):0.39%, 8172
Weight(7):0.48%, 10146
---- score ----
Conservative score = 66.406
Normal score = 81.512
Agressive score = 89.064
Insane score = 92.841

After
---- count ----
Count(1):75.71%, 178087
Count(2):1.07%, 2524
Count(3):0.55%, 1305
Count(4):6.66%, 15677
Count(5):9.83%, 23133
Count(6):2.78%, 6539
Count(7):3.37%, 7930
---- weight ----
Weight(1):96.89%, 2010118
Weight(2):0.24%, 5171
Weight(3):0.09%, 2011
Weight(4):0.78%, 16184
Weight(5):1.1%, 22972
Weight(6):0.39%, 8109
Weight(7):0.48%, 10014
---- score ----
Conservative score = 66.431
Normal score = 81.531
Agressive score = 89.08
Insane score = 92.855

No diffs for 10+ freq user log.

Change-Id: Ica4a7604507a2c91aaa7daf5bacf4fce4039f547
 2013-10-07 12:39:51 +09:00
Keisuke Kuroyanagi 3e5f4b53b0 Merge "Check the elapsed time for decaying." 2013-10-04 07:16:17 +00:00
Keisuke Kuroyanagi 67c855ea6f Check the elapsed time for decaying. 
Bug: 6669677
Change-Id: I0401a4c0908c702ce65abfa5e017dd16cb1296f9
 2013-10-04 16:11:11 +09:00
Xiaojun Bi 0b62621446 Merge "Optimize the parameters for the typing algorithm" 2013-10-04 03:25:43 +00:00
Xiaojun Bi 2ed09c214a Optimize the parameters for the typing algorithm 
The new set of parameters are optimized for both
keystroke savings and regression test scores.

This change is corresponding to the patch 6 of
Change-Id: Ib8439a85de26cbbba58837344717f0acbfcf0ab0
in the regression test repository.

Here is the link to the doc analyzing the
“nnow → know” case:
https://docs.google.com/a/google.com/document/d/1XIHfAHLlP1f3F8R-u7yxPGWcP58tokygzN_lQKXFFgM/edit

Here is the link to the spreadsheet showing
the diff of the words with frequencies >= 10
in en_user_log_phones_2011_08.csv.
https://docs.google.com/a/google.com/spreadsheet/ccc?key=0Am_RQlV4zC6SdFFVTlhJT0RLUENCWWZOZ3JQTUpMQ2c&usp=drive_web#gid=0

Updated TRT Results for this change:
[Category diff]
+1     783
-1     717
+2      20
-2      34
+3      34
-3      20
+4     575
-4     777
+5     765
-5     755
+6     785
-6     666
+7     479
-7     472

[Weighted category diff]
+1     999
-1     965
+2      21
-2      35
+3      35
-3      21
+4     804
-4     980
+5     970
-5     971
+6     954
-6     831
+7     645
-7     625

Bug: 10613502

Change-Id: I5cd447673a196ad5af641a2118602bec879bdefb
 2013-10-03 19:30:26 -07:00
Keisuke Kuroyanagi 9cb1e78a44 Fix: Native crash when an empty attribute is written. 
Bug: 10964805
Change-Id: Ia3cd891d20aee2226092f0f646d8f280901786fc
 2013-10-03 17:30:54 +09:00
Keisuke Kuroyanagi 3d92954120 Fix build. 
Change-Id: Ib33b27214e67378646e44f5b45bda298f960524f
 2013-10-02 23:02:42 +09:00
Keisuke Kuroyanagi 7c4dcf1e91 Use probability table for decaying dictionaries. 
Bug: 6669677
Change-Id: Ib5994a2e343dfcdaf23755e523f52846709b1c6c
 2013-10-02 18:01:47 +09:00
Keisuke Kuroyanagi c76bbceedc Stochastic decay. 
Bug: 6669677
Change-Id: Ib2d9228b951c77dab7a8675ce9db60677e87e771
 2013-10-02 17:51:12 +09:00
Keisuke Kuroyanagi 13d5dc914a Change name of DecayingUtils. 
Bug: 6669677
Change-Id: I97839b2184b95ae1659c55a9b31d22e2d601cba1
 2013-10-02 16:23:44 +09:00
Jean Chalard ff9476f10a Merge "Fix typos" 2013-10-01 12:10:37 +00:00
Jean Chalard 3e954347e3 Fix typos 
I think I may pass out from the sheer ugliness

Change-Id: I1e709f03ef47ec713552d41fd9e37e01b810ae88
 2013-10-01 21:08:14 +09:00
Keisuke Kuroyanagi 6995310996 Separate "GC" and "GC with decay". 
Bug: 6669677
Change-Id: I9d6aba76cef2616f0549e612db9701e1d6a19467
 2013-10-01 18:09:31 +09:00
Jean Chalard bf1beefd08 Merge "Surface the distance after first word for autocommit." 2013-09-30 14:06:02 +00:00
Jean Chalard da06e385f5 Surface the distance after first word for autocommit. 
Bug: 9059617
Change-Id: Ie9b4cc8148ae8e0ff437b3337ab6c1bde95500f5
 2013-09-30 23:02:04 +09:00
Keisuke Kuroyanagi fd02b2d6ee Implement simple dictionary decay. 
Groundwork and implement simple decay.
Increment probability when typed and decrement probability
at GC.

Bug: 6669677

Change-Id: Ib12caead0cbeef4ce7808fe8ac0b00ee331523fe
 2013-09-30 20:53:35 +09:00
Keisuke Kuroyanagi c18510049a Prepare dictionary decay. 
Bug: 6669677
Change-Id: I8fbae190dd44a6bdbee7e9b6d3a16208322727f7
 2013-09-30 13:57:54 +09:00
Keisuke Kuroyanagi 68bc4ba5c2 Merge "Add boundary checking for PtNode Array reading." 2013-09-27 14:25:24 +00:00
Keisuke Kuroyanagi 8b580819a5 Merge "Add unigram/bigram counting." 2013-09-27 14:14:49 +00:00
Keisuke Kuroyanagi 31097a57cc Add unigram/bigram counting. 
Bug: 6669677
Change-Id: I05ea2201d822dddf062b08c8467daa336760e16c
 2013-09-27 23:12:12 +09:00
Keisuke Kuroyanagi 67cb282b64 Add boundary checking for PtNode Array reading. 
Bug: 6669677
Change-Id: Iea5e19d98d2fc26f137046dd5f8e339239672351
 2013-09-27 22:28:54 +09:00
Keisuke Kuroyanagi 85463c7f40 Merge "Use base lower character to get point to key length." 2013-09-26 08:28:41 +00:00
Keisuke Kuroyanagi e639d7346c Use base lower character to get point to key length. 
Bug: 9490364
Change-Id: I3755f00ab22f926f087fc917b95a2db7d722fa00
 2013-09-26 16:36:05 +09:00
Keisuke Kuroyanagi 5ef6209656 Create empty dictionary file in native code. 
Bug: 6669677
Change-Id: I46d23deee1fd091678f4593561cb8687eb815212
 2013-09-26 12:59:02 +09:00
Keisuke Kuroyanagi 10b56df486 Merge "Prepare dictionary file creating in native code." 2013-09-26 02:38:19 +00:00
Keisuke Kuroyanagi c6015327fb Merge "Add empty PtNode checking for PatriciaTriePolicy." 2013-09-26 02:32:03 +00:00
Keisuke Kuroyanagi 9155eec0d9 Add empty PtNode checking for PatriciaTriePolicy. 
Bug: 9503570
Change-Id: I784d29c307c1040699fa0aadfc6a47d97f03ffe4
 2013-09-26 11:19:04 +09:00
Keisuke Kuroyanagi 36f4592128 Merge "Refactoring header attribute reading." 2013-09-26 01:57:22 +00:00
Keisuke Kuroyanagi 1592eb8a3a Prepare dictionary file creating in native code. 
Bug: 6669677
Change-Id: I7c476a6e99ec7ac883f05d84ef306265255154da
 2013-09-26 10:48:43 +09:00
Keisuke Kuroyanagi 989596844e Refactoring header attribute reading. 
Bug: 6669677
Change-Id: Ifc11da614d5c331ac61019a324e3a0ff187329cd
 2013-09-26 10:45:03 +09:00
Keisuke Kuroyanagi 009dcac33f Boundary check for children creating of PatriciaTriePolicy. 
Bug: 10537529
Change-Id: I0cd8b6de230795498395ca08789ccc1c74780f2c
 2013-09-25 19:54:31 +09:00
Keisuke Kuroyanagi 3d2802ef2e Allow large memory space for GC. 
Bug: 6669677
Change-Id: I021c92a1ca5cd28f745cbfe729c99e346c58074d
 2013-09-24 19:21:17 +09:00
Keisuke Kuroyanagi f3cd38f2c7 GC step 5. Add large tests and fix bugs. 
Bug: 6669677
Change-Id: I41e44a64730f9c66645995095a7ecff51ffd5351
 2013-09-24 16:32:25 +09:00
Keisuke Kuroyanagi 2cfe7f9e3b GC step 4. Update all positions in new dict and add a test. 
Bug: 6669677
Change-Id: I035d848a210653dcd3c1a1f21781968adced9225
 2013-09-24 15:29:56 +09:00
Keisuke Kuroyanagi 0355bf2d4c GC step 3. Place and write valid PtNodes. 
Bug: 6669677

Change-Id: Id2b7359bafb34acf12dc3bf4c595a03073fda805
 2013-09-20 21:11:49 +09:00
Keisuke Kuroyanagi 77ef75cbe6 GC step 2. Finding garbage bigram entries. 
Bug: 6669677
Change-Id: Ie3f406776ab56bf589ac10e15e2534ff8cb1f275
 2013-09-20 20:04:57 +09:00
Keisuke Kuroyanagi 2a64726a16 Step 1 to implement GC. Finding garbage PtNodes. 
Bug: 6669677
Change-Id: I3551fe2f16a09d2bf7761f4e1d73ebd4a03380e7
 2013-09-20 15:37:14 +09:00
Keisuke Kuroyanagi b7e8a9abf5 Groundwork for implementing GC. 
Bug: 6669677

Change-Id: Ia2b2f338795ab62cc9eef73731285b2a5a0ac5e4
 2013-09-19 14:48:59 +09:00
Keisuke Kuroyanagi e28927dfb6 Change name of header_reading_utlis.* 
Bug: 6669677
Change-Id: I91389050148237580ef640d3a58b935ea752954e
 2013-09-18 18:27:01 +09:00
Keisuke Kuroyanagi 484fa7b59c Add dictionary header writing methods. 
Bug: 6669677
Change-Id: I392ac4776b55779903cbaa17e683005d80017a41
 2013-09-18 18:08:33 +09:00
Keisuke Kuroyanagi 1560543754 Implement dictionary flush. 
Bug: 6669677
Change-Id: I1d70ac6319fb6d8700074de382568c2108887960
 2013-09-18 13:19:14 +09:00
Keisuke Kuroyanagi ac7e005ef0 Merge "Prepare dictionary saving in native code." 2013-09-17 09:18:32 +00:00
Keisuke Kuroyanagi d0246277fd Prepare dictionary saving in native code. 
Bug: 6669677

Change-Id: I0787f66409e2b183c6328fa05805945520e1e93a
 2013-09-17 18:07:16 +09:00
Keisuke Kuroyanagi d9f450ef00 Implement removeBigramWords() for DynamicPatriciaTrie. 
Bug: 6669677
Change-Id: I99cb517511b4c968b8bf937baab5d7f42b0f534e
 2013-09-17 18:05:32 +09:00
Keisuke Kuroyanagi 78b55a31cb Fix handling multi-bytes characters and add a test. 
Bug: 6669677

Change-Id: Id2154db47adea2929559a4187a726f9dfa83363e
 2013-09-17 15:11:24 +09:00
Keisuke Kuroyanagi 6d1a29eaab Fix build. 
Change-Id: I1c3c2996ca11bfe8864df774e9280893cf17c0d4
 2013-09-17 13:03:12 +09:00
Keisuke Kuroyanagi cd6a0430c7 Implement addBigramWords() for DynamicPatriciaTrie. 
Bug: 6669677
Change-Id: Ia6aa76c212165690191c65fee9dacdc44be5acd5
 2013-09-17 12:52:21 +09:00
Keisuke Kuroyanagi cb816e5e16 Merge NOT_A_VALID_WORD_POS into NOT_A_DICT_POS. 
Change-Id: I2ac3aa2a19d14e8c4159a59334e7816bd1409688
 2013-09-16 18:37:23 +09:00
Keisuke Kuroyanagi 9ea9c61c99 Move methods using ByteArrayUtils from .h to .cpp. 
Bug: 10402083
Change-Id: Ic5f1aa11eb403f5bdc83de8e38a6457200519cae
 2013-09-16 18:37:08 +09:00
Keisuke Kuroyanagi f874728496 Add boundary check for PtNode reading position. 
Bug: 10743878
Change-Id: I6727b12b0ef98bb9cfb3ca95aeab2c79749f47a2
 2013-09-16 18:34:34 +09:00
Keisuke Kuroyanagi 04bf3cd4e0 Merge "Employ "bigram link" for handling moved bigram target." 2013-09-13 09:46:27 +00:00
Keisuke Kuroyanagi 8c69ddb53e Employ "bigram link" for handling moved bigram target. 
Bug: 6669677

Change-Id: I9a6c0cdb28265e3215ced88f38344df12c25a76d
 2013-09-13 18:36:29 +09:00
Keisuke Kuroyanagi ebe3b3e8fd Merge "Fix bug related to dynamic patricia trie bigram adding." 2013-09-13 05:49:26 +00:00
Keisuke Kuroyanagi 0243c9ae3d Update children's parent position when the node get moved. 
Bug: 6669677
Change-Id: Ifcaf784afd218c7afd70167045cd2aed8b7909c2
 2013-09-13 14:26:00 +09:00
Keisuke Kuroyanagi 1c0fc852f6 Fix bug related to dynamic patricia trie bigram adding. 
Bug: 6669677

Change-Id: Ia1216a1860a7c5c12a0d8a0816dda8d6a3bea123
 2013-09-13 11:38:53 +09:00
Keisuke Kuroyanagi f38969f3f8 Fix bug of dictionary dynamic updating methods. 
Bug: 6669677
Change-Id: I5483adb03e1ac0c27bbfd99d5f4d7cc29809d70a
 2013-09-10 22:32:30 +09:00
Keisuke Kuroyanagi e531c2241e Move a flag about switching dynamic update to java. 
Bug: 6669677

Change-Id: I6aa99cae4a227f9202179c2873d13473a773e024
 2013-09-10 18:45:53 +09:00
Keisuke Kuroyanagi 210669605a Merge "Fix: ExtendableBuffer" 2013-09-10 06:45:41 +00:00
Keisuke Kuroyanagi 4e79fb24ab Fix: ExtendableBuffer 
Bug: 6669677
Change-Id: I73142fdabac1cbd4128b83c3b26b642b4b4f4da7
 2013-09-10 15:40:11 +09:00
Keisuke Kuroyanagi 216617062a Merge "Implement split and create node methods." 2013-09-10 06:38:05 +00:00
Keisuke Kuroyanagi fdcd67ba15 Implement split and create node methods. 
Bug: 6669677
Change-Id: Iac4afb637a8ba3d55f7a88f4a532831c1731f93e
 2013-09-10 15:27:19 +09:00
Keisuke Kuroyanagi c5f35a9c86 Merge "Calculate parent offset from the head of moved node." 2013-09-10 06:08:56 +00:00
Keisuke Kuroyanagi 9c820de7a1 Calculate parent offset from the head of moved node. 
Bug: 6669677
Change-Id: I710453f1c37ec14e5b6830595c1afb2caff83cf2
 2013-09-10 15:03:44 +09:00
Keisuke Kuroyanagi 1d4a07f6c7 Merge "Re-call getBuffer after writing in bigram/shortcut policy." 2013-09-10 03:27:30 +00:00
Keisuke Kuroyanagi c987120cbc Re-call getBuffer after writing in bigram/shortcut policy. 
Bug: 6669677
Change-Id: I35dc290aa9582e019681b8aae49db3f8d37c90d4
 2013-09-10 12:12:42 +09:00
Keisuke Kuroyanagi a159ad473c Implement create children array and add child method. 
Bug: 6669677
Change-Id: Ic89c3dfe01937e771db8fc59d5259496a9464a47
 2013-09-09 16:44:17 +09:00
Keisuke Kuroyanagi 7bd7dc5d0d Implement update node probability method. 
Bug: 6669677
Change-Id: I61ac0d05e362fc7d8a967ddd8286580998c70487
 2013-09-09 15:29:11 +09:00
Keisuke Kuroyanagi 3fbc5ef196 Implement inserting new node into PtNode array. 
Bug: 6669677

Change-Id: I0171476231181e41234dde76ac9061febb2e8c35
 2013-09-06 20:40:08 +09:00
Keisuke Kuroyanagi 40182c737d Merge "Compute probability using structure policy." 2013-09-06 08:10:07 +00:00
Keisuke Kuroyanagi 65d19946be Compute probability using structure policy. 
Bug: 6669677
Change-Id: Icd50fc30274cce65ebeb2e7cc8368b59e5cda05a
 2013-09-06 16:50:09 +09:00
Keisuke Kuroyanagi e4dcebe0c3 Add node flags creating methods. 
Bug: 6669677
Change-Id: Ieb4178da61fa38f6b32baefa099cc0724f9fb6be
 2013-09-06 14:56:59 +09:00
Keisuke Kuroyanagi 96a0631664 DynamicPatriciaTrieWritingUtils to write PtNodes. 
Bug: 6669677
Change-Id: I67cce3c1d338cf747ed2204c96768907d990d269
 2013-09-05 20:12:56 +09:00
Keisuke Kuroyanagi 2a41a35635 Add code point writing methods for updatable dictionaries. 
Bug: 6669677
Change-Id: If7ab6fefdb9a7f0d21c397edbb7d9e7fddc0e0db
 2013-09-05 14:24:07 +09:00
Keisuke Kuroyanagi 5901b5e1bd Check usesForgettingCurve in HeaderPolicy. 
Bug: 6669677
Change-Id: I47ebfc50f477b2a6514fba6fad421dd90f29ecb1
 2013-09-05 12:28:38 +09:00
Keisuke Kuroyanagi 91486bdb3d Merge "Employ a header attribute map in headerPolicy." 2013-09-04 10:06:05 +00:00
Keisuke Kuroyanagi 5754817a5e Employ a header attribute map in headerPolicy. 
Bug: 6669677
Change-Id: I4a084cdd7fcd7a8ed3a70fb7e365031eedd981e5
 2013-09-04 19:00:53 +09:00
Keisuke Kuroyanagi 5d8fd5457d Change name of bigram_list_reading_utils.* . 
Bug: 6669677
Change-Id: Ifd2baa11fb4ee083be1d23e8235f6f903c1b5747
 2013-09-03 20:44:58 +09:00
Keisuke Kuroyanagi 202e416b51 Add bigram writing methods. 
Bug: 6669677

Change-Id: Ib62f6c31c4bcbb41ac200dccccb13e366b3c2173
 2013-09-03 20:44:51 +09:00
Keisuke Kuroyanagi 9e3082e016 Add DynamicShortcutListPolicy::copyAllShortcuts(). 
Bug: 6669677
Change-Id: I74959fe3ca59d77ba0279c75f4d72db13735322f
 2013-09-03 18:36:53 +09:00
Keisuke Kuroyanagi f1cd7a081c Implement reading side of updating methods of dpt. 
Reading methods are used to determine the position to
be updated in dynamic patricia trie.

Bug: 6669677

Change-Id: Ie9acdcdbc73aa3fd4294c97aa39451a96b165ed9
 2013-09-03 14:47:30 +09:00
Keisuke Kuroyanagi 75d1cb667d Add buffer writing methods. 
Bug: 6669677

Change-Id: Ic3345e36a7842b77598c21de9b54f7c2863df38f
 2013-09-02 19:27:01 +09:00
Keisuke Kuroyanagi 2660f83a12 Change name of extendable_buffer.h to buffer_with_... . 
Bug: 6669677

Change-Id: Id2bd79d8f86c690d5c16d1ab726c5f6fa28ba5e3
 2013-09-02 17:18:41 +09:00
Keisuke Kuroyanagi 3e76487c6c Consolidating dict buffers into BufferWithExtendableBuffer. 
Bug: 6669677

Change-Id: Ied58c123b68199bddb91293ff5cf5d96691e1f6b
 2013-09-02 17:18:41 +09:00
Keisuke Kuroyanagi 4d814bfcb7 Introduce DynamicPatriciaTrieReadingHelper. 
It supports iterating nodes and dealing with additional
buffer. It counts nodes and node arrays to avoid infinite
loop.

Bug: 6669677

Change-Id: I322e7263c0535e098635a1e5de098838de09467d
 2013-08-30 19:41:58 +09:00
Keisuke Kuroyanagi 69ebca0643 Fix children position reading for dynamic patricia trie. 
Bug: 6669677
Change-Id: I68ad68361676f952bd25903c253c22617d3ce361
 2013-08-29 18:31:15 +09:00
Keisuke Kuroyanagi 6c4d09e9e1 Use extendable buffer for reading node info. 
Bug: 6669677
Change-Id: I78ba80100e3a38f2b49e43db1e6aef4e56ed062c
 2013-08-27 18:06:42 +09:00
Keisuke Kuroyanagi 2f0c1253e2 Use extendable buffer for reading bigram and shortcut. 
Bug: 6669677

Change-Id: I4017c0ecc5d8baff1b16ff370d3eec85de9a0f9f
 2013-08-27 17:48:04 +09:00
Keisuke Kuroyanagi 27b12933cd Rename CharGroup to PtNode in native code. 
Bug: 10233675
Change-Id: I5a4215cc79b37bfd317e60101e06b2ab9c332653
 2013-08-26 14:51:43 +09:00
Keisuke Kuroyanagi ad4e0108a9 Fix: reading uninitialized area. 
Bug: 10402083
Change-Id: I083beea29fe563b1e7739653d756b77820753e3f
 2013-08-23 19:55:55 +09:00
Keisuke Kuroyanagi 941811cbd6 Introduce ExtendableBuffer for dynamic update. 
Bug: 6669677

Change-Id: I6343c348334c0dace8b12a71eb992d6f040e8c95
 2013-08-22 18:05:15 +09:00
Keisuke Kuroyanagi a72a33388f Fix native crash by caused wrong munmap. 
Bug: 10402083
Change-Id: I5aa2cecd20cd3202c2326b1cbb6758257afd6d5d
 2013-08-21 18:57:58 +09:00
Keisuke Kuroyanagi 381c12df20 Move methods in BinaryFormat to PatriciaTriePolicy. 
Bug: 6669677

Change-Id: Ic9bc03a9d8ec789281d83d4b9e58042a083c3ba1
 2013-08-16 16:42:17 +09:00
Keisuke Kuroyanagi 0624cc6cf3 Remove BinaryDictionaryInfo and introduce MmapedBuffer. 
Bug: 6669677
Change-Id: I70e47fa20109060b6ea7a04bcc42ab8f7c543f77
 2013-08-16 12:52:04 +09:00
Ken Wakasa d4fcf1dbc5 Revert "Remove BinaryDictionaryInfo and introduce MmapedBuffer." 
This reverts commit d14ffcfcbc.

This broke a bunch of unit tests -- https://sponge.corp.google.com/target?id=a8a62bd0-3aab-4520-ae64-53f9652ef64b&target=com.google.android.inputmethod.latin.tests&searchFor=&show=FAILED&sortBy=STATUS

Change-Id: I1eae460e09808513eacefd523609330819d73d16
 2013-08-15 11:37:48 +00:00
Keisuke Kuroyanagi d14ffcfcbc Remove BinaryDictionaryInfo and introduce MmapedBuffer. 
Bug: 6669677

Change-Id: I1459bb53a9761d920427e4b5d925a2844955c88c
 2013-08-15 18:17:44 +09:00
Keisuke Kuroyanagi 6ec3f63d59 Merge "Make dictionary structure policy have updating methods." 2013-08-15 03:15:43 +00:00
Yuichiro Hanada 46ebaa49e9 Fix a comment. 
Change-Id: I5d09fd2474c7a0e6f42b6c7c6e717692636fbc6d
 2013-08-15 11:57:58 +09:00
Keisuke Kuroyanagi 7fd9667d76 Remove NodeFilter. 
Change-Id: I27d905dd3581cbb0e67d071dba04066aaa2ba22e
 2013-08-14 18:20:15 +09:00
Keisuke Kuroyanagi dfecc86f25 Fix ShortcutListPolicy::getStartPos(). 
Bug: 6669677
Change-Id: I1fa9967fc6aa0956986954c8ea1f990a47c15764
 2013-08-14 17:23:57 +09:00
Keisuke Kuroyanagi 90619ae02b Move utils for reading dict to policyimpl. 
Bug: 6669677
Change-Id: Ia1da3cb84b457665cf26172d3810e1d28329bca4
 2013-08-14 15:50:42 +09:00
Keisuke Kuroyanagi 66facd37dd Make dictionary structure policy have updating methods. 
Bug: 6669677

Change-Id: I6aa8e377c6347e0db4b7a82293b316ec64d87bec
 2013-08-14 11:34:42 +09:00
Keisuke Kuroyanagi 23d3ed962f Move files for headerPolicy to policyimpl. 
Bug: 6669677
Change-Id: Ic90feb9c3a9a7b698e588c8ad25c765a246706a8
 2013-08-13 20:35:16 +09:00
Keisuke Kuroyanagi 76e579c7ca Move header reading methods to policyimpl. 
Bug: 6669677
Change-Id: I24cb32f28cf355485b431cc56d877000e3f946d4
 2013-08-13 20:22:40 +09:00
Keisuke Kuroyanagi d81654cd61 Use DictionaryStructureWithBufferPolicy for traversing. 
Bug: 6669677
Change-Id: If7bc038278b7217ed5ba4cb77b1238b2dad0001b
 2013-08-13 19:38:20 +09:00
Keisuke Kuroyanagi f4688f8df0 Cleanup ShortcutListReadingUtils. 
Bug: 6669677
Change-Id: Ifd61022665c89f492933dde9811ec644f7e1f5c4
 2013-08-12 17:46:13 +09:00
Keisuke Kuroyanagi f1e383ed45 Change name of shortcut reading util class. 
Bug: 6669677
Change-Id: If93916f4c3d568119ff6b83e22e95c7086423d2f
 2013-08-12 17:37:59 +09:00
Keisuke Kuroyanagi 7c48352af8 Move methods for Shortcut reading to policyimpl. 
Bug: 6669677
Change-Id: If0a565de015564f803429f4fbf21b05687f80553
 2013-08-12 17:18:28 +09:00
Keisuke Kuroyanagi fd10db04e0 Move shortcut iteration methods to poilcy. 
Bug: 6669677
Change-Id: I6e535ce7962de8e153a52fd1085fe961649ffe16
 2013-08-12 15:53:09 +09:00
Keisuke Kuroyanagi 277053af7c Move bigram reading methods to BigramListReadingUtils. 
Bug: 6669677
Change-Id: Icb024a8af577323b4dcf74f4c4054cfb8b57a943
 2013-08-12 11:54:31 +09:00
Keisuke Kuroyanagi 668870be43 Move bigrams iteration methods to policy. 
To support various bigram format.

Bug: 6669677
Change-Id: Ifc8c1a855b03cd5a39d97a6e10872ef8ef76475b
 2013-08-09 20:20:34 +09:00
Keisuke Kuroyanagi e1ebef6124 Make DictionaryStructurePolicy have buffer info. 
To support various format dictionary, extendable
buffers or multiple files.

Bug: 6669677
Change-Id: I203a5a4adc049a6322cfba4eacb4cb6715f8dfc2
 2013-08-09 05:27:22 +00:00
Keisuke Kuroyanagi b2408c7f71 Fix parent position reading for ver3 dict. 
Bug: 6669677
Change-Id: Ia55d4ecb43c7d37b9bc466b848d5142bc13cd823
 2013-08-08 16:31:17 +09:00
Keisuke Kuroynagi 744084defb Implement getTerminalNodePositionOfWord for ver3 dict. 
Bug: 6669677

Change-Id: Ib9f88d23e6502b8281921ae2f1e47f930b16972f
 2013-08-07 17:59:52 +09:00
Keisuke Kuroynagi 9601df5aac Implement getCodePointsAndProbabilityAnd... for ver3 dicts. 
Bug: 6669677
Change-Id: I5c3b53d015496e11cb6c1f19938b8ee8a4c0f983
 2013-08-07 11:53:34 +09:00
Keisuke Kuroynagi 2b1dd6e253 Implement a part of ver 3 dictionary reading methods. 
Bug: 6669677
Change-Id: I83c159838ad99b2093907b9de7df6cb7a97b4165
 2013-08-06 10:18:20 +09:00
Keisuke Kuroynagi cf1f7675ea DynamicPatriciaTrieReadingUtils for version 3 dictionary. 
Bug: 6669677
Change-Id: I7105258a00159a5ddda7ff520c490c4ac9fa7902
 2013-08-01 16:15:49 +09:00
Keisuke Kuroynagi 647c000707 Give PatriciaTrieReadingUtils methods for reading nodes. 
The remaining methods in BinaryFormat will be removed.

Bug: 6669677
Change-Id: I866f828d69a600c0ac03b68d71b1f6ee2ce4dd36
 2013-08-01 16:15:35 +09:00
Keisuke Kuroynagi f0a7f70cf6 Fix: Include wrong file in dynamic_patricia_trie_policy.cpp. 
Change-Id: If994f17e41c7d07e979fa6b0aa4661da0523eb0a
 2013-08-01 14:48:20 +09:00
Keisuke Kuroynagi 8045b7cd24 Merge "Introduce DynamicPatriciaTriePolicy for ver 3 dictionary." 2013-08-01 05:39:07 +00:00
Keisuke Kuroynagi 2401f7c62d Move binary_format.h to policyimpl. 
Bug: 6669677
Change-Id: I3712f60528c7fcf364746f792cec6e180ab2dbef
 2013-07-31 21:07:32 +09:00
Keisuke Kuroynagi 26de7079b6 Introduce DynamicPatriciaTriePolicy for ver 3 dictionary. 
Bug: 6669677
Change-Id: Idd3189558aa6fbbd6b2fd007c5721bf39dde9f48
 2013-07-30 17:41:13 +09:00
Keisuke Kuroynagi 1fb11da36a Move creating children methods to structurePolicy. 
Bug: 6669677
Change-Id: I20760f8ab95b18d21383960f6466db0d79429200
 2013-07-29 19:31:27 +09:00
Keisuke Kuroynagi 4447b14b78 "much" in suggestions of "m" after "very" 
Make cache size larger for single point input.

Bug: 9823678
Change-Id: Ic6bbcebd18eccaf9f9ac07d9c0d456a62f6c5287
 2013-07-26 18:35:42 +09:00
Satoshi Kataoka 75322cecb9 Support terminal insertion error correction 
Bug: 9421356

Change-Id: I19685763ca487b5636019d62e150708c63ce6fc2
 2013-07-25 22:07:38 +09:00
Keisuke Kuroynagi cb4a7c1d87 Fix: "saomething" -> "something". 
Fixed by making MAX_CACHE_DIC_NODE_SIZE larger.

Bug: 9682742

Change-Id: I0312952c90b0820bed37df99c223921c68450570
 2013-07-24 17:55:25 +09:00
Keisuke Kuroynagi b179199830 Fix: huge bigram costs for blacklisted words. 
Bug: 8844931
Change-Id: I523005c5ed9a3d401a67b0e4e1c3ff2e4574e6df
 2013-07-22 22:05:16 +09:00
Keisuke Kuroynagi c38ec47596 Get BigramsPos and ShortcutPos via structure policy. 
Bug: 6669677
Change-Id: I924d5512377cfcfe72d3470006b81a36db3865b6
 2013-07-17 20:13:04 +09:00
Keisuke Kuroynagi b6f286bfa5 Merge "Make bigram dictionary and traverse session use structure policy." 2013-07-16 02:45:36 +00:00
Keisuke Kuroynagi 34047d8905 Quit ignoring language cost of exact matches. 
Handle exact matches in DicNode::compare() and calculateFinalScore().

Bug: 8844931
Change-Id: I17c78d4e352b0a4308727229b02a8004b38468bf
 2013-07-11 08:25:17 +00:00
Keisuke Kuroynagi 1311cdcb62 Make bigram dictionary and traverse session use structure policy. 
Bug: 6669677
Change-Id: I5a13290366afbb788da997b98d1c593963e92f3e
 2013-07-08 14:25:50 +09:00
Keisuke Kuroynagi 0933353c0c Merge "Read version 3 dictionary header." 2013-07-04 14:19:35 +00:00
Keisuke Kuroynagi a829188f54 Allow force auto commit when the top is long multiword. 
Bug: 9688196
Change-Id: I73ff2cd77b932bf7307b3541ea9da3fa4afa11b4
 2013-07-04 17:39:01 +09:00
Keisuke Kuroynagi 5ae8722bd5 Read version 3 dictionary header. 
The header structure of version 3 dictionary is same as version 2.

Bug: 6669677
Change-Id: I26c8723333938fbf102de1a40f11d97444da9075
 2013-07-03 17:10:48 +09:00
Keisuke Kuroynagi 2164856205 Merge "Move children filtering methods to DicNodeChildrenFilter." 2013-07-02 09:35:06 +00:00
Keisuke Kuroynagi 92db5e10b1 Remove format version 1 related logic. 
Removed from DictionaryStructurePolicyFactory

Bug: 9617181
Change-Id: Ice4b4a24e019c88b0ec129dc69176c7541f8530c
 2013-07-01 16:28:06 +09:00
Keisuke Kuroynagi 7a06a79287 Move children filtering methods to DicNodeChildrenFilter. 
Bug: 6669677
Change-Id: If18055e2da8dd5d783942ba384d3ea1d97030e3a
 2013-07-01 15:07:15 +09:00
Keisuke Kuroynagi c5e6efafff Introduce patriciaTrie to abstract traversing version 2 dictionary. 
Bug: 6669677
Change-Id: Ifef72f3d7a7ba67c5232b98c7835485d72d7322d
 2013-06-28 19:47:00 +09:00
Keisuke Kuroynagi 940cca4fa9 Merge "Check adjacent proximity chars for insertion for typing" 2013-06-24 00:52:02 +00:00
Keisuke Kuroynagi be2a172d4e Merge "Give the best score to the most probable exact match." 2013-06-24 00:28:02 +00:00
Keisuke Kuroynagi 24706f8f5c Check adjacent proximity chars for insertion for typing 
Change-Id: I6ce1b065bba055516fca0c8af6f14853a111aa68
 2013-06-24 09:25:46 +09:00
Keisuke Kuroynagi f0dc0b1b74 Give the best score to the most probable exact match. 
Bug: 9490364
Change-Id: If7fe5da190516d4b5f7a41b752ef5960205a5899
 2013-06-21 19:11:39 +09:00