b22f95ec8a
Remove isUpdatable from constructors of dict contents.
...
Change-Id: I2d54f477d9b341e944e265786a734f23d152bb81
2014-07-11 15:23:55 +09:00
2ac934296c
Concatenate dict buffers other than header to a single file.
...
Bug: 13664080
Change-Id: I34c9d8046b339c9b855be378a5fad907382d1359
2014-07-11 15:15:47 +09:00
198be3a6c5
Fix: wrong bigram entry pos can be used during GC.
...
Bug: 14425059
Change-Id: I4e7c977c5ef184953682175b118de5473cdb64a6
2014-07-09 11:18:35 +09:00
b00973952f
Get bigrams iterator via dict structure policy.
...
Bug: 14425059
Change-Id: Ie4c6afbb6a3b707b5aa8e9e4f941c9da6e6bcdbf
2014-07-08 16:23:31 +09:00
804f7450fc
Use linked list for bigram list.
...
BinaryDictionaryTests for VERSION4_DEV:
Before
Time: 36.461
After
Time: 33.031
Bug: 14425059
Change-Id: I9ca2714f450f61f713df6ebd34c953dece991cdb
2014-07-07 21:09:25 +09:00
f7322b166b
Make getNextWordAndNextToken output code point count.
...
Bug: 14425059
Change-Id: Id1c2927c8a1be0b1680206d444dbdf0c9453dceb
2014-06-23 19:08:30 +09:00
24d49ee577
Remove hasBigrams/hasShortcut flags from ver4 PtNode.
...
Always check bigram/shortcut content instead of having
flags in each PtNode.
Bug: 14425059
Change-Id: I9eb816681bf22fc12e3ab3c92a178a7990b90e93
2014-06-23 16:49:49 +09:00
0fbca1ac23
Remove word from personalized dicts when it's canceled.
...
Bug: 15555552
Bug: 15552347
Change-Id: Ia4d8c79d079e3ce233d46d4bbad4fff1be5c5c39
2014-06-20 14:46:13 +09:00
86fe8081b3
Merge "Add removeUnigramEntry() to native dictionary policy."
2014-06-10 22:29:12 +00:00
f12985245c
Add removeUnigramEntry() to native dictionary policy.
...
Bug: 15552347
Change-Id: Ic713f79ef26b5963060304df9c0a3a402c3cc6d4
2014-06-12 12:14:11 +09:00
22931cd941
Enable Beginning-of-Sentence prediction for contextual dict.
...
Bug: 14161647
Bug: 14119293
Change-Id: I0c00f13966db88e4de85e245e7bced43c9d474b2
2014-06-12 12:26:18 +09:00
f9ce867d80
Add boundary check for v4 bigram reading.
...
Bug: 14496386
Change-Id: Iedd3445c3222a777a2476beed7d9eb53773f406c
2014-05-27 19:29:35 +09:00
dfca51726e
Return whether the dynamic dict operation was success.
...
Bug: 12184250
Change-Id: Iee7e00c1e84c95551a077f4dd023c0a9b9ac9466
2014-05-27 17:30:00 +09:00
96990ca773
Support Beginning-of-Sentence in native code
...
Bug: 14119293
Change-Id: I0f382e33a19bf481823b23405d454de61ec835ff
2014-05-23 17:23:08 +09:00
6600340af5
Fix: PtNodeParams.representsNonWordInfo()
...
Change-Id: I7ab4fb280e8ac89f81d0b32b0663a86c9b69aa5d
2014-05-23 13:49:18 +09:00
2a015dcb25
Add Beginning-of-Sentence info in UnigramProperty.
...
Bug: 14119293
Bug: 14425059
Change-Id: I8a894352568377d32468e5563f4e89af00d22944
2014-05-23 00:07:14 +09:00
9f8c9a0161
Use PrevWordsInfo to add/remove n(bi)-gram in native code.
...
Bug: 14119293
Bug: 14425059
Change-Id: I4b9a46bfd670b35195418eaee51456d44fb91b6d
2014-05-21 18:30:34 +09:00
e810a266fd
Merge "Quit use bigram probability diff for ver4 dict."
2014-05-15 07:07:25 +00:00
64341927d2
Quit use bigram probability diff for ver4 dict.
...
Change-Id: I2cfcfbcf351877d1dff466a24974dbb05908f14e
2014-05-15 16:02:58 +09:00
cf700695ea
Always keep PtNodes that represent non-word info during GC.
...
Bug: 14119293
Change-Id: Ie50b341a512be6a6efe3e05b60e1d09560b01d1f
2014-05-14 22:29:21 +09:00
ed16af72e3
Restrict the permission of dict files created on device.
...
Bug: 13497304
Change-Id: I7bf67e2e81bc63a67bd91cbc0f753ecac88349f6
2014-05-14 17:48:28 +09:00
c1e86d3f49
Merge "Skip PtNodes with non-Unicode code points for suggestion."
2014-05-14 06:22:22 +00:00
0c8ce96bec
Add "// namespace latinime"
...
Bug: 13406708
Change-Id: I0fa85875fb5b98b4e40a149e3e33e5f56bee9a97
2014-05-13 15:16:43 +09:00
6e4b674f83
Move files under bigram/shortcut to under structure.
...
Bug: 13406708
Change-Id: I48e9d4ff30a8b50d4f1694ebbefb02cf189f7713
2014-05-13 15:16:43 +09:00
d0d0113983
Code cleanup.
...
Change-Id: Ic31b66cccfd77348f2ff7961959a3eaf9f7ad5d8
2014-05-13 01:32:38 +09:00
620a05ae59
Support bigram historical information migration.
...
Bug: 13406708
Change-Id: I4bae53e43cb7653eac3b5bd13da2d2bc8aaf88a9
2014-05-13 01:28:30 +09:00
79ba633402
Skip PtNodes with non-Unicode code points for suggestion.
...
Bug: 14119293
Change-Id: Id1d3b789b5f18757070878dba35a7980bfb44591
2014-05-12 19:38:26 +09:00
9d7e8c717f
Support unigram historical information migration.
...
Bug: 13406708
Change-Id: Ibed15b3bc5d5ae68faefa379028dbe10d32b0c0f
2014-05-12 19:21:06 +09:00
6b74f516dc
Merge "Refactoring: Use UnigramProperty to add/update unigram."
2014-05-09 11:39:17 +00:00
b7f71cdf8c
Merge "Avoid copying bigram list if possible."
2014-05-09 10:09:54 +00:00
b636e25e95
Refactoring: Use UnigramProperty to add/update unigram.
...
Bug: 13406708
Change-Id: I26fd541fb465d3543faa5f155becc455ddbb6c9c
2014-05-09 17:22:17 +09:00
455dc84cf2
Add VERSION4_TEST for testing.
...
Bug: 13406708
Change-Id: I0b434717ea3edbb131650af634c53cfbfe113c3e
2014-05-09 15:17:01 +09:00
ad518d9a5b
Avoid copying bigram list if possible.
...
Constructing en_US main dict using dicttool:
Before:
real 1m8.699s
user 1m10.600s
sys 0m2.390s
After:
real 0m17.204s
user 0m20.560s
sys 0m0.720s
Bug: 13406708
Change-Id: I3b0476be57e5cb93c6497025b3ffa7064ac326c6
2014-05-08 14:19:33 +09:00
8d8fb396a0
Add new bigram entry at the tail of existing list.
...
Bug: 13406708
Change-Id: If3162e65fc9aa2c47f046aee528276cb51fad9f4
2014-05-01 19:29:43 +09:00
8dac7ce2e2
Move PatriciaTrieReadingUtils under pt_common.
...
Change-Id: Ib8ec62b97318563785a2aafb854aabdc3f46f187
2014-04-30 14:28:08 +09:00
3b7984752c
Support creating BinaryDictionary without creating dict file.
...
Bug: 14166482
Change-Id: Ic8c78ec2c8f39358a1f085a041b608972a380eef
2014-04-22 13:18:34 -07:00
ce76821dce
Revert "Support creating BinaryDictionary without creating dict file."
...
This reverts commit 89939b8708
2014-04-22 19:34:44 +00:00
89939b8708
Support creating BinaryDictionary without creating dict file.
...
Bug: 14166482
Change-Id: Ib065279f96e227ab0fee7c8141560c4ada744d3b
2014-04-22 12:01:33 -07:00
7b43d987fa
Pass max trie structure size to on memory Ver4DictBuffers.
...
Bug: 14166482
Bug: 14161647
Change-Id: I3b0dd192d7a81c3809f1873bb66ef8fe637f8af0
2014-04-18 17:57:28 -07:00
d4e54af0ba
Merge "Make addUnigramWord use UnigramProperty."
2014-04-14 11:03:30 +00:00
793124855d
Make addUnigramWord use UnigramProperty.
...
Bug: 13406708
Change-Id: I10c108dd7047fe230c60495f13800f32fdc46d0d
2014-04-14 18:49:19 +09:00
8ca9be17db
s/hash_map_compat/unordered_map/
...
Change-Id: Icce5f9a12b04bdd7540c52750d303a585d71f28a
2014-04-11 18:07:59 +09:00
e41b2ed8d3
Separate unigram/bigram property from WordProperty.
...
Bug: 13406708
Change-Id: I48e9fccedd9dcdc1a35ffe027745b58966a83315
2014-04-08 18:24:06 +09:00
cafab169cd
s/stdint.h/cstdint/
...
Somehow, we were not able to use cstdint gcc target build. That's no
longer the case with Clang.
Removed unnecessary header inclusions too.
Change-Id: Ic83a4adf696f1d5ec7a9809253f3c95804e622e1
2014-04-07 12:39:07 +09:00
ca42ec0f44
Use emplace_back in getWordProperty().
...
Change-Id: If164d2b897bd6ad9c89baf0e9a90356100c5087b
2014-03-11 15:21:55 +09:00
a34bdc395b
Make getWordProperty return reconstructed bigram probability.
...
Bug: 13403530
Change-Id: I7ed95a54344d8b77b88922093cf3ef0e530fcae6
2014-03-11 15:20:12 +09:00
4ce480d5ce
Use unique_ptr.
...
Change-Id: Id92a5b07da4f7f95e2cd293ce8dc1a5f979b7853
2014-03-07 14:31:54 +09:00
6c7ebc4023
Use nullptr instead of 0 for pointers.
...
Change-Id: Ic9425de07b896f940ca11c51968c637ebb897514
2014-03-07 10:04:44 +09:00
5128935ac4
Add header attributes for evaluation.
...
Bug: 13197276
Change-Id: Ib5247da691ff24a73e13485288237ccc51bb54f0
2014-02-28 21:06:03 +09:00
94080a37e8
Merge "Check header attributes for forgetting curve."
2014-02-26 08:58:48 +00:00
57816c7a8b
Check header attributes for forgetting curve.
...
Bug: 13197276
Change-Id: Ib360010f309fd1ed3e81cf23aa10c1cfe82781a6
2014-02-26 17:41:31 +09:00
7b496a5861
Add boundary check for ver4 bigram/shortcut reading.
...
Bug: 13185316
Change-Id: I5236222e2a5acda69821062a140c9e36a3024bd2
2014-02-26 14:03:24 +09:00
b96012acef
Record reading error during traversing dictionaries.
...
Bug: 12916055
Change-Id: I439f21aa87cfa1f71e0354370a1514af03d68f6c
2014-02-20 19:12:32 +09:00
303e767728
Fix: crash when opening a broken dictionary.
...
Bug: 13085169
Change-Id: Icfb6184dfefc6a336432203c071d9e30ae8bf990
2014-02-19 22:11:09 +09:00
cf2b075146
Stop passing buffer directly to DynamicPtReadingHelper.
...
Bug: 12810574
Change-Id: I656d99f504d8cf4e057080713b174943afd9dd4c
2014-02-10 21:06:09 +09:00
1d6afa179c
Refactoring: extract PtNode array reading logic form helper.
...
Bug: 12810574
Change-Id: I2d2660871862f11630c3ad7bf04bb49ade57c1e7
2014-02-10 21:06:07 +09:00
941734695b
Implement Ver4PatriciaTriePolicy::getNextWordAndNextToken.
...
Bug: 12810574
Change-Id: Idea44f03c477964f58e65fbf2b55e3fcd77a2934
2014-02-04 12:42:14 +09:00
38f341a2a5
Add a jni method to iterate words in a dictionary.
...
Bug: 12810574
Change-Id: I87b13559765f0262dfd06c2ba40d43af6437ea7f
2014-02-03 14:51:58 +09:00
17e7768a3e
Fetch bigram information in getWordProperty().
...
Bug: 12810574
Change-Id: Ic67e964b01c2753f2eec82b079dc0e5650f340bb
2014-01-31 20:24:23 +09:00
20b7786ece
Add BigramProperty and ShortcutProperty in WordProperty.
...
Bug: 12810574
Change-Id: Ia9e497c49bb2cf10897bae3a79317113325db819
2014-01-31 15:26:43 +09:00
b0df28f4cc
Merge "[HD02] Make the date header attribute mandatory."
2014-01-31 03:58:55 +00:00
748f32feba
Merge "Rename UnigramProperty to WordProperty."
2014-01-31 03:06:44 +00:00
e9085da8cb
Merge "Simplify UnigramProperty."
2014-01-31 03:06:33 +00:00
080a35e959
Rename UnigramProperty to WordProperty.
...
Bug: 12810574
Change-Id: If5ddd803948aaf6e491ddcbaa5436fb3af3f7257
2014-01-31 11:06:42 +09:00
a137c21c9c
[HD02] Make the date header attribute mandatory.
...
Bug: 11281748
Change-Id: I48a6f3d95ca89ced0c51335527201ecabfb7998e
2014-01-30 18:19:21 +09:00
5f7f6a1615
Simplify UnigramProperty.
...
Bug: 12810574
Change-Id: I606f559b5dd340b7525280ecead2c2c7ec920a78
2014-01-30 17:10:23 +09:00
324adefca3
[HD01] Small initial refactoring
...
Bug: 11281748
Change-Id: I3c64c75ecd17729bdacf03048311d76ca9f37ae2
2014-01-30 17:08:13 +09:00
51c5ec10f9
Add length check for dict update operations.
...
Bug: 12602903
Change-Id: I6835dee8bf8b356f0f1cf6c0531bee5b3415a13f
2014-01-24 16:32:42 -08:00
d78a447d10
Add a dedicated method to control time in native code.
...
Bug: 12443085
Change-Id: I41a5cf6c895cb59e54af98b40efded36afcd3247
2014-01-15 14:55:57 -08:00
9b08a9e611
Fix: incompatibility of sparse table.
...
Bug: 11073222
Change-Id: Id17178efd8f9f3828d821cc718d35606e8bd4b1a
2013-12-16 22:46:29 +09:00
c1163c8518
Align block sizes for ver4 format to java side.
...
Bug: 11073222
Change-Id: I09646cbc81529576d5e34dcfa7bc29083a64076f
2013-12-16 20:42:32 +09:00
a245d15da5
Have dicttool use the native library to generate v4 dicts.
...
Yay !
Change-Id: Iea8ced9e81031b9ab7eff05ad9ef7215be248de9
2013-12-13 18:18:20 +09:00
2fa3693c26
Reset to 9bd6dac470
...
The bulk merge from -bayo to klp-dev should not have been merged to master.
Change-Id: I527a03a76f5247e4939a672f27c314dc11cbb854
2013-12-13 17:13:32 +09:00
Keisuke Kuroyanagi
Ken Wakasa
Jean Chalard