Keisuke Kuroyanagi
b22f95ec8a
Remove isUpdatable from constructors of dict contents.
...
Change-Id: I2d54f477d9b341e944e265786a734f23d152bb81
2014-07-11 15:23:55 +09:00
Keisuke Kuroyanagi
2ac934296c
Concatenate dict buffers other than header to a single file.
...
Bug: 13664080
Change-Id: I34c9d8046b339c9b855be378a5fad907382d1359
2014-07-11 15:15:47 +09:00
Keisuke Kuroyanagi
198be3a6c5
Fix: wrong bigram entry pos can be used during GC.
...
Bug: 14425059
Change-Id: I4e7c977c5ef184953682175b118de5473cdb64a6
2014-07-09 11:18:35 +09:00
Keisuke Kuroyanagi
b00973952f
Get bigrams iterator via dict structure policy.
...
Bug: 14425059
Change-Id: Ie4c6afbb6a3b707b5aa8e9e4f941c9da6e6bcdbf
2014-07-08 16:23:31 +09:00
Keisuke Kuroyanagi
804f7450fc
Use linked list for bigram list.
...
BinaryDictionaryTests for VERSION4_DEV:
Before
Time: 36.461
After
Time: 33.031
Bug: 14425059
Change-Id: I9ca2714f450f61f713df6ebd34c953dece991cdb
2014-07-07 21:09:25 +09:00
Keisuke Kuroyanagi
f7322b166b
Make getNextWordAndNextToken output code point count.
...
Bug: 14425059
Change-Id: Id1c2927c8a1be0b1680206d444dbdf0c9453dceb
2014-06-23 19:08:30 +09:00
Keisuke Kuroyanagi
24d49ee577
Remove hasBigrams/hasShortcut flags from ver4 PtNode.
...
Always check bigram/shortcut content instead of having
flags in each PtNode.
Bug: 14425059
Change-Id: I9eb816681bf22fc12e3ab3c92a178a7990b90e93
2014-06-23 16:49:49 +09:00
Keisuke Kuroyanagi
0fbca1ac23
Remove word from personalized dicts when it's canceled.
...
Bug: 15555552
Bug: 15552347
Change-Id: Ia4d8c79d079e3ce233d46d4bbad4fff1be5c5c39
2014-06-20 14:46:13 +09:00
Keisuke Kuroyanagi
86fe8081b3
Merge "Add removeUnigramEntry() to native dictionary policy."
2014-06-10 22:29:12 +00:00
Keisuke Kuroyanagi
f12985245c
Add removeUnigramEntry() to native dictionary policy.
...
Bug: 15552347
Change-Id: Ic713f79ef26b5963060304df9c0a3a402c3cc6d4
2014-06-12 12:14:11 +09:00
Keisuke Kuroyanagi
22931cd941
Enable Beginning-of-Sentence prediction for contextual dict.
...
Bug: 14161647
Bug: 14119293
Change-Id: I0c00f13966db88e4de85e245e7bced43c9d474b2
2014-06-12 12:26:18 +09:00
Keisuke Kuroyanagi
f9ce867d80
Add boundary check for v4 bigram reading.
...
Bug: 14496386
Change-Id: Iedd3445c3222a777a2476beed7d9eb53773f406c
2014-05-27 19:29:35 +09:00
Keisuke Kuroyanagi
dfca51726e
Return whether the dynamic dict operation was success.
...
Bug: 12184250
Change-Id: Iee7e00c1e84c95551a077f4dd023c0a9b9ac9466
2014-05-27 17:30:00 +09:00
Keisuke Kuroyanagi
96990ca773
Support Beginning-of-Sentence in native code
...
Bug: 14119293
Change-Id: I0f382e33a19bf481823b23405d454de61ec835ff
2014-05-23 17:23:08 +09:00
Keisuke Kuroyanagi
6600340af5
Fix: PtNodeParams.representsNonWordInfo()
...
Change-Id: I7ab4fb280e8ac89f81d0b32b0663a86c9b69aa5d
2014-05-23 13:49:18 +09:00
Keisuke Kuroyanagi
2a015dcb25
Add Beginning-of-Sentence info in UnigramProperty.
...
Bug: 14119293
Bug: 14425059
Change-Id: I8a894352568377d32468e5563f4e89af00d22944
2014-05-23 00:07:14 +09:00
Keisuke Kuroyanagi
9f8c9a0161
Use PrevWordsInfo to add/remove n(bi)-gram in native code.
...
Bug: 14119293
Bug: 14425059
Change-Id: I4b9a46bfd670b35195418eaee51456d44fb91b6d
2014-05-21 18:30:34 +09:00
Keisuke Kuroyanagi
e810a266fd
Merge "Quit use bigram probability diff for ver4 dict."
2014-05-15 07:07:25 +00:00
Keisuke Kuroyanagi
64341927d2
Quit use bigram probability diff for ver4 dict.
...
Change-Id: I2cfcfbcf351877d1dff466a24974dbb05908f14e
2014-05-15 16:02:58 +09:00
Keisuke Kuroyanagi
cf700695ea
Always keep PtNodes that represent non-word info during GC.
...
Bug: 14119293
Change-Id: Ie50b341a512be6a6efe3e05b60e1d09560b01d1f
2014-05-14 22:29:21 +09:00
Keisuke Kuroyanagi
ed16af72e3
Restrict the permission of dict files created on device.
...
Bug: 13497304
Change-Id: I7bf67e2e81bc63a67bd91cbc0f753ecac88349f6
2014-05-14 17:48:28 +09:00
Keisuke Kuroyanagi
c1e86d3f49
Merge "Skip PtNodes with non-Unicode code points for suggestion."
2014-05-14 06:22:22 +00:00
Keisuke Kuroyanagi
0c8ce96bec
Add "// namespace latinime"
...
Bug: 13406708
Change-Id: I0fa85875fb5b98b4e40a149e3e33e5f56bee9a97
2014-05-13 15:16:43 +09:00
Keisuke Kuroyanagi
6e4b674f83
Move files under bigram/shortcut to under structure.
...
Bug: 13406708
Change-Id: I48e9d4ff30a8b50d4f1694ebbefb02cf189f7713
2014-05-13 15:16:43 +09:00
Keisuke Kuroyanagi
d0d0113983
Code cleanup.
...
Change-Id: Ic31b66cccfd77348f2ff7961959a3eaf9f7ad5d8
2014-05-13 01:32:38 +09:00
Keisuke Kuroyanagi
620a05ae59
Support bigram historical information migration.
...
Bug: 13406708
Change-Id: I4bae53e43cb7653eac3b5bd13da2d2bc8aaf88a9
2014-05-13 01:28:30 +09:00
Keisuke Kuroyanagi
79ba633402
Skip PtNodes with non-Unicode code points for suggestion.
...
Bug: 14119293
Change-Id: Id1d3b789b5f18757070878dba35a7980bfb44591
2014-05-12 19:38:26 +09:00
Keisuke Kuroyanagi
9d7e8c717f
Support unigram historical information migration.
...
Bug: 13406708
Change-Id: Ibed15b3bc5d5ae68faefa379028dbe10d32b0c0f
2014-05-12 19:21:06 +09:00
Keisuke Kuroyanagi
6b74f516dc
Merge "Refactoring: Use UnigramProperty to add/update unigram."
2014-05-09 11:39:17 +00:00
Keisuke Kuroyanagi
b7f71cdf8c
Merge "Avoid copying bigram list if possible."
2014-05-09 10:09:54 +00:00
Keisuke Kuroyanagi
b636e25e95
Refactoring: Use UnigramProperty to add/update unigram.
...
Bug: 13406708
Change-Id: I26fd541fb465d3543faa5f155becc455ddbb6c9c
2014-05-09 17:22:17 +09:00
Keisuke Kuroyanagi
455dc84cf2
Add VERSION4_TEST for testing.
...
Bug: 13406708
Change-Id: I0b434717ea3edbb131650af634c53cfbfe113c3e
2014-05-09 15:17:01 +09:00
Keisuke Kuroyanagi
ad518d9a5b
Avoid copying bigram list if possible.
...
Constructing en_US main dict using dicttool:
Before:
real 1m8.699s
user 1m10.600s
sys 0m2.390s
After:
real 0m17.204s
user 0m20.560s
sys 0m0.720s
Bug: 13406708
Change-Id: I3b0476be57e5cb93c6497025b3ffa7064ac326c6
2014-05-08 14:19:33 +09:00
Keisuke Kuroyanagi
8d8fb396a0
Add new bigram entry at the tail of existing list.
...
Bug: 13406708
Change-Id: If3162e65fc9aa2c47f046aee528276cb51fad9f4
2014-05-01 19:29:43 +09:00
Keisuke Kuroyanagi
8dac7ce2e2
Move PatriciaTrieReadingUtils under pt_common.
...
Change-Id: Ib8ec62b97318563785a2aafb854aabdc3f46f187
2014-04-30 14:28:08 +09:00
Keisuke Kuroyanagi
3b7984752c
Support creating BinaryDictionary without creating dict file.
...
Bug: 14166482
Change-Id: Ic8c78ec2c8f39358a1f085a041b608972a380eef
2014-04-22 13:18:34 -07:00
Keisuke Kuroyanagi
ce76821dce
Revert "Support creating BinaryDictionary without creating dict file."
...
This reverts commit 89939b8708
.
Change-Id: I8ca1b55609ad305a93178063bd88bd1cede872af
2014-04-22 19:34:44 +00:00
Keisuke Kuroyanagi
89939b8708
Support creating BinaryDictionary without creating dict file.
...
Bug: 14166482
Change-Id: Ib065279f96e227ab0fee7c8141560c4ada744d3b
2014-04-22 12:01:33 -07:00
Keisuke Kuroyanagi
7b43d987fa
Pass max trie structure size to on memory Ver4DictBuffers.
...
Bug: 14166482
Bug: 14161647
Change-Id: I3b0dd192d7a81c3809f1873bb66ef8fe637f8af0
2014-04-18 17:57:28 -07:00
Keisuke Kuroyanagi
d4e54af0ba
Merge "Make addUnigramWord use UnigramProperty."
2014-04-14 11:03:30 +00:00
Keisuke Kuroyanagi
793124855d
Make addUnigramWord use UnigramProperty.
...
Bug: 13406708
Change-Id: I10c108dd7047fe230c60495f13800f32fdc46d0d
2014-04-14 18:49:19 +09:00
Ken Wakasa
8ca9be17db
s/hash_map_compat/unordered_map/
...
Change-Id: Icce5f9a12b04bdd7540c52750d303a585d71f28a
2014-04-11 18:07:59 +09:00
Keisuke Kuroyanagi
e41b2ed8d3
Separate unigram/bigram property from WordProperty.
...
Bug: 13406708
Change-Id: I48e9fccedd9dcdc1a35ffe027745b58966a83315
2014-04-08 18:24:06 +09:00
Ken Wakasa
cafab169cd
s/stdint.h/cstdint/
...
Somehow, we were not able to use cstdint gcc target build. That's no
longer the case with Clang.
Removed unnecessary header inclusions too.
Change-Id: Ic83a4adf696f1d5ec7a9809253f3c95804e622e1
2014-04-07 12:39:07 +09:00
Keisuke Kuroyanagi
ca42ec0f44
Use emplace_back in getWordProperty().
...
Change-Id: If164d2b897bd6ad9c89baf0e9a90356100c5087b
2014-03-11 15:21:55 +09:00
Keisuke Kuroyanagi
a34bdc395b
Make getWordProperty return reconstructed bigram probability.
...
Bug: 13403530
Change-Id: I7ed95a54344d8b77b88922093cf3ef0e530fcae6
2014-03-11 15:20:12 +09:00
Keisuke Kuroyanagi
4ce480d5ce
Use unique_ptr.
...
Change-Id: Id92a5b07da4f7f95e2cd293ce8dc1a5f979b7853
2014-03-07 14:31:54 +09:00
Keisuke Kuroyanagi
6c7ebc4023
Use nullptr instead of 0 for pointers.
...
Change-Id: Ic9425de07b896f940ca11c51968c637ebb897514
2014-03-07 10:04:44 +09:00
Keisuke Kuroyanagi
5128935ac4
Add header attributes for evaluation.
...
Bug: 13197276
Change-Id: Ib5247da691ff24a73e13485288237ccc51bb54f0
2014-02-28 21:06:03 +09:00
Keisuke Kuroyanagi
94080a37e8
Merge "Check header attributes for forgetting curve."
2014-02-26 08:58:48 +00:00