d58a07666e
Merge "Rename jni method for n-gram supporting."
2014-06-24 01:52:26 +00:00
83b6ee2538
Rename jni method for n-gram supporting.
...
Bug: 14425059
Change-Id: I242d1de5d9db37344f02d8fba9035829794a56c8
2014-06-25 14:14:40 +09:00
0aef59746c
Make Beginning-of-Sentence prediction require two exposures.
...
Bug: 15868192
Change-Id: I4b577ad11c992d872d6f902fd50527d9ca5cc1ba
2014-06-25 20:20:44 +09:00
88fa47a27d
Support migration/dump of Beginning-of-Sentence entries.
...
Bug: 14119293
Change-Id: Ie975138f819794d5c34a7a547be5a6117050e084
2014-06-24 12:37:07 +09:00
f7322b166b
Make getNextWordAndNextToken output code point count.
...
Bug: 14425059
Change-Id: Id1c2927c8a1be0b1680206d444dbdf0c9453dceb
2014-06-23 19:08:30 +09:00
24d49ee577
Remove hasBigrams/hasShortcut flags from ver4 PtNode.
...
Always check bigram/shortcut content instead of having
flags in each PtNode.
Bug: 14425059
Change-Id: I9eb816681bf22fc12e3ab3c92a178a7990b90e93
2014-06-23 16:49:49 +09:00
0fbca1ac23
Remove word from personalized dicts when it's canceled.
...
Bug: 15555552
Bug: 15552347
Change-Id: Ia4d8c79d079e3ce233d46d4bbad4fff1be5c5c39
2014-06-20 14:46:13 +09:00
89899e14df
Merge "Update unit test for beginning-of-sentence."
2014-06-10 22:31:57 +00:00
e041bfab5c
Update unit test for beginning-of-sentence.
...
Bug: 14119293
Change-Id: I594a1ca1096f0c12c678dd30eb75de905e55c5bb
2014-06-12 16:07:39 +09:00
22931cd941
Enable Beginning-of-Sentence prediction for contextual dict.
...
Bug: 14161647
Bug: 14119293
Change-Id: I0c00f13966db88e4de85e245e7bced43c9d474b2
2014-06-12 12:26:18 +09:00
86fe8081b3
Merge "Add removeUnigramEntry() to native dictionary policy."
2014-06-10 22:29:12 +00:00
f12985245c
Add removeUnigramEntry() to native dictionary policy.
...
Bug: 15552347
Change-Id: Ic713f79ef26b5963060304df9c0a3a402c3cc6d4
2014-06-12 12:14:11 +09:00
447c19364a
Merge "Enable Beginning-of-Sentence prediction for contextual dict."
2014-06-10 22:31:57 +00:00
59ed0c2db2
Block offensive words in native code.
...
Bug: 15300452
Change-Id: Ic6c91c47453e19fb5db2c62b66659619e72b1ecf
2014-06-12 11:36:00 +09:00
07e1412631
Purge dict reading code for 401 and create for 402.
...
Change-Id: Ib0150b3306bb2db8aff443d1158fecc8464cbbdb
2014-06-10 19:22:40 +09:00
d9b8602f48
Implement getMaxProbabilityOfExactMatches().
...
Bug: 13142176
Bug: 15428247
Change-Id: I5be6d683be95505412615ca7c88260de1ea05f54
2014-06-06 17:37:46 +09:00
7852765a50
Add length check for attaching a BoS marker.
...
Bug: 15297872
Change-Id: I7b0a27409b81b76561d19c6caa18a5011d3e90f6
2014-05-28 20:20:39 +09:00
f9ce867d80
Add boundary check for v4 bigram reading.
...
Bug: 14496386
Change-Id: Iedd3445c3222a777a2476beed7d9eb53773f406c
2014-05-27 19:29:35 +09:00
dfca51726e
Return whether the dynamic dict operation was success.
...
Bug: 12184250
Change-Id: Iee7e00c1e84c95551a077f4dd023c0a9b9ac9466
2014-05-27 17:30:00 +09:00
a6278eb9c1
Use whether it's exact match to detect distracters.
...
Bug: 13142176
Change-Id: Id5b7286d28897931f7bfe571be45d46ffeef4adf
2014-05-26 21:56:20 +09:00
a37f374ad1
Version up dynamic dict format from 401 to 402.
...
Change-Id: Ibea36af905ade773ae3db3a5456f7b5a0ad7d220
2014-05-23 20:20:56 +09:00
1adca93381
Extend jni methods and enable Beginning-of-Sentence.
...
Bug: 14119293
Change-Id: I78fc877367dd0d6240eeacb750b6d2d0b93cba83
2014-05-23 19:58:58 +09:00
96990ca773
Support Beginning-of-Sentence in native code
...
Bug: 14119293
Change-Id: I0f382e33a19bf481823b23405d454de61ec835ff
2014-05-23 17:23:08 +09:00
6600340af5
Fix: PtNodeParams.representsNonWordInfo()
...
Change-Id: I7ab4fb280e8ac89f81d0b32b0663a86c9b69aa5d
2014-05-23 13:49:18 +09:00
2a015dcb25
Add Beginning-of-Sentence info in UnigramProperty.
...
Bug: 14119293
Bug: 14425059
Change-Id: I8a894352568377d32468e5563f4e89af00d22944
2014-05-23 00:07:14 +09:00
9f8c9a0161
Use PrevWordsInfo to add/remove n(bi)-gram in native code.
...
Bug: 14119293
Bug: 14425059
Change-Id: I4b9a46bfd670b35195418eaee51456d44fb91b6d
2014-05-21 18:30:34 +09:00
fa7db65dec
Support multiple previous words in DicNode.
...
Bug: 14425059
Change-Id: Ib8682befe4d7d9fe5122eb538e7c804f75ded463
2014-05-19 11:47:10 +09:00
45d1a936a7
Move prev word related logic to PrevWordsInfo.
...
Bug: 14119293
Bug: 14425059
Change-Id: I1bbd7ab4ace2c475f27bc468cb7b4d67e1ae2f9f
2014-05-16 22:31:45 +09:00
b94ec1437b
Introduce MAX_PREV_WORD_COUNT_FOR_N_GRAM.
...
Bug: 14425059
Change-Id: I587f90df026a14359d2ee452040bbfaf02c1ae51
2014-05-15 18:45:49 +09:00
b87fffb8be
Refactoring: Move prev word information into PrevWordsInfo.
...
Bug: 14119293
Bug: 14425059
Change-Id: I5a5f81c6b189e3ad1da093343a3121463f87c21c
2014-05-15 18:39:23 +09:00
d6fdd58cc0
Cleanup DicTraverseSession.
...
Change-Id: I953665e1958e7214e9365e2bc9d3f3843016dc81
2014-05-15 17:41:23 +09:00
e810a266fd
Merge "Quit use bigram probability diff for ver4 dict."
2014-05-15 07:07:25 +00:00
64341927d2
Quit use bigram probability diff for ver4 dict.
...
Change-Id: I2cfcfbcf351877d1dff466a24974dbb05908f14e
2014-05-15 16:02:58 +09:00
a709bc7f93
Merge "Reduce the cost for handling intentional omission."
2014-05-15 03:31:39 +00:00
39e5a15af0
Merge "Always keep PtNodes that represent non-word info during GC."
2014-05-15 02:38:57 +00:00
91d7538225
Merge "Add VERSION_4_DEV(402) in native code."
2014-05-14 13:40:10 +00:00
7116ea98f4
Add VERSION_4_DEV(402) in native code.
...
Bug: 13406708
Change-Id: I96cfacf524d670a6a5637a96a63bcd47aaf09ca0
2014-05-14 22:37:05 +09:00
cf700695ea
Always keep PtNodes that represent non-word info during GC.
...
Bug: 14119293
Change-Id: Ie50b341a512be6a6efe3e05b60e1d09560b01d1f
2014-05-14 22:29:21 +09:00
2dc8e36f8e
Use nullptr instead of MmappedBufferPtr(nullptr).
...
Change-Id: I9bc35f723f1c105ef6155a49630b8cd492900bc8
2014-05-14 22:05:00 +09:00
32bdf9f344
Merge "Use JniDataUtils::putXxxToArray() to output a value."
2014-05-14 11:12:09 +00:00
5c00bde9bc
Merge "Filter out control characters and non-unicode code points."
2014-05-14 11:07:32 +00:00
304a71d171
Use JniDataUtils::putXxxToArray() to output a value.
...
Change-Id: Ifb7bed6623b01acb524566a0fe00dcbdf7273888
2014-05-14 20:05:15 +09:00
e9121a68a6
Filter out control characters and non-unicode code points.
...
Bug: 14119293
Bug: 13758761
Change-Id: I6b79cff0714152807f5e20b7a75060a0a772b28b
2014-05-14 20:05:10 +09:00
728141d55a
Merge "Generate dict code for version 401."
2014-05-14 10:51:12 +00:00
6bf268132d
Generate dict code for version 401.
...
Bug: 13406708
Change-Id: I769ac84d54d997e7aefab0c9c16727455a132e0b
2014-05-14 19:09:01 +09:00
72d6061efb
Merge "Use JniDataUtils::outputCodePoints() to output code points"
2014-05-14 09:07:45 +00:00
ed16af72e3
Restrict the permission of dict files created on device.
...
Bug: 13497304
Change-Id: I7bf67e2e81bc63a67bd91cbc0f753ecac88349f6
2014-05-14 17:48:28 +09:00
d2f7cd28ae
Use JniDataUtils::outputCodePoints() to output code points
...
Bug: 14119293
Bug: 13758761
Change-Id: I516d2f96e219e6e65246f502186ccb9bd5f0670f
2014-05-14 17:42:47 +09:00
c1e86d3f49
Merge "Skip PtNodes with non-Unicode code points for suggestion."
2014-05-14 06:22:22 +00:00
14dd663fe5
Reduce the cost for handling intentional omission.
...
Bug: 14602770
Change-Id: I3a678a3e9b227fc4c3b8d23012aba18ca2e52da5
2014-05-14 13:54:32 +09:00
0c8ce96bec
Add "// namespace latinime"
...
Bug: 13406708
Change-Id: I0fa85875fb5b98b4e40a149e3e33e5f56bee9a97
2014-05-13 15:16:43 +09:00
6e4b674f83
Move files under bigram/shortcut to under structure.
...
Bug: 13406708
Change-Id: I48e9d4ff30a8b50d4f1694ebbefb02cf189f7713
2014-05-13 15:16:43 +09:00
d0d0113983
Code cleanup.
...
Change-Id: Ic31b66cccfd77348f2ff7961959a3eaf9f7ad5d8
2014-05-13 01:32:38 +09:00
620a05ae59
Support bigram historical information migration.
...
Bug: 13406708
Change-Id: I4bae53e43cb7653eac3b5bd13da2d2bc8aaf88a9
2014-05-13 01:28:30 +09:00
79ba633402
Skip PtNodes with non-Unicode code points for suggestion.
...
Bug: 14119293
Change-Id: Id1d3b789b5f18757070878dba35a7980bfb44591
2014-05-12 19:38:26 +09:00
9d7e8c717f
Support unigram historical information migration.
...
Bug: 13406708
Change-Id: Ibed15b3bc5d5ae68faefa379028dbe10d32b0c0f
2014-05-12 19:21:06 +09:00
6b74f516dc
Merge "Refactoring: Use UnigramProperty to add/update unigram."
2014-05-09 11:39:17 +00:00
b7f71cdf8c
Merge "Avoid copying bigram list if possible."
2014-05-09 10:09:54 +00:00
b636e25e95
Refactoring: Use UnigramProperty to add/update unigram.
...
Bug: 13406708
Change-Id: I26fd541fb465d3543faa5f155becc455ddbb6c9c
2014-05-09 17:22:17 +09:00
eaa347bc1a
Merge "Add VERSION4_TEST for testing."
2014-05-09 07:06:06 +00:00
455dc84cf2
Add VERSION4_TEST for testing.
...
Bug: 13406708
Change-Id: I0b434717ea3edbb131650af634c53cfbfe113c3e
2014-05-09 15:17:01 +09:00
08ea74eb2c
Merge "Implement migrateNative()."
2014-05-08 05:21:06 +00:00
ad518d9a5b
Avoid copying bigram list if possible.
...
Constructing en_US main dict using dicttool:
Before:
real 1m8.699s
user 1m10.600s
sys 0m2.390s
After:
real 0m17.204s
user 0m20.560s
sys 0m0.720s
Bug: 13406708
Change-Id: I3b0476be57e5cb93c6497025b3ffa7064ac326c6
2014-05-08 14:19:33 +09:00
8a9d9de72a
Implement migrateNative().
...
Bug: 13406708
Change-Id: Ie786a68e780baa298fedf82d21390ba45d871a52
2014-05-08 12:31:32 +09:00
ad8f2201e2
Make initial additional buffer size 0.
...
Bug: 14113369
Change-Id: I3b7b3a452544725577101dc3b9bcb85b3e08afe9
2014-05-07 23:44:32 +09:00
8e8a5a58c4
Merge "Add new bigram entry at the tail of existing list."
2014-05-01 11:19:36 +00:00
8d8fb396a0
Add new bigram entry at the tail of existing list.
...
Bug: 13406708
Change-Id: If3162e65fc9aa2c47f046aee528276cb51fad9f4
2014-05-01 19:29:43 +09:00
ab5839ae0b
Merge "Add CODE_POINT_BEGINNING_OF_SENTENCE in native code."
2014-05-01 02:08:27 +00:00
4162cfdc59
Add CODE_POINT_BEGINNING_OF_SENTENCE in native code.
...
Bug: 14119293
Change-Id: Ib101435e2840172ba004ead081022669647a8ad9
2014-04-30 20:40:02 +09:00
fc2858c55b
Small clean up for LARGEST_INT_DIGIT_COUNT
...
Change-Id: I4a61c037370bf57b396d80ba7212d94328eaf7b9
2014-04-30 19:41:42 +09:00
8dac7ce2e2
Move PatriciaTrieReadingUtils under pt_common.
...
Change-Id: Ib8ec62b97318563785a2aafb854aabdc3f46f187
2014-04-30 14:28:08 +09:00
9e15b4551e
Small clean up on MAX_LOCALE_STRING_LENGTH
...
Change-Id: I77f93879e841f33b81f6d193d1bd5782ad2124f5
2014-04-28 16:50:06 +09:00
3b7984752c
Support creating BinaryDictionary without creating dict file.
...
Bug: 14166482
Change-Id: Ic8c78ec2c8f39358a1f085a041b608972a380eef
2014-04-22 13:18:34 -07:00
ce76821dce
Revert "Support creating BinaryDictionary without creating dict file."
...
This reverts commit 89939b8708
2014-04-22 19:34:44 +00:00
89939b8708
Support creating BinaryDictionary without creating dict file.
...
Bug: 14166482
Change-Id: Ib065279f96e227ab0fee7c8141560c4ada744d3b
2014-04-22 12:01:33 -07:00
903be5bbd3
Implement latinime_BinaryDictionary_createOnMemory().
...
Bug: 14166482
Change-Id: If7ec3345ab34edcd6bc5cef9e72580ced894a0e3
2014-04-22 12:01:22 -07:00
edc7d02d76
Separate JniDataUtils::constructAttributeMap().
...
Bug: 14166482
Change-Id: Ic2dc407231743ea668fe0ab8869eb8a7f8b9625f
2014-04-21 19:59:21 -07:00
7b43d987fa
Pass max trie structure size to on memory Ver4DictBuffers.
...
Bug: 14166482
Bug: 14161647
Change-Id: I3b0dd192d7a81c3809f1873bb66ef8fe637f8af0
2014-04-18 17:57:28 -07:00
d4e54af0ba
Merge "Make addUnigramWord use UnigramProperty."
2014-04-14 11:03:30 +00:00
793124855d
Make addUnigramWord use UnigramProperty.
...
Bug: 13406708
Change-Id: I10c108dd7047fe230c60495f13800f32fdc46d0d
2014-04-14 18:49:19 +09:00
8ca9be17db
s/hash_map_compat/unordered_map/
...
Change-Id: Icce5f9a12b04bdd7540c52750d303a585d71f28a
2014-04-11 18:07:59 +09:00
63d33f42c8
Increase terminal cache size for gesture.
...
Before:
(0) 670.00 (10.52%)
(1) 5530.00 (86.81%)
(2) 120.00 (1.88%)
Total 6370.00 (sum of others 6320.00)
After:
(0) 730.00 (11.06%)
(1) 5750.00 (87.12%)
(2) 100.00 (1.52%)
Total 6600.00 (sum of others 6580.00)
Bug: 13773693
Bug: 10701902
Bug: 9505397
Change-Id: Ie53640994180a7acabe05da544ab2166403e2394
2014-04-09 16:50:31 +09:00
bc9500fb14
Quit limiting gesture search space by distance.
...
Before:
(0) 470.00 (8.25%)
(1) 5160.00 (90.53%)
(2) 50.00 (0.88%)
(66) 10.00 (0.18%)
Total 5700.00 (sum of others 5690.00)
After:
(0) 670.00 (10.52%)
(1) 5530.00 (86.81%)
(2) 120.00 (1.88%)
Total 6370.00 (sum of others 6320.00)
Bug: 13773693
Bug: 10701902
Bug: 9505397
Change-Id: Ie168ac4ae0ca0e954512b08e3cc0f6399a3c163d
2014-04-09 16:50:31 +09:00
26c806620c
Use 2D normal distribution for gesture.
...
Bug: 13799846
Bug: 10701902
Bug: 9505397
Change-Id: I6c3f84f035f2310f2f7dfec4432ebdb6e50d5df0
2014-04-09 16:50:31 +09:00
e3d57ae792
Merge "Use bitset for BloomFilter."
2014-04-09 07:39:49 +00:00
2fd0bf9a37
Use bitset for BloomFilter.
...
Before:
(0) 660.00 (1.43%)
(1) 45320.00 (98.18%)
(2) 80.00 (0.17%)
Total 46160.00 (sum of others 46060.00)
After:
(0) 620.00 (1.34%)
(1) 45310.00 (98.05%)
(2) 130.00 (0.28%)
Total 46210.00 (sum of others 46060.00)
Change-Id: I936b639c50e15208aee999a929b33983c6caa59d
2014-04-09 16:18:23 +09:00
b417d7d69f
Make NELEMS type-safe
...
Change-Id: I867271665b3c595dc40ac63e93e68c87dc213d08
2014-04-09 15:51:39 +09:00
e41b2ed8d3
Separate unigram/bigram property from WordProperty.
...
Bug: 13406708
Change-Id: I48e9fccedd9dcdc1a35ffe027745b58966a83315
2014-04-08 18:24:06 +09:00
cafab169cd
s/stdint.h/cstdint/
...
Somehow, we were not able to use cstdint gcc target build. That's no
longer the case with Clang.
Removed unnecessary header inclusions too.
Change-Id: Ic83a4adf696f1d5ec7a9809253f3c95804e622e1
2014-04-07 12:39:07 +09:00
3bf9647a26
Separate NormalDistribution from ProximityInfoUtils.
...
Bug: 13773693
Change-Id: I9e835cc92e16f8ae0e0492af0715d0ef5cd84a35
2014-04-04 18:49:12 +09:00
5f6a247744
Remove unused code.
...
Bug: 13773693
Change-Id: Ic9d7f81b0eed40e945b04a91fd97ba40fa1cd262
2014-04-03 14:09:25 +09:00
de2b312c6d
Use same language weight for all dictionaries.
...
Bug: 8187060
Change-Id: Ib9d8a8aed2c141137c1bb3c748a89fb8216293e7
2014-04-03 01:43:36 +09:00
77de3a4b65
Change wrongly used "lookahead" to "completion".
...
Change-Id: I111cf59d0d24fa869217788c314fffa94ee9f9f9
2014-04-02 12:44:41 +09:00
543fea98a4
Merge "Parameter tuning for typing."
2014-04-01 09:45:17 +00:00
7dc7a9a398
Parameter tuning for typing.
...
accidently -> accidentally (top suggest)
convienent -> convenient (top suggest)
dissapear -> disappear (top suggest)
Execution time: +31%
Before:
(0) 610.00 (1.66%)
(1) 35820.00 (97.66%)
(2) 130.00 (0.35%)
Total 36680.00 (sum of others 36560.00)
After:
(0) 820.00 (1.69%)
(1) 47200.00 (97.50%)
(2) 210.00 (0.43%)
Total 48410.00 (sum of others 48230.00)
Bug: 13588483
Bug: 13456384
Bug: 9816611
Change-Id: Ia17491e135a63aaca6e980c3a59bd829441dac40
2014-04-01 17:25:43 +09:00
da93bc54b2
Merge "Remove useless methods from DicNode."
2014-04-01 05:40:29 +00:00
94c7bd471f
Merge "Separate DicNodePool from DicNodePriorityQueue."
2014-04-01 05:36:35 +00:00
ef1d4ddc8f
Remove useless methods from DicNode.
...
Change-Id: I6ea5eb7e848dbce0f0d468160c16e5da76b23b20
2014-04-01 14:09:02 +09:00
67ff21f321
Separate DicNodePool from DicNodePriorityQueue.
...
Change-Id: Ie500a4185792c2ffa0f5185ea5919626fe31c6fe
2014-04-01 14:08:59 +09:00
ff1b3947c6
Split SuggestionsOutputUtils::outputSuggestions.
...
Bug: 13333066
Change-Id: Ie5e513dacdc5502e7263ddf709824bbd6bc6b74a
2014-03-28 20:11:27 +09:00
Keisuke Kuroyanagi
Ken Wakasa
Yohei Yukawa