96d47fe745
Change getUnigramProbabilityOfPtNode to getProbabilityOfPtNode()
...
Bug: 14425059
Change-Id: I92403d750d2979408b41554dcd79b0ed31b41161
2014-07-29 11:35:57 +09:00
179f38cc8c
Merge "Change getUnigramProbabilityOfPtNode to getProbabilityOfPtNode()" into lmp-dev
2014-07-25 11:19:20 +00:00
3225b6fe66
Add boundary check for ver2 bigram reading.
...
Bug: 16330528
Change-Id: I6aca6c7a735e2a652eb325572d44dff660789cff
2014-07-28 21:48:00 +09:00
d4c9d50ef0
Add BufferWithExtendablebufferTest
...
Bug: 13754552
Change-Id: Ib9c330dc85bdb5d4d3d5bc15cc3ce5f28faa0b12
2014-07-25 07:02:10 +09:00
ff6ca09905
Add BufferWithExtendableBuffer::extend()
...
Bug: 14425059
Change-Id: I13dd8412ba4d16a1325ac2210b3852d580e71ad3
2014-07-25 07:02:05 +09:00
6810e8df1e
Refactoring checkAndPrepareWriting.
...
Bug: 14425059
Change-Id: I98940f3841a6c61b0482440b4da21f415dc843f6
2014-07-24 10:24:42 +09:00
b22f95ec8a
Remove isUpdatable from constructors of dict contents.
...
Change-Id: I2d54f477d9b341e944e265786a734f23d152bb81
2014-07-11 15:23:55 +09:00
2ac934296c
Concatenate dict buffers other than header to a single file.
...
Bug: 13664080
Change-Id: I34c9d8046b339c9b855be378a5fad907382d1359
2014-07-11 15:15:47 +09:00
198be3a6c5
Fix: wrong bigram entry pos can be used during GC.
...
Bug: 14425059
Change-Id: I4e7c977c5ef184953682175b118de5473cdb64a6
2014-07-09 11:18:35 +09:00
b00973952f
Get bigrams iterator via dict structure policy.
...
Bug: 14425059
Change-Id: Ie4c6afbb6a3b707b5aa8e9e4f941c9da6e6bcdbf
2014-07-08 16:23:31 +09:00
804f7450fc
Use linked list for bigram list.
...
BinaryDictionaryTests for VERSION4_DEV:
Before
Time: 36.461
After
Time: 33.031
Bug: 14425059
Change-Id: I9ca2714f450f61f713df6ebd34c953dece991cdb
2014-07-07 21:09:25 +09:00
6afd14dc94
Remove unused method.
...
Bug: 13664080
Change-Id: I47fd90f01ea9a18b11037c966150fa5fd926101b
2014-06-27 15:23:38 +09:00
8681bef03c
Remove BigramDictionary form native code.
...
Bug: 14425059
Change-Id: I48d1a4ae6a4317afaaf67748060618f8f9485d40
2014-06-26 15:15:13 +09:00
f7322b166b
Make getNextWordAndNextToken output code point count.
...
Bug: 14425059
Change-Id: Id1c2927c8a1be0b1680206d444dbdf0c9453dceb
2014-06-23 19:08:30 +09:00
24d49ee577
Remove hasBigrams/hasShortcut flags from ver4 PtNode.
...
Always check bigram/shortcut content instead of having
flags in each PtNode.
Bug: 14425059
Change-Id: I9eb816681bf22fc12e3ab3c92a178a7990b90e93
2014-06-23 16:49:49 +09:00
0fbca1ac23
Remove word from personalized dicts when it's canceled.
...
Bug: 15555552
Bug: 15552347
Change-Id: Ia4d8c79d079e3ce233d46d4bbad4fff1be5c5c39
2014-06-20 14:46:13 +09:00
89899e14df
Merge "Update unit test for beginning-of-sentence."
2014-06-10 22:31:57 +00:00
e041bfab5c
Update unit test for beginning-of-sentence.
...
Bug: 14119293
Change-Id: I594a1ca1096f0c12c678dd30eb75de905e55c5bb
2014-06-12 16:07:39 +09:00
22931cd941
Enable Beginning-of-Sentence prediction for contextual dict.
...
Bug: 14161647
Bug: 14119293
Change-Id: I0c00f13966db88e4de85e245e7bced43c9d474b2
2014-06-12 12:26:18 +09:00
86fe8081b3
Merge "Add removeUnigramEntry() to native dictionary policy."
2014-06-10 22:29:12 +00:00
f12985245c
Add removeUnigramEntry() to native dictionary policy.
...
Bug: 15552347
Change-Id: Ic713f79ef26b5963060304df9c0a3a402c3cc6d4
2014-06-12 12:14:11 +09:00
07e1412631
Purge dict reading code for 401 and create for 402.
...
Change-Id: Ib0150b3306bb2db8aff443d1158fecc8464cbbdb
2014-06-10 19:22:40 +09:00
f9ce867d80
Add boundary check for v4 bigram reading.
...
Bug: 14496386
Change-Id: Iedd3445c3222a777a2476beed7d9eb53773f406c
2014-05-27 19:29:35 +09:00
dfca51726e
Return whether the dynamic dict operation was success.
...
Bug: 12184250
Change-Id: Iee7e00c1e84c95551a077f4dd023c0a9b9ac9466
2014-05-27 17:30:00 +09:00
a37f374ad1
Version up dynamic dict format from 401 to 402.
...
Change-Id: Ibea36af905ade773ae3db3a5456f7b5a0ad7d220
2014-05-23 20:20:56 +09:00
1adca93381
Extend jni methods and enable Beginning-of-Sentence.
...
Bug: 14119293
Change-Id: I78fc877367dd0d6240eeacb750b6d2d0b93cba83
2014-05-23 19:58:58 +09:00
96990ca773
Support Beginning-of-Sentence in native code
...
Bug: 14119293
Change-Id: I0f382e33a19bf481823b23405d454de61ec835ff
2014-05-23 17:23:08 +09:00
6600340af5
Fix: PtNodeParams.representsNonWordInfo()
...
Change-Id: I7ab4fb280e8ac89f81d0b32b0663a86c9b69aa5d
2014-05-23 13:49:18 +09:00
2a015dcb25
Add Beginning-of-Sentence info in UnigramProperty.
...
Bug: 14119293
Bug: 14425059
Change-Id: I8a894352568377d32468e5563f4e89af00d22944
2014-05-23 00:07:14 +09:00
9f8c9a0161
Use PrevWordsInfo to add/remove n(bi)-gram in native code.
...
Bug: 14119293
Bug: 14425059
Change-Id: I4b9a46bfd670b35195418eaee51456d44fb91b6d
2014-05-21 18:30:34 +09:00
e810a266fd
Merge "Quit use bigram probability diff for ver4 dict."
2014-05-15 07:07:25 +00:00
64341927d2
Quit use bigram probability diff for ver4 dict.
...
Change-Id: I2cfcfbcf351877d1dff466a24974dbb05908f14e
2014-05-15 16:02:58 +09:00
39e5a15af0
Merge "Always keep PtNodes that represent non-word info during GC."
2014-05-15 02:38:57 +00:00
91d7538225
Merge "Add VERSION_4_DEV(402) in native code."
2014-05-14 13:40:10 +00:00
7116ea98f4
Add VERSION_4_DEV(402) in native code.
...
Bug: 13406708
Change-Id: I96cfacf524d670a6a5637a96a63bcd47aaf09ca0
2014-05-14 22:37:05 +09:00
cf700695ea
Always keep PtNodes that represent non-word info during GC.
...
Bug: 14119293
Change-Id: Ie50b341a512be6a6efe3e05b60e1d09560b01d1f
2014-05-14 22:29:21 +09:00
2dc8e36f8e
Use nullptr instead of MmappedBufferPtr(nullptr).
...
Change-Id: I9bc35f723f1c105ef6155a49630b8cd492900bc8
2014-05-14 22:05:00 +09:00
728141d55a
Merge "Generate dict code for version 401."
2014-05-14 10:51:12 +00:00
6bf268132d
Generate dict code for version 401.
...
Bug: 13406708
Change-Id: I769ac84d54d997e7aefab0c9c16727455a132e0b
2014-05-14 19:09:01 +09:00
ed16af72e3
Restrict the permission of dict files created on device.
...
Bug: 13497304
Change-Id: I7bf67e2e81bc63a67bd91cbc0f753ecac88349f6
2014-05-14 17:48:28 +09:00
c1e86d3f49
Merge "Skip PtNodes with non-Unicode code points for suggestion."
2014-05-14 06:22:22 +00:00
0c8ce96bec
Add "// namespace latinime"
...
Bug: 13406708
Change-Id: I0fa85875fb5b98b4e40a149e3e33e5f56bee9a97
2014-05-13 15:16:43 +09:00
6e4b674f83
Move files under bigram/shortcut to under structure.
...
Bug: 13406708
Change-Id: I48e9d4ff30a8b50d4f1694ebbefb02cf189f7713
2014-05-13 15:16:43 +09:00
d0d0113983
Code cleanup.
...
Change-Id: Ic31b66cccfd77348f2ff7961959a3eaf9f7ad5d8
2014-05-13 01:32:38 +09:00
620a05ae59
Support bigram historical information migration.
...
Bug: 13406708
Change-Id: I4bae53e43cb7653eac3b5bd13da2d2bc8aaf88a9
2014-05-13 01:28:30 +09:00
79ba633402
Skip PtNodes with non-Unicode code points for suggestion.
...
Bug: 14119293
Change-Id: Id1d3b789b5f18757070878dba35a7980bfb44591
2014-05-12 19:38:26 +09:00
9d7e8c717f
Support unigram historical information migration.
...
Bug: 13406708
Change-Id: Ibed15b3bc5d5ae68faefa379028dbe10d32b0c0f
2014-05-12 19:21:06 +09:00
6b74f516dc
Merge "Refactoring: Use UnigramProperty to add/update unigram."
2014-05-09 11:39:17 +00:00
b7f71cdf8c
Merge "Avoid copying bigram list if possible."
2014-05-09 10:09:54 +00:00
b636e25e95
Refactoring: Use UnigramProperty to add/update unigram.
...
Bug: 13406708
Change-Id: I26fd541fb465d3543faa5f155becc455ddbb6c9c
2014-05-09 17:22:17 +09:00
eaa347bc1a
Merge "Add VERSION4_TEST for testing."
2014-05-09 07:06:06 +00:00
455dc84cf2
Add VERSION4_TEST for testing.
...
Bug: 13406708
Change-Id: I0b434717ea3edbb131650af634c53cfbfe113c3e
2014-05-09 15:17:01 +09:00
08ea74eb2c
Merge "Implement migrateNative()."
2014-05-08 05:21:06 +00:00
ad518d9a5b
Avoid copying bigram list if possible.
...
Constructing en_US main dict using dicttool:
Before:
real 1m8.699s
user 1m10.600s
sys 0m2.390s
After:
real 0m17.204s
user 0m20.560s
sys 0m0.720s
Bug: 13406708
Change-Id: I3b0476be57e5cb93c6497025b3ffa7064ac326c6
2014-05-08 14:19:33 +09:00
8a9d9de72a
Implement migrateNative().
...
Bug: 13406708
Change-Id: Ie786a68e780baa298fedf82d21390ba45d871a52
2014-05-08 12:31:32 +09:00
ad8f2201e2
Make initial additional buffer size 0.
...
Bug: 14113369
Change-Id: I3b7b3a452544725577101dc3b9bcb85b3e08afe9
2014-05-07 23:44:32 +09:00
8e8a5a58c4
Merge "Add new bigram entry at the tail of existing list."
2014-05-01 11:19:36 +00:00
8d8fb396a0
Add new bigram entry at the tail of existing list.
...
Bug: 13406708
Change-Id: If3162e65fc9aa2c47f046aee528276cb51fad9f4
2014-05-01 19:29:43 +09:00
fc2858c55b
Small clean up for LARGEST_INT_DIGIT_COUNT
...
Change-Id: I4a61c037370bf57b396d80ba7212d94328eaf7b9
2014-04-30 19:41:42 +09:00
8dac7ce2e2
Move PatriciaTrieReadingUtils under pt_common.
...
Change-Id: Ib8ec62b97318563785a2aafb854aabdc3f46f187
2014-04-30 14:28:08 +09:00
3b7984752c
Support creating BinaryDictionary without creating dict file.
...
Bug: 14166482
Change-Id: Ic8c78ec2c8f39358a1f085a041b608972a380eef
2014-04-22 13:18:34 -07:00
ce76821dce
Revert "Support creating BinaryDictionary without creating dict file."
...
This reverts commit 89939b8708
2014-04-22 19:34:44 +00:00
89939b8708
Support creating BinaryDictionary without creating dict file.
...
Bug: 14166482
Change-Id: Ib065279f96e227ab0fee7c8141560c4ada744d3b
2014-04-22 12:01:33 -07:00
903be5bbd3
Implement latinime_BinaryDictionary_createOnMemory().
...
Bug: 14166482
Change-Id: If7ec3345ab34edcd6bc5cef9e72580ced894a0e3
2014-04-22 12:01:22 -07:00
7b43d987fa
Pass max trie structure size to on memory Ver4DictBuffers.
...
Bug: 14166482
Bug: 14161647
Change-Id: I3b0dd192d7a81c3809f1873bb66ef8fe637f8af0
2014-04-18 17:57:28 -07:00
d4e54af0ba
Merge "Make addUnigramWord use UnigramProperty."
2014-04-14 11:03:30 +00:00
793124855d
Make addUnigramWord use UnigramProperty.
...
Bug: 13406708
Change-Id: I10c108dd7047fe230c60495f13800f32fdc46d0d
2014-04-14 18:49:19 +09:00
8ca9be17db
s/hash_map_compat/unordered_map/
...
Change-Id: Icce5f9a12b04bdd7540c52750d303a585d71f28a
2014-04-11 18:07:59 +09:00
e41b2ed8d3
Separate unigram/bigram property from WordProperty.
...
Bug: 13406708
Change-Id: I48e9fccedd9dcdc1a35ffe027745b58966a83315
2014-04-08 18:24:06 +09:00
cafab169cd
s/stdint.h/cstdint/
...
Somehow, we were not able to use cstdint gcc target build. That's no
longer the case with Clang.
Removed unnecessary header inclusions too.
Change-Id: Ic83a4adf696f1d5ec7a9809253f3c95804e622e1
2014-04-07 12:39:07 +09:00
cf88cf6593
Add a check for ForgettingCurveUtils::decodeProbability.
...
Bug: 13607383
Change-Id: I1a56c52173d0e458864e65fb1c1f4603c0c81c6f
2014-03-24 11:11:07 +00:00
ef665816d0
Add boundary check for SparseTable.
...
Bug: 13592288
Change-Id: I8bc4ff778c6073d22f86b6050a880b6fb69de31e
2014-03-24 09:17:53 +00:00
4c93770430
Fix: wrong size checking in SparseTable.
...
Bug: 13592288
Change-Id: I52f03081c5abf25e3c160e2b8fff5e98f8448499
2014-03-24 14:28:36 +09:00
73a2426d45
Change default decaying parameters.
...
Bug: 13458617
Change-Id: I4942af4a48881fb6ca866c74140b262e6d323f4a
2014-03-14 12:23:24 +09:00
4bc35daaa2
Check "." and ".." to delete all files in a dir.
...
Bug: 13421999
Change-Id: I9d0f65cfc9a0f18c7affc152a5b54f9970048e9c
2014-03-12 15:31:49 +09:00
ca42ec0f44
Use emplace_back in getWordProperty().
...
Change-Id: If164d2b897bd6ad9c89baf0e9a90356100c5087b
2014-03-11 15:21:55 +09:00
a34bdc395b
Make getWordProperty return reconstructed bigram probability.
...
Bug: 13403530
Change-Id: I7ed95a54344d8b77b88922093cf3ef0e530fcae6
2014-03-11 15:20:12 +09:00
c36c97b4f7
Merge "Use unique_ptr."
2014-03-07 09:06:58 +00:00
6767f88e3c
Address warnings pointed out by clang
...
Change-Id: I7943f6c7f9ba08f283a3116adc2af29495f04a21
2014-03-07 17:05:09 +09:00
4ce480d5ce
Use unique_ptr.
...
Change-Id: Id92a5b07da4f7f95e2cd293ce8dc1a5f979b7853
2014-03-07 14:31:54 +09:00
865e6cf497
Revert "Revert "Use std::min() and std::max()""
...
This reverts commit f63000abea
2014-03-07 14:19:56 +09:00
6c7ebc4023
Use nullptr instead of 0 for pointers.
...
Change-Id: Ic9425de07b896f940ca11c51968c637ebb897514
2014-03-07 10:04:44 +09:00
5128935ac4
Add header attributes for evaluation.
...
Bug: 13197276
Change-Id: Ib5247da691ff24a73e13485288237ccc51bb54f0
2014-02-28 21:06:03 +09:00
3d70932857
Use MAX_PROBABILITY for the limit of probabilities.
...
Bug: 13197276
Change-Id: Ic3f855e566603b395395dff1f860f3d76ecb019c
2014-02-26 20:48:53 +09:00
7919a594fc
Merge "Create more aggressive probability tables."
2014-02-26 11:18:44 +00:00
b368089dbf
Stop dividing unigram probability by 2 for backoff.
...
Bug: 13197276
Change-Id: I8eaa24b6f710383a8d257e3ec28c37a1a1da8e31
2014-02-26 19:40:18 +09:00
70566266be
Create more aggressive probability tables.
...
Bug: 13197276
Change-Id: I1451dcbe61088d7301bbef5ceedc72f5649e6764
2014-02-26 19:36:57 +09:00
2b27e06733
Merge "Read and write forgetting curve header attributes."
2014-02-26 09:06:56 +00:00
6112cc26a7
Read and write forgetting curve header attributes.
...
Bug: 13197276
Change-Id: I6c38a127e1745903594b7bf0ec80ef179e794feb
2014-02-26 18:04:12 +09:00
94080a37e8
Merge "Check header attributes for forgetting curve."
2014-02-26 08:58:48 +00:00
57816c7a8b
Check header attributes for forgetting curve.
...
Bug: 13197276
Change-Id: Ib360010f309fd1ed3e81cf23aa10c1cfe82781a6
2014-02-26 17:41:31 +09:00
7b496a5861
Add boundary check for ver4 bigram/shortcut reading.
...
Bug: 13185316
Change-Id: I5236222e2a5acda69821062a140c9e36a3024bd2
2014-02-26 14:03:24 +09:00
be81b75dec
Add boundary check for ver2 dict reading.
...
Bug: 12916055
Change-Id: I78ad1f98a5401f920dcfc3379aa431eb2311ae02
2014-02-21 13:26:01 +09:00
b96012acef
Record reading error during traversing dictionaries.
...
Bug: 12916055
Change-Id: I439f21aa87cfa1f71e0354370a1514af03d68f6c
2014-02-20 19:12:32 +09:00
303e767728
Fix: crash when opening a broken dictionary.
...
Bug: 13085169
Change-Id: Icfb6184dfefc6a336432203c071d9e30ae8bf990
2014-02-19 22:11:09 +09:00
0fc93fe445
Implement PatriciaTriePolicy::getNextWordAndNextToken().
...
Bug: 12810574
Change-Id: Id1d44f90de9455d9cbe7b6e0a161cae91d6d422c
2014-02-15 17:39:20 +09:00
c63d183473
Implement PatriciaTriePolicy::getWordProperty().
...
Bug: 12810574
Change-Id: I7bcccfd3641ebbcf2b8d857d33bb4734c42af5eb
2014-02-14 17:56:45 +09:00
be61170588
Create Ver2PtNodeArrayReader.
...
Bug: 12810574
Change-Id: I7708d24d735680b2fe9e6700316076018e88c98d
2014-02-10 21:09:55 +09:00
cf2b075146
Stop passing buffer directly to DynamicPtReadingHelper.
...
Bug: 12810574
Change-Id: I656d99f504d8cf4e057080713b174943afd9dd4c
2014-02-10 21:06:09 +09:00
1e2752924d
Create Ver2ParticiaTrieNodeReader.
...
Bug: 12810574
Change-Id: I7d3298b5f419d557755ae433c8b8cc0d145f4cc3
2014-02-10 21:06:09 +09:00
1d6afa179c
Refactoring: extract PtNode array reading logic form helper.
...
Bug: 12810574
Change-Id: I2d2660871862f11630c3ad7bf04bb49ade57c1e7
2014-02-10 21:06:07 +09:00
fc9ca59cc1
Implement getHeaderInfoNative.
...
Bug: 11281877
Bug: 12810574
Change-Id: Ia3d85ae2cfdb486e74b8636a62431eae883c85da
2014-02-06 18:19:42 +09:00
43cf9076b2
[HD04] Make the locale mandatory.
...
Bug: 11281748
Change-Id: I69281b0053bec404c3e3c713ade3f65a140f51b1
2014-02-05 17:35:25 +09:00
1d3058daa1
Merge "Don't boost exact matches for personalized dicts."
2014-02-04 09:13:48 +00:00
54622d38c2
Don't boost exact matches for personalized dicts.
...
Bug: 12800726
Change-Id: Ia3a2f82db6cdc9645cad0e329c0b79328d7452cc
2014-02-04 17:47:14 +09:00
941734695b
Implement Ver4PatriciaTriePolicy::getNextWordAndNextToken.
...
Bug: 12810574
Change-Id: Idea44f03c477964f58e65fbf2b55e3fcd77a2934
2014-02-04 12:42:14 +09:00
38f341a2a5
Add a jni method to iterate words in a dictionary.
...
Bug: 12810574
Change-Id: I87b13559765f0262dfd06c2ba40d43af6437ea7f
2014-02-03 14:51:58 +09:00
17e7768a3e
Fetch bigram information in getWordProperty().
...
Bug: 12810574
Change-Id: Ic67e964b01c2753f2eec82b079dc0e5650f340bb
2014-01-31 20:24:23 +09:00
20b7786ece
Add BigramProperty and ShortcutProperty in WordProperty.
...
Bug: 12810574
Change-Id: Ia9e497c49bb2cf10897bae3a79317113325db819
2014-01-31 15:26:43 +09:00
b0df28f4cc
Merge "[HD02] Make the date header attribute mandatory."
2014-01-31 03:58:55 +00:00
748f32feba
Merge "Rename UnigramProperty to WordProperty."
2014-01-31 03:06:44 +00:00
e9085da8cb
Merge "Simplify UnigramProperty."
2014-01-31 03:06:33 +00:00
080a35e959
Rename UnigramProperty to WordProperty.
...
Bug: 12810574
Change-Id: If5ddd803948aaf6e491ddcbaa5436fb3af3f7257
2014-01-31 11:06:42 +09:00
a137c21c9c
[HD02] Make the date header attribute mandatory.
...
Bug: 11281748
Change-Id: I48a6f3d95ca89ced0c51335527201ecabfb7998e
2014-01-30 18:19:21 +09:00
5f7f6a1615
Simplify UnigramProperty.
...
Bug: 12810574
Change-Id: I606f559b5dd340b7525280ecead2c2c7ec920a78
2014-01-30 17:10:23 +09:00
324adefca3
[HD01] Small initial refactoring
...
Bug: 11281748
Change-Id: I3c64c75ecd17729bdacf03048311d76ca9f37ae2
2014-01-30 17:08:13 +09:00
51c5ec10f9
Add length check for dict update operations.
...
Bug: 12602903
Change-Id: I6835dee8bf8b356f0f1cf6c0531bee5b3415a13f
2014-01-24 16:32:42 -08:00
d78a447d10
Add a dedicated method to control time in native code.
...
Bug: 12443085
Change-Id: I41a5cf6c895cb59e54af98b40efded36afcd3247
2014-01-15 14:55:57 -08:00
05113c1847
Check whether the word is OOV or not for amanatto update.
...
Bug: 11736529
Change-Id: I27973212da7e922b2185dbb4febfb5bb4dfbdcbb
2014-01-08 21:02:35 +09:00
b5215d456a
Increment version number of ver4 format to 401.
...
Bug: 11073222
Change-Id: I3878aef02040b1ef73165645fc70ede586289c51
2013-12-16 23:02:54 +09:00
9b08a9e611
Fix: incompatibility of sparse table.
...
Bug: 11073222
Change-Id: Id17178efd8f9f3828d821cc718d35606e8bd4b1a
2013-12-16 22:46:29 +09:00
c1163c8518
Align block sizes for ver4 format to java side.
...
Bug: 11073222
Change-Id: I09646cbc81529576d5e34dcfa7bc29083a64076f
2013-12-16 20:42:32 +09:00
a245d15da5
Have dicttool use the native library to generate v4 dicts.
...
Yay !
Change-Id: Iea8ced9e81031b9ab7eff05ad9ef7215be248de9
2013-12-13 18:18:20 +09:00
2fa3693c26
Reset to 9bd6dac470
...
The bulk merge from -bayo to klp-dev should not have been merged to master.
Change-Id: I527a03a76f5247e4939a672f27c314dc11cbb854
2013-12-13 17:13:32 +09:00
6bc5acaa79
Fix: Suggested words from user history are invalid.
...
- Suggestions form user history can contain invalid words.
- isValidWord always returns false.
Bug: 11139426
Change-Id: I6075b275603332ddb00f4a9284afcaa82d824270
2013-10-22 05:23:48 -07:00
6d91e4ce74
Fix offdevice test build.
...
Change-Id: I42143bdc4f6f5aa4f55530de84819ab722a22f72
2013-10-08 16:05:42 +09:00
802e6383ea
Fix a bug: wrong parent position is written when moving.
...
Change-Id: I93d33ba9105eb85327108cca5706d8db3399de6a
2013-10-08 12:44:00 +09:00
cfb018ba6d
Fix bug and Add large test for decaying dictionary.
...
- GC gets failure when the dictionary become empty.
- Useless unigrams are sometimes not removed.
Bug: 10197478
Change-Id: I8d1479c01efba61a81f03bc077da6bcb4797a940
2013-10-07 17:31:01 +09:00
3e5f4b53b0
Merge "Check the elapsed time for decaying."
2013-10-04 07:16:17 +00:00
67c855ea6f
Check the elapsed time for decaying.
...
Bug: 6669677
Change-Id: I0401a4c0908c702ce65abfa5e017dd16cb1296f9
2013-10-04 16:11:11 +09:00
9cb1e78a44
Fix: Native crash when an empty attribute is written.
...
Bug: 10964805
Change-Id: Ia3cd891d20aee2226092f0f646d8f280901786fc
2013-10-03 17:30:54 +09:00
3d92954120
Fix build.
...
Change-Id: Ib33b27214e67378646e44f5b45bda298f960524f
2013-10-02 23:02:42 +09:00
7c4dcf1e91
Use probability table for decaying dictionaries.
...
Bug: 6669677
Change-Id: Ib5994a2e343dfcdaf23755e523f52846709b1c6c
2013-10-02 18:01:47 +09:00
c76bbceedc
Stochastic decay.
...
Bug: 6669677
Change-Id: Ib2d9228b951c77dab7a8675ce9db60677e87e771
2013-10-02 17:51:12 +09:00
13d5dc914a
Change name of DecayingUtils.
...
Bug: 6669677
Change-Id: I97839b2184b95ae1659c55a9b31d22e2d601cba1
2013-10-02 16:23:44 +09:00
6995310996
Separate "GC" and "GC with decay".
...
Bug: 6669677
Change-Id: I9d6aba76cef2616f0549e612db9701e1d6a19467
2013-10-01 18:09:31 +09:00
fd02b2d6ee
Implement simple dictionary decay.
...
Groundwork and implement simple decay.
Increment probability when typed and decrement probability
at GC.
Bug: 6669677
Change-Id: Ib12caead0cbeef4ce7808fe8ac0b00ee331523fe
2013-09-30 20:53:35 +09:00
c18510049a
Prepare dictionary decay.
...
Bug: 6669677
Change-Id: I8fbae190dd44a6bdbee7e9b6d3a16208322727f7
2013-09-30 13:57:54 +09:00
68bc4ba5c2
Merge "Add boundary checking for PtNode Array reading."
2013-09-27 14:25:24 +00:00
31097a57cc
Add unigram/bigram counting.
...
Bug: 6669677
Change-Id: I05ea2201d822dddf062b08c8467daa336760e16c
2013-09-27 23:12:12 +09:00
67cb282b64
Add boundary checking for PtNode Array reading.
...
Bug: 6669677
Change-Id: Iea5e19d98d2fc26f137046dd5f8e339239672351
2013-09-27 22:28:54 +09:00
5ef6209656
Create empty dictionary file in native code.
...
Bug: 6669677
Change-Id: I46d23deee1fd091678f4593561cb8687eb815212
2013-09-26 12:59:02 +09:00
10b56df486
Merge "Prepare dictionary file creating in native code."
2013-09-26 02:38:19 +00:00
c6015327fb
Merge "Add empty PtNode checking for PatriciaTriePolicy."
2013-09-26 02:32:03 +00:00
9155eec0d9
Add empty PtNode checking for PatriciaTriePolicy.
...
Bug: 9503570
Change-Id: I784d29c307c1040699fa0aadfc6a47d97f03ffe4
2013-09-26 11:19:04 +09:00
36f4592128
Merge "Refactoring header attribute reading."
2013-09-26 01:57:22 +00:00
1592eb8a3a
Prepare dictionary file creating in native code.
...
Bug: 6669677
Change-Id: I7c476a6e99ec7ac883f05d84ef306265255154da
2013-09-26 10:48:43 +09:00
989596844e
Refactoring header attribute reading.
...
Bug: 6669677
Change-Id: Ifc11da614d5c331ac61019a324e3a0ff187329cd
2013-09-26 10:45:03 +09:00
009dcac33f
Boundary check for children creating of PatriciaTriePolicy.
...
Bug: 10537529
Change-Id: I0cd8b6de230795498395ca08789ccc1c74780f2c
2013-09-25 19:54:31 +09:00
3d2802ef2e
Allow large memory space for GC.
...
Bug: 6669677
Change-Id: I021c92a1ca5cd28f745cbfe729c99e346c58074d
2013-09-24 19:21:17 +09:00
f3cd38f2c7
GC step 5. Add large tests and fix bugs.
...
Bug: 6669677
Change-Id: I41e44a64730f9c66645995095a7ecff51ffd5351
2013-09-24 16:32:25 +09:00
2cfe7f9e3b
GC step 4. Update all positions in new dict and add a test.
...
Bug: 6669677
Change-Id: I035d848a210653dcd3c1a1f21781968adced9225
2013-09-24 15:29:56 +09:00
0355bf2d4c
GC step 3. Place and write valid PtNodes.
...
Bug: 6669677
Change-Id: Id2b7359bafb34acf12dc3bf4c595a03073fda805
2013-09-20 21:11:49 +09:00
77ef75cbe6
GC step 2. Finding garbage bigram entries.
...
Bug: 6669677
Change-Id: Ie3f406776ab56bf589ac10e15e2534ff8cb1f275
2013-09-20 20:04:57 +09:00
2a64726a16
Step 1 to implement GC. Finding garbage PtNodes.
...
Bug: 6669677
Change-Id: I3551fe2f16a09d2bf7761f4e1d73ebd4a03380e7
2013-09-20 15:37:14 +09:00
b7e8a9abf5
Groundwork for implementing GC.
...
Bug: 6669677
Change-Id: Ia2b2f338795ab62cc9eef73731285b2a5a0ac5e4
2013-09-19 14:48:59 +09:00
e28927dfb6
Change name of header_reading_utlis.*
...
Bug: 6669677
Change-Id: I91389050148237580ef640d3a58b935ea752954e
2013-09-18 18:27:01 +09:00
484fa7b59c
Add dictionary header writing methods.
...
Bug: 6669677
Change-Id: I392ac4776b55779903cbaa17e683005d80017a41
2013-09-18 18:08:33 +09:00
1560543754
Implement dictionary flush.
...
Bug: 6669677
Change-Id: I1d70ac6319fb6d8700074de382568c2108887960
2013-09-18 13:19:14 +09:00
ac7e005ef0
Merge "Prepare dictionary saving in native code."
2013-09-17 09:18:32 +00:00
d0246277fd
Prepare dictionary saving in native code.
...
Bug: 6669677
Change-Id: I0787f66409e2b183c6328fa05805945520e1e93a
2013-09-17 18:07:16 +09:00
d9f450ef00
Implement removeBigramWords() for DynamicPatriciaTrie.
...
Bug: 6669677
Change-Id: I99cb517511b4c968b8bf937baab5d7f42b0f534e
2013-09-17 18:05:32 +09:00
78b55a31cb
Fix handling multi-bytes characters and add a test.
...
Bug: 6669677
Change-Id: Id2154db47adea2929559a4187a726f9dfa83363e
2013-09-17 15:11:24 +09:00
6d1a29eaab
Fix build.
...
Change-Id: I1c3c2996ca11bfe8864df774e9280893cf17c0d4
2013-09-17 13:03:12 +09:00
cd6a0430c7
Implement addBigramWords() for DynamicPatriciaTrie.
...
Bug: 6669677
Change-Id: Ia6aa76c212165690191c65fee9dacdc44be5acd5
2013-09-17 12:52:21 +09:00
cb816e5e16
Merge NOT_A_VALID_WORD_POS into NOT_A_DICT_POS.
...
Change-Id: I2ac3aa2a19d14e8c4159a59334e7816bd1409688
2013-09-16 18:37:23 +09:00
9ea9c61c99
Move methods using ByteArrayUtils from .h to .cpp.
...
Bug: 10402083
Change-Id: Ic5f1aa11eb403f5bdc83de8e38a6457200519cae
2013-09-16 18:37:08 +09:00
f874728496
Add boundary check for PtNode reading position.
...
Bug: 10743878
Change-Id: I6727b12b0ef98bb9cfb3ca95aeab2c79749f47a2
2013-09-16 18:34:34 +09:00
04bf3cd4e0
Merge "Employ "bigram link" for handling moved bigram target."
2013-09-13 09:46:27 +00:00
8c69ddb53e
Employ "bigram link" for handling moved bigram target.
...
Bug: 6669677
Change-Id: I9a6c0cdb28265e3215ced88f38344df12c25a76d
2013-09-13 18:36:29 +09:00
ebe3b3e8fd
Merge "Fix bug related to dynamic patricia trie bigram adding."
2013-09-13 05:49:26 +00:00
0243c9ae3d
Update children's parent position when the node get moved.
...
Bug: 6669677
Change-Id: Ifcaf784afd218c7afd70167045cd2aed8b7909c2
2013-09-13 14:26:00 +09:00
1c0fc852f6
Fix bug related to dynamic patricia trie bigram adding.
...
Bug: 6669677
Change-Id: Ia1216a1860a7c5c12a0d8a0816dda8d6a3bea123
2013-09-13 11:38:53 +09:00
f38969f3f8
Fix bug of dictionary dynamic updating methods.
...
Bug: 6669677
Change-Id: I5483adb03e1ac0c27bbfd99d5f4d7cc29809d70a
2013-09-10 22:32:30 +09:00
e531c2241e
Move a flag about switching dynamic update to java.
...
Bug: 6669677
Change-Id: I6aa99cae4a227f9202179c2873d13473a773e024
2013-09-10 18:45:53 +09:00
210669605a
Merge "Fix: ExtendableBuffer"
2013-09-10 06:45:41 +00:00
4e79fb24ab
Fix: ExtendableBuffer
...
Bug: 6669677
Change-Id: I73142fdabac1cbd4128b83c3b26b642b4b4f4da7
2013-09-10 15:40:11 +09:00
216617062a
Merge "Implement split and create node methods."
2013-09-10 06:38:05 +00:00
fdcd67ba15
Implement split and create node methods.
...
Bug: 6669677
Change-Id: Iac4afb637a8ba3d55f7a88f4a532831c1731f93e
2013-09-10 15:27:19 +09:00
c5f35a9c86
Merge "Calculate parent offset from the head of moved node."
2013-09-10 06:08:56 +00:00
9c820de7a1
Calculate parent offset from the head of moved node.
...
Bug: 6669677
Change-Id: I710453f1c37ec14e5b6830595c1afb2caff83cf2
2013-09-10 15:03:44 +09:00
1d4a07f6c7
Merge "Re-call getBuffer after writing in bigram/shortcut policy."
2013-09-10 03:27:30 +00:00
c987120cbc
Re-call getBuffer after writing in bigram/shortcut policy.
...
Bug: 6669677
Change-Id: I35dc290aa9582e019681b8aae49db3f8d37c90d4
2013-09-10 12:12:42 +09:00
a159ad473c
Implement create children array and add child method.
...
Bug: 6669677
Change-Id: Ic89c3dfe01937e771db8fc59d5259496a9464a47
2013-09-09 16:44:17 +09:00
7bd7dc5d0d
Implement update node probability method.
...
Bug: 6669677
Change-Id: I61ac0d05e362fc7d8a967ddd8286580998c70487
2013-09-09 15:29:11 +09:00
3fbc5ef196
Implement inserting new node into PtNode array.
...
Bug: 6669677
Change-Id: I0171476231181e41234dde76ac9061febb2e8c35
2013-09-06 20:40:08 +09:00
40182c737d
Merge "Compute probability using structure policy."
2013-09-06 08:10:07 +00:00
65d19946be
Compute probability using structure policy.
...
Bug: 6669677
Change-Id: Icd50fc30274cce65ebeb2e7cc8368b59e5cda05a
2013-09-06 16:50:09 +09:00
e4dcebe0c3
Add node flags creating methods.
...
Bug: 6669677
Change-Id: Ieb4178da61fa38f6b32baefa099cc0724f9fb6be
2013-09-06 14:56:59 +09:00
96a0631664
DynamicPatriciaTrieWritingUtils to write PtNodes.
...
Bug: 6669677
Change-Id: I67cce3c1d338cf747ed2204c96768907d990d269
2013-09-05 20:12:56 +09:00
2a41a35635
Add code point writing methods for updatable dictionaries.
...
Bug: 6669677
Change-Id: If7ab6fefdb9a7f0d21c397edbb7d9e7fddc0e0db
2013-09-05 14:24:07 +09:00
5901b5e1bd
Check usesForgettingCurve in HeaderPolicy.
...
Bug: 6669677
Change-Id: I47ebfc50f477b2a6514fba6fad421dd90f29ecb1
2013-09-05 12:28:38 +09:00
91486bdb3d
Merge "Employ a header attribute map in headerPolicy."
2013-09-04 10:06:05 +00:00
5754817a5e
Employ a header attribute map in headerPolicy.
...
Bug: 6669677
Change-Id: I4a084cdd7fcd7a8ed3a70fb7e365031eedd981e5
2013-09-04 19:00:53 +09:00
5d8fd5457d
Change name of bigram_list_reading_utils.* .
...
Bug: 6669677
Change-Id: Ifd2baa11fb4ee083be1d23e8235f6f903c1b5747
2013-09-03 20:44:58 +09:00
202e416b51
Add bigram writing methods.
...
Bug: 6669677
Change-Id: Ib62f6c31c4bcbb41ac200dccccb13e366b3c2173
2013-09-03 20:44:51 +09:00
9e3082e016
Add DynamicShortcutListPolicy::copyAllShortcuts().
...
Bug: 6669677
Change-Id: I74959fe3ca59d77ba0279c75f4d72db13735322f
2013-09-03 18:36:53 +09:00
f1cd7a081c
Implement reading side of updating methods of dpt.
...
Reading methods are used to determine the position to
be updated in dynamic patricia trie.
Bug: 6669677
Change-Id: Ie9acdcdbc73aa3fd4294c97aa39451a96b165ed9
2013-09-03 14:47:30 +09:00
75d1cb667d
Add buffer writing methods.
...
Bug: 6669677
Change-Id: Ic3345e36a7842b77598c21de9b54f7c2863df38f
2013-09-02 19:27:01 +09:00
2660f83a12
Change name of extendable_buffer.h to buffer_with_... .
...
Bug: 6669677
Change-Id: Id2bd79d8f86c690d5c16d1ab726c5f6fa28ba5e3
2013-09-02 17:18:41 +09:00
Keisuke Kuroyanagi
Ken Wakasa
Jean Chalard