nyancrimew/LatinIME
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
32307 commits 1 branch 0 tags 363 MiB
Commit graph

1188 commits

Author SHA1 Message Date
Keisuke Kuroyanagi
b22f95ec8a Remove isUpdatable from constructors of dict contents. 
Change-Id: I2d54f477d9b341e944e265786a734f23d152bb81
 2014-07-11 15:23:55 +09:00
Keisuke Kuroyanagi
2ac934296c Concatenate dict buffers other than header to a single file. 
Bug: 13664080
Change-Id: I34c9d8046b339c9b855be378a5fad907382d1359
 2014-07-11 15:15:47 +09:00
Keisuke Kuroyanagi
198be3a6c5 Fix: wrong bigram entry pos can be used during GC. 
Bug: 14425059
Change-Id: I4e7c977c5ef184953682175b118de5473cdb64a6
 2014-07-09 11:18:35 +09:00
Keisuke Kuroyanagi
3b8f2ece40 Merge "Move phrase gesture from additional features to public." 2014-07-02 20:35:42 +00:00
Keisuke Kuroyanagi
b8a9479b57 Move phrase gesture from additional features to public. 
Bug: 15758242
Change-Id: I703dc2703afdf9402c5d2ac27411b74ece993dba
 2014-07-08 16:36:06 +09:00
Keisuke Kuroyanagi
b00973952f Get bigrams iterator via dict structure policy. 
Bug: 14425059
Change-Id: Ie4c6afbb6a3b707b5aa8e9e4f941c9da6e6bcdbf
 2014-07-08 16:23:31 +09:00
Keisuke Kuroyanagi
804f7450fc Use linked list for bigram list. 
BinaryDictionaryTests for VERSION4_DEV:
Before
Time: 36.461
After
Time: 33.031

Bug: 14425059

Change-Id: I9ca2714f450f61f713df6ebd34c953dece991cdb
 2014-07-07 21:09:25 +09:00
Keisuke Kuroyanagi
6afd14dc94 Remove unused method. 
Bug: 13664080
Change-Id: I47fd90f01ea9a18b11037c966150fa5fd926101b
 2014-06-27 15:23:38 +09:00
Keisuke Kuroyanagi
c152d7c19d Fix unit tests that use getNgramProbabilityNative(). 
Change-Id: I8548094bc8d1a0ea125b29e2a0ea82a6834b3e48
 2014-06-26 19:34:27 +09:00
Keisuke Kuroyanagi
05b1e0d42f Extend jni method to use multiple previous words. 
Bug: 14425059
Change-Id: I8150e8812ee32f8c713315a234b52e5b3a79d7a0
 2014-06-26 18:47:25 +09:00
Keisuke Kuroyanagi
8681bef03c Remove BigramDictionary form native code. 
Bug: 14425059
Change-Id: I48d1a4ae6a4317afaaf67748060618f8f9485d40
 2014-06-26 15:15:13 +09:00
Keisuke Kuroyanagi
d58a07666e Merge "Rename jni method for n-gram supporting." 2014-06-24 01:52:26 +00:00
Keisuke Kuroyanagi
83b6ee2538 Rename jni method for n-gram supporting. 
Bug: 14425059
Change-Id: I242d1de5d9db37344f02d8fba9035829794a56c8
 2014-06-25 14:14:40 +09:00
Keisuke Kuroyanagi
520765491c Merge "Make Beginning-of-Sentence prediction require two exposures." 2014-06-24 01:52:26 +00:00
Keisuke Kuroyanagi
0aef59746c Make Beginning-of-Sentence prediction require two exposures. 
Bug: 15868192
Change-Id: I4b577ad11c992d872d6f902fd50527d9ca5cc1ba
 2014-06-25 20:20:44 +09:00
Yohei Yukawa
5961f2dfac Merge "Simplify a make file" 2014-06-24 01:59:00 +00:00
Yohei Yukawa
cc9b60401e Simplify a make file 
This CL simplifies the makefile for host unittest.

BUG: 13754552
Change-Id: Id09da77b45610db2747897533c18a6d0d81c59d7
 2014-06-25 23:10:19 +09:00
Keisuke Kuroyanagi
88fa47a27d Support migration/dump of Beginning-of-Sentence entries. 
Bug: 14119293
Change-Id: Ie975138f819794d5c34a7a547be5a6117050e084
 2014-06-24 12:37:07 +09:00
Keisuke Kuroyanagi
f7322b166b Make getNextWordAndNextToken output code point count. 
Bug: 14425059
Change-Id: Id1c2927c8a1be0b1680206d444dbdf0c9453dceb
 2014-06-23 19:08:30 +09:00
Keisuke Kuroyanagi
a807663aae Merge "Remove hasBigrams/hasShortcut flags from ver4 PtNode." 2014-06-19 23:04:31 +00:00
Keisuke Kuroyanagi
24d49ee577 Remove hasBigrams/hasShortcut flags from ver4 PtNode. 
Always check bigram/shortcut content instead of having
flags in each PtNode.

Bug: 14425059
Change-Id: I9eb816681bf22fc12e3ab3c92a178a7990b90e93
 2014-06-23 16:49:49 +09:00
Yohei Yukawa
762a356a45 Merge "Add native unittest support on target devices for LatinIME" 2014-06-19 22:58:18 +00:00
Yohei Yukawa
7f4cca0cd1 Add native unittest support on target devices for LatinIME 
This CL adds native unittest support on target devices for
LatinIME.

Note that you need to specify --target option to run-tests.sh
to run native unittest on the target device.

BUG: 13754552
Change-Id: I1dfb78ae1461163db8d47c3ba9141d4812070fd0
 2014-06-23 14:41:34 +09:00
Keisuke Kuroyanagi
0fbca1ac23 Remove word from personalized dicts when it's canceled. 
Bug: 15555552
Bug: 15552347
Change-Id: Ia4d8c79d079e3ce233d46d4bbad4fff1be5c5c39
 2014-06-20 14:46:13 +09:00
Keisuke Kuroyanagi
89899e14df Merge "Update unit test for beginning-of-sentence." 2014-06-10 22:31:57 +00:00
Keisuke Kuroyanagi
e041bfab5c Update unit test for beginning-of-sentence. 
Bug: 14119293
Change-Id: I594a1ca1096f0c12c678dd30eb75de905e55c5bb
 2014-06-12 16:07:39 +09:00
Keisuke Kuroyanagi
22931cd941 Enable Beginning-of-Sentence prediction for contextual dict. 
Bug: 14161647
Bug: 14119293
Change-Id: I0c00f13966db88e4de85e245e7bced43c9d474b2
 2014-06-12 12:26:18 +09:00
Keisuke Kuroyanagi
da0ea7603b Merge "Add jni method for removeUnigramEntry()." 2014-06-10 22:31:57 +00:00
Keisuke Kuroyanagi
b4d77eca55 Add jni method for removeUnigramEntry(). 
Bug: 15552347
Change-Id: Iadcdc73d3335d561a27b9314d3fcdab0b0c96e49
 2014-06-12 12:21:44 +09:00
Keisuke Kuroyanagi
f12985245c Add removeUnigramEntry() to native dictionary policy. 
Bug: 15552347
Change-Id: Ic713f79ef26b5963060304df9c0a3a402c3cc6d4
 2014-06-12 12:14:11 +09:00
Keisuke Kuroyanagi
86fe8081b3 Merge "Add removeUnigramEntry() to native dictionary policy." 2014-06-10 22:29:12 +00:00
Keisuke Kuroyanagi
447c19364a Merge "Enable Beginning-of-Sentence prediction for contextual dict." 2014-06-10 22:31:57 +00:00
Keisuke Kuroyanagi
59ed0c2db2 Block offensive words in native code. 
Bug: 15300452

Change-Id: Ic6c91c47453e19fb5db2c62b66659619e72b1ecf
 2014-06-12 11:36:00 +09:00
Keisuke Kuroyanagi
07e1412631 Purge dict reading code for 401 and create for 402. 
Change-Id: Ib0150b3306bb2db8aff443d1158fecc8464cbbdb
 2014-06-10 19:22:40 +09:00
Keisuke Kuroyanagi
d9b8602f48 Implement getMaxProbabilityOfExactMatches(). 
Bug: 13142176
Bug: 15428247
Change-Id: I5be6d683be95505412615ca7c88260de1ea05f54
 2014-06-06 17:37:46 +09:00
Keisuke Kuroyanagi
d6b89e17a6 Add jni method for getMaxFrequencyOfExactMatches(). 
Bug: 13142176
Bug: 15428247
Change-Id: I2ae25090b67ad0c6df97dec7712e2f7e0bb08c40
 2014-06-05 18:16:11 +09:00
Keisuke Kuroyanagi
7852765a50 Add length check for attaching a BoS marker. 
Bug: 15297872
Change-Id: I7b0a27409b81b76561d19c6caa18a5011d3e90f6
 2014-05-28 20:20:39 +09:00
Keisuke Kuroyanagi
f9ce867d80 Add boundary check for v4 bigram reading. 
Bug: 14496386
Change-Id: Iedd3445c3222a777a2476beed7d9eb53773f406c
 2014-05-27 19:29:35 +09:00
Keisuke Kuroyanagi
dfca51726e Return whether the dynamic dict operation was success. 
Bug: 12184250
Change-Id: Iee7e00c1e84c95551a077f4dd023c0a9b9ac9466
 2014-05-27 17:30:00 +09:00
Keisuke Kuroyanagi
0088d1f668 Merge "Use whether it's exact match to detect distracters." 2014-05-26 13:53:19 +00:00
Keisuke Kuroyanagi
a6278eb9c1 Use whether it's exact match to detect distracters. 
Bug: 13142176
Change-Id: Id5b7286d28897931f7bfe571be45d46ffeef4adf
 2014-05-26 21:56:20 +09:00
Yohei Yukawa
67e02a6aaa Specify c++_static for bundled Latin IME as well 
APP_STL := c++_static has been specified for unbundled
Latin IME but somehow not for bundled build.

This CL fixes the inconsistency.

Change-Id: Ie1f4aba27e30981d3c2aa322a16c1cd18413e920
 2014-05-26 18:30:48 +09:00
Keisuke Kuroyanagi
a37f374ad1 Version up dynamic dict format from 401 to 402. 
Change-Id: Ibea36af905ade773ae3db3a5456f7b5a0ad7d220
 2014-05-23 20:20:56 +09:00
Keisuke Kuroyanagi
1adca93381 Extend jni methods and enable Beginning-of-Sentence. 
Bug: 14119293
Change-Id: I78fc877367dd0d6240eeacb750b6d2d0b93cba83
 2014-05-23 19:58:58 +09:00
Keisuke Kuroyanagi
96990ca773 Support Beginning-of-Sentence in native code 
Bug: 14119293
Change-Id: I0f382e33a19bf481823b23405d454de61ec835ff
 2014-05-23 17:23:08 +09:00
Keisuke Kuroyanagi
6600340af5 Fix: PtNodeParams.representsNonWordInfo() 
Change-Id: I7ab4fb280e8ac89f81d0b32b0663a86c9b69aa5d
 2014-05-23 13:49:18 +09:00
Keisuke Kuroyanagi
2a015dcb25 Add Beginning-of-Sentence info in UnigramProperty. 
Bug: 14119293
Bug: 14425059
Change-Id: I8a894352568377d32468e5563f4e89af00d22944
 2014-05-23 00:07:14 +09:00
Keisuke Kuroyanagi
9f8c9a0161 Use PrevWordsInfo to add/remove n(bi)-gram in native code. 
Bug: 14119293
Bug: 14425059
Change-Id: I4b9a46bfd670b35195418eaee51456d44fb91b6d
 2014-05-21 18:30:34 +09:00
Keisuke Kuroyanagi
fa7db65dec Support multiple previous words in DicNode. 
Bug: 14425059
Change-Id: Ib8682befe4d7d9fe5122eb538e7c804f75ded463
 2014-05-19 11:47:10 +09:00
Keisuke Kuroyanagi
45d1a936a7 Move prev word related logic to PrevWordsInfo. 
Bug: 14119293
Bug: 14425059
Change-Id: I1bbd7ab4ace2c475f27bc468cb7b4d67e1ae2f9f
 2014-05-16 22:31:45 +09:00
Keisuke Kuroyanagi
b94ec1437b Introduce MAX_PREV_WORD_COUNT_FOR_N_GRAM. 
Bug: 14425059
Change-Id: I587f90df026a14359d2ee452040bbfaf02c1ae51
 2014-05-15 18:45:49 +09:00
Keisuke Kuroyanagi
b87fffb8be Refactoring: Move prev word information into PrevWordsInfo. 
Bug: 14119293
Bug: 14425059

Change-Id: I5a5f81c6b189e3ad1da093343a3121463f87c21c
 2014-05-15 18:39:23 +09:00
Keisuke Kuroyanagi
d6fdd58cc0 Cleanup DicTraverseSession. 
Change-Id: I953665e1958e7214e9365e2bc9d3f3843016dc81
 2014-05-15 17:41:23 +09:00
Keisuke Kuroyanagi
e810a266fd Merge "Quit use bigram probability diff for ver4 dict." 2014-05-15 07:07:25 +00:00
Keisuke Kuroyanagi
64341927d2 Quit use bigram probability diff for ver4 dict. 
Change-Id: I2cfcfbcf351877d1dff466a24974dbb05908f14e
 2014-05-15 16:02:58 +09:00
Keisuke Kuroyanagi
a709bc7f93 Merge "Reduce the cost for handling intentional omission." 2014-05-15 03:31:39 +00:00
Keisuke Kuroyanagi
39e5a15af0 Merge "Always keep PtNodes that represent non-word info during GC." 2014-05-15 02:38:57 +00:00
Keisuke Kuroyanagi
91d7538225 Merge "Add VERSION_4_DEV(402) in native code." 2014-05-14 13:40:10 +00:00
Keisuke Kuroyanagi
7116ea98f4 Add VERSION_4_DEV(402) in native code. 
Bug: 13406708
Change-Id: I96cfacf524d670a6a5637a96a63bcd47aaf09ca0
 2014-05-14 22:37:05 +09:00
Keisuke Kuroyanagi
cf700695ea Always keep PtNodes that represent non-word info during GC. 
Bug: 14119293
Change-Id: Ie50b341a512be6a6efe3e05b60e1d09560b01d1f
 2014-05-14 22:29:21 +09:00
Keisuke Kuroyanagi
2dc8e36f8e Use nullptr instead of MmappedBufferPtr(nullptr). 
Change-Id: I9bc35f723f1c105ef6155a49630b8cd492900bc8
 2014-05-14 22:05:00 +09:00
Keisuke Kuroyanagi
32bdf9f344 Merge "Use JniDataUtils::putXxxToArray() to output a value." 2014-05-14 11:12:09 +00:00
Keisuke Kuroyanagi
5c00bde9bc Merge "Filter out control characters and non-unicode code points." 2014-05-14 11:07:32 +00:00
Keisuke Kuroyanagi
304a71d171 Use JniDataUtils::putXxxToArray() to output a value. 
Change-Id: Ifb7bed6623b01acb524566a0fe00dcbdf7273888
 2014-05-14 20:05:15 +09:00
Keisuke Kuroyanagi
e9121a68a6 Filter out control characters and non-unicode code points. 
Bug: 14119293
Bug: 13758761
Change-Id: I6b79cff0714152807f5e20b7a75060a0a772b28b
 2014-05-14 20:05:10 +09:00
Keisuke Kuroyanagi
728141d55a Merge "Generate dict code for version 401." 2014-05-14 10:51:12 +00:00
Keisuke Kuroyanagi
6bf268132d Generate dict code for version 401. 
Bug: 13406708
Change-Id: I769ac84d54d997e7aefab0c9c16727455a132e0b
 2014-05-14 19:09:01 +09:00
Keisuke Kuroyanagi
72d6061efb Merge "Use JniDataUtils::outputCodePoints() to output code points" 2014-05-14 09:07:45 +00:00
Keisuke Kuroyanagi
ed16af72e3 Restrict the permission of dict files created on device. 
Bug: 13497304
Change-Id: I7bf67e2e81bc63a67bd91cbc0f753ecac88349f6
 2014-05-14 17:48:28 +09:00
Keisuke Kuroyanagi
d2f7cd28ae Use JniDataUtils::outputCodePoints() to output code points 
Bug: 14119293
Bug: 13758761

Change-Id: I516d2f96e219e6e65246f502186ccb9bd5f0670f
 2014-05-14 17:42:47 +09:00
Keisuke Kuroyanagi
c1e86d3f49 Merge "Skip PtNodes with non-Unicode code points for suggestion." 2014-05-14 06:22:22 +00:00
Keisuke Kuroyanagi
14dd663fe5 Reduce the cost for handling intentional omission. 
Bug: 14602770

Change-Id: I3a678a3e9b227fc4c3b8d23012aba18ca2e52da5
 2014-05-14 13:54:32 +09:00
Keisuke Kuroyanagi
0c8ce96bec Add "// namespace latinime" 
Bug: 13406708
Change-Id: I0fa85875fb5b98b4e40a149e3e33e5f56bee9a97
 2014-05-13 15:16:43 +09:00
Keisuke Kuroyanagi
6e4b674f83 Move files under bigram/shortcut to under structure. 
Bug: 13406708
Change-Id: I48e9d4ff30a8b50d4f1694ebbefb02cf189f7713
 2014-05-13 15:16:43 +09:00
Keisuke Kuroyanagi
d0d0113983 Code cleanup. 
Change-Id: Ic31b66cccfd77348f2ff7961959a3eaf9f7ad5d8
 2014-05-13 01:32:38 +09:00
Keisuke Kuroyanagi
620a05ae59 Support bigram historical information migration. 
Bug: 13406708
Change-Id: I4bae53e43cb7653eac3b5bd13da2d2bc8aaf88a9
 2014-05-13 01:28:30 +09:00
Keisuke Kuroyanagi
79ba633402 Skip PtNodes with non-Unicode code points for suggestion. 
Bug: 14119293
Change-Id: Id1d3b789b5f18757070878dba35a7980bfb44591
 2014-05-12 19:38:26 +09:00
Keisuke Kuroyanagi
9d7e8c717f Support unigram historical information migration. 
Bug: 13406708
Change-Id: Ibed15b3bc5d5ae68faefa379028dbe10d32b0c0f
 2014-05-12 19:21:06 +09:00
Keisuke Kuroyanagi
6b74f516dc Merge "Refactoring: Use UnigramProperty to add/update unigram." 2014-05-09 11:39:17 +00:00
Keisuke Kuroyanagi
b7f71cdf8c Merge "Avoid copying bigram list if possible." 2014-05-09 10:09:54 +00:00
Keisuke Kuroyanagi
b636e25e95 Refactoring: Use UnigramProperty to add/update unigram. 
Bug: 13406708
Change-Id: I26fd541fb465d3543faa5f155becc455ddbb6c9c
 2014-05-09 17:22:17 +09:00
Keisuke Kuroyanagi
eaa347bc1a Merge "Add VERSION4_TEST for testing." 2014-05-09 07:06:06 +00:00
Keisuke Kuroyanagi
455dc84cf2 Add VERSION4_TEST for testing. 
Bug: 13406708
Change-Id: I0b434717ea3edbb131650af634c53cfbfe113c3e
 2014-05-09 15:17:01 +09:00
Keisuke Kuroyanagi
08ea74eb2c Merge "Implement migrateNative()." 2014-05-08 05:21:06 +00:00
Keisuke Kuroyanagi
08d12a1be3 Merge "Extend jni interface for dictionary migration." 2014-05-08 05:20:33 +00:00
Keisuke Kuroyanagi
ad518d9a5b Avoid copying bigram list if possible. 
Constructing en_US main dict using dicttool:
Before:
real    1m8.699s
user    1m10.600s
sys     0m2.390s
After:
real    0m17.204s
user    0m20.560s
sys     0m0.720s


Bug: 13406708
Change-Id: I3b0476be57e5cb93c6497025b3ffa7064ac326c6
 2014-05-08 14:19:33 +09:00
Keisuke Kuroyanagi
8a9d9de72a Implement migrateNative(). 
Bug: 13406708
Change-Id: Ie786a68e780baa298fedf82d21390ba45d871a52
 2014-05-08 12:31:32 +09:00
Keisuke Kuroyanagi
1471fbad66 Extend jni interface for dictionary migration. 
Bug: 13406708

Change-Id: Iadad6df296467d0e5b631fba4a409d53c8a5a6f2
 2014-05-08 12:31:04 +09:00
Keisuke Kuroyanagi
ad8f2201e2 Make initial additional buffer size 0. 
Bug: 14113369
Change-Id: I3b7b3a452544725577101dc3b9bcb85b3e08afe9
 2014-05-07 23:44:32 +09:00
Keisuke Kuroyanagi
8e8a5a58c4 Merge "Add new bigram entry at the tail of existing list." 2014-05-01 11:19:36 +00:00
Keisuke Kuroyanagi
8d8fb396a0 Add new bigram entry at the tail of existing list. 
Bug: 13406708
Change-Id: If3162e65fc9aa2c47f046aee528276cb51fad9f4
 2014-05-01 19:29:43 +09:00
Keisuke Kuroyanagi
ab5839ae0b Merge "Add CODE_POINT_BEGINNING_OF_SENTENCE in native code." 2014-05-01 02:08:27 +00:00
Keisuke Kuroyanagi
4162cfdc59 Add CODE_POINT_BEGINNING_OF_SENTENCE in native code. 
Bug: 14119293
Change-Id: Ib101435e2840172ba004ead081022669647a8ad9
 2014-04-30 20:40:02 +09:00
Ken Wakasa
fc2858c55b Small clean up for LARGEST_INT_DIGIT_COUNT 
Change-Id: I4a61c037370bf57b396d80ba7212d94328eaf7b9
 2014-04-30 19:41:42 +09:00
Keisuke Kuroyanagi
8dac7ce2e2 Move PatriciaTrieReadingUtils under pt_common. 
Change-Id: Ib8ec62b97318563785a2aafb854aabdc3f46f187
 2014-04-30 14:28:08 +09:00
Ken Wakasa
9e15b4551e Small clean up on MAX_LOCALE_STRING_LENGTH 
Change-Id: I77f93879e841f33b81f6d193d1bd5782ad2124f5
 2014-04-28 16:50:06 +09:00
Keisuke Kuroyanagi
3b7984752c Support creating BinaryDictionary without creating dict file. 
Bug: 14166482
Change-Id: Ic8c78ec2c8f39358a1f085a041b608972a380eef
 2014-04-22 13:18:34 -07:00
Keisuke Kuroyanagi
ce76821dce Revert "Support creating BinaryDictionary without creating dict file." 
This reverts commit 89939b8708.

Change-Id: I8ca1b55609ad305a93178063bd88bd1cede872af
 2014-04-22 19:34:44 +00:00
Keisuke Kuroyanagi
89939b8708 Support creating BinaryDictionary without creating dict file. 
Bug: 14166482
Change-Id: Ib065279f96e227ab0fee7c8141560c4ada744d3b
 2014-04-22 12:01:33 -07:00
Keisuke Kuroyanagi
903be5bbd3 Implement latinime_BinaryDictionary_createOnMemory(). 
Bug: 14166482
Change-Id: If7ec3345ab34edcd6bc5cef9e72580ced894a0e3
 2014-04-22 12:01:22 -07:00