Commit Graph

1095 Commits (1adca93381d261a6070be2721dbf8b8abafbfe01)

Author SHA1 Message Date
Keisuke Kuroyanagi 1adca93381 Extend jni methods and enable Beginning-of-Sentence.
Bug: 14119293
Change-Id: I78fc877367dd0d6240eeacb750b6d2d0b93cba83
2014-05-23 19:58:58 +09:00
Keisuke Kuroyanagi 96990ca773 Support Beginning-of-Sentence in native code
Bug: 14119293
Change-Id: I0f382e33a19bf481823b23405d454de61ec835ff
2014-05-23 17:23:08 +09:00
Keisuke Kuroyanagi 6600340af5 Fix: PtNodeParams.representsNonWordInfo()
Change-Id: I7ab4fb280e8ac89f81d0b32b0663a86c9b69aa5d
2014-05-23 13:49:18 +09:00
Keisuke Kuroyanagi 2a015dcb25 Add Beginning-of-Sentence info in UnigramProperty.
Bug: 14119293
Bug: 14425059
Change-Id: I8a894352568377d32468e5563f4e89af00d22944
2014-05-23 00:07:14 +09:00
Keisuke Kuroyanagi 9f8c9a0161 Use PrevWordsInfo to add/remove n(bi)-gram in native code.
Bug: 14119293
Bug: 14425059
Change-Id: I4b9a46bfd670b35195418eaee51456d44fb91b6d
2014-05-21 18:30:34 +09:00
Keisuke Kuroyanagi fa7db65dec Support multiple previous words in DicNode.
Bug: 14425059
Change-Id: Ib8682befe4d7d9fe5122eb538e7c804f75ded463
2014-05-19 11:47:10 +09:00
Keisuke Kuroyanagi 45d1a936a7 Move prev word related logic to PrevWordsInfo.
Bug: 14119293
Bug: 14425059
Change-Id: I1bbd7ab4ace2c475f27bc468cb7b4d67e1ae2f9f
2014-05-16 22:31:45 +09:00
Keisuke Kuroyanagi b94ec1437b Introduce MAX_PREV_WORD_COUNT_FOR_N_GRAM.
Bug: 14425059
Change-Id: I587f90df026a14359d2ee452040bbfaf02c1ae51
2014-05-15 18:45:49 +09:00
Keisuke Kuroyanagi b87fffb8be Refactoring: Move prev word information into PrevWordsInfo.
Bug: 14119293
Bug: 14425059

Change-Id: I5a5f81c6b189e3ad1da093343a3121463f87c21c
2014-05-15 18:39:23 +09:00
Keisuke Kuroyanagi d6fdd58cc0 Cleanup DicTraverseSession.
Change-Id: I953665e1958e7214e9365e2bc9d3f3843016dc81
2014-05-15 17:41:23 +09:00
Keisuke Kuroyanagi e810a266fd Merge "Quit use bigram probability diff for ver4 dict." 2014-05-15 07:07:25 +00:00
Keisuke Kuroyanagi 64341927d2 Quit use bigram probability diff for ver4 dict.
Change-Id: I2cfcfbcf351877d1dff466a24974dbb05908f14e
2014-05-15 16:02:58 +09:00
Keisuke Kuroyanagi a709bc7f93 Merge "Reduce the cost for handling intentional omission." 2014-05-15 03:31:39 +00:00
Keisuke Kuroyanagi 39e5a15af0 Merge "Always keep PtNodes that represent non-word info during GC." 2014-05-15 02:38:57 +00:00
Keisuke Kuroyanagi 91d7538225 Merge "Add VERSION_4_DEV(402) in native code." 2014-05-14 13:40:10 +00:00
Keisuke Kuroyanagi 7116ea98f4 Add VERSION_4_DEV(402) in native code.
Bug: 13406708
Change-Id: I96cfacf524d670a6a5637a96a63bcd47aaf09ca0
2014-05-14 22:37:05 +09:00
Keisuke Kuroyanagi cf700695ea Always keep PtNodes that represent non-word info during GC.
Bug: 14119293
Change-Id: Ie50b341a512be6a6efe3e05b60e1d09560b01d1f
2014-05-14 22:29:21 +09:00
Keisuke Kuroyanagi 2dc8e36f8e Use nullptr instead of MmappedBufferPtr(nullptr).
Change-Id: I9bc35f723f1c105ef6155a49630b8cd492900bc8
2014-05-14 22:05:00 +09:00
Keisuke Kuroyanagi 32bdf9f344 Merge "Use JniDataUtils::putXxxToArray() to output a value." 2014-05-14 11:12:09 +00:00
Keisuke Kuroyanagi 5c00bde9bc Merge "Filter out control characters and non-unicode code points." 2014-05-14 11:07:32 +00:00
Keisuke Kuroyanagi 304a71d171 Use JniDataUtils::putXxxToArray() to output a value.
Change-Id: Ifb7bed6623b01acb524566a0fe00dcbdf7273888
2014-05-14 20:05:15 +09:00
Keisuke Kuroyanagi e9121a68a6 Filter out control characters and non-unicode code points.
Bug: 14119293
Bug: 13758761
Change-Id: I6b79cff0714152807f5e20b7a75060a0a772b28b
2014-05-14 20:05:10 +09:00
Keisuke Kuroyanagi 728141d55a Merge "Generate dict code for version 401." 2014-05-14 10:51:12 +00:00
Keisuke Kuroyanagi 6bf268132d Generate dict code for version 401.
Bug: 13406708
Change-Id: I769ac84d54d997e7aefab0c9c16727455a132e0b
2014-05-14 19:09:01 +09:00
Keisuke Kuroyanagi 72d6061efb Merge "Use JniDataUtils::outputCodePoints() to output code points" 2014-05-14 09:07:45 +00:00
Keisuke Kuroyanagi ed16af72e3 Restrict the permission of dict files created on device.
Bug: 13497304
Change-Id: I7bf67e2e81bc63a67bd91cbc0f753ecac88349f6
2014-05-14 17:48:28 +09:00
Keisuke Kuroyanagi d2f7cd28ae Use JniDataUtils::outputCodePoints() to output code points
Bug: 14119293
Bug: 13758761

Change-Id: I516d2f96e219e6e65246f502186ccb9bd5f0670f
2014-05-14 17:42:47 +09:00
Keisuke Kuroyanagi c1e86d3f49 Merge "Skip PtNodes with non-Unicode code points for suggestion." 2014-05-14 06:22:22 +00:00
Keisuke Kuroyanagi 14dd663fe5 Reduce the cost for handling intentional omission.
Bug: 14602770

Change-Id: I3a678a3e9b227fc4c3b8d23012aba18ca2e52da5
2014-05-14 13:54:32 +09:00
Keisuke Kuroyanagi 0c8ce96bec Add "// namespace latinime"
Bug: 13406708
Change-Id: I0fa85875fb5b98b4e40a149e3e33e5f56bee9a97
2014-05-13 15:16:43 +09:00
Keisuke Kuroyanagi 6e4b674f83 Move files under bigram/shortcut to under structure.
Bug: 13406708
Change-Id: I48e9d4ff30a8b50d4f1694ebbefb02cf189f7713
2014-05-13 15:16:43 +09:00
Keisuke Kuroyanagi d0d0113983 Code cleanup.
Change-Id: Ic31b66cccfd77348f2ff7961959a3eaf9f7ad5d8
2014-05-13 01:32:38 +09:00
Keisuke Kuroyanagi 620a05ae59 Support bigram historical information migration.
Bug: 13406708
Change-Id: I4bae53e43cb7653eac3b5bd13da2d2bc8aaf88a9
2014-05-13 01:28:30 +09:00
Keisuke Kuroyanagi 79ba633402 Skip PtNodes with non-Unicode code points for suggestion.
Bug: 14119293
Change-Id: Id1d3b789b5f18757070878dba35a7980bfb44591
2014-05-12 19:38:26 +09:00
Keisuke Kuroyanagi 9d7e8c717f Support unigram historical information migration.
Bug: 13406708
Change-Id: Ibed15b3bc5d5ae68faefa379028dbe10d32b0c0f
2014-05-12 19:21:06 +09:00
Keisuke Kuroyanagi 6b74f516dc Merge "Refactoring: Use UnigramProperty to add/update unigram." 2014-05-09 11:39:17 +00:00
Keisuke Kuroyanagi b7f71cdf8c Merge "Avoid copying bigram list if possible." 2014-05-09 10:09:54 +00:00
Keisuke Kuroyanagi b636e25e95 Refactoring: Use UnigramProperty to add/update unigram.
Bug: 13406708
Change-Id: I26fd541fb465d3543faa5f155becc455ddbb6c9c
2014-05-09 17:22:17 +09:00
Keisuke Kuroyanagi eaa347bc1a Merge "Add VERSION4_TEST for testing." 2014-05-09 07:06:06 +00:00
Keisuke Kuroyanagi 455dc84cf2 Add VERSION4_TEST for testing.
Bug: 13406708
Change-Id: I0b434717ea3edbb131650af634c53cfbfe113c3e
2014-05-09 15:17:01 +09:00
Keisuke Kuroyanagi 08ea74eb2c Merge "Implement migrateNative()." 2014-05-08 05:21:06 +00:00
Keisuke Kuroyanagi 08d12a1be3 Merge "Extend jni interface for dictionary migration." 2014-05-08 05:20:33 +00:00
Keisuke Kuroyanagi ad518d9a5b Avoid copying bigram list if possible.
Constructing en_US main dict using dicttool:
Before:
real    1m8.699s
user    1m10.600s
sys     0m2.390s
After:
real    0m17.204s
user    0m20.560s
sys     0m0.720s


Bug: 13406708
Change-Id: I3b0476be57e5cb93c6497025b3ffa7064ac326c6
2014-05-08 14:19:33 +09:00
Keisuke Kuroyanagi 8a9d9de72a Implement migrateNative().
Bug: 13406708
Change-Id: Ie786a68e780baa298fedf82d21390ba45d871a52
2014-05-08 12:31:32 +09:00
Keisuke Kuroyanagi 1471fbad66 Extend jni interface for dictionary migration.
Bug: 13406708

Change-Id: Iadad6df296467d0e5b631fba4a409d53c8a5a6f2
2014-05-08 12:31:04 +09:00
Keisuke Kuroyanagi ad8f2201e2 Make initial additional buffer size 0.
Bug: 14113369
Change-Id: I3b7b3a452544725577101dc3b9bcb85b3e08afe9
2014-05-07 23:44:32 +09:00
Keisuke Kuroyanagi 8e8a5a58c4 Merge "Add new bigram entry at the tail of existing list." 2014-05-01 11:19:36 +00:00
Keisuke Kuroyanagi 8d8fb396a0 Add new bigram entry at the tail of existing list.
Bug: 13406708
Change-Id: If3162e65fc9aa2c47f046aee528276cb51fad9f4
2014-05-01 19:29:43 +09:00
Keisuke Kuroyanagi ab5839ae0b Merge "Add CODE_POINT_BEGINNING_OF_SENTENCE in native code." 2014-05-01 02:08:27 +00:00
Keisuke Kuroyanagi 4162cfdc59 Add CODE_POINT_BEGINNING_OF_SENTENCE in native code.
Bug: 14119293
Change-Id: Ib101435e2840172ba004ead081022669647a8ad9
2014-04-30 20:40:02 +09:00