64341927d2
Quit use bigram probability diff for ver4 dict.
...
Change-Id: I2cfcfbcf351877d1dff466a24974dbb05908f14e
2014-05-15 16:02:58 +09:00
04536f14b0
Add VERSION4_DEV(402) in Java side and use it for tests.
...
Bug: 13406708
Change-Id: I7c9c53bf34d12dced7331a359a83f6444490783a
2014-05-14 22:37:07 +09:00
3c7f140e8e
Add large unit test for dictionary migration.
...
Bug: 13406708
Change-Id: Ic1eb627139a1ff57c0000d95d9bd57aa3609f753
2014-05-13 15:16:30 +09:00
455dc84cf2
Add VERSION4_TEST for testing.
...
Bug: 13406708
Change-Id: I0b434717ea3edbb131650af634c53cfbfe113c3e
2014-05-09 15:17:01 +09:00
3b7984752c
Support creating BinaryDictionary without creating dict file.
...
Bug: 14166482
Change-Id: Ic8c78ec2c8f39358a1f085a041b608972a380eef
2014-04-22 13:18:34 -07:00
ce76821dce
Revert "Support creating BinaryDictionary without creating dict file."
...
This reverts commit 89939b8708
2014-04-22 19:34:44 +00:00
89939b8708
Support creating BinaryDictionary without creating dict file.
...
Bug: 14166482
Change-Id: Ib065279f96e227ab0fee7c8141560c4ada744d3b
2014-04-22 12:01:33 -07:00
98705b6bf5
Fix unit tests.
...
Change-Id: I2a85763358e1eef0f76ef827413afffeb4cb8058
2014-03-14 12:27:23 +09:00
61467744e7
Unit test update for I7ed95a54344d8b.
...
Change-Id: Ic8af56b8741ee7c231d73ce6a660fd32c9f5a3b3
2014-03-12 15:41:20 +09:00
516f86815d
Separate WeightedString from FusionDictionary.
...
Bug: 8187060
Change-Id: I40c1dafca3eb52244c64fdb4c1db30a56385d678
2014-03-06 18:53:06 +09:00
e784148ae6
Separate utility methods from BinaryDictionary.
...
Bug: 8187060
Change-Id: Ice2984e332b7bd3bb17174aefc80b5635b72fc50
2014-03-05 18:19:34 +09:00
da973e75dc
Make InputLogicTest more robust
...
Change-Id: I134f14971126cbeed05b472c08747f2b88ad30e6
2014-02-13 19:38:51 +09:00
337dce8074
Merge "Consolidate WordProperty and Word."
2014-02-06 07:02:07 +00:00
5f5feeba13
Consolidate WordProperty and Word.
...
Bug: 11281877
Bug: 12810574
Change-Id: I9dc99188f80f25a8780c1860dab46e4aa80a23e5
2014-02-06 15:13:33 +09:00
239f7daf4a
Merge "Make WeightedString have ProbabilityInfo."
2014-02-05 12:53:14 +00:00
df1d3e733e
Make WeightedString have ProbabilityInfo.
...
Bug: 11281877
Bug: 12810574
Change-Id: I265e3d8654c75766cd0e0d09d67ef62b4566298a
2014-02-05 21:44:55 +09:00
43cf9076b2
[HD04] Make the locale mandatory.
...
Bug: 11281748
Change-Id: I69281b0053bec404c3e3c713ade3f65a140f51b1
2014-02-05 17:35:25 +09:00
941734695b
Implement Ver4PatriciaTriePolicy::getNextWordAndNextToken.
...
Bug: 12810574
Change-Id: Idea44f03c477964f58e65fbf2b55e3fcd77a2934
2014-02-04 12:42:14 +09:00
9221772ab7
Get bigram information via getWordProperty().
...
Bug: 12810574
Change-Id: I2750a5659ccbc3e31307c28e35dd9a1dbdffbeee
2014-02-03 11:47:26 +09:00
94d9a2309f
Extend jni interface to get bigrams in WordProperty.
...
Bug: 12810574
Change-Id: Ia4b88d02ea8790a5c47d32376cc0b84c3e071ddd
2014-01-31 20:32:44 +09:00
080a35e959
Rename UnigramProperty to WordProperty.
...
Bug: 12810574
Change-Id: If5ddd803948aaf6e491ddcbaa5436fb3af3f7257
2014-01-31 11:06:42 +09:00
51c5ec10f9
Add length check for dict update operations.
...
Bug: 12602903
Change-Id: I6835dee8bf8b356f0f1cf6c0531bee5b3415a13f
2014-01-24 16:32:42 -08:00
d78a447d10
Add a dedicated method to control time in native code.
...
Bug: 12443085
Change-Id: I41a5cf6c895cb59e54af98b40efded36afcd3247
2014-01-15 14:55:57 -08:00
7b9b095090
Move LanguageModelParams class to utility package
...
This change must be checked in together with Ic82d951237.
Change-Id: I2ab451330f85a0147563b805682f26edecb71a29
2014-01-15 16:14:53 +09:00
d77ad1902b
Fix NPE in a unit test.
...
Change-Id: I0dfc23232ed77e1ff1aedecfa6ac022020cb1c19
2013-12-26 17:01:48 +09:00
7b55cd3e2b
Remove flags from Java side.
...
This simplifies the code quite a bit.
- GERMAN_UMLAUTS are now handled through a key-value attribute.
The dictionary generator does not need to know about it any more.
- FRENCH_LIGATURES are deprecated as we handle them with shortcuts now.
- CONTAINS_BIGRAMS is deprecated. Bigram processing is always applied
regardless of this flag.
Bug: 11281748
Change-Id: If567e52e245a9342adc7f3104a0f7d8d782df8c1
2013-12-13 18:15:05 +09:00
2fa3693c26
Reset to 9bd6dac470
...
The bulk merge from -bayo to klp-dev should not have been merged to master.
Change-Id: I527a03a76f5247e4939a672f27c314dc11cbb854
2013-12-13 17:13:32 +09:00
bcb196e3b5
Make BinaryDictionaryTests use current time as a seed.
...
Bug: 10197478
Change-Id: Ib79cd3a81b01b16259df29b36e1df6b8f0affca8
2013-10-04 16:48:22 +09:00
b698e9c1fa
Fix build.
...
Change-Id: If7c76ca5f7030ba2003228ed1877bdb948b4b99d
2013-09-30 14:21:48 +09:00
31097a57cc
Add unigram/bigram counting.
...
Bug: 6669677
Change-Id: I05ea2201d822dddf062b08c8467daa336760e16c
2013-09-27 23:12:12 +09:00
5ef6209656
Create empty dictionary file in native code.
...
Bug: 6669677
Change-Id: I46d23deee1fd091678f4593561cb8687eb815212
2013-09-26 12:59:02 +09:00
3d2802ef2e
Allow large memory space for GC.
...
Bug: 6669677
Change-Id: I021c92a1ca5cd28f745cbfe729c99e346c58074d
2013-09-24 19:21:17 +09:00
f3cd38f2c7
GC step 5. Add large tests and fix bugs.
...
Bug: 6669677
Change-Id: I41e44a64730f9c66645995095a7ecff51ffd5351
2013-09-24 16:32:25 +09:00
2cfe7f9e3b
GC step 4. Update all positions in new dict and add a test.
...
Bug: 6669677
Change-Id: I035d848a210653dcd3c1a1f21781968adced9225
2013-09-24 15:29:56 +09:00
484fa7b59c
Add dictionary header writing methods.
...
Bug: 6669677
Change-Id: I392ac4776b55779903cbaa17e683005d80017a41
2013-09-18 18:08:33 +09:00
72e53a1e38
Merge "Implement dictionary flush."
2013-09-18 05:20:05 +00:00
1560543754
Implement dictionary flush.
...
Bug: 6669677
Change-Id: I1d70ac6319fb6d8700074de382568c2108887960
2013-09-18 13:19:14 +09:00
4d02a2d44d
Add BinaryDictionary.getBigramProbabilityNative().
...
Bug: 10197478
Change-Id: Id16e3ee22a17679c46cc7a0199a9e28065177aa3
2013-09-18 11:18:28 +09:00
d9f450ef00
Implement removeBigramWords() for DynamicPatriciaTrie.
...
Bug: 6669677
Change-Id: I99cb517511b4c968b8bf937baab5d7f42b0f534e
2013-09-17 18:05:32 +09:00
71cce2dd3e
Add a large test for BinaryDictionary.addBigramWords().
...
Bug; 10197478
Change-Id: I52590dd1ff9469557672a4bef4f898ad94b47b6f
2013-09-17 15:32:37 +09:00
78b55a31cb
Fix handling multi-bytes characters and add a test.
...
Bug: 6669677
Change-Id: Id2154db47adea2929559a4187a726f9dfa83363e
2013-09-17 15:11:24 +09:00
cd6a0430c7
Implement addBigramWords() for DynamicPatriciaTrie.
...
Bug: 6669677
Change-Id: Ia6aa76c212165690191c65fee9dacdc44be5acd5
2013-09-17 12:52:21 +09:00
5b51355b9e
Add tests for BinaryDictionary.addUnigramWord().
...
Bug: 10197478
Change-Id: Ie46f360b8c75183b49d42d00fefed4a8531f64a9
2013-09-10 22:46:10 +09:00
4be103df0e
Add binary dictionary tests.
...
Bug: 10197478
Change-Id: I13cf9b67d1b61b8e89fe90b431c2d6e066ca8e7a
2013-09-10 19:16:33 +09:00
Keisuke Kuroyanagi
Tadashi G. Takaoka
Jean Chalard
Ken Wakasa