6a1b37353d
Fix dicttool build
...
This CL partially reverts
- Id88b02b74bdfe4ca05b08181ceb6b34d5652fc0c
- I05c7d8429e8d9a26139456763c77997340fea8c2
And followup (remove shortcut support)
- I73b7dc008a5acaf75a31a36a2d332b5afabd82d0
Bug: 28255684
Test: make -j10 dicttool_aosp
Change-Id: I2e01ed86b9517a1141aee35ea6d8ef39258981d1
2018-10-29 15:59:05 +09:00
77fd5dbf3e
Remove dead code, including tests.
...
Change-Id: Id88b02b74bdfe4ca05b08181ceb6b34d5652fc0c
2015-03-10 17:22:41 -07:00
12d80ebead
Remove shortcut support from LatinIME.
...
Note this change does not affect the native decoder interface.
Change-Id: I73b7dc008a5acaf75a31a36a2d332b5afabd82d0
2015-02-11 13:02:41 -08:00
1da382b81e
Remove unused import
...
Change-Id: I9e2a79534673b2c6587f57fad663b85d971658b9
2014-11-12 16:27:14 -08:00
e1652d7570
Fix the build : move offdevice tests in an off device file
...
Also fix a failing test
Change-Id: I280aa6b5836c7332da08a88f1258546edeb25a37
2014-11-06 22:33:57 +09:00
5b33d197ba
Add a header command to dicttool.
...
This will allow to greatly improve the performance of the
metadata-generating files, as they won't have to wait for
the info command to read the entire dictionary when the
header is all we need.
Also add tests, and while we're at it, use the seed as
intended to enable reproducible tests.
Change-Id: I0ba79ef62f0292b23e63aed57ff565bb102281a2
2014-11-06 18:50:59 +09:00
88480f39f6
Update unit tests for new static dict version.
...
Bug: 11031090
Change-Id: Ifd7b241f1a0a30678ca7de97cc6e96927a9b5c8f
2014-10-31 14:50:18 +09:00
54a1b8ec54
Add latinime-common local lib
...
This is just a placeholder in this CL. It will be shared by LatinIME,
dicttool, and tests.
Change-Id: I6fb3516a5061555f6f24b29141c2871d2319b023
2014-10-22 19:26:21 +09:00
5f00fe09e9
Fix some compiler warnings
...
This CL fixes the following compiler warnings.
- Indirect access to static member
- Access to a non-accessible member of an enclosing type
- Parameter assignment
- Method can be static
- Local variable declaration hides another field or variable
- Value of local variable is not used
- Unused import
- Unused private member
- Unnecessary 'else' statement
- Unnecessary declaration of throw exception
- Redundant type arguments
- Missing '@Override' annotation
- Unused '@SuppressWarning' annotations
Bug: 18003991
Change-Id: Icfebe753e53a2cc621848f769d6a3d7ce501ebc7
2014-10-21 19:28:37 +09:00
05172bf1a5
Renaming "blacklist" flag to "possibly offensive"
...
No behaviour changes.
Unified the overloaded FusionDictionary::add method to always take an
isPossiblyOffensive argument.
Bug: 11031090
Change-Id: I5741a023ca1ce842d2cf10d4f6c926b0efabaa78
2014-10-21 11:51:47 +09:00
3a41528cbe
Merge "Fix testCharacterTableIsPresent to use BinaryDictUtils.getDictFile"
2014-10-03 12:04:05 +00:00
6164f1fcb9
Fix testCharacterTableIsPresent to use BinaryDictUtils.getDictFile
...
Bug:17097992
Change-Id: Ic26c95fccb63d592883f85fe34f4570f1861b8c3
2014-10-03 20:07:41 +09:00
80861cde9d
Merge "Fix bug of testCharacterTableIsPresent (dicttool test)"
2014-10-03 10:07:04 +00:00
71f9b8d50b
Fix bug of testCharacterTableIsPresent (dicttool test)
...
Bug:17097992
Change-Id: Ibe343b8cc704631e84ed9d3a39feee30a54bdc38
2014-10-03 18:56:41 +09:00
b28d1cc487
Fix a behavior change in dicttool
...
The behavior change was introduced by I8b3458ad. Concretely,
empty bigram lists would end up as empty lists instead of null.
Change-Id: Ibcdf7e6aabc6aba3f5db0477335882394e050ce5
2014-10-03 18:04:10 +09:00
fb051c3957
Merge changes I3c1f5ac1,I269c9aa8
...
* changes:
Switch code point table
Test for code point table (dicttool test)
2014-10-03 07:55:59 +00:00
7e5614520a
Merge "Include a code point table in the binary dictionary."
2014-10-02 08:55:18 +00:00
25c884ba65
Test for code point table (dicttool test)
...
Bug:17097992
Change-Id: I269c9aa86378f32083f8688f4ce91862d47dd181
2014-10-02 12:28:04 +09:00
9168ab60cf
Include a code point table in the binary dictionary.
...
Bug:17097992
Change-Id: I677a5eb3a704e4386f6573360e44ca335d81d2df
2014-10-02 12:27:49 +09:00
c6a6f6a990
Introduce NgramProperty in Java side.
...
Bug: 14425059
Change-Id: I8b3458ad22730b3dccbe0caea2c5930f5276dc82
2014-10-01 11:21:08 +09:00
88fa47a27d
Support migration/dump of Beginning-of-Sentence entries.
...
Bug: 14119293
Change-Id: Ie975138f819794d5c34a7a547be5a6117050e084
2014-06-24 12:37:07 +09:00
a91561aa58
Use Java 7 diamond operator
...
Change-Id: If16ef50ae73147594615d0f49d6a22621eaf1aef
2014-05-24 01:05:42 +09:00
93cda5bb39
Move code only used for dicttool and tests under tests.
...
Bug: 13035567
Change-Id: I13c6df013ef2b67c9bf67455d9c32d283bf9ea2e
2014-03-27 15:30:32 +09:00
516f86815d
Separate WeightedString from FusionDictionary.
...
Bug: 8187060
Change-Id: I40c1dafca3eb52244c64fdb4c1db30a56385d678
2014-03-06 18:53:06 +09:00
e784148ae6
Separate utility methods from BinaryDictionary.
...
Bug: 8187060
Change-Id: Ice2984e332b7bd3bb17174aefc80b5635b72fc50
2014-03-05 18:19:34 +09:00
890b44e537
Correctly read the header of APK-embedded dicts
...
Bug: 13164518
Change-Id: I8768ad887af8b89ad9f29637f606c3c68629c7ca
2014-02-24 22:54:01 +09:00
95d16561e0
Remove unused code.
...
Bug: 12810574
Change-Id: I9c7fff60ae0e94d52f3bd19c3e88de5a53b917d7
2014-02-15 17:39:24 +09:00
0fc93fe445
Implement PatriciaTriePolicy::getNextWordAndNextToken().
...
Bug: 12810574
Change-Id: Id1d44f90de9455d9cbe7b6e0a161cae91d6d422c
2014-02-15 17:39:20 +09:00
85fe06e759
Merge "Remove unused argument from readDictionaryBinary."
2014-02-14 10:37:56 +00:00
8e3a1d0f89
Remove unused argument from readDictionaryBinary.
...
Bug: 12810574
Change-Id: Ice415ebd8d11162facca3fe8927ef8a616b11424
2014-02-14 19:02:15 +09:00
8fa7a09f1e
Merge "Implement PatriciaTriePolicy::getWordProperty()."
2014-02-14 09:08:09 +00:00
c63d183473
Implement PatriciaTriePolicy::getWordProperty().
...
Bug: 12810574
Change-Id: I7bcccfd3641ebbcf2b8d857d33bb4734c42af5eb
2014-02-14 17:56:45 +09:00
da973e75dc
Make InputLogicTest more robust
...
Change-Id: I134f14971126cbeed05b472c08747f2b88ad30e6
2014-02-13 19:38:51 +09:00
8ffc631826
Make PtNode have ProbabilityInfo instead of raw value.
...
Bug: 11281877
Bug: 12810574
Change-Id: Id1cda0afc74c4e30633c735729143491b2274a7b
2014-02-10 15:05:08 +09:00
ab6a93773b
Use native logic to read Ver4 dict.
...
Bug: 11281877
Bug: 12810574
Change-Id: Ief371d3ef61818e4e031de4659aee3c9584c7379
2014-02-06 21:55:37 +09:00
b986f78ba8
Separate header class from FormatSpec.
...
Bug: 12810574
Change-Id: Iacf1cd05a268bf690ab864b5e32a18a4b0ccc693
2014-02-04 21:36:04 +09:00
5cb7509314
Fix BinaryDictDecoderEncoderTests.
...
Bug: 12809791
Change-Id: I04313df78692b01e153a34c932a37f079a924105
2014-01-31 19:44:17 +09:00
26bd46095a
Reading dictionary containing timestamps in Java Side.
...
Just skipping historical information fields.
Bug: 11281877
Change-Id: I43d2adaa576b7da11ed3ca54990265dbb6f53b08
2014-01-29 20:19:24 +09:00
c2fd53ee0e
Remove ver4 dict updater.
...
Change-Id: I468994c98d091be621b9fb3fbe6405c67fc6a465
2013-12-17 18:17:51 +09:00
42334bb493
Quit checking bigram order in BinaryDictDecoderEncoderTests.
...
Change-Id: I1b8eb6ab2ea797d2590495b1f57f9ec9560ea817
2013-12-17 17:38:24 +09:00
4fdcefe504
Move DictUpdater to the tests directory.
...
Bug: 11245133
Change-Id: I0907a091ac3ae960eaf3b27da78dbb48a24b2ea1
2013-12-17 14:31:25 +09:00
b868375763
Fix failing tests
...
- Version 3 is not supported
- Now passing the right string to open v4 dicts. Fix the tests for this.
Change-Id: I7829330c3568a715b96396ba4e4e69c6e17775ab
2013-12-16 14:32:19 +09:00
7b55cd3e2b
Remove flags from Java side.
...
This simplifies the code quite a bit.
- GERMAN_UMLAUTS are now handled through a key-value attribute.
The dictionary generator does not need to know about it any more.
- FRENCH_LIGATURES are deprecated as we handle them with shortcuts now.
- CONTAINS_BIGRAMS is deprecated. Bigram processing is always applied
regardless of this flag.
Bug: 11281748
Change-Id: If567e52e245a9342adc7f3104a0f7d8d782df8c1
2013-12-13 18:15:05 +09:00
2fa3693c26
Reset to 9bd6dac470
...
The bulk merge from -bayo to klp-dev should not have been merged to master.
Change-Id: I527a03a76f5247e4939a672f27c314dc11cbb854
2013-12-13 17:13:32 +09:00
9514ed5c2a
Add the new format of bigram entries.
...
In new format, each bigram entry has flags (1 byte), a terminal id (3 byte),
a time-stamp (4 byte), a counter (1 byte) and a level (1 byte).
Bug: 10920255
Bug: 10920165
Change-Id: I0f7fc125a6178e6d25a07e8462afc41a7f57e3e1
2013-10-11 14:50:41 +09:00
e4e0add9fb
Add Ver4DictUpdater.
...
Change-Id: I986ab26faf535fc4bc98443053f534eced9d048f
2013-10-04 17:33:29 +09:00
75d60e821c
Refactor BinaryDictIOUtilsTests.
...
Change-Id: I2208378b33038771b460abb33f9a690872e998e2
2013-10-04 14:19:13 +09:00
d6e307a4b7
Add DictUpdater.
...
Change-Id: Ic586e46e5a9f59de53d53e59886d635345940974
2013-10-03 20:16:34 +09:00
3aa8977cb2
Remove some unused variables.
...
Change-Id: Iaf1556fec194d17cb4318f2bdcc837f8d79449ef
2013-10-02 18:26:03 +09:00
fa946d4a0f
Fix a test and crash with a better error message when reading
...
When there are too many bigrams, we stop reading the file,
so the file pointer is in an inconsistent place. This means we
have no idea what's going to happen next. It's better to crash
right away.
Change-Id: Id3b7b78cbe4fda3493b3c9c46758763e1ab5f6a3
2013-10-02 11:48:47 +09:00
Tadashi G. Takaoka
Dan Zivkovic
Jean Chalard
Adrian Velicu
Ken Wakasa
Akifumi Yoshimoto
Keisuke Kuroyanagi
Yuichiro Hanada