Jean Chalard
75de9fb7ff
Remove flags from Java side.
...
This simplifies the code quite a bit.
- GERMAN_UMLAUTS are now handled through a key-value attribute.
The dictionary generator does not need to know about it any more.
- FRENCH_LIGATURES are deprecated as we handle them with shortcuts now.
- CONTAINS_BIGRAMS is deprecated. Bigram processing is always applied
regardless of this flag.
Bug: 11281748
Change-Id: I55a11ba61d3589c1584a3fa6c941374b349b7b5c
2013-12-12 17:01:29 +09:00
Yuichiro Hanada
48e01ec111
Make dicttool read the compressed combined format.
...
Change-Id: Ib39fa110402895a655f4e705caae53397ace9259
2013-09-30 14:59:19 +09:00
Yuichiro Hanada
0e40cd0c40
Add getDictDecoder.
...
Bug: 9618601
Change-Id: I173100ac704c03f7d5d0d53477e83cab5d1110d4
2013-09-12 20:14:09 +09:00
Yuichiro Hanada
95bc256f41
Add a flag to readDictioanryBinary in DictDecoder.
...
Change-Id: I356adb72047ebc43c924fbff1ff45e7460508a31
2013-09-11 18:20:56 +09:00
Yuichiro Hanada
576f625ee1
Rename CharGroup to PtNode.
...
Bug: 10233675
Change-Id: I7b0eb07d195cd386cd0d9e97cd59bf48fcf24107
2013-08-26 15:58:30 +09:00
Yuichiro Hanada
e9a10ff0f0
Add DictDecoder.readDictionaryBinary.
...
Bug: 10434720
Change-Id: I14690a6e0f922ed1bab3a4b6c9a457ae84d4c1a4
2013-08-23 20:29:25 +09:00
Yuichiro Hanada
c922c8a504
Add DictEncoder.
...
Change-Id: I41049b9118b58838e5dedf8e5618d939ca70c5ef
2013-08-22 11:53:41 +09:00
Yuichiro Hanada
a306e08753
Rename BinaryDictEncoder to BinaryDictEncoderUtils.
...
Change-Id: I4dabf17da7003b1d8204a83dbd10e5be6e8fd805
2013-08-21 18:54:34 +09:00
Yuichiro Hanada
112257e40f
Rename BinaryDictDecoder to Ver3DictDecoder.
...
Change-Id: Ibf9b95b658df6e2c2218bdb62e2380f326a03832
2013-08-20 17:11:51 +09:00
Yuichiro Hanada
77bce05e6f
[Refactor] Rename BinaryDictReader and BinaryDictDecoder.
...
BinaryDictReader -> BinaryDictDecoder.
BinaryDictDecoder -> BianryDictDecoderUtils.
Change-Id: Iadf2153b379b760538ecda488dda4f17225e5f37
2013-08-19 19:36:31 +09:00
Jean Chalard
af30cbf0ee
Rename Node to PtNodeArray
...
Bug: 10247660
Change-Id: I1a0ac19f58f96adb5efac5fd35c6404831618c99
2013-08-16 16:24:54 +09:00
Yuichiro Hanada
a94eb97413
Remove unused imports.
...
Change-Id: Ib46da6efeaa8f83a9cda24c947fc2e667a1468e8
2013-08-16 11:19:53 +09:00
Yuichiro Hanada
94460eba11
[Refactor] Divide BinaryDictInputOutput into BinaryDictEncoder and BinaryDictDecoder.
...
Change-Id: I7c3269d77e3e3b567e459dcaa1bc029903941744
2013-08-15 20:23:07 +09:00
Ken Wakasa
117f18e844
Revert "[Refactor] Divide BinaryDictInputOutput into BinaryDictInputUtils and BinaryDictOutputUtils."
...
This reverts commit 4c63d0614e
2013-08-15 08:54:29 +00:00
Yuichiro Hanada
4c63d0614e
[Refactor] Divide BinaryDictInputOutput into BinaryDictInputUtils and BinaryDictOutputUtils.
...
Change-Id: I0d476abe763c11ba9005152f928e8dccf15ac9de
2013-08-15 15:46:58 +09:00
Yuichiro Hanada
3feacba1eb
Add BinaryDictReader.
...
Bug: 9618601
Change-Id: Ief07fa0c3c4f7f5999a3fafcef4e47b6b6fd8143
2013-08-13 19:55:05 +09:00
Jean Chalard
23d4eb55ba
Add tests to dicttool test.
...
Bug: 8526576
Change-Id: Idd6f9cd076d5915361c68f5c29afbba67dd54eba
2013-06-20 17:29:37 +09:00
Jean Chalard
a411595b16
Fix two nasty bugs with surrogate pairs.
...
The important bug is in findWordInTree. The problem, which is
not obvious, is that we were calling codePointAt() with the
code point index in the string, instead of the char index.
The other bug this change fixes was harmless in the practice,
because it's in the iteration which is only used for debug and
pretty printing purposes. It's very similar in that it would
substract a length in code point to a length in chars and
truncate a StringBuilder at that length, so it would fail in a
quite similar manner. This changes the meaning of the "length"
attribute in Position, but it's clearer this way anyway.
Bug: 8450145
Change-Id: If396f883a9e6449de39351553ba83f5be5bd30f0
2013-04-01 17:06:19 +09:00
Jean Chalard
a8058d169d
Cleanup after dicttool
...
Arrange for temporary files to be deleted.
Also, add a security on stack overflows.
Change-Id: I9da2ebefb06409a71b235243ea835ce10d6a9b81
2012-10-30 19:57:02 +09:00
Jean Chalard
b3c98901c5
Add auto detection and decoding of dictionary files. (A2)
...
Bug: 7388852
Change-Id: I25e755fc15f5b383acc046f668e9681efa4f0c2f
2012-10-25 16:40:15 +09:00
Tadashi G. Takaoka
15f6d4ae34
Add @UsedForTesting and @ExternallyReferenced annotations
...
Bug: 7268357
Change-Id: I0b7e0c19f04af9ae30874d0a4c26ad81bc80be8c
2012-10-22 11:18:43 -07:00
Jean Chalard
72b1c93941
Reinstate the shortcut-only attribute
...
Also add the blacklist attribute
Bug: 7005742
Bug: 2704000
Change-Id: Icbe60bdf25bfb098d9e3f20870be30d6aef07c9d
2012-08-31 22:11:52 +09:00
Jean Chalard
3598053d41
Move dicttool tests and make a script to run them (A4)
...
Change-Id: If582e81741c5edaf345dcb55517dcad0baa64c9e
2012-08-04 01:55:33 +09:00
