nyancrimew/LatinIME
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
29312 commits 1 branch 0 tags 363 MiB
Commit graph

80 commits

Author SHA1 Message Date
Keisuke Kuroyanagi
42334bb493 Quit checking bigram order in BinaryDictDecoderEncoderTests. 
Change-Id: I1b8eb6ab2ea797d2590495b1f57f9ec9560ea817
 2013-12-17 17:38:24 +09:00
Keisuke Kuroyanagi
4fdcefe504 Move DictUpdater to the tests directory. 
Bug: 11245133
Change-Id: I0907a091ac3ae960eaf3b27da78dbb48a24b2ea1
 2013-12-17 14:31:25 +09:00
Jean Chalard
b868375763 Fix failing tests 
- Version 3 is not supported
- Now passing the right string to open v4 dicts. Fix the tests for this.

Change-Id: I7829330c3568a715b96396ba4e4e69c6e17775ab
 2013-12-16 14:32:19 +09:00
Jean Chalard
1dc7eca114 Remove references to the v3 dictionary. 
Change-Id: I811c8c923ad67a6d9bfdd11bdef8991eb7135c27
 2013-12-13 18:53:41 +09:00
Jean Chalard
a245d15da5 Have dicttool use the native library to generate v4 dicts. 
Yay !

Change-Id: Iea8ced9e81031b9ab7eff05ad9ef7215be248de9
 2013-12-13 18:18:20 +09:00
Jean Chalard
7b55cd3e2b Remove flags from Java side. 
This simplifies the code quite a bit.
- GERMAN_UMLAUTS are now handled through a key-value attribute.
  The dictionary generator does not need to know about it any more.
- FRENCH_LIGATURES are deprecated as we handle them with shortcuts now.
- CONTAINS_BIGRAMS is deprecated. Bigram processing is always applied
  regardless of this flag.

Bug: 11281748
Change-Id: If567e52e245a9342adc7f3104a0f7d8d782df8c1
 2013-12-13 18:15:05 +09:00
Ken Wakasa
2fa3693c26 Reset to 9bd6dac470 
The bulk merge from -bayo to klp-dev should not have been merged to master.

Change-Id: I527a03a76f5247e4939a672f27c314dc11cbb854
 2013-12-13 17:13:32 +09:00
Yuichiro Hanada
9514ed5c2a Add the new format of bigram entries. 
In new format, each bigram entry has flags (1 byte), a terminal id (3 byte),
a time-stamp (4 byte), a counter (1 byte) and a level (1 byte).

Bug: 10920255
Bug: 10920165
Change-Id: I0f7fc125a6178e6d25a07e8462afc41a7f57e3e1
 2013-10-11 14:50:41 +09:00
Keisuke Kuroyanagi
61aae2b450 Merge "Add Ime language switching test." 2013-10-08 05:35:25 +00:00
Keisuke Kuroyanagi
72bc9ad37e Add Ime language switching test. 
Change-Id: I6a96dc5fdd533899353d537382608c2759faff1c
 2013-10-08 14:00:57 +09:00
Yuichiro Hanada
e4e0add9fb Add Ver4DictUpdater. 
Change-Id: I986ab26faf535fc4bc98443053f534eced9d048f
 2013-10-04 17:33:29 +09:00
Yuichiro Hanada
75d60e821c Refactor BinaryDictIOUtilsTests. 
Change-Id: I2208378b33038771b460abb33f9a690872e998e2
 2013-10-04 14:19:13 +09:00
Yuichiro Hanada
d6e307a4b7 Add DictUpdater. 
Change-Id: Ic586e46e5a9f59de53d53e59886d635345940974
 2013-10-03 20:16:34 +09:00
Yuichiro Hanada
3aa8977cb2 Remove some unused variables. 
Change-Id: Iaf1556fec194d17cb4318f2bdcc837f8d79449ef
 2013-10-02 18:26:03 +09:00
Yuichiro Hanada
4284e9aae7 Make SparseTable have multiple content tables. 
Bug: 10920165
Change-Id: Ie9008452ee292fb0b1fec66e2ffed228c4af6c3e
 2013-10-02 15:36:13 +09:00
Jean Chalard
fa946d4a0f Fix a test and crash with a better error message when reading 
When there are too many bigrams, we stop reading the file,
so the file pointer is in an inconsistent place. This means we
have no idea what's going to happen next. It's better to crash
right away.

Change-Id: Id3b7b78cbe4fda3493b3c9c46758763e1ab5f6a3
 2013-10-02 11:48:47 +09:00
Yuichiro Hanada
d188af7022 Add SparseTable. 
Bug: 10920165
Change-Id: I749dd0269e788799e30b10beb2671813d40ce15f
 2013-09-26 16:16:30 +09:00
Yuichiro Hanada
1625aeafd2 Fix runReadUnigramsAndBigramsTests. 
Change-Id: Idd9176c9943dfacac5a06957f1a07187b642b207
 2013-09-24 12:31:45 +09:00
Yuichiro Hanada
14087ba52c Add Ver4DictDecoder. 
Bug: 9618601
Change-Id: I43c5840505c6a847aaf4893a400392ccd45903c0
 2013-09-19 16:11:23 +09:00
Keisuke Kuroyanagi
78b55a31cb Fix handling multi-bytes characters and add a test. 
Bug: 6669677

Change-Id: Id2154db47adea2929559a4187a726f9dfa83363e
 2013-09-17 15:11:24 +09:00
Yuichiro Hanada
a141d8ef7d Add Ver4DictEncoder. 
Bug: 9618601
Change-Id: I161d2845906f07c1251deb8005fdffe49c5b7940
 2013-09-13 17:33:51 +09:00
Yuichiro Hanada
0e40cd0c40 Add getDictDecoder. 
Bug: 9618601
Change-Id: I173100ac704c03f7d5d0d53477e83cab5d1110d4
 2013-09-12 20:14:09 +09:00
Yuichiro Hanada
95bc256f41 Add a flag to readDictioanryBinary in DictDecoder. 
Change-Id: I356adb72047ebc43c924fbff1ff45e7460508a31
 2013-09-11 18:20:56 +09:00
Yuichiro Hanada
2232a70806 Clean up unused imports. 
Change-Id: I7147ca237b99399e79210852aa5bf5a01101d779
 2013-08-26 08:29:16 +00:00
Yuichiro Hanada
752a33640c [Refactor] Add DictDecoder.readUnigramsAndBigramsBinary. 
Change-Id: I259db91d837c67cbcb3b6dc504b21dca23a6a5be
 2013-08-26 17:24:38 +09:00
Yuichiro Hanada
bb5b84a826 [Refactor] Add DictDecoder.getTerminalPosition. 
Change-Id: I9d04f64a58f5481cbb64cf1c09b5c485dd4176b4
 2013-08-26 16:14:59 +09:00
Yuichiro Hanada
576f625ee1 Rename CharGroup to PtNode. 
Bug: 10233675
Change-Id: I7b0eb07d195cd386cd0d9e97cd59bf48fcf24107
 2013-08-26 15:58:30 +09:00
Yuichiro Hanada
e9a10ff0f0 Add DictDecoder.readDictionaryBinary. 
Bug: 10434720
Change-Id: I14690a6e0f922ed1bab3a4b6c9a457ae84d4c1a4
 2013-08-23 20:29:25 +09:00
Yuichiro Hanada
373c492a02 Add an unit test for CharEncoding. 
Change-Id: Ifb1cc01fa5bc2d6d69671f1acb9b9675a4081d32
 2013-08-22 23:05:09 +09:00
Yuichiro Hanada
aa4168ee09 Fix writePlacedNode. 
Change-Id: I1d6b086f1d9f0dbd8d74f964e29ae62c533af978
 2013-08-22 23:02:08 +09:00
Yuichiro Hanada
e301085a70 Move findWordByBinaryDictReader to BinaryDictIOUtilsTests. 
Change-Id: I443238fd816dea9650dcbbeb3ea757f9674fa52f
 2013-08-22 14:49:49 +09:00
Yuichiro Hanada
c922c8a504 Add DictEncoder. 
Change-Id: I41049b9118b58838e5dedf8e5618d939ca70c5ef
 2013-08-22 11:53:41 +09:00
Yuichiro Hanada
558e34c7bd Make readPtNode be called with the address from the beginning of the file. 
Change-Id: I8939fdfb4f79e55bcd7393633784effb30df3f8f
 2013-08-21 20:02:18 +09:00
Yuichiro Hanada
a306e08753 Rename BinaryDictEncoder to BinaryDictEncoderUtils. 
Change-Id: I4dabf17da7003b1d8204a83dbd10e5be6e8fd805
 2013-08-21 18:54:34 +09:00
Yuichiro Hanada
107a5f6fb8 Add PtNodeReader. 
Change-Id: Ic918822fc1b3a8a7c39ffbcf7defde2c5bf888db
 2013-08-21 18:43:18 +09:00
Yuichiro Hanada
065aad9501 Add DictDecoder. 
Change-Id: Ia1c32f21fe07081ce04d093660e18146b93275a4
 2013-08-20 17:43:13 +09:00
Yuichiro Hanada
112257e40f Rename BinaryDictDecoder to Ver3DictDecoder. 
Change-Id: Ibf9b95b658df6e2c2218bdb62e2380f326a03832
 2013-08-20 17:11:51 +09:00
Yuichiro Hanada
66004ce2de Remove populateOptions. 
Change-Id: I1a1830aaa8ea586b68fc34ff3a27ae52b810e8af
 2013-08-20 16:06:52 +09:00
Yuichiro Hanada
77bce05e6f [Refactor] Rename BinaryDictReader and BinaryDictDecoder. 
BinaryDictReader -> BinaryDictDecoder.
BinaryDictDecoder -> BianryDictDecoderUtils.

Change-Id: Iadf2153b379b760538ecda488dda4f17225e5f37
 2013-08-19 19:36:31 +09:00
Ken Wakasa
a83e25642f Merge "Add HeaderReaderInterface." 2013-08-19 02:34:23 +00:00
Yuichiro Hanada
d794b42f98 Add HeaderReaderInterface. 
Change-Id: I298f86b70d18cd08b240509b6f757c72e1a59ffe
 2013-08-19 11:15:03 +09:00
Yuichiro Hanada
8aaae56cf6 Fix unit test. 
Change-Id: Ib104d5de71c2ab1a07921b407c74c21b0409d9af
 2013-08-19 11:10:28 +09:00
Yuichiro Hanada
3a73b37b30 Make BinaryDictIOUtils and DynamicBinaryIOUtils use BinaryDictReader. 
Change-Id: I191dfe0e05ff3c2c5af99e8beebbb73b097748a3
 2013-08-16 21:06:23 +09:00
Jean Chalard
af30cbf0ee Rename Node to PtNodeArray 
Bug: 10247660
Change-Id: I1a0ac19f58f96adb5efac5fd35c6404831618c99
 2013-08-16 16:24:54 +09:00
Yuichiro Hanada
94460eba11 [Refactor] Divide BinaryDictInputOutput into BinaryDictEncoder and BinaryDictDecoder. 
Change-Id: I7c3269d77e3e3b567e459dcaa1bc029903941744
 2013-08-15 20:23:07 +09:00
Ken Wakasa
117f18e844 Revert "[Refactor] Divide BinaryDictInputOutput into BinaryDictInputUtils and BinaryDictOutputUtils." 
This reverts commit 4c63d0614e.

Change-Id: I1fa277d720bab4d895259df7d6d82eebfa5eb6c5
 2013-08-15 08:54:29 +00:00
Yuichiro Hanada
4c63d0614e [Refactor] Divide BinaryDictInputOutput into BinaryDictInputUtils and BinaryDictOutputUtils. 
Change-Id: I0d476abe763c11ba9005152f928e8dccf15ac9de
 2013-08-15 15:46:58 +09:00
Yuichiro Hanada
3edb62c69b Move some methods in BinaryDictIOUtils to DynamicBinaryDictIOUtils. 
Change-Id: I9ba55582c533fef0eb3e60c46bf23c8b16ee1ff4
 2013-08-14 19:33:36 +09:00
Yuichiro Hanada
bbc8a930f7 Add FusionDictionaryBufferFromWritableByteBufferFactory. 
Change-Id: I23de0a178e7f11f2cf301fd433cde60c6152055b
 2013-08-14 17:07:44 +09:00
Yuichiro Hanada
3feacba1eb Add BinaryDictReader. 
Bug: 9618601

Change-Id: Ief07fa0c3c4f7f5999a3fafcef4e47b6b6fd8143
 2013-08-13 19:55:05 +09:00