Commit graph

177 commits

Author SHA1 Message Date
Yuichiro Hanada
14d31d4640 Add AbstractDictDecoder.
Change-Id: Ie69d84e090f69dc3ea1f5de73ad8c954ecd2c6a7
2013-10-04 14:16:59 +09:00
Yuichiro Hanada
73b9d3b879 Separate shortcut targets from the trie file.
Bug: 10920165
Change-Id: I340759eadbde7fb64cb3b9a3c619ee3a768cedf8
2013-10-04 11:16:10 +09:00
Yuichiro Hanada
3dd77a6d66 Add SparseTableContentWriter to Ver4DictEncoder.
Bug: 10920165
Change-Id: I6372492e97297baad4c5aeeb3fb36dcccd7a944b
2013-10-03 20:31:01 +09:00
Yuichiro Hanada
d6e307a4b7 Add DictUpdater.
Change-Id: Ic586e46e5a9f59de53d53e59886d635345940974
2013-10-03 20:16:34 +09:00
Yuichiro Hanada
3aa8977cb2 Remove some unused variables.
Change-Id: Iaf1556fec194d17cb4318f2bdcc837f8d79449ef
2013-10-02 18:26:03 +09:00
Yuichiro Hanada
4284e9aae7 Make SparseTable have multiple content tables.
Bug: 10920165
Change-Id: Ie9008452ee292fb0b1fec66e2ffed228c4af6c3e
2013-10-02 15:36:13 +09:00
Jean Chalard
fa946d4a0f Fix a test and crash with a better error message when reading
When there are too many bigrams, we stop reading the file,
so the file pointer is in an inconsistent place. This means we
have no idea what's going to happen next. It's better to crash
right away.

Change-Id: Id3b7b78cbe4fda3493b3c9c46758763e1ab5f6a3
2013-10-02 11:48:47 +09:00
Yuichiro Hanada
bc4926235d Add skipPtNode to DictDecoders.
Change-Id: I042ff041b68572182c87dc87db6a6aa2bbbefc6c
2013-10-01 17:21:21 +09:00
Ken Wakasa
25f4782853 Merge "Separate bigram entries from the trie file." 2013-09-30 08:00:01 +00:00
Yuichiro Hanada
fd46e87da2 Separate bigram entries from the trie file.
Bug: 10920165
Change-Id: I42eb7f228994a56c2f31a1c83904c8c360b6d088
2013-09-30 16:42:42 +09:00
Yuichiro Hanada
dc4f357f6b Change scopes of some methods in DynamicBinaryDictIOUtils.
Change-Id: I1371d41760b38834d1d5ff40e682ac0a6fc9f2e9
2013-09-30 11:32:07 +09:00
Keisuke Kuroyanagi
5ed30a7660 Set header attributes for ExpandableBinaryDictionary.
Bug: 6669677

Change-Id: I2462777eaa07600f1c774226adfc4f351882f523
2013-09-28 12:50:09 +09:00
Keisuke Kuroyanagi
e3e10317b0 Merge "Create empty dictionary file in native code." 2013-09-26 08:03:38 +00:00
Yuichiro Hanada
d188af7022 Add SparseTable.
Bug: 10920165
Change-Id: I749dd0269e788799e30b10beb2671813d40ce15f
2013-09-26 16:16:30 +09:00
Keisuke Kuroyanagi
5ef6209656 Create empty dictionary file in native code.
Bug: 6669677
Change-Id: I46d23deee1fd091678f4593561cb8687eb815212
2013-09-26 12:59:02 +09:00
Yuichiro Hanada
22c5c450fe Make Ver4DictEncoder write an address table of terminal nodes.
Bug: 10920165
Change-Id: I86017456ea9fa5d6e12b57172c34f5ed4f88ef94
2013-09-25 17:03:54 +09:00
Yuichiro Hanada
14087ba52c Add Ver4DictDecoder.
Bug: 9618601
Change-Id: I43c5840505c6a847aaf4893a400392ccd45903c0
2013-09-19 16:11:23 +09:00
Yuichiro Hanada
d0c87576ff Fix getChildrenPosition.
Change-Id: Ic39592dbf0d4e6d2e4879cdb1a95b65d46a57e92
2013-09-18 15:50:10 +09:00
Yuichiro Hanada
a141d8ef7d Add Ver4DictEncoder.
Bug: 9618601
Change-Id: I161d2845906f07c1251deb8005fdffe49c5b7940
2013-09-13 17:33:51 +09:00
Yuichiro Hanada
0e40cd0c40 Add getDictDecoder.
Bug: 9618601
Change-Id: I173100ac704c03f7d5d0d53477e83cab5d1110d4
2013-09-12 20:14:09 +09:00
Yuichiro Hanada
be470f06e4 Hide DictBuffer from BinaryDictDecoderUtils.
Bug: 9618601
Change-Id: I87515bc1555bdfe09426a056ceda74d8e63eba82
2013-09-12 16:49:54 +09:00
Yuichiro Hanada
7547a7042c Add writePtNode to DictEncoder.
Bug: 9618601
Change-Id: Ie4529444780edee8d0f0d5f23c5619c1a3344286
2013-09-11 18:27:06 +09:00
Yuichiro Hanada
95bc256f41 Add a flag to readDictioanryBinary in DictDecoder.
Change-Id: I356adb72047ebc43c924fbff1ff45e7460508a31
2013-09-11 18:20:56 +09:00
Yuichiro Hanada
8a06b26cb4 Check the version of format options before writing.
Change-Id: I3951e4395a2da225e240e856c23339b679e6155e
2013-09-10 18:59:31 +09:00
Yuichiro Hanada
70e81af726 Hide physical structure of the dictionary from BinaryDictEncoderUtils.
Bug: 9618601
Change-Id: I01b8da42058d7c05bf7f4e6d43718032c2ad12c5
2013-09-09 14:55:33 +09:00
Yuichiro Hanada
563bcb5117 Rename write(|Signed)VariableAddress and to write(|Signed)ChildrenPosition.
Change-Id: Ib018bde346744b470a4fb13b0c57e3a633b8bb20
2013-09-06 23:53:58 +09:00
Yuichiro Hanada
6156892b7e Add computeChildrenPosition and simplify writePlacedNode.
Change-Id: Idcf0e056bf61e29844d4d0e91c482cd78c41704c
2013-09-06 23:52:10 +09:00
Yuichiro Hanada
55f5f7a005 [Refactor] Add writeDictionaryHeader.
Change-Id: I69026c47ce1d23f5c39c99ace76fa6b96cd1ce1b
2013-09-05 16:19:26 +09:00
Yuichiro Hanada
663f5375d1 Add comments about how to calculate parent and children addresses.
Change-Id: Idf37d9a08a7b464a14c8755bedb755005e49ba33
2013-09-05 15:10:28 +09:00
Yuichiro Hanada
d5a3c593db Remove a broken dictionary file when fail to read a dictionary file.
Bug: 10434720
Change-Id: Ibdf05a39113538546b8fcf9d59af7dddf7ca27fc
2013-08-26 18:08:21 +09:00
Yuichiro Hanada
b50a7a4bc7 Simplify computeActualNodeArraySize.
Change-Id: I128d02866c2cc70c42263496927327b8a2130139
2013-08-26 08:30:23 +00:00
Yuichiro Hanada
752a33640c [Refactor] Add DictDecoder.readUnigramsAndBigramsBinary.
Change-Id: I259db91d837c67cbcb3b6dc504b21dca23a6a5be
2013-08-26 17:24:38 +09:00
Yuichiro Hanada
bb5b84a826 [Refactor] Add DictDecoder.getTerminalPosition.
Change-Id: I9d04f64a58f5481cbb64cf1c09b5c485dd4176b4
2013-08-26 16:14:59 +09:00
Yuichiro Hanada
576f625ee1 Rename CharGroup to PtNode.
Bug: 10233675
Change-Id: I7b0eb07d195cd386cd0d9e97cd59bf48fcf24107
2013-08-26 15:58:30 +09:00
Yuichiro Hanada
8f4d5d5f23 Fix getDictionaryFileHeader.
Change-Id: I5186f41b0f38eb08f4154f7b3de7ffc2119fc44b
2013-08-23 22:39:36 +09:00
Yuichiro Hanada
e9a10ff0f0 Add DictDecoder.readDictionaryBinary.
Bug: 10434720
Change-Id: I14690a6e0f922ed1bab3a4b6c9a457ae84d4c1a4
2013-08-23 20:29:25 +09:00
Yuichiro Hanada
aa4168ee09 Fix writePlacedNode.
Change-Id: I1d6b086f1d9f0dbd8d74f964e29ae62c533af978
2013-08-22 23:02:08 +09:00
Yuichiro Hanada
e301085a70 Move findWordByBinaryDictReader to BinaryDictIOUtilsTests.
Change-Id: I443238fd816dea9650dcbbeb3ea757f9674fa52f
2013-08-22 14:49:49 +09:00
Yuichiro Hanada
c922c8a504 Add DictEncoder.
Change-Id: I41049b9118b58838e5dedf8e5618d939ca70c5ef
2013-08-22 11:53:41 +09:00
Yuichiro Hanada
558e34c7bd Make readPtNode be called with the address from the beginning of the file.
Change-Id: I8939fdfb4f79e55bcd7393633784effb30df3f8f
2013-08-21 20:02:18 +09:00
Yuichiro Hanada
a306e08753 Rename BinaryDictEncoder to BinaryDictEncoderUtils.
Change-Id: I4dabf17da7003b1d8204a83dbd10e5be6e8fd805
2013-08-21 18:54:34 +09:00
Yuichiro Hanada
107a5f6fb8 Add PtNodeReader.
Change-Id: Ic918822fc1b3a8a7c39ffbcf7defde2c5bf888db
2013-08-21 18:43:18 +09:00
Yuichiro Hanada
065aad9501 Add DictDecoder.
Change-Id: Ia1c32f21fe07081ce04d093660e18146b93275a4
2013-08-20 17:43:13 +09:00
Yuichiro Hanada
112257e40f Rename BinaryDictDecoder to Ver3DictDecoder.
Change-Id: Ibf9b95b658df6e2c2218bdb62e2380f326a03832
2013-08-20 17:11:51 +09:00
Yuichiro Hanada
66004ce2de Remove populateOptions.
Change-Id: I1a1830aaa8ea586b68fc34ff3a27ae52b810e8af
2013-08-20 16:06:52 +09:00
Yuichiro Hanada
77bce05e6f [Refactor] Rename BinaryDictReader and BinaryDictDecoder.
BinaryDictReader -> BinaryDictDecoder.
BinaryDictDecoder -> BianryDictDecoderUtils.

Change-Id: Iadf2153b379b760538ecda488dda4f17225e5f37
2013-08-19 19:36:31 +09:00
Yuichiro Hanada
d794b42f98 Add HeaderReaderInterface.
Change-Id: I298f86b70d18cd08b240509b6f757c72e1a59ffe
2013-08-19 11:15:03 +09:00
Yuichiro Hanada
3a73b37b30 Make BinaryDictIOUtils and DynamicBinaryIOUtils use BinaryDictReader.
Change-Id: I191dfe0e05ff3c2c5af99e8beebbb73b097748a3
2013-08-16 21:06:23 +09:00
Yuichiro Hanada
e72c4e5fc7 Remove a static buffer for thread safety.
Change-Id: I335c35eb182ff63abb8a5b04c053a98d44b7c6ce
2013-08-16 20:22:46 +09:00
Ken Wakasa
47bac6ebf2 Merge "Remove unnecessary caching." 2013-08-16 08:28:19 +00:00