Commit graph

150 commits

Author SHA1 Message Date
Yuichiro Hanada
55f5f7a005 [Refactor] Add writeDictionaryHeader.
Change-Id: I69026c47ce1d23f5c39c99ace76fa6b96cd1ce1b
2013-09-05 16:19:26 +09:00
Yuichiro Hanada
663f5375d1 Add comments about how to calculate parent and children addresses.
Change-Id: Idf37d9a08a7b464a14c8755bedb755005e49ba33
2013-09-05 15:10:28 +09:00
Yuichiro Hanada
d5a3c593db Remove a broken dictionary file when fail to read a dictionary file.
Bug: 10434720
Change-Id: Ibdf05a39113538546b8fcf9d59af7dddf7ca27fc
2013-08-26 18:08:21 +09:00
Yuichiro Hanada
b50a7a4bc7 Simplify computeActualNodeArraySize.
Change-Id: I128d02866c2cc70c42263496927327b8a2130139
2013-08-26 08:30:23 +00:00
Yuichiro Hanada
752a33640c [Refactor] Add DictDecoder.readUnigramsAndBigramsBinary.
Change-Id: I259db91d837c67cbcb3b6dc504b21dca23a6a5be
2013-08-26 17:24:38 +09:00
Yuichiro Hanada
bb5b84a826 [Refactor] Add DictDecoder.getTerminalPosition.
Change-Id: I9d04f64a58f5481cbb64cf1c09b5c485dd4176b4
2013-08-26 16:14:59 +09:00
Yuichiro Hanada
576f625ee1 Rename CharGroup to PtNode.
Bug: 10233675
Change-Id: I7b0eb07d195cd386cd0d9e97cd59bf48fcf24107
2013-08-26 15:58:30 +09:00
Yuichiro Hanada
8f4d5d5f23 Fix getDictionaryFileHeader.
Change-Id: I5186f41b0f38eb08f4154f7b3de7ffc2119fc44b
2013-08-23 22:39:36 +09:00
Yuichiro Hanada
e9a10ff0f0 Add DictDecoder.readDictionaryBinary.
Bug: 10434720
Change-Id: I14690a6e0f922ed1bab3a4b6c9a457ae84d4c1a4
2013-08-23 20:29:25 +09:00
Yuichiro Hanada
aa4168ee09 Fix writePlacedNode.
Change-Id: I1d6b086f1d9f0dbd8d74f964e29ae62c533af978
2013-08-22 23:02:08 +09:00
Yuichiro Hanada
e301085a70 Move findWordByBinaryDictReader to BinaryDictIOUtilsTests.
Change-Id: I443238fd816dea9650dcbbeb3ea757f9674fa52f
2013-08-22 14:49:49 +09:00
Yuichiro Hanada
c922c8a504 Add DictEncoder.
Change-Id: I41049b9118b58838e5dedf8e5618d939ca70c5ef
2013-08-22 11:53:41 +09:00
Yuichiro Hanada
558e34c7bd Make readPtNode be called with the address from the beginning of the file.
Change-Id: I8939fdfb4f79e55bcd7393633784effb30df3f8f
2013-08-21 20:02:18 +09:00
Yuichiro Hanada
a306e08753 Rename BinaryDictEncoder to BinaryDictEncoderUtils.
Change-Id: I4dabf17da7003b1d8204a83dbd10e5be6e8fd805
2013-08-21 18:54:34 +09:00
Yuichiro Hanada
107a5f6fb8 Add PtNodeReader.
Change-Id: Ic918822fc1b3a8a7c39ffbcf7defde2c5bf888db
2013-08-21 18:43:18 +09:00
Yuichiro Hanada
065aad9501 Add DictDecoder.
Change-Id: Ia1c32f21fe07081ce04d093660e18146b93275a4
2013-08-20 17:43:13 +09:00
Yuichiro Hanada
112257e40f Rename BinaryDictDecoder to Ver3DictDecoder.
Change-Id: Ibf9b95b658df6e2c2218bdb62e2380f326a03832
2013-08-20 17:11:51 +09:00
Yuichiro Hanada
66004ce2de Remove populateOptions.
Change-Id: I1a1830aaa8ea586b68fc34ff3a27ae52b810e8af
2013-08-20 16:06:52 +09:00
Yuichiro Hanada
77bce05e6f [Refactor] Rename BinaryDictReader and BinaryDictDecoder.
BinaryDictReader -> BinaryDictDecoder.
BinaryDictDecoder -> BianryDictDecoderUtils.

Change-Id: Iadf2153b379b760538ecda488dda4f17225e5f37
2013-08-19 19:36:31 +09:00
Yuichiro Hanada
d794b42f98 Add HeaderReaderInterface.
Change-Id: I298f86b70d18cd08b240509b6f757c72e1a59ffe
2013-08-19 11:15:03 +09:00
Yuichiro Hanada
3a73b37b30 Make BinaryDictIOUtils and DynamicBinaryIOUtils use BinaryDictReader.
Change-Id: I191dfe0e05ff3c2c5af99e8beebbb73b097748a3
2013-08-16 21:06:23 +09:00
Yuichiro Hanada
e72c4e5fc7 Remove a static buffer for thread safety.
Change-Id: I335c35eb182ff63abb8a5b04c053a98d44b7c6ce
2013-08-16 20:22:46 +09:00
Ken Wakasa
47bac6ebf2 Merge "Remove unnecessary caching." 2013-08-16 08:28:19 +00:00
Yuichiro Hanada
6e26cc3f5d Remove unnecessary caching.
Change-Id: Ic4ccab9d344b30b72fca1503827eec1c628fa4ac
2013-08-16 17:10:45 +09:00
Jean Chalard
af30cbf0ee Rename Node to PtNodeArray
Bug: 10247660
Change-Id: I1a0ac19f58f96adb5efac5fd35c6404831618c99
2013-08-16 16:24:54 +09:00
Yuichiro Hanada
94460eba11 [Refactor] Divide BinaryDictInputOutput into BinaryDictEncoder and BinaryDictDecoder.
Change-Id: I7c3269d77e3e3b567e459dcaa1bc029903941744
2013-08-15 20:23:07 +09:00
Jean Chalard
e7870a2c0d Add an initial JNI interface to dicttool.
Bug: 10100269
Change-Id: I883992c2033e7d9e7c754c0bf653767728b221b6
2013-08-15 17:58:55 +09:00
Ken Wakasa
117f18e844 Revert "[Refactor] Divide BinaryDictInputOutput into BinaryDictInputUtils and BinaryDictOutputUtils."
This reverts commit 4c63d0614e.

Change-Id: I1fa277d720bab4d895259df7d6d82eebfa5eb6c5
2013-08-15 08:54:29 +00:00
Yuichiro Hanada
4c63d0614e [Refactor] Divide BinaryDictInputOutput into BinaryDictInputUtils and BinaryDictOutputUtils.
Change-Id: I0d476abe763c11ba9005152f928e8dccf15ac9de
2013-08-15 15:46:58 +09:00
Yuichiro Hanada
1db93c9c04 [Refactor] Move some helper methods to BinaryDictIOUtils.
Change-Id: Ib817a975dc1f82241f732b236c44b042fda25b3c
2013-08-15 10:49:40 +09:00
Yuichiro Hanada
3edb62c69b Move some methods in BinaryDictIOUtils to DynamicBinaryDictIOUtils.
Change-Id: I9ba55582c533fef0eb3e60c46bf23c8b16ee1ff4
2013-08-14 19:33:36 +09:00
Ken Wakasa
f795f2b789 Merge "Add FusionDictionaryBufferFromWritableByteBufferFactory." 2013-08-14 10:26:21 +00:00
Yuichiro Hanada
665592774c Move some constants in BinaryDictInputOutput to FormatSpec.
Change-Id: I6b12faf35b65238b9a64c82d4d1a6050f980e72e
2013-08-14 19:19:27 +09:00
Yuichiro Hanada
bbc8a930f7 Add FusionDictionaryBufferFromWritableByteBufferFactory.
Change-Id: I23de0a178e7f11f2cf301fd433cde60c6152055b
2013-08-14 17:07:44 +09:00
Yuichiro Hanada
3feacba1eb Add BinaryDictReader.
Bug: 9618601

Change-Id: Ief07fa0c3c4f7f5999a3fafcef4e47b6b6fd8143
2013-08-13 19:55:05 +09:00
Yuichiro Hanada
b7bb9c9722 Make readHeader check the header size before using it.
Change-Id: I5dc3e2b674f7343ef57317fde6bdb7349a7fe04c
2013-08-13 17:06:25 +09:00
Yuichiro Hanada
7ec9db2c34 Remove the code and comments about version 1 format.
Change-Id: I827052f234eeaa4dbcfd37da69a99866896a158b
2013-08-09 16:05:07 +09:00
Yuichiro Hanada
7d1ae52ded Fix unit tests.
Change-Id: Ic0013089625e112aaccc888d462330640ef7cc6f
2013-08-08 19:12:35 +09:00
Jean Chalard
93445b4821 Fix some warnings
Change-Id: I7290cd1fb675a1b85b9b6ac2d464c932b5bca1dd
2013-07-31 16:17:01 +09:00
Satoshi Kataoka
ffcbbaf127 Refactor on UserHistoryDictionary
Bug: 9429906
Change-Id: I576a91643bdaf5017cc826ac2e07a74a9a275d60
2013-07-26 13:00:19 +09:00
Jean Chalard
25de86a6a2 [FD4] Separate cached address before/after update for groups
This should fix bug#8526576 for good.

Bug: 8526576
Change-Id: I473aad26b69d64efa09d2ec9d8e69f29f5cf4819
2013-07-24 18:40:14 +09:00
Jean Chalard
429db8d61e [FD3] Split stackNodes into two methods.
In the future we need to have a method that computes only
from the size, as we used to have, to initialize the cached
addresses, and a much simpler and faster method to copy
the cached sizes.

Bug: 8526576
Change-Id: I6a5a790303ab8f3bf957c7ca266eb12da7c1ad9e
2013-07-24 17:26:16 +09:00
Jean Chalard
91cbe3566d [FD2] Separate cached address before/after update for nodes.
Bug: 8526576
Change-Id: Ib9f8594a9e12dc75eba296faff2612c4bd7483d3
2013-07-23 17:52:54 +09:00
Jean Chalard
257750d988 [FD1] Move parents' address computation outside
There is no need to do it repeatedly in this loop: it's
clearer and faster to do it at the end only.

Bug: 8526576
Change-Id: I707571179c89479830891ec6d4fd06a9fffed7c1
2013-07-17 20:47:53 +09:00
Jean Chalard
c2e9c511cb Fix Binary dict tests
There are two problems here. The first one is the tests would send
an invalid unicode character. Although we could want dicttool to
handle this more gracefully, it's fine for now.

The second problem is much more serious. If a node has more than
128 children, then the java code will crash trying to read the
dictionary back because of a bug that this change fixes. In
theory, it's possible that happens when we try to load the user
history dictionary back from the disk - native code is not affected
so there is no other point that may cause a problem.
In the practice, that means you'd need to have 129 words with a
common prefix (including empty string) but all different after
this. It's almost impossible with Google Keyboard since there are
only so many keys on the keyboard that you can make a word out
of, and then again you'd have to do it repeatedly until it
actually enters the user history dictionary, wait for it to get
saved on the disk.
The bad news is, if you manage to get this far, the keyboard will
crash every time and won't be able to get up until you clear
data for the package.
The good news is, the dictionary itself is not corrupted and only
the reading code is wrong. So updating to a newer version would
actually even recover from this situation.

All in all, considering how almost-impossible this is to trigger,
I don't think even a single user actually did hit this bug.

Bug: 8583091
Change-Id: Iabb2a7f47cbd9ed3193d2a3487318d280753e071
2013-04-15 12:48:16 +09:00
Jean Chalard
ca0fdbbe2e Fix two bugs in dicttool
Both bugs only affect debug mode. One has the wrong object tested
with equals, the other has the iteration failing in some cases.

Change-Id: Ie9100d257a3f9e3be340cf3e38116f63417bdc1a
2013-04-10 22:10:31 +09:00
Jean Chalard
a411595b16 Fix two nasty bugs with surrogate pairs.
The important bug is in findWordInTree. The problem, which is
not obvious, is that we were calling codePointAt() with the
code point index in the string, instead of the char index.
The other bug this change fixes was harmless in the practice,
because it's in the iteration which is only used for debug and
pretty printing purposes. It's very similar in that it would
substract a length in code point to a length in chars and
truncate a StringBuilder at that length, so it would fail in a
quite similar manner. This changes the meaning of the "length"
attribute in Position, but it's clearer this way anyway.

Bug: 8450145
Change-Id: If396f883a9e6449de39351553ba83f5be5bd30f0
2013-04-01 17:06:19 +09:00
Jean Chalard
c6799ffeab Send the dictionaries descriptions to the dict pack
Bug: 8255795
Change-Id: I12a5922f50c2d2e3aa639457abcc1483e6a48721
2013-02-23 01:46:39 -08:00
Jean Chalard
2521edec09 Fix a bug with the passed dictionary id
We used to make the dictionary that we passed to the
dictionary pack as an initial value based on the locale.
This is wrong - it should be read from the dictionary.
This change fixes that.

Bug: 7005813
Change-Id: Ib08ed31dd9c216f6f7b9c6c3174ca514bf96e06f
2013-02-22 20:49:48 -08:00
Jean Chalard
af4a7e8c4b Create methods in LatinIME to make the current dict lists
Bug: 7005813
Change-Id: I82232af8e3071333b6fd01e4453b6b3c0a3ddb1f
2013-01-31 09:26:52 +09:00