nyancrimew/LatinIME
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
14605 commits 1 branch 0 tags 363 MiB
Commit graph

77 commits

Author SHA1 Message Date
Yuichiro Hanada
665592774c Move some constants in BinaryDictInputOutput to FormatSpec. 
Change-Id: I6b12faf35b65238b9a64c82d4d1a6050f980e72e
 2013-08-14 19:19:27 +09:00
Yuichiro Hanada
3feacba1eb Add BinaryDictReader. 
Bug: 9618601

Change-Id: Ief07fa0c3c4f7f5999a3fafcef4e47b6b6fd8143
 2013-08-13 19:55:05 +09:00
Yuichiro Hanada
b7bb9c9722 Make readHeader check the header size before using it. 
Change-Id: I5dc3e2b674f7343ef57317fde6bdb7349a7fe04c
 2013-08-13 17:06:25 +09:00
Yuichiro Hanada
7ec9db2c34 Remove the code and comments about version 1 format. 
Change-Id: I827052f234eeaa4dbcfd37da69a99866896a158b
 2013-08-09 16:05:07 +09:00
Jean Chalard
25de86a6a2 [FD4] Separate cached address before/after update for groups 
This should fix bug#8526576 for good.

Bug: 8526576
Change-Id: I473aad26b69d64efa09d2ec9d8e69f29f5cf4819
 2013-07-24 18:40:14 +09:00
Jean Chalard
429db8d61e [FD3] Split stackNodes into two methods. 
In the future we need to have a method that computes only
from the size, as we used to have, to initialize the cached
addresses, and a much simpler and faster method to copy
the cached sizes.

Bug: 8526576
Change-Id: I6a5a790303ab8f3bf957c7ca266eb12da7c1ad9e
 2013-07-24 17:26:16 +09:00
Jean Chalard
91cbe3566d [FD2] Separate cached address before/after update for nodes. 
Bug: 8526576
Change-Id: Ib9f8594a9e12dc75eba296faff2612c4bd7483d3
 2013-07-23 17:52:54 +09:00
Jean Chalard
257750d988 [FD1] Move parents' address computation outside 
There is no need to do it repeatedly in this loop: it's
clearer and faster to do it at the end only.

Bug: 8526576
Change-Id: I707571179c89479830891ec6d4fd06a9fffed7c1
 2013-07-17 20:47:53 +09:00
Jean Chalard
c2e9c511cb Fix Binary dict tests 
There are two problems here. The first one is the tests would send
an invalid unicode character. Although we could want dicttool to
handle this more gracefully, it's fine for now.

The second problem is much more serious. If a node has more than
128 children, then the java code will crash trying to read the
dictionary back because of a bug that this change fixes. In
theory, it's possible that happens when we try to load the user
history dictionary back from the disk - native code is not affected
so there is no other point that may cause a problem.
In the practice, that means you'd need to have 129 words with a
common prefix (including empty string) but all different after
this. It's almost impossible with Google Keyboard since there are
only so many keys on the keyboard that you can make a word out
of, and then again you'd have to do it repeatedly until it
actually enters the user history dictionary, wait for it to get
saved on the disk.
The bad news is, if you manage to get this far, the keyboard will
crash every time and won't be able to get up until you clear
data for the package.
The good news is, the dictionary itself is not corrupted and only
the reading code is wrong. So updating to a newer version would
actually even recover from this situation.

All in all, considering how almost-impossible this is to trigger,
I don't think even a single user actually did hit this bug.

Bug: 8583091
Change-Id: Iabb2a7f47cbd9ed3193d2a3487318d280753e071
 2013-04-15 12:48:16 +09:00
Tadashi G. Takaoka
8aa9963a89 Fix Apache license comment 
Change-Id: Ic56167f952a7f4449da366e1e81610e72c966086
 2013-01-21 22:23:37 +09:00
Ken Wakasa
b6ca354431 Small code cleanups 
Multi-project commit with I249d5fbe

Change-Id: Ia28c4e970992aa1299a30e604eaa5d096655c3a5
 2013-01-07 12:13:42 +09:00
Jean Chalard
f1b464da31 Remove a useless member 
Change-Id: Id13e0aeec6ec3655d6bb0edc7f8f7821e7dc5a36
 2012-12-11 19:15:24 +09:00
Jean Chalard
2da8866518 Remove a couple Eclipse and Android Lint warnings 
Change-Id: I0c29c5d2abcbf80759b996d34b534deb083cd7d3
 2012-12-06 21:30:51 +09:00
Jean Chalard
f41389a74b Remove warnings 
Thanks Eclipse

Change-Id: I88e3979ed22be5d8be5a5accdde417c6b1a8bf2d
 2012-10-29 14:24:16 +09:00
Jean Chalard
b3c98901c5 Add auto detection and decoding of dictionary files. (A2) 
Bug: 7388852
Change-Id: I25e755fc15f5b383acc046f668e9681efa4f0c2f
 2012-10-25 16:40:15 +09:00
Jean Chalard
c59c741987 Return the correct bigram frequency 
The "correct" bigram frequency is now returned by the reading
code. However, as the binary format represents the frequency
in a lossy manner, the frequency is not guaranteed to be the
exact same as the one in the source text format - only a close
enough value. It is however the exact same value seen by the
native code.

Bug: 7395653
Change-Id: I49199ef18901c671189912b3550623e9643baedd
 2012-10-23 17:17:37 +09:00
Tadashi G. Takaoka
15f6d4ae34 Add @UsedForTesting and @ExternallyReferenced annotations 
Bug: 7268357
Change-Id: I0b7e0c19f04af9ae30874d0a4c26ad81bc80be8c
 2012-10-22 11:18:43 -07:00
Yuichiro Hanada
3c6d9fe148 Add insertWord. 
bug: 6669677

Change-Id: Ide55a4931071de9cd42c1cddae63ddd531d2feba
 2012-10-04 17:19:47 +09:00
Yuichiro Hanada
c3a98ca306 Add writeNode. 
Change-Id: I088bb6ea43ce0841d725e48b677d429e1155569d
 2012-10-04 14:28:42 +09:00
Yuichiro Hanada
38712ff27d Add updateParentAddresses. 
Change-Id: Iac210131b7c003ef363e1138bf22f777a37c6a89
 2012-10-03 19:37:17 +09:00
Yuichiro Hanada
a853356b82 Add isDeletedGroup. 
Change-Id: I83f09c068868e5e6e1b46f494a6ef957f0b466d8
 2012-10-03 02:19:41 -07:00
Yuichiro Hanada
7223cc2ef1 Add MAX_BIGRAMS_IN_A_GROUP. 
Change-Id: I128d5deb8e523045d7ad77d7a8fd3db944f71238
 2012-10-03 18:10:06 +09:00
Yuichiro Hanada
4ad4ff618f Add makeCharGroupFlags. 
Change-Id: Id2c580f21b77f66a97c5fbdf4542fdafe6c43614
 2012-10-03 14:33:59 +09:00
Yuichiro Hanada
fb7e08ea8f Add writeCharGroup. 
bug: 6669677

Change-Id: I36792ba9c511a5148c963096cc93ca8c2e0ee04e
 2012-10-01 21:50:38 +09:00
Yuichiro Hanada
f3aed3ea26 Add updateChildrenAddress. 
Change-Id: Ic06a755d85612476e719e580469dc1cd9447286c
 2012-09-28 18:45:56 +09:00
Tadashi G. Takaoka
a28a05e971 Cleanup: Make some classes as final 
Change-Id: I6009b3c1950ba32b7f1e205a3db2307fe0cd688e
 2012-09-27 19:03:30 +09:00
Yuichiro Hanada
2ee70804e9 Add moved char groups. 
bug: 6669677

Change-Id: I372f841044fe8e076a50a80ac10b715e5f8fd4eb
 2012-09-26 17:01:48 +09:00
Yuichiro Hanada
a161bdac88 add capacity to FusionDictionaryBufferInterface. 
bug: 6669677

Change-Id: I4627093811a19c46ce13fe351d1db63cbd78cf4a
 2012-09-25 21:47:11 +09:00
Yuichiro Hanada
8ec0064c49 Make children addresses and parent addresses use signed addresses. 
Signed addresses are used only in version 3 with dynamic update.

bug: 6669677

Change-Id: Iadaeab199b5019d2330b4573c24da74d64f0945e
 2012-09-25 12:55:14 +09:00
Yuichiro Hanada
82d9deaaf2 Combine mHasParentAddress with mHasLinkedListNode into mSupportsDynamicUpdate. 
bug: 6669677

Change-Id: I82799af199358420f09ac34fc005091e202c5d3b
 2012-09-24 13:17:44 +09:00
Yuichiro Hanada
66597f5e5f Add deleteWord. 
bug: 6669677

Change-Id: I1a5b90ee05e5cffd74a5c140384a3e37c79e7e70
 2012-09-21 12:40:07 +09:00
Yuichiro Hanada
d36245fad2 Add getTerminalPosition. 
Change-Id: If04d779db23b1aea2cc12e5e9b8cecfcb35a5737
 2012-09-20 18:02:16 +09:00
Yuichiro Hanada
65feee12e5 Make BinaryDictIOUtils. 
Change-Id: I45830235ee738233e8eb2bd91d659705b698f58c
 2012-09-19 15:37:37 +09:00
Yuichiro Hanada
c2fdf0dfbf Make readNode read linked list nodes. 
Change-Id: Ia5eaae0653179b2eb74c53b0823beaf80377a389
 2012-09-19 14:49:23 +09:00
Yuichiro Hanada
a149c53c8e add limit to FusionDictionaryBufferInterface. 
Change-Id: Ic9ff717a9751023d47b02ff3b9d1fbf3115c2501
 2012-09-19 12:28:19 +09:00
Yuichiro Hanada
b686df15fc Add a new flag for linked list nodes. 
Change-Id: Ib2f194775cfe5ab05481ac95cd709d6e8e8dd3c6
 2012-09-18 22:01:49 +09:00
Yuichiro Hanada
bf45dc4860 Make writePlacedNode write the linked-list node. 
Change-Id: I60feda815ea08cf73300fccca1ae12b97550f116
 2012-09-18 21:20:07 +09:00
Yuichiro Hanada
8adc0154e6 Remove populateOptions(final ByteBuffer buffer). 
Change-Id: Ifc4c64c9cffe4f343c5a604c192db010a1792acc
 2012-09-18 14:42:52 +09:00
Yuichiro Hanada
cc958dd96e Refactor BinaryDictInputOutput. 
Change-Id: Idb4b635fcac70cc988e0dd3ce3bf121fba12099c
 2012-09-14 11:08:01 +09:00
Yuichiro Hanada
1a347723c5 Move FormatOptions and FileHeader to FormatSpec. 
Change-Id: I232e35598635113bf2c81825669c744aadc79efe
 2012-09-13 16:35:41 +09:00
Yuichiro Hanada
81d97eec0e Move constants and comments. 
Change-Id: Ifd66bda7d528827ba61c60531121ea206a2325be
 2012-09-13 14:28:39 +09:00
Yuichiro Hanada
8d031a63b4 Add put method to FusionDictionaryBufferInterface. 
Change-Id: Iac0b35d2da05e81237d105e8fe13c56d16038de1
 2012-09-12 15:41:21 +09:00
Yuichiro Hanada
e55b644aef Add new binary dictionary format. 
Change-Id: Ia99411d4009857d5e420ca87ef8acf1f1826d3ed
 2012-09-10 13:05:46 +09:00
Yuichiro Hanada
eae7b293e4 Check the length of the word when add to FusionDictionary. 
Change-Id: Id98d18e90a8b83b597507728b467f56888c8fd12
 2012-09-10 12:35:53 +09:00
Yuichiro Hanada
83dfe0fd8c Add FormatOptions. 
Change-Id: Ibad05a5f9143de1156b2c897593ec89b0a0b07e7
 2012-09-05 18:05:43 +09:00
Jean Chalard
2035b946a3 Merge "Reinstate the shortcut-only attribute" into jb-mr1-dev 2012-09-02 19:28:01 -07:00
Jean Chalard
72b1c93941 Reinstate the shortcut-only attribute 
Also add the blacklist attribute

Bug: 7005742
Bug: 2704000
Change-Id: Icbe60bdf25bfb098d9e3f20870be30d6aef07c9d
 2012-08-31 22:11:52 +09:00
Yuichiro Hanada
666a433802 add UserHistoryDictIOUtils. 
Change-Id: I8a70e43b23f65b5fd5f0ee0b30a94ad8f5ef8a8a
 2012-08-31 15:08:57 +09:00
Yuichiro Hanada
b2a43a2ed4 add readUnigramsAndBigramsBinary. 
Change-Id: I7967f11211221d4877bf0a0c30183af885f45390
 2012-08-31 14:39:19 +09:00
Yuichiro Hanada
62ed901100 add readHeader. 
Change-Id: I5be5d62a63ca897e36fe93200ffdca6befb363aa
 2012-08-30 14:17:50 +09:00