nyancrimew/LatinIME
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
30179 commits 1 branch 0 tags 363 MiB
Commit graph

227 commits

Author SHA1 Message Date
Keisuke Kuroyanagi
5128935ac4 Add header attributes for evaluation. 
Bug: 13197276
Change-Id: Ib5247da691ff24a73e13485288237ccc51bb54f0
 2014-02-28 21:06:03 +09:00
Keisuke Kuroyanagi
ea89cb40b7 Use dynamic operations to construct all ver4 dicts. 
Bug: 8187060
Bug: 13127350
Change-Id: I081ee904c41898128efa8ba7a1bf3fa0a46c6231
 2014-02-27 23:21:09 +09:00
Keisuke Kuroyanagi
6112cc26a7 Read and write forgetting curve header attributes. 
Bug: 13197276
Change-Id: I6c38a127e1745903594b7bf0ec80ef179e794feb
 2014-02-26 18:04:12 +09:00
Jean Chalard
890b44e537 Correctly read the header of APK-embedded dicts 
Bug: 13164518
Change-Id: I8768ad887af8b89ad9f29637f606c3c68629c7ca
 2014-02-24 22:54:01 +09:00
Keisuke Kuroyanagi
d24a99cff6 Fix: file descriptor leaking. 
File descriptors will be closed in
BinaryDictionary.finalize(); but, this leads to unit test fails.

Bug: 13066902
Change-Id: I2b0d3e54ee91fa844530df54596f86bbbbac81a5
 2014-02-21 17:25:34 +09:00
Keisuke Kuroyanagi
afd9b62f00 Throw IOException when dict header cannot be read. 
Bug: 13106638
Change-Id: Ib48ef0a191a436552c49b6e817dfa13fe1567ae8
 2014-02-20 18:11:17 +09:00
Keisuke Kuroyanagi
97a40d030f Throw exception when the header cannot be read. 
Bug: 13066902
Change-Id: I8fd0f361a4ca55aaab8a2bd2e11f48f92b87fc93
 2014-02-19 12:35:58 +09:00
Keisuke Kuroyanagi
95d16561e0 Remove unused code. 
Bug: 12810574
Change-Id: I9c7fff60ae0e94d52f3bd19c3e88de5a53b917d7
 2014-02-15 17:39:24 +09:00
Keisuke Kuroyanagi
0fc93fe445 Implement PatriciaTriePolicy::getNextWordAndNextToken(). 
Bug: 12810574
Change-Id: Id1d44f90de9455d9cbe7b6e0a161cae91d6d422c
 2014-02-15 17:39:20 +09:00
Keisuke Kuroyanagi
8e3a1d0f89 Remove unused argument from readDictionaryBinary. 
Bug: 12810574
Change-Id: Ice415ebd8d11162facca3fe8927ef8a616b11424
 2014-02-14 19:02:15 +09:00
Keisuke Kuroyanagi
69ccac6e51 Remove unused code. 
Bug: 12810574
Change-Id: If0ef02a984469a3b6e0c00b1c3c8d98d0d2b5466
 2014-02-10 15:05:11 +09:00
Keisuke Kuroyanagi
8ffc631826 Make PtNode have ProbabilityInfo instead of raw value. 
Bug: 11281877
Bug: 12810574
Change-Id: Id1cda0afc74c4e30633c735729143491b2274a7b
 2014-02-10 15:05:08 +09:00
Keisuke Kuroyanagi
ab6a93773b Use native logic to read Ver4 dict. 
Bug: 11281877
Bug: 12810574
Change-Id: Ief371d3ef61818e4e031de4659aee3c9584c7379
 2014-02-06 21:55:37 +09:00
Keisuke Kuroyanagi
fd018c1588 Merge "Use CombinedFormatUtils to convert dict elements to strings." 2014-02-06 07:29:30 +00:00
Keisuke Kuroyanagi
b24de426fc Use CombinedFormatUtils to convert dict elements to strings. 
Bug: 11281877
Bug: 12810574
Change-Id: Ib631f75eab73abc9877a7698171c45e8f2fc7600
 2014-02-06 16:09:25 +09:00
Keisuke Kuroyanagi
337dce8074 Merge "Consolidate WordProperty and Word." 2014-02-06 07:02:07 +00:00
Keisuke Kuroyanagi
5f5feeba13 Consolidate WordProperty and Word. 
Bug: 11281877
Bug: 12810574
Change-Id: I9dc99188f80f25a8780c1860dab46e4aa80a23e5
 2014-02-06 15:13:33 +09:00
Keisuke Kuroyanagi
239f7daf4a Merge "Make WeightedString have ProbabilityInfo." 2014-02-05 12:53:14 +00:00
Keisuke Kuroyanagi
df1d3e733e Make WeightedString have ProbabilityInfo. 
Bug: 11281877
Bug: 12810574
Change-Id: I265e3d8654c75766cd0e0d09d67ef62b4566298a
 2014-02-05 21:44:55 +09:00
Keisuke Kuroyanagi
e42380a471 Merge "Separate ProbabilityInfo form WordProperty." 2014-02-05 12:37:49 +00:00
Keisuke Kuroyanagi
75a3df30f6 Separate ProbabilityInfo form WordProperty. 
Bug: 11281877
Bug: 12810574
Change-Id: I0c77d976c915bdf2a2cddabd0cbedc2fb6691c7b
 2014-02-05 21:31:35 +09:00
Jean Chalard
43cf9076b2 [HD04] Make the locale mandatory. 
Bug: 11281748
Change-Id: I69281b0053bec404c3e3c713ade3f65a140f51b1
 2014-02-05 17:35:25 +09:00
Keisuke Kuroyanagi
b986f78ba8 Separate header class from FormatSpec. 
Bug: 12810574
Change-Id: Iacf1cd05a268bf690ab864b5e32a18a4b0ccc693
 2014-02-04 21:36:04 +09:00
Jean Chalard
79b2e4d86c [HD03] Straighten out attribute key names in Java. 
Bug: 11281748
Change-Id: I1d813bdacd45bcfd9c4cc73ac1d67c5c89854e86
 2014-01-31 14:46:07 +09:00
Keisuke Kuroyanagi
971bbcf562 Fix historical info of bigrams reading. 
Bug: 12809791
Change-Id: I879279a0d0e1007a7c6cb87a627bf046340dda98
 2014-01-30 16:19:22 +09:00
Keisuke Kuroyanagi
26bd46095a Reading dictionary containing timestamps in Java Side. 
Just skipping historical information fields.

Bug: 11281877
Change-Id: I43d2adaa576b7da11ed3ca54990265dbb6f53b08
 2014-01-29 20:19:24 +09:00
Keisuke Kuroyanagi
fde4c5f128 Fix unit tests by adding @UsedForTesting 
Change-Id: I2d2d226e21a46239077394ab09db88a4bd563460
 2014-01-19 13:37:23 -08:00
Keisuke Kuroyanagi
8214a8c2cf Fix possible NPE. 
Bug: 11933670
Change-Id: Ib0ce853e691bdadffb73874ea9f188cfbec8bc93
 2014-01-06 17:56:49 +09:00
Jean Chalard
4196360912 Run GC when needed, instead of too late. 
Bug: 12164855
Change-Id: Ie4d8be9f2652b01470bcc06911e2191e09fb3149
 2013-12-17 18:08:48 +09:00
Jean Chalard
97a553ae69 Fix wrong dereference reading code. 
Bug: 12164855
Change-Id: I26faf27edfa303510197bccaa3521578f649741d
 2013-12-17 16:42:22 +09:00
Keisuke Kuroyanagi
b54c747d7c Merge "Move DictUpdater to the tests directory." 2013-12-17 05:33:16 +00:00
Keisuke Kuroyanagi
4fdcefe504 Move DictUpdater to the tests directory. 
Bug: 11245133
Change-Id: I0907a091ac3ae960eaf3b27da78dbb48a24b2ea1
 2013-12-17 14:31:25 +09:00
Keisuke Kuroyanagi
b5215d456a Increment version number of ver4 format to 401. 
Bug: 11073222
Change-Id: I3878aef02040b1ef73165645fc70ede586289c51
 2013-12-16 23:02:54 +09:00
Keisuke Kuroyanagi
56e7e38d37 Fix: incompatible PtNode array size reading. 
Bug: 11073222
Change-Id: I681be725b5c6dcc10aa536b00327698a7df75277
 2013-12-16 22:42:01 +09:00
Jean Chalard
5590e9d040 Merge "Make the bad header a checked exception and fix dicttool test" 2013-12-16 07:18:44 +00:00
Jean Chalard
3df617323e Make the bad header a checked exception and fix dicttool test 
Change-Id: If780a2a1cb722aee83d910904c0a6315add9552d
 2013-12-16 15:40:45 +09:00
Jean Chalard
b868375763 Fix failing tests 
- Version 3 is not supported
- Now passing the right string to open v4 dicts. Fix the tests for this.

Change-Id: I7829330c3568a715b96396ba4e4e69c6e17775ab
 2013-12-16 14:32:19 +09:00
Jean Chalard
1dc7eca114 Remove references to the v3 dictionary. 
Change-Id: I811c8c923ad67a6d9bfdd11bdef8991eb7135c27
 2013-12-13 18:53:41 +09:00
Jean Chalard
a245d15da5 Have dicttool use the native library to generate v4 dicts. 
Yay !

Change-Id: Iea8ced9e81031b9ab7eff05ad9ef7215be248de9
 2013-12-13 18:18:20 +09:00
Jean Chalard
7b55cd3e2b Remove flags from Java side. 
This simplifies the code quite a bit.
- GERMAN_UMLAUTS are now handled through a key-value attribute.
  The dictionary generator does not need to know about it any more.
- FRENCH_LIGATURES are deprecated as we handle them with shortcuts now.
- CONTAINS_BIGRAMS is deprecated. Bigram processing is always applied
  regardless of this flag.

Bug: 11281748
Change-Id: If567e52e245a9342adc7f3104a0f7d8d782df8c1
 2013-12-13 18:15:05 +09:00
Ken Wakasa
2fa3693c26 Reset to 9bd6dac470 
The bulk merge from -bayo to klp-dev should not have been merged to master.

Change-Id: I527a03a76f5247e4939a672f27c314dc11cbb854
 2013-12-13 17:13:32 +09:00
Keisuke Kuroyanagi
db4f373004 Fix: PtNode array size writirng when array size > 127. DO NOT MERGE. 
This is a manual cherrypick of Ib729ceedbc8ef837e50490439817b36039ae2b4e.

Bug: 11772864
Change-Id: I5ecbe729dbdd24e194e48b4d68b17af8549c4726
 2013-11-21 08:45:51 +00:00
Ken Wakasa
7abdab1c6f Merge "Add a time stamp for unigrams." 2013-10-11 10:17:01 +00:00
Ken Wakasa
be22a01df3 Merge "Add resolveBigramPositions." 2013-10-11 07:45:20 +00:00
Yuichiro Hanada
c32962b8f1 Add a time stamp for unigrams. 
Bug: 10920255
Change-Id: I26d2cce3c322a4ff39a614f8615f43fb7bd3baed
 2013-10-11 15:08:09 +09:00
Yuichiro Hanada
9514ed5c2a Add the new format of bigram entries. 
In new format, each bigram entry has flags (1 byte), a terminal id (3 byte),
a time-stamp (4 byte), a counter (1 byte) and a level (1 byte).

Bug: 10920255
Bug: 10920165
Change-Id: I0f7fc125a6178e6d25a07e8462afc41a7f57e3e1
 2013-10-11 14:50:41 +09:00
Yuichiro Hanada
67c758f160 Add resolveBigramPositions. 
Bug: 11073222
Change-Id: I680e4304c6ab701e9aa132e1e140d789097a53ae
 2013-10-09 17:39:22 +09:00
Jean Chalard
ca9c3c0613 Fix a possible IOOB 
We want to use StringUtils here, but it's full of references to
stuff not accessible host-side like JsonReader and TextUtils
and SettingsValues :/

Bug: 11061476
Change-Id: I3c0194979833ede283b4f9190335dba5376fe6fc
 2013-10-07 23:16:57 +09:00
Yuichiro Hanada
e4e0add9fb Add Ver4DictUpdater. 
Change-Id: I986ab26faf535fc4bc98443053f534eced9d048f
 2013-10-04 17:33:29 +09:00
Yuichiro Hanada
75d60e821c Refactor BinaryDictIOUtilsTests. 
Change-Id: I2208378b33038771b460abb33f9a690872e998e2
 2013-10-04 14:19:13 +09:00