nyancrimew/LatinIME
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
14723 commits 1 branch 0 tags 363 MiB
Commit graph

69 commits

Author SHA1 Message Date
Yuichiro Hanada
c922c8a504 Add DictEncoder. 
Change-Id: I41049b9118b58838e5dedf8e5618d939ca70c5ef
 2013-08-22 11:53:41 +09:00
Yuichiro Hanada
a306e08753 Rename BinaryDictEncoder to BinaryDictEncoderUtils. 
Change-Id: I4dabf17da7003b1d8204a83dbd10e5be6e8fd805
 2013-08-21 18:54:34 +09:00
Yuichiro Hanada
112257e40f Rename BinaryDictDecoder to Ver3DictDecoder. 
Change-Id: Ibf9b95b658df6e2c2218bdb62e2380f326a03832
 2013-08-20 17:11:51 +09:00
Yuichiro Hanada
77bce05e6f [Refactor] Rename BinaryDictReader and BinaryDictDecoder. 
BinaryDictReader -> BinaryDictDecoder.
BinaryDictDecoder -> BianryDictDecoderUtils.

Change-Id: Iadf2153b379b760538ecda488dda4f17225e5f37
 2013-08-19 19:36:31 +09:00
Jean Chalard
af30cbf0ee Rename Node to PtNodeArray 
Bug: 10247660
Change-Id: I1a0ac19f58f96adb5efac5fd35c6404831618c99
 2013-08-16 16:24:54 +09:00
Yuichiro Hanada
a94eb97413 Remove unused imports. 
Change-Id: Ib46da6efeaa8f83a9cda24c947fc2e667a1468e8
 2013-08-16 11:19:53 +09:00
Yuichiro Hanada
35f41a9f18 Fix build. 
Change-Id: I80d177f3c9bff1fb90029edeafa4a62d0b99e09e
 2013-08-15 20:47:42 +09:00
Yuichiro Hanada
94460eba11 [Refactor] Divide BinaryDictInputOutput into BinaryDictEncoder and BinaryDictDecoder. 
Change-Id: I7c3269d77e3e3b567e459dcaa1bc029903941744
 2013-08-15 20:23:07 +09:00
Jean Chalard
e7870a2c0d Add an initial JNI interface to dicttool. 
Bug: 10100269
Change-Id: I883992c2033e7d9e7c754c0bf653767728b221b6
 2013-08-15 17:58:55 +09:00
Ken Wakasa
117f18e844 Revert "[Refactor] Divide BinaryDictInputOutput into BinaryDictInputUtils and BinaryDictOutputUtils." 
This reverts commit 4c63d0614e.

Change-Id: I1fa277d720bab4d895259df7d6d82eebfa5eb6c5
 2013-08-15 08:54:29 +00:00
Yuichiro Hanada
4c63d0614e [Refactor] Divide BinaryDictInputOutput into BinaryDictInputUtils and BinaryDictOutputUtils. 
Change-Id: I0d476abe763c11ba9005152f928e8dccf15ac9de
 2013-08-15 15:46:58 +09:00
Yuichiro Hanada
3feacba1eb Add BinaryDictReader. 
Bug: 9618601

Change-Id: Ief07fa0c3c4f7f5999a3fafcef4e47b6b6fd8143
 2013-08-13 19:55:05 +09:00
Ken Wakasa
b03447e1af Move a couple classes to the utils package 
Change-Id: Ia14a2011d79bad7cd02697b9254705f6e2099442
 2013-07-19 10:46:46 +09:00
Ken Wakasa
4d5939fc0a Remove dependencies to android source in dicttool 
bug: 9889653
Change-Id: I2630c56548f9146ade85b17f891932dae2082a85
 2013-07-17 13:16:02 +09:00
Ying Wang
cc54032b39 Fix build. 
Change-Id: I51c0b7ba5641c1e7c1d63a4e5a6307b08d6f4886
 2013-07-16 19:09:14 -07:00
Jean Chalard
b6cc4333a8 Generalize constuctors & add tests 
If there is any two-args constructor with the right arg types
in a test class, use it. Also, add a test suite to dicttool test.

Also, have the return value of dicttool reflect success or
failure.

Change-Id: I51ab2a4abb93a0a483e8d6ef3c39d1ff1bce1dbd
 2013-07-04 17:13:58 +09:00
Jean Chalard
4b7acd1df6 Add args to dicttool test. 
Change-Id: I0667e0a5a6f6db3964cfcca5c8f083b9ceb41a2e
 2013-07-01 15:28:32 +09:00
Ken Wakasa
e28eba5074 Move util classes to the latin/utils directory 
Change-Id: I1c5b27c8edf231680edb8d96f63b9d04cfc6a6fa
 2013-06-24 17:04:40 +09:00
Jean Chalard
23d4eb55ba Add tests to dicttool test. 
Bug: 8526576
Change-Id: Idd6f9cd076d5915361c68f5c29afbba67dd54eba
 2013-06-20 17:29:37 +09:00
Jean Chalard
d055cca7d2 Add an initial test command to dicttool. 
Also fix some warnings
This is a preliminary change to help track b/8526576

Bug: 8526576
Change-Id: If2ed74bb92f9b5fb7bbaf9c4851d9d0b5514b405
 2013-06-19 20:08:54 +09:00
Jean Chalard
a411595b16 Fix two nasty bugs with surrogate pairs. 
The important bug is in findWordInTree. The problem, which is
not obvious, is that we were calling codePointAt() with the
code point index in the string, instead of the char index.
The other bug this change fixes was harmless in the practice,
because it's in the iteration which is only used for debug and
pretty printing purposes. It's very similar in that it would
substract a length in code point to a length in chars and
truncate a StringBuilder at that length, so it would fail in a
quite similar manner. This changes the meaning of the "length"
attribute in Position, but it's clearer this way anyway.

Bug: 8450145
Change-Id: If396f883a9e6449de39351553ba83f5be5bd30f0
 2013-04-01 17:06:19 +09:00
Ken Wakasa
02996dc56a Clean up dependency for dicttool 
bug: 7635215
Change-Id: Ie50297ab592261390885c593341b6c46161b2e63
 2012-11-29 19:09:01 +09:00
Jean Chalard
5b1037fa94 Fix a bug in the diff tool 
Fix reporting of removed attributes multiple times instead
of just one in some cases. Also, report the frequency of
the removed bigram in the same cases.

Change-Id: I1455fa7553fcebc3005c222bf5aa9afcf1678c31
 2012-11-08 11:57:45 +09:00
Jean Chalard
271e731eb4 Add a missing report, and fix a wrong report 
Deleted words would be reported, but added words wouldn't
Also, shortcuts would be reported as bigrams

Change-Id: I017653f2afa65cf16924017a0be2ee2cbd6ca10d
 2012-11-01 14:25:53 +09:00
Jean Chalard
8c02f3005a Add a "no changes" message in the diff command 
Also fix a bug where no locale would be interpreted as a
different locale instead of the same.

Change-Id: I296feef45e39a9e89509cb315e789676755141ab
 2012-11-01 13:16:58 +09:00
Jean Chalard
acafc073b9 Fix an NPE in recent code 
Change-Id: I5c76edff5c3c250167d425a737b09b10e25f9f84
 2012-11-01 12:48:37 +09:00
Jean Chalard
1b67a2bd8b Merge "Use spaces instead of tabs in the combined format." 2012-10-31 02:33:55 -07:00
Jean Chalard
8f18c261cd Use spaces instead of tabs in the combined format. 
Change-Id: Ie8a4cdc44f87839e29bfbffc03d4853b7b5cc5f6
 2012-10-31 18:32:09 +09:00
Jean Chalard
51a0ef8c59 Add a plumbing option to dicttool info. 
Also align the `porcelain' option to the diff command
that was used mistakenly.

Bug: 7388665
Change-Id: Ic0e1b98c62ce37b2e909384a0370af4458563703
 2012-10-31 16:35:22 +09:00
Jean Chalard
a8058d169d Cleanup after dicttool 
Arrange for temporary files to be deleted.
Also, add a security on stack overflows.

Change-Id: I9da2ebefb06409a71b235243ea835ce10d6a9b81
 2012-10-30 19:57:02 +09:00
Jean Chalard
ee0b5488b5 Implement the package command 
Bug: 7388852
Change-Id: Ib6a50e02894961b4c7d3f57bc0fc70c3b3bfe4e9
 2012-10-30 18:42:58 +09:00
Jean Chalard
ba98b63983 Add the unpackage command 
Bug: 7388852
Change-Id: I7e5376f5d646e669884ef78e4e0dee62c99d4d6b
 2012-10-30 18:20:26 +09:00
Jean Chalard
e44358d2eb Add an empty crypt command to aosp dicttool 
This also makes the AdditionalCommandList useless,
so let's remove it

Bug: 7388852
Change-Id: I308f54ca6f778afe01c1233801a9cc22ebb3d1e9
 2012-10-30 17:44:39 +09:00
Jean Chalard
e5b68fb51d Accept comments in the combined file format 
Change-Id: I17b745281b8b7b1922e1c2d3717986e9bb11aa0b
 2012-10-29 18:08:50 +09:00
Jean Chalard
f41389a74b Remove warnings 
Thanks Eclipse

Change-Id: I88e3979ed22be5d8be5a5accdde417c6b1a8bf2d
 2012-10-29 14:24:16 +09:00
Jean Chalard
74d66a5513 Add word arguments to the info command (A10) 
With this, it's now possible to get all the information about a word from
any dictionary.

Bug: 7388857
Change-Id: Ifc9d539f52f87044a1974d95bf35d87465e84c47
 2012-10-29 12:41:57 +09:00
Jean Chalard
a23e333079 Implement the word-level diff (A9) 
Bug: 7388857
Change-Id: I4c4560d4f4b579936a44cdf409a4c27300b65610
 2012-10-29 12:31:22 +09:00
Jean Chalard
0c5b93800e Add the dicttool diff command. (A8) 
This is an initial implementation, it does not
do everything just yet.

Bug: 7388857
Change-Id: I11543973a4657e122ca3475225059b06b9a3f2b9
 2012-10-29 11:15:15 +09:00
Jean Chalard
6ecc50a867 Add a reporting option to the getDictionary method (A7) 
This is refactoring to help future changes

Change-Id: I8c4372383a2fff5805b96cc47df5012e4b231004
 2012-10-29 10:39:40 +09:00
Jean Chalard
47cac57e45 Finish up the "info" command in dicttool. (A6) 
Bug: 7388857
Change-Id: I704f12a6be76ce1644ec5e8dd3b667f112e9c04a
 2012-10-25 19:15:24 +09:00
Jean Chalard
f1d35ac5dc Implement a part of the "info" command (A5) 
Bug: 7388857
Change-Id: Id1bba7a33ab6195d5711bebb1df5c18c9c08925c
 2012-10-25 18:19:10 +09:00
Jean Chalard
3862b9e819 Add symmetry and pre-emptively improve performance (A4) 
The test for binary dictionary will soon entail decrypting and
uncompressing files if necessary to test for their headers, and will
become much slower than it is. It's better to be able to detect the
XML format too, and leave the slower test for last.

Bug: 7388852
Change-Id: I6b9a7944de80217e1571cab65dcd1cff347b3046
 2012-10-25 17:20:24 +09:00
Jean Chalard
0044df6cf2 Add automatic decryption capabilities. (A3) 
Bug: 7388852
Change-Id: Ie952b1ad6cfad2f83793bd968e159d3cfbf20a47
 2012-10-25 16:47:32 +09:00
Jean Chalard
b3c98901c5 Add auto detection and decoding of dictionary files. (A2) 
Bug: 7388852
Change-Id: I25e755fc15f5b383acc046f668e9681efa4f0c2f
 2012-10-25 16:40:15 +09:00
Jean Chalard
77fe603a3d Some initial refactoring in dicttool. (A1) 
Bug: 7388852
Change-Id: I5ff70d12f3a8096ae6fb8cd4883a32ffe1683c9b
 2012-10-25 16:40:11 +09:00
Jean Chalard
9bb4eebf48 Add reader and writer for the combined dict format. 
This introduces a new textual format for the dictionary that combines
words, bigrams and shortcuts to avoid complexity. It is also extensible
to n-grams to fool-prof for the future, and easier to read than XML.

Bug: 7388540
Change-Id: I942bbad51bd0c905a5a54c278667563fd6dd66ec
 2012-10-23 15:06:25 +09:00
Jean Chalard
cbb3252731 Merge "Use a buffered writer for XML output" 2012-10-22 18:54:38 -07:00
Jean Chalard
000a020e32 Use a buffered writer for XML output 
Change-Id: I85aa133985282993542659db2ee61960b02042d7
 2012-10-23 10:42:22 +09:00
Tadashi G. Takaoka
15f6d4ae34 Add @UsedForTesting and @ExternallyReferenced annotations 
Bug: 7268357
Change-Id: I0b7e0c19f04af9ae30874d0a4c26ad81bc80be8c
 2012-10-22 11:18:43 -07:00
Tadashi G. Takaoka
243c1fecc6 Sync package name and source path 
Change-Id: I8d3ded01c508c3afaa0d8afa9ce1445117626ddc
 2012-10-19 18:14:19 -07:00