nyancrimew/LatinIME
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
27590 commits 1 branch 0 tags 363 MiB
Commit graph

424 commits

Author SHA1 Message Date
Yuichiro Hanada
4284e9aae7 Make SparseTable have multiple content tables. 
Bug: 10920165
Change-Id: Ie9008452ee292fb0b1fec66e2ffed228c4af6c3e
 2013-10-02 15:36:13 +09:00
Jean Chalard
fa946d4a0f Fix a test and crash with a better error message when reading 
When there are too many bigrams, we stop reading the file,
so the file pointer is in an inconsistent place. This means we
have no idea what's going to happen next. It's better to crash
right away.

Change-Id: Id3b7b78cbe4fda3493b3c9c46758763e1ab5f6a3
 2013-10-02 11:48:47 +09:00
Keisuke Kuroyanagi
6995310996 Separate "GC" and "GC with decay". 
Bug: 6669677
Change-Id: I9d6aba76cef2616f0549e612db9701e1d6a19467
 2013-10-01 18:09:31 +09:00
Keisuke Kuroyanagi
fd02b2d6ee Implement simple dictionary decay. 
Groundwork and implement simple decay.
Increment probability when typed and decrement probability
at GC.

Bug: 6669677

Change-Id: Ib12caead0cbeef4ce7808fe8ac0b00ee331523fe
 2013-09-30 20:53:35 +09:00
Keisuke Kuroyanagi
b698e9c1fa Fix build. 
Change-Id: If7c76ca5f7030ba2003228ed1877bdb948b4b99d
 2013-09-30 14:21:48 +09:00
Keisuke Kuroyanagi
1cf4789ba6 Merge "Set header attributes for ExpandableBinaryDictionary." 2013-09-29 05:14:22 +00:00
Keisuke Kuroyanagi
5ed30a7660 Set header attributes for ExpandableBinaryDictionary. 
Bug: 6669677

Change-Id: I2462777eaa07600f1c774226adfc4f351882f523
 2013-09-28 12:50:09 +09:00
Keisuke Kuroyanagi
8b580819a5 Merge "Add unigram/bigram counting." 2013-09-27 14:14:49 +00:00
Keisuke Kuroyanagi
31097a57cc Add unigram/bigram counting. 
Bug: 6669677
Change-Id: I05ea2201d822dddf062b08c8467daa336760e16c
 2013-09-27 23:12:12 +09:00
Yuichiro Hanada
a328f538c3 Fix PrioritizedSerialExecutor. 
It was possible that fetchNextTasks() would be called by multiple
threads concurrently.
If it happens, some tasks in the task queues might be ignored.

Change-Id: Idc81c43c45e382da3850cc55b9a42c281548d2a8
 2013-09-27 16:46:34 +09:00
Keisuke Kuroyanagi
e3e10317b0 Merge "Create empty dictionary file in native code." 2013-09-26 08:03:38 +00:00
Yuichiro Hanada
d188af7022 Add SparseTable. 
Bug: 10920165
Change-Id: I749dd0269e788799e30b10beb2671813d40ce15f
 2013-09-26 16:16:30 +09:00
Keisuke Kuroyanagi
5ef6209656 Create empty dictionary file in native code. 
Bug: 6669677
Change-Id: I46d23deee1fd091678f4593561cb8687eb815212
 2013-09-26 12:59:02 +09:00
Ken Wakasa
f56b82f809 Fix the offdevice regression test build 
Followup to If4e44eca3cdc5bb02cf2e0c8c44ecd4bf27fae57

bug: 10622489
Change-Id: If98b2c75725f8692f0c2b41c33e448086404479b
 2013-09-25 19:13:47 +09:00
Keisuke Kuroyanagi
2e58670da9 Quit using ExpandableDictionary. 
Bug: 6669677
Change-Id: Ie90417fa9b726454fe729a665fcd549efabb9e94
 2013-09-24 22:57:15 +09:00
Keisuke Kuroyanagi
3e6b68bc0e Merge "Allow large memory space for GC." 2013-09-24 10:24:28 +00:00
Keisuke Kuroyanagi
3d2802ef2e Allow large memory space for GC. 
Bug: 6669677
Change-Id: I021c92a1ca5cd28f745cbfe729c99e346c58074d
 2013-09-24 19:21:17 +09:00
Jean Chalard
3d68b06662 Copy only the spans we are interested in. 
The PARAGRAPH type of span is dangerous, as concatenating
CharSequences that contain it may crash. We also don't use
other spans than SuggestionSpans, so we don't copy them.

Bug: 10622489
Change-Id: If4e44eca3cdc5bb02cf2e0c8c44ecd4bf27fae57
 2013-09-24 19:04:51 +09:00
Keisuke Kuroyanagi
f3cd38f2c7 GC step 5. Add large tests and fix bugs. 
Bug: 6669677
Change-Id: I41e44a64730f9c66645995095a7ecff51ffd5351
 2013-09-24 16:32:25 +09:00
Keisuke Kuroyanagi
dd7d95bd4a Merge "GC step 4. Update all positions in new dict and add a test." 2013-09-24 06:45:25 +00:00
Keisuke Kuroyanagi
2cfe7f9e3b GC step 4. Update all positions in new dict and add a test. 
Bug: 6669677
Change-Id: I035d848a210653dcd3c1a1f21781968adced9225
 2013-09-24 15:29:56 +09:00
Yuichiro Hanada
1625aeafd2 Fix runReadUnigramsAndBigramsTests. 
Change-Id: Idd9176c9943dfacac5a06957f1a07187b642b207
 2013-09-24 12:31:45 +09:00
Yuichiro Hanada
14087ba52c Add Ver4DictDecoder. 
Bug: 9618601
Change-Id: I43c5840505c6a847aaf4893a400392ccd45903c0
 2013-09-19 16:11:23 +09:00
Keisuke Kuroyanagi
561a83118c Merge "Add dictionary header writing methods." 2013-09-18 09:17:54 +00:00
Keisuke Kuroyanagi
484fa7b59c Add dictionary header writing methods. 
Bug: 6669677
Change-Id: I392ac4776b55779903cbaa17e683005d80017a41
 2013-09-18 18:08:33 +09:00
Satoshi Kataoka
8837deab73 Merge "Fix Emoji recents" 2013-09-18 08:46:25 +00:00
Satoshi Kataoka
ea7bfa5cd5 Fix Emoji recents 
Bug: 10538430

Change-Id: Ia2e423eb1843195b3c3218afc79a5820b8cd729b
 2013-09-18 17:44:16 +09:00
Keisuke Kuroyanagi
72e53a1e38 Merge "Implement dictionary flush." 2013-09-18 05:20:05 +00:00
Keisuke Kuroyanagi
1560543754 Implement dictionary flush. 
Bug: 6669677
Change-Id: I1d70ac6319fb6d8700074de382568c2108887960
 2013-09-18 13:19:14 +09:00
Keisuke Kuroyanagi
9465819cf6 Merge "Add BinaryDictionary.getBigramProbabilityNative()." 2013-09-18 04:10:38 +00:00
Keisuke Kuroyanagi
4d02a2d44d Add BinaryDictionary.getBigramProbabilityNative(). 
Bug: 10197478
Change-Id: Id16e3ee22a17679c46cc7a0199a9e28065177aa3
 2013-09-18 11:18:28 +09:00
Jean Chalard
e8f717943f Improve trailing quotes processing 
Bug: 10445496
Change-Id: Ib7bd05aea59ae6c9b8ce31299ef2235521b0e350
 2013-09-17 22:17:22 +09:00
Keisuke Kuroyanagi
d9f450ef00 Implement removeBigramWords() for DynamicPatriciaTrie. 
Bug: 6669677
Change-Id: I99cb517511b4c968b8bf937baab5d7f42b0f534e
 2013-09-17 18:05:32 +09:00
Jean Chalard
7e51989b3f Merge "[AC7] Actually auto-commit." 2013-09-17 08:20:47 +00:00
Keisuke Kuroyanagi
e8754aba1c [AC7] Actually auto-commit. 
Bug: 9059617
Change-Id: I7e5d89a9037b9181a0a6456c12043b4bfda8fe4b
 2013-09-17 16:44:50 +09:00
Keisuke Kuroyanagi
71cce2dd3e Add a large test for BinaryDictionary.addBigramWords(). 
Bug; 10197478

Change-Id: I52590dd1ff9469557672a4bef4f898ad94b47b6f
 2013-09-17 15:32:37 +09:00
Keisuke Kuroyanagi
78b55a31cb Fix handling multi-bytes characters and add a test. 
Bug: 6669677

Change-Id: Id2154db47adea2929559a4187a726f9dfa83363e
 2013-09-17 15:11:24 +09:00
Keisuke Kuroyanagi
cd6a0430c7 Implement addBigramWords() for DynamicPatriciaTrie. 
Bug: 6669677
Change-Id: Ia6aa76c212165690191c65fee9dacdc44be5acd5
 2013-09-17 12:52:21 +09:00
Jean Chalard
41aaff6cdf Merge "[AC4] Add native methods necessary for auto-commit" 2013-09-13 11:41:55 +00:00
Jean Chalard
24aad5a4d5 [AC4] Add native methods necessary for auto-commit 
Bug: 9059617
Change-Id: I7a47b0675446fc4f39628c60d16de9aea90d1b4d
 2013-09-13 20:15:15 +09:00
Ken Wakasa
fbc7e61a3e Merge "Add Ver4DictEncoder." 2013-09-13 08:51:58 +00:00
Yuichiro Hanada
a099a3e341 Add shutdown and isTerminated to PrioritizedSerialExecutor. 
Change-Id: Id185224d135c8571a39efe8a41a66ce769142903
 2013-09-13 17:46:50 +09:00
Yuichiro Hanada
a141d8ef7d Add Ver4DictEncoder. 
Bug: 9618601
Change-Id: I161d2845906f07c1251deb8005fdffe49c5b7940
 2013-09-13 17:33:51 +09:00
Yuichiro Hanada
0e40cd0c40 Add getDictDecoder. 
Bug: 9618601
Change-Id: I173100ac704c03f7d5d0d53477e83cab5d1110d4
 2013-09-12 20:14:09 +09:00
Yuichiro Hanada
95bc256f41 Add a flag to readDictioanryBinary in DictDecoder. 
Change-Id: I356adb72047ebc43c924fbff1ff45e7460508a31
 2013-09-11 18:20:56 +09:00
Keisuke Kuroyanagi
5b51355b9e Add tests for BinaryDictionary.addUnigramWord(). 
Bug: 10197478
Change-Id: Ie46f360b8c75183b49d42d00fefed4a8531f64a9
 2013-09-10 22:46:10 +09:00
Keisuke Kuroyanagi
b8e857f799 Merge "Add binary dictionary tests." 2013-09-10 11:35:58 +00:00
Keisuke Kuroyanagi
4be103df0e Add binary dictionary tests. 
Bug: 10197478

Change-Id: I13cf9b67d1b61b8e89fe90b431c2d6e066ca8e7a
 2013-09-10 19:16:33 +09:00
Yuichiro Hanada
e193c5255d Revert Ida230ca42 and I6adf7d08f. 
Change-Id: I2ddb250d5a473ea955a5171656974de7288a13f7
 2013-09-10 19:08:43 +09:00
Yuichiro Hanada
ef073f4024 Use PrioritizedSerialExecutor instead of ReentrantReadWriteLock. 
Change-Id: I003b769e408ebd692085a7e17fed6d9396876202
 2013-09-10 19:05:41 +09:00
Yuichiro Hanada
acdabb25f4 Add AsyncResultHolder. 
Change-Id: Icfa685bcda2f5c74f5649f09098d00b4bd321c5a
 2013-09-10 15:48:06 +09:00
Satoshi Kataoka
be96361156 Merge "Check contents in user history dictionary tests" 2013-09-09 06:46:45 +00:00
Satoshi Kataoka
e5a35711b8 Check contents in user history dictionary tests 
Bug: 10667710

Change-Id: I45ebb08cb4ef9a9f6d2da31e2c50e740f88209c2
 2013-09-09 15:28:50 +09:00
Yuichiro Hanada
cc2751ba03 Make commitCurrentAutoCorrection asynchronous. 
Change-Id: Ida230ca4243347fb3ab9fda7de3a9a18f886cd1c
 2013-09-09 14:47:26 +09:00
Keisuke Kuroyanagi
6e04d65932 Make DynamicPred...Base extend ExpandableBinaryDictionary. 
Bug: 6669677
Change-Id: I06afad35d3eb73510c34d10cd4116f5bcf934f7c
 2013-08-26 18:50:22 +09:00
Yuichiro Hanada
2232a70806 Clean up unused imports. 
Change-Id: I7147ca237b99399e79210852aa5bf5a01101d779
 2013-08-26 08:29:16 +00:00
Yuichiro Hanada
752a33640c [Refactor] Add DictDecoder.readUnigramsAndBigramsBinary. 
Change-Id: I259db91d837c67cbcb3b6dc504b21dca23a6a5be
 2013-08-26 17:24:38 +09:00
Yuichiro Hanada
bb5b84a826 [Refactor] Add DictDecoder.getTerminalPosition. 
Change-Id: I9d04f64a58f5481cbb64cf1c09b5c485dd4176b4
 2013-08-26 16:14:59 +09:00
Yuichiro Hanada
576f625ee1 Rename CharGroup to PtNode. 
Bug: 10233675
Change-Id: I7b0eb07d195cd386cd0d9e97cd59bf48fcf24107
 2013-08-26 15:58:30 +09:00
Yuichiro Hanada
e9a10ff0f0 Add DictDecoder.readDictionaryBinary. 
Bug: 10434720
Change-Id: I14690a6e0f922ed1bab3a4b6c9a457ae84d4c1a4
 2013-08-23 20:29:25 +09:00
Yuichiro Hanada
1b50a42b7c Fix addWord. 
Change-Id: Ib71dbd5954f44ca80733fa4b1048c829e1df138a
 2013-08-23 16:56:48 +09:00
Yuichiro Hanada
373c492a02 Add an unit test for CharEncoding. 
Change-Id: Ifb1cc01fa5bc2d6d69671f1acb9b9675a4081d32
 2013-08-22 23:05:09 +09:00
Yuichiro Hanada
aa4168ee09 Fix writePlacedNode. 
Change-Id: I1d6b086f1d9f0dbd8d74f964e29ae62c533af978
 2013-08-22 23:02:08 +09:00
Yuichiro Hanada
e301085a70 Move findWordByBinaryDictReader to BinaryDictIOUtilsTests. 
Change-Id: I443238fd816dea9650dcbbeb3ea757f9674fa52f
 2013-08-22 14:49:49 +09:00
Yuichiro Hanada
c922c8a504 Add DictEncoder. 
Change-Id: I41049b9118b58838e5dedf8e5618d939ca70c5ef
 2013-08-22 11:53:41 +09:00
Yuichiro Hanada
558e34c7bd Make readPtNode be called with the address from the beginning of the file. 
Change-Id: I8939fdfb4f79e55bcd7393633784effb30df3f8f
 2013-08-21 20:02:18 +09:00
Yuichiro Hanada
a306e08753 Rename BinaryDictEncoder to BinaryDictEncoderUtils. 
Change-Id: I4dabf17da7003b1d8204a83dbd10e5be6e8fd805
 2013-08-21 18:54:34 +09:00
Yuichiro Hanada
107a5f6fb8 Add PtNodeReader. 
Change-Id: Ic918822fc1b3a8a7c39ffbcf7defde2c5bf888db
 2013-08-21 18:43:18 +09:00
Jean Chalard
e38097dc2a Add tests for start composing 
Bug: 10403176
Change-Id: I510bf1dccf6795426e6fb1b36181b18039a0f435
 2013-08-21 16:19:31 +09:00
Jean Chalard
e8ef095670 [AC2] Reference a dict rather than a string in suggestion infos 
Bug: 9059617
Change-Id: Ic17bc0fd5d812268fd37d7fd35b4e9ebfb95fa5e
 2013-08-20 19:18:09 +09:00
Jean Chalard
ef1e363016 [AC1] Add a memory of where to cut the first word. 
Bug: 9059617
Change-Id: I931b6d1eb49f9f7dd7d5b18956e35751d1b2b631
 2013-08-20 18:43:19 +09:00
Yuichiro Hanada
065aad9501 Add DictDecoder. 
Change-Id: Ia1c32f21fe07081ce04d093660e18146b93275a4
 2013-08-20 17:43:13 +09:00
Yuichiro Hanada
112257e40f Rename BinaryDictDecoder to Ver3DictDecoder. 
Change-Id: Ibf9b95b658df6e2c2218bdb62e2380f326a03832
 2013-08-20 17:11:51 +09:00
Yuichiro Hanada
66004ce2de Remove populateOptions. 
Change-Id: I1a1830aaa8ea586b68fc34ff3a27ae52b810e8af
 2013-08-20 16:06:52 +09:00
Yuichiro Hanada
77bce05e6f [Refactor] Rename BinaryDictReader and BinaryDictDecoder. 
BinaryDictReader -> BinaryDictDecoder.
BinaryDictDecoder -> BianryDictDecoderUtils.

Change-Id: Iadf2153b379b760538ecda488dda4f17225e5f37
 2013-08-19 19:36:31 +09:00
Yuichiro Hanada
9204303206 Remove an unnecessary debug output. 
Change-Id: If92bf1e3908257c4c0cd68d041b122404e301e08
 2013-08-19 17:04:32 +09:00
Ken Wakasa
a83e25642f Merge "Add HeaderReaderInterface." 2013-08-19 02:34:23 +00:00
Yuichiro Hanada
d794b42f98 Add HeaderReaderInterface. 
Change-Id: I298f86b70d18cd08b240509b6f757c72e1a59ffe
 2013-08-19 11:15:03 +09:00
Yuichiro Hanada
8aaae56cf6 Fix unit test. 
Change-Id: Ib104d5de71c2ab1a07921b407c74c21b0409d9af
 2013-08-19 11:10:28 +09:00
Ken Wakasa
606a056b53 Merge "Make BinaryDictIOUtils and DynamicBinaryIOUtils use BinaryDictReader." 2013-08-16 12:46:07 +00:00
Yuichiro Hanada
3a73b37b30 Make BinaryDictIOUtils and DynamicBinaryIOUtils use BinaryDictReader. 
Change-Id: I191dfe0e05ff3c2c5af99e8beebbb73b097748a3
 2013-08-16 21:06:23 +09:00
Jean Chalard
e32475611c Merge "Support languages that don't use spaces." 2013-08-16 12:05:16 +00:00
Jean Chalard
c239a34262 Support languages that don't use spaces. 
Thai is an example.

Bug: 10138062
Change-Id: Ib9a8264c77ed42b4256432d7c8a60d08575dcdc7
 2013-08-16 21:01:08 +09:00
Ken Wakasa
c2f2d0d211 Merge "Limit the number of waiting UpdateBinaryTask to at most 1." 2013-08-16 11:58:57 +00:00
Yuichiro Hanada
0d70bcc821 Limit the number of waiting UpdateBinaryTask to at most 1. 
Change-Id: Ia588d8f29ab58983ea5321ff8c63ad273c33eab3
 2013-08-16 19:44:47 +09:00
Jean Chalard
21dddb1462 Merge "Rename Node to PtNodeArray" 2013-08-16 08:00:55 +00:00
Satoshi Kataoka
ad4c90934b Merge "Add hex string utils" 2013-08-16 07:34:42 +00:00
Jean Chalard
af30cbf0ee Rename Node to PtNodeArray 
Bug: 10247660
Change-Id: I1a0ac19f58f96adb5efac5fd35c6404831618c99
 2013-08-16 16:24:54 +09:00
Satoshi Kataoka
283cf9cfc9 Add hex string utils 
Bug: 9429906
Change-Id: I3267d1d86122ab471c3e63008c73e9f9b758fc8e
 2013-08-16 16:17:48 +09:00
Tadashi G. Takaoka
5080983ea6 Fix SubtypeLocaleUtilsTests 
Change-Id: Ib1005ee7e99e526aad5099b160051b12e7614e81
 2013-08-16 11:21:32 +09:00
Ken Wakasa
33fce975c9 Merge "[Refactor] Divide BinaryDictInputOutput into BinaryDictEncoder and BinaryDictDecoder." 2013-08-15 11:26:06 +00:00
Yuichiro Hanada
94460eba11 [Refactor] Divide BinaryDictInputOutput into BinaryDictEncoder and BinaryDictDecoder. 
Change-Id: I7c3269d77e3e3b567e459dcaa1bc029903941744
 2013-08-15 20:23:07 +09:00
Tadashi G. Takaoka
7dc60f9db7 Add getters and copy constructor to Key class 
Change-Id: I78eb96dcc26378926dc63acbfeb4828cbd44435d
 2013-08-15 20:12:46 +09:00
Ken Wakasa
117f18e844 Revert "[Refactor] Divide BinaryDictInputOutput into BinaryDictInputUtils and BinaryDictOutputUtils." 
This reverts commit 4c63d0614e.

Change-Id: I1fa277d720bab4d895259df7d6d82eebfa5eb6c5
 2013-08-15 08:54:29 +00:00
Yuichiro Hanada
4c63d0614e [Refactor] Divide BinaryDictInputOutput into BinaryDictInputUtils and BinaryDictOutputUtils. 
Change-Id: I0d476abe763c11ba9005152f928e8dccf15ac9de
 2013-08-15 15:46:58 +09:00
Yuichiro Hanada
3edb62c69b Move some methods in BinaryDictIOUtils to DynamicBinaryDictIOUtils. 
Change-Id: I9ba55582c533fef0eb3e60c46bf23c8b16ee1ff4
 2013-08-14 19:33:36 +09:00
Ken Wakasa
f795f2b789 Merge "Add FusionDictionaryBufferFromWritableByteBufferFactory." 2013-08-14 10:26:21 +00:00
Jean Chalard
af5fbe70ac Add looksValidForDictionaryInsertion 
...and test it.

Also at the same time, add a facility to create a SettingsValues for
test, and some minor performance improvement to surrounding
methods.

Change-Id: I13b629ae14755c244af2a9406a7e9b4a4a16090f
 2013-08-14 18:39:28 +09:00
Yuichiro Hanada
bbc8a930f7 Add FusionDictionaryBufferFromWritableByteBufferFactory. 
Change-Id: I23de0a178e7f11f2cf301fd433cde60c6152055b
 2013-08-14 17:07:44 +09:00
Yuichiro Hanada
3feacba1eb Add BinaryDictReader. 
Bug: 9618601

Change-Id: Ief07fa0c3c4f7f5999a3fafcef4e47b6b6fd8143
 2013-08-13 19:55:05 +09:00