nyancrimew/LatinIME
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
35487 commits 1 branch 0 tags 363 MiB
Commit graph

1280 commits

Author SHA1 Message Date
Keisuke Kuroyanagi
ac983b13a9 Use word id to get shortcut iterator. 
Bug: 14425059
Change-Id: I1b35a139bd29f70b328cbc82648783b99f633d72
 2014-09-03 18:33:10 +09:00
Keisuke Kuroyanagi
847a026cd8 Make dictionary structure policy return shortcut iterator. 
Bug: 14425059

Change-Id: I0da22c41f818673430c285103af340397aaba9fb
 2014-09-03 18:20:14 +09:00
Keisuke Kuroyanagi
259ee830f5 Fix: Missing include. 
Change-Id: I23cb5fffa474d916f3e80b951bfb30902498e043
 2014-09-03 17:03:04 +09:00
Keisuke Kuroyanagi
cc6081c51b Merge "Use word id for methods related to n-grams." 2014-09-03 07:42:02 +00:00
Keisuke Kuroyanagi
89a003b12b Use word id for methods related to n-grams. 
Bug: 14425059

Change-Id: I81e5d3793527776d3c9faa5594005ddbd4a71354
 2014-09-03 16:32:43 +09:00
Keisuke Kuroyanagi
29dcf97e74 Add CharUtilsTest. 
Change-Id: I69b57d88a5376393b5d09d8b8e056c82cb8b8a3c
 2014-09-03 13:57:32 +09:00
Keisuke Kuroyanagi
69732169cc Add DamerauLevenshteinEditDistancePolicyTest 
Change-Id: Ia36bc6ddd0ec553fee151153061428c7c66d32d2
 2014-09-02 19:47:43 +09:00
Keisuke Kuroyanagi
985b2c2e91 Add GeometryUtilsTest. 
Change-Id: I6ea9ef137c2930ca8e8745fec829b9bcaf1d10b9
 2014-09-02 17:10:42 +09:00
Keisuke Kuroyanagi
93e3b5a16f Add TerminalPositionLookupTableTest. 
Change-Id: I4a3ab4c94a7759d7f24c7edc9c167fe6bbdd3eb7
 2014-08-29 14:16:15 +09:00
Keisuke Kuroyanagi
40b6d6cbd1 Add DicNodePoolTest. 
Change-Id: I9bec708def3af6bea1c82b9576b80ce1b7901ddc
 2014-08-29 13:02:22 +09:00
Keisuke Kuroyanagi
e394a0c6ff Implement IntArrayView::skip(). 
Change-Id: Ib7f13963ac1d9c64eff2c08391154c6a25024126
 2014-08-28 12:12:56 +09:00
Keisuke Kuroyanagi
733a023b64 Remove redundant code. 
Change-Id: I173eac15dc128e84893881ec4ee0ad1ad2fce44b
 2014-08-27 20:28:48 +09:00
Keisuke Kuroyanagi
6ae4d79d81 Use IntArrayView for dictionary structure policy. 
Change-Id: I0dc94908259d70d5085ff22abf422d90affb1452
 2014-08-27 20:28:43 +09:00
Keisuke Kuroyanagi
b85bf4ebb8 Merge "Use word id to construct DicNode instead of isTerminal flag." 2014-08-27 11:21:41 +00:00
Keisuke Kuroyanagi
fe395232d6 Remove bigram dict content. 
Bug: 14425059
Change-Id: I75918c6761a50832da511088eb83becd56b23662
 2014-08-27 20:05:59 +09:00
Keisuke Kuroyanagi
758d093644 Get entry count after truncation using LanguageModelDictContent. 
Bug: 14425059
Change-Id: I41b237c1c22c21740946d52e3be9d6f963c9cd54
 2014-08-27 20:04:39 +09:00
Keisuke Kuroyanagi
c7f1de826c Merge "Use LanguageModelDictContent in getWordProperty()." 2014-08-27 10:58:28 +00:00
Keisuke Kuroyanagi
7c08ede12b Merge "Use LanguageModelDictContent in iterateNgramEntries()." 2014-08-27 10:44:21 +00:00
Keisuke Kuroyanagi
82f7d3a9de Merge "Add a method to iterate entries in LanguageModelDictContent." 2014-08-27 10:35:38 +00:00
Keisuke Kuroyanagi
78aea9f133 Add TimeKeeperTest. 
Change-Id: Id35823414f8552fd6820a5a559d6c9d54c7d6cb6
 2014-08-27 14:39:27 +09:00
Keisuke Kuroyanagi
7d47500357 Use word id to construct DicNode instead of isTerminal flag. 
Bug: 14425059
Change-Id: I8484d34756bd76668ece34211e7366a4758d7bf5
 2014-08-26 14:33:19 +09:00
Keisuke Kuroyanagi
8b4409f4b9 Use LanguageModelDictContent in getWordProperty(). 
Bug: 14425059
Change-Id: Ic230f764ff5570f24ce6ce930023798718f326df
 2014-08-26 12:05:02 +09:00
Keisuke Kuroyanagi
1c14effab5 Use LanguageModelDictContent in iterateNgramEntries(). 
Bug: 14425059
Change-Id: I8a5028657ea5a4eeee8ef40db52a80f09b878972
 2014-08-26 12:03:14 +09:00
Keisuke Kuroyanagi
07b3b41c25 Add a method to iterate entries in LanguageModelDictContent. 
Bug: 14425059
Change-Id: I4e9c3a97891c020f762fa709f806d333c067f496
 2014-08-26 12:01:08 +09:00
Keisuke Kuroyanagi
30e6b21af0 Add SparseTableTest. 
Change-Id: I5684a24f9584cd326e014274c2298773aa3a91ca
 2014-08-25 22:27:01 +09:00
Keisuke Kuroyanagi
c04ae77583 Add ByteArrayUtilsTest. 
Change-Id: I4695a2573b4d2d8331fc9ce1e9ff83397791a586
 2014-08-25 16:52:12 +09:00
Keisuke Kuroyanagi
295e60234c Get bigram probability using language model dict content. 
Bug: 14425059
Change-Id: Ia0493ed94e9bd47e9396d8cda94511b47820db0d
 2014-08-22 20:13:06 +09:00
Keisuke Kuroyanagi
063f86d40f Truncate entries in language model dict content. 
Bug: 14425059

Change-Id: I023c1d5109a2c43fcea3bb11a0fd7198c82891ba
 2014-08-22 20:13:04 +09:00
Keisuke Kuroyanagi
9aa6699107 Update probabilities in language model dict content for GC. 
Bug: 14425059
Change-Id: I354408afd8e5c1955ff0acea3d0243d628fe3843
 2014-08-22 20:07:54 +09:00
Keisuke Kuroyanagi
9c9f2d06bc Fix TrieMap: Cannot create next level map of small value entry. 
Bug: 14425059
Change-Id: I864f81ea7dd1d847ef2b016366ce046b1c43d5e5
 2014-08-20 20:48:25 +09:00
Keisuke Kuroyanagi
ace03d7919 Merge "Add BoS flag in probability entry." 2014-08-16 04:15:21 +00:00
Keisuke Kuroyanagi
623067a183 Add BoS flag in probability entry. 
Bug: 14425059

Change-Id: I50439630034ada0280c44cbbb308aa0b95b72048
 2014-08-19 11:49:05 +09:00
Keisuke Kuroyanagi
bfcd5efd50 Merge "Use byte array view in ver4 dict contents." 2014-08-16 04:15:21 +00:00
Keisuke Kuroyanagi
1f6e52ef02 Use byte array view in ver4 dict contents. 
Change-Id: Icf79a51a200f7ccd775264d1a83dd61e7dcfbab2
 2014-08-18 22:46:10 +09:00
Keisuke Kuroyanagi
2f34f0d1a8 Merge "Remove entry from language model dict content." 2014-08-16 04:15:21 +00:00
Keisuke Kuroyanagi
d3097c67ca Remove entry from language model dict content. 
Bug: 14425059
Change-Id: Iea51c0ae908d499da19839de06222a1c4d19088e
 2014-08-18 12:34:50 +09:00
Keisuke Kuroyanagi
b4531d861e Add method to remove entry from language model dict content. 
Bug: 14425059
Change-Id: Id21af0110e770caa3e95cb5d7ba8b3d1af8e0b12
 2014-08-18 12:34:48 +09:00
Keisuke Kuroyanagi
0db136e51f Merge "Add method to remove entry from language model dict content." 2014-08-16 04:12:14 +00:00
Keisuke Kuroyanagi
7be252d479 Use %zd to print size_t value. 
Change-Id: Id46f50353b489d84b9a97dd4ddc5519bd99492cd
 2014-08-15 16:50:23 +09:00
Keisuke Kuroyanagi
ed1ab3b363 Improve accent error handling in exact match logic. 
Bug: 16963362
Change-Id: I50a48806bc3147388d165faab6b572415a851f29
 2014-08-15 16:50:23 +09:00
Keisuke Kuroyanagi
5409957440 Merge "Remove unigram for v402 with historical info." 2014-08-15 05:02:04 +00:00
Keisuke Kuroyanagi
8890b01550 Remove unigram for v402 with historical info. 
Bug: 15531638
Change-Id: If1b73ac693e45a80df987ea16b2dece5597697e8
 2014-08-15 15:47:53 +09:00
Keisuke Kuroyanagi
d9d94f40c7 Merge "Make FormatUtils use ByteArrayView." 2014-08-15 05:03:06 +00:00
Keisuke Kuroyanagi
4fbb2148ee Make FormatUtils use ByteArrayView. 
Change-Id: I472b238a0d59f0092ee1f5f3b12ad63823025fae
 2014-08-14 18:09:30 +09:00
Keisuke Kuroyanagi
5fe1bed2e4 Add TrieMap::remove(). 
Bug: 14425059
Change-Id: Ifb9a873eab79efdd6bee8e0da73c0cf37727350e
 2014-08-13 21:10:48 +09:00
Keisuke Kuroyanagi
9a23f0fba2 Add bigrams to language model content. 
Bug: 14425059

Change-Id: Id81e3775ea0104750a23e3dca62c00681ed8dc2e
 2014-08-12 20:32:42 +09:00
Keisuke Kuroyanagi
9069d30043 Use IntArrayView to add/remove n-gram entry. 
Bug: 14425059
Change-Id: I734f12dfde5161abdfb019a6391677dba8984ff4
 2014-08-06 12:52:01 +09:00
Keisuke Kuroyanagi
da8cda0578 Clean up IntArrayViewTest 
Change-Id: I30138f0722571100fe13efc96f8332efde846c2e
 2014-08-06 10:09:02 +09:00
Keisuke Kuroyanagi
03dc44f543 Add/Get n-gram probability entry in languageModelDictContent 
Bug: 14425059
Change-Id: I7926c3812f89b9a71fe1873a5bc32f793f91b640
 2014-08-06 00:42:56 +00:00
Keisuke Kuroyanagi
851e0458fe Remove ProbabilityDictContent and use LanguageModelDictContent 
Bug: 14425059
Change-Id: I1bb9e78ecb24139b87c99be6722e37eec0a2285d
 2014-08-05 14:13:07 +09:00
Keisuke Kuroyanagi
0889484266 Add methods for unigrams to LanguageModelDictContent. 
Bug: 14425059
Change-Id: I0a6b480a3d4735787ffac68c47b4ffefc3f1b8a5
 2014-08-05 12:38:55 +09:00
Keisuke Kuroyanagi
c4696b2eb6 Save language model in the body buffer. 
Bug: 14425059
Change-Id: Iaec277f7bed03d6c6780c6ce90fbe5fe799e175e
 2014-08-01 20:19:16 +09:00
Keisuke Kuroyanagi
c0c674cdc0 Make MmappedBuffer use byte array view. 
Bug: 16691311
Change-Id: I2122c01ee27c33e11dec52643925c069927bea2b
 2014-08-01 19:26:01 +09:00
Keisuke Kuroyanagi
0257e40345 Use ReadWriteByteArrayView in BufferWithExtendableBuffer. 
Bug: 16691311
Change-Id: Ibd68354a53cebf284a886d76bd8ce9848a51346f
 2014-08-01 08:44:32 +00:00
Keisuke Kuroyanagi
df11f9b8e6 Merge "Add LanguageModelDictContent." into lmp-dev 2014-07-24 16:04:26 +00:00
Keisuke Kuroyanagi
dc3856d758 Add LanguageModelDictContent. 
This class will replace BigramDictContent and
ProbabilityDictContent.

Bug: 14425059
Change-Id: I3d15c833957e27b2f5999386db042188272bbb4b
 2014-08-01 12:45:00 +09:00
Keisuke Kuroyanagi
90b7c1729f Remove DictContent. 
Bug: 14425059
Change-Id: I74fa4b6ba4605447c1c87427371e4be5eb8e7ae6
 2014-08-01 12:06:21 +09:00
Keisuke Kuroyanagi
0e6a1d1020 Remove getBigramsIteratorOfPtNode from structure policy. 
Bug: 14425059
Change-Id: I427fbfd026c1e33b5e9cf78949fe862d8748b7ec
 2014-08-01 12:06:21 +09:00
Keisuke Kuroyanagi
35c62b2cc9 Use NgramListener in MultiBigramMap. 
Bug: 14425059
Change-Id: I425536290111f2a8172f31370706f858a1e07f6e
 2014-08-01 02:51:07 +00:00
Keisuke Kuroyanagi
de5c3a2562 Construct TrieMap from existing buffer. 
Bug: 14425059
Change-Id: Ia1c58d88540f55fb38fe0bdfcd7c0a34ebc7415e
 2014-07-31 09:06:22 +00:00
Keisuke Kuroyanagi
e7c9415688 Merge "Remove DictContent." into lmp-dev 2014-07-24 16:04:26 +00:00
Keisuke Kuroyanagi
4a1ed92ea8 Merge "Remove getBigramsIteratorOfPtNode from structure policy." into lmp-dev 2014-07-25 11:19:20 +00:00
Keisuke Kuroyanagi
3437032a91 Merge "Add TrieMap.save()." into lmp-dev 2014-07-24 16:04:26 +00:00
Keisuke Kuroyanagi
60ae3e0be5 Add TrieMap.save(). 
Bug: 14425059
Change-Id: Ie0b4fe8b9a157da0401bb277c703ab084d5bf9cc
 2014-07-31 06:01:52 +00:00
Keisuke Kuroyanagi
5c1decfbb9 Add entry iteration method to TrieMap. 
Bug: 14425059
Change-Id: I79420b755f29f651d8eed61e7e48b6eb001d8dd2
 2014-07-30 19:04:07 +09:00
Keisuke Kuroyanagi
57574b98c2 Merge "Remove doesAutoCorrectValidWord()." into lmp-dev 2014-07-25 11:19:20 +00:00
Keisuke Kuroyanagi
5b69472d56 Remove doesAutoCorrectValidWord(). 
Possibly offensive words are wrongly demoted when input by
gesture.

Change-Id: Ibb2eb73d4191dea6cb767643341fe96465c1f70c
 2014-07-31 14:48:16 +09:00
Keisuke Kuroyanagi
d3d7c31ca9 Fix native debug build. 
Change-Id: I95ff6b14d7f444911ec76698381b8d2d8da94776
 2014-07-31 14:36:12 +09:00
Keisuke Kuroyanagi
d624091afb Merge "Fix native debug build." into lmp-dev 2014-07-25 11:19:20 +00:00
Keisuke Kuroyanagi
211370a3da Merge "Add memory view class for int and uint8_t." into lmp-dev 2014-07-24 16:04:26 +00:00
Keisuke Kuroyanagi
75d8c20d03 Add memory view class for int and uint8_t. 
Bug: 16691311
Change-Id: I32eb692cdb758861e24b772fdcda57642b95ae1b
 2014-08-01 12:39:31 +09:00
Keisuke Kuroyanagi
f40adc97cb Merge "Add a method to iterate ngram entries." into lmp-dev 2014-07-24 16:04:26 +00:00
Keisuke Kuroyanagi
2d57b3339a Add a method to iterate ngram entries. 
Bug: 14425059
Change-Id: I9962c546504288f6c22b1a7368f775decd229c62
 2014-07-30 10:52:56 +09:00
Keisuke Kuroyanagi
5a7b634aaf Merge "Add entry iteration method to TrieMap." into lmp-dev 2014-07-24 16:04:26 +00:00
Keisuke Kuroyanagi
a704ad254e Merge "Remove PrevWordsInfo.getBigramsIteratorForPrediction()." into lmp-dev 2014-07-25 11:19:20 +00:00
Keisuke Kuroyanagi
bd1f59bda5 Remove PrevWordsInfo.getBigramsIteratorForPrediction(). 
This changes the behavior of predictions for some rare cases.
For example, let's consider the case of the prev word is
"ABC", "ABC" and "abc" is in the dict, "ABC" doesn't have
any bigrams, and "abc" has bigrams.
Without this change, "abc"'s bigrams are used for
prediction. But, with this change, "abc"'s bigrams are not
used for prediction. This strategy is the same as the the
current way of bigram probability computation during
traversal (for suggestion).

Bug: 14425059
Change-Id: I857ff39aa70aea265efa651451a63b759bb47d48
 2014-07-30 17:26:26 +09:00
Ken Wakasa
0b1fa0c1c7 Cleanup: C++11 doesn't require a space between angle brackets 
Change-Id: Id0777cf3deac94395bd63aee40b49375dff3e35a
 2014-07-30 11:32:05 +09:00
Keisuke Kuroyanagi
6c3b0d3caa Merge "Add TrieMap." into lmp-dev 2014-07-25 11:19:20 +00:00
Keisuke Kuroyanagi
de3121dead Add TrieMap. 
Bug: 14425059
Change-Id: Ief36f808ed4f3e35bfab07837ea314d2564973ab
 2014-07-29 17:00:29 +09:00
Keisuke Kuroyanagi
2e32e83b6e Merge "Get n-gram probability in structure policy." into lmp-dev 2014-07-24 16:04:26 +00:00
Keisuke Kuroyanagi
1229879e7c Get n-gram probability in structure policy. 
Bug: 14425059
Change-Id: Id955a2e07dbdfb90ae55720f0c92597e0594c0d9
 2014-07-29 12:01:18 +09:00
Keisuke Kuroyanagi
96d47fe745 Change getUnigramProbabilityOfPtNode to getProbabilityOfPtNode() 
Bug: 14425059
Change-Id: I92403d750d2979408b41554dcd79b0ed31b41161
 2014-07-29 11:35:57 +09:00
Keisuke Kuroyanagi
179f38cc8c Merge "Change getUnigramProbabilityOfPtNode to getProbabilityOfPtNode()" into lmp-dev 2014-07-25 11:19:20 +00:00
Keisuke Kuroyanagi
3225b6fe66 Add boundary check for ver2 bigram reading. 
Bug: 16330528
Change-Id: I6aca6c7a735e2a652eb325572d44dff660789cff
 2014-07-28 21:48:00 +09:00
Keisuke Kuroyanagi
825243bfdd Merge "Add BufferWithExtendablebufferTest" into lmp-dev 2014-07-24 15:37:54 +00:00
Keisuke Kuroyanagi
d4c9d50ef0 Add BufferWithExtendablebufferTest 
Bug: 13754552
Change-Id: Ib9c330dc85bdb5d4d3d5bc15cc3ce5f28faa0b12
 2014-07-25 07:02:10 +09:00
Keisuke Kuroyanagi
ff6ca09905 Add BufferWithExtendableBuffer::extend() 
Bug: 14425059
Change-Id: I13dd8412ba4d16a1325ac2210b3852d580e71ad3
 2014-07-25 07:02:05 +09:00
Keisuke Kuroyanagi
dfbe2a8ca8 Merge "Add BufferWithExtendableBuffer::extend()" into lmp-dev 2014-07-24 15:35:58 +00:00
Keisuke Kuroyanagi
f78eb27c7e Merge "Refactoring checkAndPrepareWriting." into lmp-dev 2014-07-24 15:35:58 +00:00
Stephen Hines
5a289ed293 Revert "Revert "Revert "Use mclinker""" 
This reverts commit b48c93b3fc.

MCLinker is not a valid option for clang 3.5.
 2014-07-23 19:58:17 -07:00
Keisuke Kuroyanagi
6810e8df1e Refactoring checkAndPrepareWriting. 
Bug: 14425059
Change-Id: I98940f3841a6c61b0482440b4da21f415dc843f6
 2014-07-24 10:24:42 +09:00
Ken Wakasa
81a48d47d1 Add -mstackrealign to avoid crashes with SSE instructions on pre-K x86 devices. 
bug: 16213554
Change-Id: Icd67ed4fd57fcc5f5bf8039b7892d0d446210247
 2014-07-15 20:26:55 +09:00
Keisuke Kuroyanagi
b22f95ec8a Remove isUpdatable from constructors of dict contents. 
Change-Id: I2d54f477d9b341e944e265786a734f23d152bb81
 2014-07-11 15:23:55 +09:00
Keisuke Kuroyanagi
2ac934296c Concatenate dict buffers other than header to a single file. 
Bug: 13664080
Change-Id: I34c9d8046b339c9b855be378a5fad907382d1359
 2014-07-11 15:15:47 +09:00
Keisuke Kuroyanagi
198be3a6c5 Fix: wrong bigram entry pos can be used during GC. 
Bug: 14425059
Change-Id: I4e7c977c5ef184953682175b118de5473cdb64a6
 2014-07-09 11:18:35 +09:00
Keisuke Kuroyanagi
3b8f2ece40 Merge "Move phrase gesture from additional features to public." 2014-07-02 20:35:42 +00:00
Keisuke Kuroyanagi
b8a9479b57 Move phrase gesture from additional features to public. 
Bug: 15758242
Change-Id: I703dc2703afdf9402c5d2ac27411b74ece993dba
 2014-07-08 16:36:06 +09:00
Keisuke Kuroyanagi
b00973952f Get bigrams iterator via dict structure policy. 
Bug: 14425059
Change-Id: Ie4c6afbb6a3b707b5aa8e9e4f941c9da6e6bcdbf
 2014-07-08 16:23:31 +09:00
Keisuke Kuroyanagi
804f7450fc Use linked list for bigram list. 
BinaryDictionaryTests for VERSION4_DEV:
Before
Time: 36.461
After
Time: 33.031

Bug: 14425059

Change-Id: I9ca2714f450f61f713df6ebd34c953dece991cdb
 2014-07-07 21:09:25 +09:00
Keisuke Kuroyanagi
6afd14dc94 Remove unused method. 
Bug: 13664080
Change-Id: I47fd90f01ea9a18b11037c966150fa5fd926101b
 2014-06-27 15:23:38 +09:00