Keisuke Kuroyanagi
45783013bf
Rename prev_words_info.h to ngram_context.h
...
Bug: 14425059
Change-Id: I0e906631ecad2361a8198b3f9e3394bb22c5bf83
2014-10-09 21:28:19 +09:00
Adrian Velicu
44efbe64b1
Fixing misspelled word
...
Change-Id: I51d77e271143d40256b39e5c60a3065d9fdf63fb
2014-10-09 19:26:54 +09:00
Keisuke Kuroyanagi
72e2383d11
Rename PrevWordsInfo to NgramContext.
...
Bug: 14425059
Change-Id: I30703fc80e9450d4e2dbfec965e7f9f4468f6a11
2014-10-09 17:34:32 +09:00
Keisuke Kuroyanagi
ab4437f468
Rename updateCounter to updateEntriesForWordWithNgramContext.
...
Bug: 14425059
Change-Id: Id9b0dd7e32c711ed4292981517c3febd5fe9e897
2014-10-09 17:34:29 +09:00
Keisuke Kuroyanagi
948ef10d03
Merge "Improve bigram probability computation for decaying dicts."
2014-10-06 13:06:29 +00:00
Keisuke Kuroyanagi
aae1a062eb
Improve bigram probability computation for decaying dicts.
...
Without personalization:
Total words: 1079345, Success Num: 819749, Success Percentage: 75.949%
Bad Failures, with auto-correction (typed word == expected word, output word != expected word): 1754, Bad Failure Percentage: 0.163%
Failures, with auto-correction (F-C): 28463, F-C Percentage: 2.637%
Max Keystrokes: 6074285, Min Keystrokes: 4649326, Keystroke Saving Percentage:23.459%
With current probability computing logic:
Total words: 1079382, Success Num: 838329, Success Percentage: 77.667%
Bad Failures, with auto-correction (typed word == expected word, output word != expected word): 1332, Bad Failure Percentage: 0.123%
Failures, with auto-correction (F-C): 28558, F-C Percentage: 2.646%
Max Keystrokes: 6074503, Min Keystrokes: 4474102, Keystroke Saving Percentage:26.346%
Remove isof files.
With new probability computing logic:
Total words: 1079356, Success Num: 844954, Success Percentage: 78.283%
Bad Failures, with auto-correction (typed word == expected word, output word != expected word): 1306, Bad Failure Percentage: 0.121%
Failures, with auto-correction (F-C): 27214, F-C Percentage: 2.521%
Max Keystrokes: 6074477, Min Keystrokes: 4243021, Keystroke Saving Percentage:30.150%
Remove isof files.
Bug: 16547409
Change-Id: I3d2a49c7aaa2c0f6835c52ef72d22466ee225789
2014-10-06 22:03:11 +09:00
Keisuke Kuroyanagi
552470c882
Merge "Make sure to suppress BoS prediction until input twice."
2014-10-06 10:46:31 +00:00
Keisuke Kuroyanagi
c7d199e770
Merge "Fix: BoS prediction is shown after inputting just once."
2014-10-06 10:38:18 +00:00
Keisuke Kuroyanagi
1c2f00f6b5
Make sure to suppress BoS prediction until input twice.
...
Change-Id: I98d91f264d5d1d3f5bcda1fd9ec885779ba2f746
2014-10-06 19:36:04 +09:00
Keisuke Kuroyanagi
ca17ed7d9d
Fix: BoS prediction is shown after inputting just once.
...
Change-Id: Ibba209f47cb5b1a4b08281689d607711b8dcfad4
2014-10-06 19:28:39 +09:00
Keisuke Kuroyanagi
36c4eaadfb
Show prediction results in debug build.
...
Bug: 16547409
Change-Id: If85418583998cd639c794bf5d5cfbbb972c34f72
2014-10-06 18:36:54 +09:00
Keisuke Kuroyanagi
16e1615301
Fix: ProbabilityDictContent can be wrongly large.
...
It can be twice as large as it should be (80KB larger).
Change-Id: If94f748f8c48a442b3c95ac989099aaed2aa2f86
2014-10-06 11:55:07 +09:00
Jean Chalard
4ef27c0358
[ML13] Fix the locale passing in ProximityInfo
...
The locale is used to determine additional proximity characters. This
is dependent on the dictionary language, but was passed as a function
of the layout, which is wrong and would have given bad suggestions in
multi-lingual mode.
Ideally, additional proximity characters should be inserted in the
dictionary header, but for now it's a rather simple change to get
it from the dictionary's locale instead of the proximity info locale.
Also, that allows us to remove completely the locale parameter from
proximity info, which is a much needed change.
This change has zero effect on unit tests and on regression tests.
Bug: 11230254
Change-Id: If95157155db7dccd1f00b8ba55ccb3600283f9e4
2014-10-03 18:16:34 +09:00
Keisuke Kuroyanagi
29777e3a8a
Implement updateCounter() by using existing entry adding methods.
...
Bug: 14425059
Change-Id: I0b6cb80e1fb8f738e9c7d9e80fbc0c479546b879
2014-10-01 19:59:39 +09:00
Keisuke Kuroyanagi
287e155e44
Move HistoricalInfo to property and use it in *Property.
...
Bug: 14425059
Change-Id: Icccccabad98fb543c6a6be2844cfc0086d80b739
2014-10-01 11:39:33 +09:00
Keisuke Kuroyanagi
79bb37d499
Rename BigramProperty to NgramProperty.
...
Remaining work is changing bigram to ngram for supporting
ngram entry counting, dumping, and migration.
Bug: 14425059
Change-Id: Ifba288a1166996d62a5e57698f63537ea0a2a8ee
2014-09-29 19:10:39 +09:00
Keisuke Kuroyanagi
cb4f544198
Quit reading unigram probability in Ver4PatriciaTrieNodeReader.
...
Bug: 14425059
Change-Id: I4fc7b0e236151a2c64e7131772264024c6597633
2014-09-25 11:41:50 +09:00
Keisuke Kuroyanagi
2842e50c4b
Use std::move for dictionary properties.
...
Change-Id: I15056b36b7493f4bac1dbcbb46a0b44343ede153
2014-09-25 11:36:52 +09:00
Keisuke Kuroyanagi
80d139a694
Use CodePointArrayView in WordProperty.
...
Change-Id: I45a9755c413003831788d190beb499fee8ce63aa
2014-09-24 14:15:36 +09:00
Keisuke Kuroyanagi
65a7ccfa00
Refactoring method to get code points and code point count.
...
Bug: 14425059
Change-Id: I4731bd6076d34556e46e6714180fed324fb6aba3
2014-09-24 14:15:36 +09:00
Keisuke Kuroyanagi
7d911d6f91
Move word flags to language model dict content.
...
Bug: 14425059
Change-Id: I64712e5c83d0bc241e6f0f16117ab47b5d75bd4b
2014-09-24 14:15:34 +09:00
Keisuke Kuroyanagi
ddfaeff544
Prepare supporting n-gram for user history dictionary.
...
Bug:17097992
Change-Id: Ic8bfde3d4cc0e720bf7681e08e16fb2ad94d5670
2014-09-22 18:18:50 +09:00
Keisuke Kuroyanagi
5a20827fc7
Fix native crash.
...
Change-Id: Ia8871db0248ddbc821605079e2a3e93859f4c9b7
2014-09-19 21:04:15 +09:00
Akifumi Yoshimoto
5c6db929e4
Merge "Native side reads character table"
2014-09-19 09:58:40 +00:00
Akifumi Yoshimoto
fb2bde5a68
Native side reads character table
...
Bug:17097992
Change-Id: Ibcfc67833a6754fe6a2d82a3e3023b33886b9ea2
2014-09-19 18:00:16 +09:00
Jean Chalard
6da9b21191
[ML8] Add a language weight
...
...and rename an improperly named normalization value
Bug: 11230254
Change-Id: I0f5633148a9f66dbfd7d28540b8a8985131c4549
2014-09-19 13:44:42 +09:00
Keisuke Kuroyanagi
9db2cb4a37
Make MAX_PREV_WORD_COUNT_FOR_N_GRAM 2.
...
Bug: 14425059
Change-Id: Iefacd7e4a6c256658a58856240310eeb91ae32db
2014-09-18 19:18:55 +09:00
Keisuke Kuroyanagi
8806248386
Pass prevWordCount to native code for getSuggestion.
...
Bug: 14425059
Change-Id: Id2b49a448982c997e5c0a22445b115e675156c3e
2014-09-18 19:18:55 +09:00
Keisuke Kuroyanagi
b349b57476
Merge "Use ReadOnlyByteArrayView in Ver2ParticiaTrieNodeReader."
2014-09-17 12:53:54 +00:00
Keisuke Kuroyanagi
a533e18c07
Merge "Use ReadOnlyByteArrayView in Ver2PtNodeArrayReader."
2014-09-17 12:53:48 +00:00
Keisuke Kuroyanagi
4a55d41410
Merge "Use ReadOnlyByteArrayView in ShortcutListPolicy"
2014-09-17 12:51:06 +00:00
Keisuke Kuroyanagi
c1040a255c
Merge "Use ReadOnlyByteArrayView in BigramListPolicy."
2014-09-17 12:50:51 +00:00
Keisuke Kuroyanagi
5849feeee1
Merge "Use ReadOnlyByteArrayView in PatriciaTriePolicy."
2014-09-17 12:50:16 +00:00
Keisuke Kuroyanagi
198a47a161
Merge "Add firstOrDefault and lastOrDefault to IntArrayView."
2014-09-17 12:43:10 +00:00
Keisuke Kuroyanagi
6258c57c32
Use ReadOnlyByteArrayView in Ver2ParticiaTrieNodeReader.
...
Change-Id: Ie3325bb9666c7ff6d519a1b72ed2fe9fc4f140a9
2014-09-17 21:42:30 +09:00
Keisuke Kuroyanagi
a892628fa3
Use ReadOnlyByteArrayView in Ver2PtNodeArrayReader.
...
Change-Id: Ia1065465abdf7c64fadb3ff1efd2a4d2df343264
2014-09-17 21:42:30 +09:00
Keisuke Kuroyanagi
59ebd51718
Use ReadOnlyByteArrayView in ShortcutListPolicy
...
Change-Id: I03a6f49c9005306bcc5fce8b7e4d37b8d30b9faa
2014-09-17 21:42:29 +09:00
Keisuke Kuroyanagi
d01eb3c94c
Use ReadOnlyByteArrayView in BigramListPolicy.
...
Change-Id: I0f8e8044c8803a7a542f97be5b8f08fdb5ae3926
2014-09-17 21:42:26 +09:00
Keisuke Kuroyanagi
180e7b4c07
Use ReadOnlyByteArrayView in PatriciaTriePolicy.
...
Change-Id: I4ef01c0af73f27c256cc1a719343be836913dc71
2014-09-17 21:38:46 +09:00
Keisuke Kuroyanagi
09c154925f
Add firstOrDefault and lastOrDefault to IntArrayView.
...
Change-Id: I854c02eff3fa0b53c72a5f1cabce001f4854ada0
2014-09-17 21:16:31 +09:00
Keisuke Kuroyanagi
fc7d0540fe
Use CodePointArrayView in DictionaryUtils.
...
Change-Id: I9ae308e60124ea5acb4ee09847c4fdd58ff168e2
2014-09-17 20:13:36 +09:00
Keisuke Kuroyanagi
3e75c59133
Use CodePointArrayView in Dictionary.
...
Change-Id: I63fa0a8348f6de6ec7a424a8033e936b4af72beb
2014-09-17 20:13:36 +09:00
Keisuke Kuroyanagi
89a074fade
Use IntArrayView in DynamicPtUpdatingHelper.
...
Change-Id: Ifa185eba7dd9abee1b35e49fe559be1042ca63d7
2014-09-17 20:13:36 +09:00
Keisuke Kuroyanagi
4926b90ec5
Support n-gram for look-up.
...
Bug: 14425059
Change-Id: I19523c29fb802cd65158c7540d1608e7f55c4ca7
2014-09-17 16:20:00 +09:00
Keisuke Kuroyanagi
5520e84e16
Support removing n-gram entry in structure policy.
...
Bug: 14425059
Change-Id: Ifd2ac97a15dcbb936e2f068866a9d1ee68759ba2
2014-09-16 15:46:28 +09:00
Keisuke Kuroyanagi
0c0b8207cd
Support adding n-gram entry in structure policy.
...
Bug: 14425059
Change-Id: Ia207d3c4735b1d6d43f18d18a70d28af613cb458
2014-09-16 15:37:09 +09:00
Keisuke Kuroyanagi
7eb6e28b9e
Update input checking logic for adding/removing n-gram.
...
Bug: 14425059
Change-Id: Ifb4e3c3f06aaf9a16f632ff208ba76cf741cea83
2014-09-16 14:14:22 +09:00
Keisuke Kuroyanagi
d2230525bc
Have mPrevWordCount in DicNodeProperties.
...
Bug: 14425059
Change-Id: I5ce22bace4ec08d0da4e5c167288a742c4426c33
2014-09-16 12:46:16 +09:00
Keisuke Kuroyanagi
36ba139ca6
Support decaying dict in getWordProbability().
...
Bug: 14425059
Change-Id: I24db3f9131c2999fc388035dc365c7faaef3bdb1
2014-09-14 17:29:50 +09:00
Keisuke Kuroyanagi
c43b6664fa
Use passed previous word count in PrevWordsInfo.
...
Bug: 14425059
Change-Id: I04007bdacf0176a05be7a27ef1c20c5b851d8bed
2014-09-14 17:29:38 +09:00
Keisuke Kuroyanagi
537f6eea8a
Use WordIdArrayView for prevWordIds.
...
Bug: 14425059
Change-Id: Ia84fb997d89564e60111b46ca83bbfa3b187f316
2014-09-11 19:36:22 +09:00
Keisuke Kuroyanagi
a3b0eb1685
Implement IntArrayView.copyToBuffer().
...
Bug: 14425059
Change-Id: Ic1b7de72701eae0c0b1bb28d12514099446a66a9
2014-09-11 15:21:52 +09:00
Keisuke Kuroyanagi
d53aea5af9
Remove unigram probability from dicNode.
...
Bug: 14425059
Change-Id: Ie848e8568bb4dbb1d8358e823a881d9157a1aad3
2014-09-10 21:21:25 +09:00
Keisuke Kuroyanagi
c32356c229
Quit using dicNode.getUnigramProbability().
...
Bug: 14425059
Change-Id: I192070cc11e5d46c8413ebc19982d6a8c93577fc
2014-09-10 21:21:25 +09:00
Keisuke Kuroyanagi
521e2382da
Use CodePointArrayView to create children DicNodes.
...
Change-Id: Ie940b6595f3f3f804fbb8dd03c710ea062b75af3
2014-09-10 21:21:23 +09:00
Keisuke Kuroyanagi
87a5c76906
Use WordAttributes for checking flags.
...
Bug: 14425059
Change-Id: Idee84478a482a0e7b5cc53e5dbd4e2484584ba79
2014-09-10 19:51:57 +09:00
Keisuke Kuroyanagi
2111e3abc9
Introduce WordAttributes to get word probability and flags.
...
Bug: 14425059
Change-Id: Iee11d038e0893d7ddd6c52447907f8c55fecb6a5
2014-09-10 19:51:48 +09:00
Keisuke Kuroyanagi
11a48f92a5
Use getProbabilityOfWordInContext for prediction.
...
Bug: 14425059
Change-Id: I9d5c905a0adda3503c593bfbf0bb9af8d1686f5d
2014-09-10 19:51:14 +09:00
Keisuke Kuroyanagi
395fe8e98d
Implement LanguageModelDictContent.getWordProbability().
...
Bug: 14425059
Change-Id: I290a05cee6f341caa25fb222892505529cef1eb7
2014-09-10 19:51:12 +09:00
Keisuke Kuroyanagi
9f8da0f833
Use MultiBigramMap in structure policy.
...
Bug: 14425059
Change-Id: I4d78da4839ef177e0223e6e5bcf0ebd7315c3099
2014-09-09 17:53:44 +09:00
Keisuke Kuroyanagi
138647876b
Merge "Rename probability to unigramProbability."
2014-09-09 06:16:08 +00:00
Keisuke Kuroyanagi
72ee1c7ee5
Merge "Remove mHasChildrenPtNodes from DicNodeProperties."
2014-09-09 06:03:36 +00:00
Keisuke Kuroyanagi
9c42ad47d4
Rename probability to unigramProbability.
...
Bug: 14425059
Change-Id: I6a204c3b8fb257d037ad95a1a455ae6fb89068fd
2014-09-09 14:09:01 +09:00
Keisuke Kuroyanagi
d028294890
Remove mHasChildrenPtNodes from DicNodeProperties.
...
Bug: 14425059
Change-Id: I3a9511e7f7c3a722f9942f525530f04def5965da
2014-09-09 14:08:41 +09:00
Keisuke Kuroyanagi
3f779718cc
Merge "Remove DicNode.getPtNodePos()."
2014-09-05 10:15:45 +00:00
Keisuke Kuroyanagi
9ff6fee838
Remove DicNode.getPtNodePos().
...
Bug: 14425059
Change-Id: If6e291d23e68342792febb85f8a576ce785b3845
2014-09-05 17:27:01 +09:00
Keisuke Kuroyanagi
5cfa023777
Merge "Add HeaderReadWriteUtilsTest"
2014-09-04 07:30:59 +00:00
Keisuke Kuroyanagi
fbddeedff6
Add HeaderReadWriteUtilsTest
...
Change-Id: I08aeaa3e5852008874e38ed4799ab8989759e861
2014-09-04 15:32:30 +09:00
Keisuke Kuroyanagi
94e4cd25a8
Use word id to get code ponits of the word.
...
Bug: 14425059
Change-Id: I81accffcdf5abe447c33ffc3a8e8315f9a4cde7f
2014-09-03 18:55:31 +09:00
Keisuke Kuroyanagi
ac983b13a9
Use word id to get shortcut iterator.
...
Bug: 14425059
Change-Id: I1b35a139bd29f70b328cbc82648783b99f633d72
2014-09-03 18:33:10 +09:00
Keisuke Kuroyanagi
847a026cd8
Make dictionary structure policy return shortcut iterator.
...
Bug: 14425059
Change-Id: I0da22c41f818673430c285103af340397aaba9fb
2014-09-03 18:20:14 +09:00
Keisuke Kuroyanagi
259ee830f5
Fix: Missing include.
...
Change-Id: I23cb5fffa474d916f3e80b951bfb30902498e043
2014-09-03 17:03:04 +09:00
Keisuke Kuroyanagi
cc6081c51b
Merge "Use word id for methods related to n-grams."
2014-09-03 07:42:02 +00:00
Keisuke Kuroyanagi
89a003b12b
Use word id for methods related to n-grams.
...
Bug: 14425059
Change-Id: I81e5d3793527776d3c9faa5594005ddbd4a71354
2014-09-03 16:32:43 +09:00
Keisuke Kuroyanagi
29dcf97e74
Add CharUtilsTest.
...
Change-Id: I69b57d88a5376393b5d09d8b8e056c82cb8b8a3c
2014-09-03 13:57:32 +09:00
Keisuke Kuroyanagi
985b2c2e91
Add GeometryUtilsTest.
...
Change-Id: I6ea9ef137c2930ca8e8745fec829b9bcaf1d10b9
2014-09-02 17:10:42 +09:00
Keisuke Kuroyanagi
93e3b5a16f
Add TerminalPositionLookupTableTest.
...
Change-Id: I4a3ab4c94a7759d7f24c7edc9c167fe6bbdd3eb7
2014-08-29 14:16:15 +09:00
Keisuke Kuroyanagi
e394a0c6ff
Implement IntArrayView::skip().
...
Change-Id: Ib7f13963ac1d9c64eff2c08391154c6a25024126
2014-08-28 12:12:56 +09:00
Keisuke Kuroyanagi
733a023b64
Remove redundant code.
...
Change-Id: I173eac15dc128e84893881ec4ee0ad1ad2fce44b
2014-08-27 20:28:48 +09:00
Keisuke Kuroyanagi
6ae4d79d81
Use IntArrayView for dictionary structure policy.
...
Change-Id: I0dc94908259d70d5085ff22abf422d90affb1452
2014-08-27 20:28:43 +09:00
Keisuke Kuroyanagi
b85bf4ebb8
Merge "Use word id to construct DicNode instead of isTerminal flag."
2014-08-27 11:21:41 +00:00
Keisuke Kuroyanagi
fe395232d6
Remove bigram dict content.
...
Bug: 14425059
Change-Id: I75918c6761a50832da511088eb83becd56b23662
2014-08-27 20:05:59 +09:00
Keisuke Kuroyanagi
758d093644
Get entry count after truncation using LanguageModelDictContent.
...
Bug: 14425059
Change-Id: I41b237c1c22c21740946d52e3be9d6f963c9cd54
2014-08-27 20:04:39 +09:00
Keisuke Kuroyanagi
c7f1de826c
Merge "Use LanguageModelDictContent in getWordProperty()."
2014-08-27 10:58:28 +00:00
Keisuke Kuroyanagi
7c08ede12b
Merge "Use LanguageModelDictContent in iterateNgramEntries()."
2014-08-27 10:44:21 +00:00
Keisuke Kuroyanagi
82f7d3a9de
Merge "Add a method to iterate entries in LanguageModelDictContent."
2014-08-27 10:35:38 +00:00
Keisuke Kuroyanagi
7d47500357
Use word id to construct DicNode instead of isTerminal flag.
...
Bug: 14425059
Change-Id: I8484d34756bd76668ece34211e7366a4758d7bf5
2014-08-26 14:33:19 +09:00
Keisuke Kuroyanagi
8b4409f4b9
Use LanguageModelDictContent in getWordProperty().
...
Bug: 14425059
Change-Id: Ic230f764ff5570f24ce6ce930023798718f326df
2014-08-26 12:05:02 +09:00
Keisuke Kuroyanagi
1c14effab5
Use LanguageModelDictContent in iterateNgramEntries().
...
Bug: 14425059
Change-Id: I8a5028657ea5a4eeee8ef40db52a80f09b878972
2014-08-26 12:03:14 +09:00
Keisuke Kuroyanagi
07b3b41c25
Add a method to iterate entries in LanguageModelDictContent.
...
Bug: 14425059
Change-Id: I4e9c3a97891c020f762fa709f806d333c067f496
2014-08-26 12:01:08 +09:00
Keisuke Kuroyanagi
30e6b21af0
Add SparseTableTest.
...
Change-Id: I5684a24f9584cd326e014274c2298773aa3a91ca
2014-08-25 22:27:01 +09:00
Keisuke Kuroyanagi
c04ae77583
Add ByteArrayUtilsTest.
...
Change-Id: I4695a2573b4d2d8331fc9ce1e9ff83397791a586
2014-08-25 16:52:12 +09:00
Keisuke Kuroyanagi
295e60234c
Get bigram probability using language model dict content.
...
Bug: 14425059
Change-Id: Ia0493ed94e9bd47e9396d8cda94511b47820db0d
2014-08-22 20:13:06 +09:00
Keisuke Kuroyanagi
063f86d40f
Truncate entries in language model dict content.
...
Bug: 14425059
Change-Id: I023c1d5109a2c43fcea3bb11a0fd7198c82891ba
2014-08-22 20:13:04 +09:00
Keisuke Kuroyanagi
9aa6699107
Update probabilities in language model dict content for GC.
...
Bug: 14425059
Change-Id: I354408afd8e5c1955ff0acea3d0243d628fe3843
2014-08-22 20:07:54 +09:00
Keisuke Kuroyanagi
9c9f2d06bc
Fix TrieMap: Cannot create next level map of small value entry.
...
Bug: 14425059
Change-Id: I864f81ea7dd1d847ef2b016366ce046b1c43d5e5
2014-08-20 20:48:25 +09:00
Keisuke Kuroyanagi
ace03d7919
Merge "Add BoS flag in probability entry."
2014-08-16 04:15:21 +00:00
Keisuke Kuroyanagi
623067a183
Add BoS flag in probability entry.
...
Bug: 14425059
Change-Id: I50439630034ada0280c44cbbb308aa0b95b72048
2014-08-19 11:49:05 +09:00
Keisuke Kuroyanagi
bfcd5efd50
Merge "Use byte array view in ver4 dict contents."
2014-08-16 04:15:21 +00:00
Keisuke Kuroyanagi
1f6e52ef02
Use byte array view in ver4 dict contents.
...
Change-Id: Icf79a51a200f7ccd775264d1a83dd61e7dcfbab2
2014-08-18 22:46:10 +09:00