Commit graph

867 commits

Author SHA1 Message Date
Keisuke Kuroyanagi
e3d57ae792 Merge "Use bitset for BloomFilter." 2014-04-09 07:39:49 +00:00
Keisuke Kuroyanagi
2fd0bf9a37 Use bitset for BloomFilter.
Before:
(0)  660.00 (1.43%)
(1)  45320.00 (98.18%)
(2)  80.00 (0.17%)
Total 46160.00 (sum of others 46060.00)

After:
(0)  620.00 (1.34%)
(1)  45310.00 (98.05%)
(2)  130.00 (0.28%)
Total 46210.00 (sum of others 46060.00)

Change-Id: I936b639c50e15208aee999a929b33983c6caa59d
2014-04-09 16:18:23 +09:00
Yohei Yukawa
b417d7d69f Make NELEMS type-safe
Change-Id: I867271665b3c595dc40ac63e93e68c87dc213d08
2014-04-09 15:51:39 +09:00
Keisuke Kuroyanagi
e41b2ed8d3 Separate unigram/bigram property from WordProperty.
Bug: 13406708

Change-Id: I48e9fccedd9dcdc1a35ffe027745b58966a83315
2014-04-08 18:24:06 +09:00
Ken Wakasa
cafab169cd s/stdint.h/cstdint/
Somehow, we were not able to use cstdint gcc target build. That's no
longer the case with Clang.

Removed unnecessary header inclusions too.

Change-Id: Ic83a4adf696f1d5ec7a9809253f3c95804e622e1
2014-04-07 12:39:07 +09:00
Ken Wakasa
694114feca Remove redundant LOCAL_IS_HOST_MODULE assignments
Also, remove irrelevant LOCAL_NDK_STL_VARIANT for host modules.

Change-Id: I45e916c11f7c1f9b13956f01274a82ee7de1f558
2014-04-04 22:32:45 +09:00
Keisuke Kuroyanagi
3bf9647a26 Separate NormalDistribution from ProximityInfoUtils.
Bug: 13773693
Change-Id: I9e835cc92e16f8ae0e0492af0715d0ef5cd84a35
2014-04-04 18:49:12 +09:00
Keisuke Kuroyanagi
4df668d197 Merge "Remove unused code." 2014-04-03 05:20:20 +00:00
Keisuke Kuroyanagi
5f6a247744 Remove unused code.
Bug: 13773693
Change-Id: Ic9d7f81b0eed40e945b04a91fd97ba40fa1cd262
2014-04-03 14:09:25 +09:00
Ken Wakasa
ed2781cf91 Exclude liblatinime_host_unittests from Mac build
Change-Id: I786c0ad92ba7ce95182e617686c6466a8b2f400e
2014-04-03 13:07:10 +09:00
Keisuke Kuroyanagi
de2b312c6d Use same language weight for all dictionaries.
Bug: 8187060
Change-Id: Ib9d8a8aed2c141137c1bb3c748a89fb8216293e7
2014-04-03 01:43:36 +09:00
Keisuke Kuroyanagi
35314af0d8 Merge "Extend jni interface to input/output languageWeight." 2014-04-02 16:20:28 +00:00
Keisuke Kuroyanagi
1de95ceada Extend jni interface to input/output languageWeight.
Bug: 8187060
Change-Id: Ic55d328b87f258f4d65fff207233789a53b2b0ff
2014-04-03 01:01:51 +09:00
Yohei Yukawa
c73b46f40a Do not call exit 1 when loaded with source command
BUG: 13754552
Change-Id: I3d5e4c0786547521325405100c175dd1b21754df
2014-04-02 19:47:08 +09:00
Yohei Yukawa
8855306cf9 Add a script to run native tests
This script saves your time to remember how to build and run
native unittests for LatinIME.

BUG: 13754552
Change-Id: I98398648f0eb0a0100a8729df26f71d53b54c5e3
2014-04-02 16:28:34 +09:00
Yohei Yukawa
711ed47d24 Merge "Add an initial native test on host environment" 2014-04-02 07:11:08 +00:00
Yohei Yukawa
c72652cb00 Add an initial native test on host environment
This CL adds an initial native test that runs on host
environment, from which we can extend the coverage of native
test.

BUG: 13754552
Change-Id: I0af9d1b5dd049c83b955f5b05add4b39f3cfcc6c
2014-04-02 14:23:53 +09:00
Keisuke Kuroyanagi
77de3a4b65 Change wrongly used "lookahead" to "completion".
Change-Id: I111cf59d0d24fa869217788c314fffa94ee9f9f9
2014-04-02 12:44:41 +09:00
Keisuke Kuroyanagi
543fea98a4 Merge "Parameter tuning for typing." 2014-04-01 09:45:17 +00:00
Keisuke Kuroyanagi
7dc7a9a398 Parameter tuning for typing.
accidently -> accidentally (top suggest)
convienent -> convenient (top suggest)
dissapear -> disappear (top suggest)

Execution time: +31%

Before:
(0)  610.00 (1.66%)
(1)  35820.00 (97.66%)
(2)  130.00 (0.35%)
Total 36680.00 (sum of others 36560.00)

After:
(0)  820.00 (1.69%)
(1)  47200.00 (97.50%)
(2)  210.00 (0.43%)
Total 48410.00 (sum of others 48230.00)

Bug: 13588483
Bug: 13456384
Bug: 9816611

Change-Id: Ia17491e135a63aaca6e980c3a59bd829441dac40
2014-04-01 17:25:43 +09:00
Ken Wakasa
f6c29aa52b Remove -std=c++11 as it is enabled by default now
Change-Id: I53edb2f3d281d78c0301474d65e60f86cc6ff363
2014-04-01 17:07:00 +09:00
Keisuke Kuroyanagi
da93bc54b2 Merge "Remove useless methods from DicNode." 2014-04-01 05:40:29 +00:00
Keisuke Kuroyanagi
94c7bd471f Merge "Separate DicNodePool from DicNodePriorityQueue." 2014-04-01 05:36:35 +00:00
Keisuke Kuroyanagi
ef1d4ddc8f Remove useless methods from DicNode.
Change-Id: I6ea5eb7e848dbce0f0d468160c16e5da76b23b20
2014-04-01 14:09:02 +09:00
Keisuke Kuroyanagi
67ff21f321 Separate DicNodePool from DicNodePriorityQueue.
Change-Id: Ie500a4185792c2ffa0f5185ea5919626fe31c6fe
2014-04-01 14:08:59 +09:00
Keisuke Kuroyanagi
ff1b3947c6 Split SuggestionsOutputUtils::outputSuggestions.
Bug: 13333066
Change-Id: Ie5e513dacdc5502e7263ddf709824bbd6bc6b74a
2014-03-28 20:11:27 +09:00
Keisuke Kuroyanagi
a0ea92f76c Remove mPartiallyCommited.
Bug: 13333066

Change-Id: I4e3aa31aa501866c23cc4b86093b866306161bfd
2014-03-28 20:08:51 +09:00
Keisuke Kuroyanagi
655b65cb0b Cleanup DicNodePriorityQueue.
Change-Id: Ia44ca0e14f8aad090ae2b8007771dea6d9b41221
2014-03-28 17:19:53 +09:00
Keisuke Kuroyanagi
144b9a6336 Remove DicNode.managedDelete.
Change-Id: I8146975d72878ba34d1c5054a7cf5fdcc41a3332
2014-03-28 17:19:28 +09:00
Keisuke Kuroyanagi
d73edf23ac Use SuggestionResults to get suggestion.
Bug: 8187060
Bug: 13333066
Change-Id: I435096ecf8422453f9b167adb0ca3b9c8a840018
2014-03-27 20:05:33 +09:00
Keisuke Kuroyanagi
2426bc2943 Merge "Remove DicNodeUtils::appendTwoWords()." 2014-03-27 09:49:50 +00:00
Keisuke Kuroyanagi
a83b013eec Remove DicNodeUtils::appendTwoWords().
Change-Id: I7402793b3f71ef5d786abea2f97bd4b624e4405f
2014-03-27 17:54:14 +09:00
Keisuke Kuroyanagi
b4a983d400 Merge "Remove DicNode.isFirstWord()." 2014-03-27 08:17:05 +00:00
Ken Wakasa
63b48311b0 RegisterNatives() returns 0 on success.
cf. http://docs.oracle.com/javase/6/docs/technotes/guides/jni/spec/functions.html#wp5833

Change-Id: I2c599f2d8765a7475c7076e6b049f77ad79feb79
2014-03-26 20:47:14 +09:00
Keisuke Kuroyanagi
90f6411899 Remove DicNode.isFirstWord().
Change-Id: I825c0f7cf1fcf66c6309e0bab8e0fdd767170107
2014-03-25 18:07:14 +09:00
Keisuke Kuroyanagi
eddbb7ac88 Merge DicNodeStatePrevWord into DicNoteStateOutput.
Before:
(0)  2232.70 (0.86%)
(1)  255258.50 (98.89%)
(2)  585.73 (0.23%)
(66)  0.26 (0.00%)
Total 258126.46 (sum of others 258077.18)

After:
(0)  2249.23 (0.93%)
(1)  239883.63 (98.83%)
(2)  554.82 (0.23%)
(66)  0.35 (0.00%)
Total 242734.38 (sum of others 242688.04)

Change-Id: I9760cae5b98b3d1f4804b6b60317887eaa3ff71c
2014-03-25 18:07:09 +09:00
Keisuke Kuroyanagi
cf88cf6593 Add a check for ForgettingCurveUtils::decodeProbability.
Bug: 13607383
Change-Id: I1a56c52173d0e458864e65fb1c1f4603c0c81c6f
2014-03-24 11:11:07 +00:00
Keisuke Kuroyanagi
ef665816d0 Add boundary check for SparseTable.
Bug: 13592288
Change-Id: I8bc4ff778c6073d22f86b6050a880b6fb69de31e
2014-03-24 09:17:53 +00:00
Keisuke Kuroyanagi
4c93770430 Fix: wrong size checking in SparseTable.
Bug: 13592288
Change-Id: I52f03081c5abf25e3c160e2b8fff5e98f8448499
2014-03-24 14:28:36 +09:00
Keisuke Kuroyanagi
7c20f140e0 Merge "Remove unused partial commit." 2014-03-20 09:54:15 +00:00
Keisuke Kuroyanagi
5a0381f7d0 Remove unused partial commit.
Change-Id: Ia9341a5cb24947d93056810890d94c1f66481d8c
2014-03-20 18:47:45 +09:00
Keisuke Kuroyanagi
18ba146b72 Stop apply Completion for DicNodes without any proximity chars.
Bug: 13538127
Change-Id: Ifffc0cde1ea8a0ceb13e4e4e57c151b3ea33a522
2014-03-20 18:06:06 +09:00
Ken Wakasa
46b6f60551 Switch to Clang and libc++
Change-Id: Id425997af58967e49094ab593ed8f0b70c72bbbc
2014-03-18 12:09:10 +09:00
Ken Wakasa
622589ab43 Several cleanups
Remove obsolete comment and unused macros. Cosmetic fixes as well.

Change-Id: I9843331c8ce90d9b10b54f425ea1c7416b57c541
2014-03-17 16:33:37 +09:00
Keisuke Kuroyanagi
73a2426d45 Change default decaying parameters.
Bug: 13458617
Change-Id: I4942af4a48881fb6ca866c74140b262e6d323f4a
2014-03-14 12:23:24 +09:00
Keisuke Kuroyanagi
4bc35daaa2 Check "." and ".." to delete all files in a dir.
Bug: 13421999
Change-Id: I9d0f65cfc9a0f18c7affc152a5b54f9970048e9c
2014-03-12 15:31:49 +09:00
Keisuke Kuroyanagi
5dc17c22ca Fix: wrong suggestion comparator.
Change-Id: I8829c7cb3060955e345fbdc003cf77f8469d37c0
2014-03-11 11:57:56 +00:00
Keisuke Kuroyanagi
34ab038414 Merge "Stop new BigramDictionary." 2014-03-11 09:38:14 +00:00
Keisuke Kuroyanagi
e7ddf49d50 Merge "Use emplace_back in getWordProperty()." 2014-03-11 09:38:00 +00:00
Keisuke Kuroyanagi
ed72a768d6 Merge "Make getWordProperty return reconstructed bigram probability." 2014-03-11 09:37:49 +00:00
Keisuke Kuroyanagi
ca42ec0f44 Use emplace_back in getWordProperty().
Change-Id: If164d2b897bd6ad9c89baf0e9a90356100c5087b
2014-03-11 15:21:55 +09:00
Keisuke Kuroyanagi
a34bdc395b Make getWordProperty return reconstructed bigram probability.
Bug: 13403530
Change-Id: I7ed95a54344d8b77b88922093cf3ef0e530fcae6
2014-03-11 15:20:12 +09:00
Keisuke Kuroyanagi
b43ea5c335 Use isFirstCharUppercase instead of get first code point.
Change-Id: I0a28305397bfc1e7218e16df00839781a74f5599
2014-03-10 18:39:45 +09:00
Keisuke Kuroyanagi
0a5efa31de Add getter methods for public member variables.
Change-Id: Ic6656bb3db6db992b37923db53816e0a73a7621f
2014-03-10 18:39:45 +09:00
Keisuke Kuroyanagi
6066069f35 Move initialization logic for from DicNode to DicNodeState.
Change-Id: Ic7075349e2e9c7f4ca707efa12ad1d24b49a2c87
2014-03-10 18:39:43 +09:00
Keisuke Kuroyanagi
258260bef2 Remove unused member of DicNodeStatePrevWord.
Change-Id: I265855da58e3a2e070e525e116ac0d18d1a10d18
2014-03-10 18:39:15 +09:00
Keisuke Kuroyanagi
632c9aca5b Improve DicNode handling.
- Stop creating useless DicNode and DicNodeVector.
- Remove useless virtual.
- Implement copy constructor and assignment operator of DicNodeState.
- Remove useless memset.

Before:
(0)  2266.21 (0.79%)
(1)  285422.05 (98.97%)
(2)  642.62 (0.22%)
(66)  0.19 (0.00%)
Total 288384.35 (sum of others 288331.07)

After:
(0)  2232.70 (0.86%)
(1)  255258.50 (98.89%)
(2)  585.73 (0.23%)
(66)  0.26 (0.00%)
Total 258126.46 (sum of others 258077.18)


Change-Id: I0bb1e9de8b38a6743a11aaeb2b17bd0da5b7ad34
2014-03-10 17:57:53 +09:00
Keisuke Kuroyanagi
87db47d175 Use emplace_back to push DicNode to vector.
Before:
(0)  2234.43 (0.71%)
(1)  310083.71 (99.07%)
(2)  626.88 (0.20%)
(66)  0.26 (0.00%)
Total 312995.33 (sum of others 312945.28)

After
(0)  2266.21 (0.79%)
(1)  285422.05 (98.97%)
(2)  642.62 (0.22%)
(66)  0.19 (0.00%)
Total 288384.35 (sum of others 288331.07)


Change-Id: I5ab98076c54d147ff7ec2a29280859f4558e24d0
2014-03-10 15:41:33 +09:00
Keisuke Kuroyanagi
66cc9dec55 Stop new BigramDictionary.
BigramDictionary is allocated inside of Dictionary.

Change-Id: If224b4c408403f43eb3d2e292c0e0ecb86429290
2014-03-10 05:42:36 +00:00
Keisuke Kuroyanagi
e137ec0a91 Introduce SuggestionResults and use it for predictions.
Bug: 8187060
Bug: 13333066
Change-Id: I1ead897024508b7e40fbd93af7d14bfe74b93826
2014-03-07 19:36:19 +09:00
Keisuke Kuroyanagi
c36c97b4f7 Merge "Use unique_ptr." 2014-03-07 09:06:58 +00:00
Ken Wakasa
6767f88e3c Address warnings pointed out by clang
Change-Id: I7943f6c7f9ba08f283a3116adc2af29495f04a21
2014-03-07 17:05:09 +09:00
Keisuke Kuroyanagi
4ce480d5ce Use unique_ptr.
Change-Id: Id92a5b07da4f7f95e2cd293ce8dc1a5f979b7853
2014-03-07 14:31:54 +09:00
Keisuke Kuroyanagi
865e6cf497 Revert "Revert "Use std::min() and std::max()""
This reverts commit f63000abea.

Change-Id: I6056d176a13c6ecad38e658ab8778bd898291d02
2014-03-07 14:19:56 +09:00
Keisuke Kuroyanagi
a1e0ef4836 Fix: point to line segment distance can be NaN.
Change-Id: I7db6bfea925a0eda82bfb1f7b5e1110b193dabdb
2014-03-07 14:19:52 +09:00
Keisuke Kuroyanagi
3339224813 Merge "Use nullptr instead of 0 for pointers." 2014-03-07 03:56:56 +00:00
Ken Wakasa
f63000abea Revert "Use std::min() and std::max()"
This somehow breaks TRT.

bug: 13350753

This reverts commit cf612a3abf.

Change-Id: I812f067e7cc8106b054527732dc6fe4efd7cc0fe
2014-03-07 02:30:18 +00:00
Keisuke Kuroyanagi
6c7ebc4023 Use nullptr instead of 0 for pointers.
Change-Id: Ic9425de07b896f940ca11c51968c637ebb897514
2014-03-07 10:04:44 +09:00
Ken Wakasa
cf612a3abf Use std::min() and std::max()
Change-Id: I2992fa16692ace2a6febedc4393812faf763638f
2014-03-06 23:19:54 +09:00
Yohei Yukawa
72c362985c Use C++11 deleted functions to implement DISALLOW_ macros
With C++ deleted functions, compilers can generate more
descriptive error messages when we are doing something disallowed.

Change-Id: I48d23e5eced545732558e56f5ff9ff70c95a6ff0
2014-03-06 20:48:43 +09:00
Ken Wakasa
ca0a0da864 Switch to C++11
Change-Id: Ib2970327c36422d7b11103d52ba7e9e52db5554f
2014-03-06 20:32:20 +09:00
Keisuke Kuroyanagi
9fd50e34a8 Fix: Bug of I39e905b6ddfc8d3.
primaryKey have to be the first elemnet of the proximities.

Bug: 13315755
Change-Id: I6c94408a2ef2750fa85ff12b93bed8b8d874717e
2014-03-06 18:25:15 +09:00
Keisuke Kuroyanagi
2c7e97dcdd Merge "Create suggest/core/result directory." 2014-03-06 07:32:19 +00:00
Keisuke Kuroyanagi
fb1588dd3c Create suggest/core/result directory.
Bug: 8187060
Bug: 13333066
Change-Id: I185c93bd4542d997be101b61561a00d69111c00d
2014-03-06 14:35:47 +09:00
Keisuke Kuroyanagi
a84f7848d8 Merge "Fix: The top left key is a proximity of nonexistent keys." 2014-03-05 10:27:27 +00:00
Keisuke Kuroyanagi
6501255111 Fix: The top left key is a proximity of nonexistent keys.
Bug: 13315755
Change-Id: I39e905b6ddfc8d3510c408a3bea853390a75d088
2014-03-05 19:06:18 +09:00
Keisuke Kuroyanagi
e784148ae6 Separate utility methods from BinaryDictionary.
Bug: 8187060
Change-Id: Ice2984e332b7bd3bb17174aefc80b5635b72fc50
2014-03-05 18:19:34 +09:00
Keisuke Kuroyanagi
9e51c6c0c6 Separate utility methods from BinaryDictionary in native code.
Bug: 8187060
Change-Id: I099de557d2aaee95c6b809450108139550e9b0aa
2014-03-05 12:49:04 +09:00
Keisuke Kuroyanagi
5128935ac4 Add header attributes for evaluation.
Bug: 13197276
Change-Id: Ib5247da691ff24a73e13485288237ccc51bb54f0
2014-02-28 21:06:03 +09:00
Keisuke Kuroyanagi
3d70932857 Use MAX_PROBABILITY for the limit of probabilities.
Bug: 13197276
Change-Id: Ic3f855e566603b395395dff1f860f3d76ecb019c
2014-02-26 20:48:53 +09:00
Keisuke Kuroyanagi
7919a594fc Merge "Create more aggressive probability tables." 2014-02-26 11:18:44 +00:00
Keisuke Kuroyanagi
b368089dbf Stop dividing unigram probability by 2 for backoff.
Bug: 13197276
Change-Id: I8eaa24b6f710383a8d257e3ec28c37a1a1da8e31
2014-02-26 19:40:18 +09:00
Keisuke Kuroyanagi
70566266be Create more aggressive probability tables.
Bug: 13197276

Change-Id: I1451dcbe61088d7301bbef5ceedc72f5649e6764
2014-02-26 19:36:57 +09:00
Keisuke Kuroyanagi
2b27e06733 Merge "Read and write forgetting curve header attributes." 2014-02-26 09:06:56 +00:00
Keisuke Kuroyanagi
6112cc26a7 Read and write forgetting curve header attributes.
Bug: 13197276
Change-Id: I6c38a127e1745903594b7bf0ec80ef179e794feb
2014-02-26 18:04:12 +09:00
Keisuke Kuroyanagi
94080a37e8 Merge "Check header attributes for forgetting curve." 2014-02-26 08:58:48 +00:00
Keisuke Kuroyanagi
57816c7a8b Check header attributes for forgetting curve.
Bug: 13197276
Change-Id: Ib360010f309fd1ed3e81cf23aa10c1cfe82781a6
2014-02-26 17:41:31 +09:00
Keisuke Kuroyanagi
7b496a5861 Add boundary check for ver4 bigram/shortcut reading.
Bug: 13185316
Change-Id: I5236222e2a5acda69821062a140c9e36a3024bd2
2014-02-26 14:03:24 +09:00
Keisuke Kuroyanagi
33e38096ad Fix: -1 can be used as a code point of most probable string.
Bug: 11797518
Change-Id: Id2e8bce92d000f39d506af7cb844447696bcc019
2014-02-24 20:40:16 +09:00
Keisuke Kuroyanagi
d934fb43ad Remove corrupted ver4 dictionaries.
Bug: 12916055
Change-Id: I2c390ab1dc483915691b47a605772cbc2dfeaf09
2014-02-21 15:09:37 +09:00
Keisuke Kuroyanagi
be81b75dec Add boundary check for ver2 dict reading.
Bug: 12916055
Change-Id: I78ad1f98a5401f920dcfc3379aa431eb2311ae02
2014-02-21 13:26:01 +09:00
Keisuke Kuroyanagi
b96012acef Record reading error during traversing dictionaries.
Bug: 12916055
Change-Id: I439f21aa87cfa1f71e0354370a1514af03d68f6c
2014-02-20 19:12:32 +09:00
Keisuke Kuroyanagi
303e767728 Fix: crash when opening a broken dictionary.
Bug: 13085169
Change-Id: Icfb6184dfefc6a336432203c071d9e30ae8bf990
2014-02-19 22:11:09 +09:00
Keisuke Kuroyanagi
f2a6f2db3e Merge "Fix: using wrong method to call ArrayList.add." 2014-02-19 02:27:33 +00:00
Keisuke Kuroyanagi
14fe931362 Fix: using wrong method to call ArrayList.add.
bug: 13072808
Change-Id: I60dc5ae1e535f2f0b3784a12c40ac082560a436f
2014-02-19 11:24:39 +09:00
Ken Wakasa
7e79814aa3 Fix wrong Java method call function
bug: 13072808

Change-Id: I189fda85b59ce40e3657bfbbcc313e7e1225a0da
2014-02-18 23:40:24 +09:00
Jean Chalard
2046bd6b85 Merge "Make dotted i the base character of dotless i." 2014-02-17 09:44:17 +00:00
Jean Chalard
220dc2d7a8 Make dotted i the base character of dotless i.
This is already the case for the capital version, and natives
Turkish speakers seem to think it should be the case. An
important reason to make it so is that the Turkish keyboard
has only the dotless i as a "more keys" key, so it's really
hard to type words with a dotless i without this, especially
those with several of them.

Bug: 11519119
Change-Id: I4df2a289783e4eb3c1291b8ad0a2ba5947bf7671
2014-02-17 18:24:28 +09:00
Keisuke Kuroyanagi
0fc93fe445 Implement PatriciaTriePolicy::getNextWordAndNextToken().
Bug: 12810574
Change-Id: Id1d44f90de9455d9cbe7b6e0a161cae91d6d422c
2014-02-15 17:39:20 +09:00
Keisuke Kuroyanagi
8fa7a09f1e Merge "Implement PatriciaTriePolicy::getWordProperty()." 2014-02-14 09:08:09 +00:00