Allow common redis and leveldb connections (#12385)
* Allow common redis and leveldb connections Prevents multiple reopening of redis and leveldb connections to the same place by sharing connections. Further allows for more configurable redis connection type using the redisURI and a leveldbURI scheme. Signed-off-by: Andrew Thornton <art27@cantab.net> * add unit-test Signed-off-by: Andrew Thornton <art27@cantab.net> * as per @lunny Signed-off-by: Andrew Thornton <art27@cantab.net> * add test Signed-off-by: Andrew Thornton <art27@cantab.net> * Update modules/cache/cache_redis.go * Update modules/queue/queue_disk.go * Update modules/cache/cache_redis.go * Update modules/cache/cache_redis.go * Update modules/queue/unique_queue_disk.go * Update modules/queue/queue_disk.go * Update modules/queue/unique_queue_disk.go * Update modules/session/redis.go Co-authored-by: techknowlogick <techknowlogick@gitea.io> Co-authored-by: Lauris BH <lauris@nix.lv>release/v1.15
parent
f404bdde9b
commit
7f8e3192cd
|
@ -467,8 +467,10 @@ LENGTH = 20
|
||||||
BATCH_LENGTH = 20
|
BATCH_LENGTH = 20
|
||||||
; Connection string for redis queues this will store the redis connection string.
|
; Connection string for redis queues this will store the redis connection string.
|
||||||
CONN_STR = "addrs=127.0.0.1:6379 db=0"
|
CONN_STR = "addrs=127.0.0.1:6379 db=0"
|
||||||
; Provide the suffix of the default redis queue name - specific queues can be overriden within in their [queue.name] sections.
|
; Provides the suffix of the default redis/disk queue name - specific queues can be overriden within in their [queue.name] sections.
|
||||||
QUEUE_NAME = "_queue"
|
QUEUE_NAME = "_queue"
|
||||||
|
; Provides the suffix of the default redis/disk unique queue set name - specific queues can be overriden within in their [queue.name] sections.
|
||||||
|
SET_NAME = "_unique"
|
||||||
; If the queue cannot be created at startup - level queues may need a timeout at startup - wrap the queue:
|
; If the queue cannot be created at startup - level queues may need a timeout at startup - wrap the queue:
|
||||||
WRAP_IF_NECESSARY = true
|
WRAP_IF_NECESSARY = true
|
||||||
; Attempt to create the wrapped queue at max
|
; Attempt to create the wrapped queue at max
|
||||||
|
|
|
@ -308,15 +308,13 @@ relation to port exhaustion.
|
||||||
## Queue (`queue` and `queue.*`)
|
## Queue (`queue` and `queue.*`)
|
||||||
|
|
||||||
- `TYPE`: **persistable-channel**: General queue type, currently support: `persistable-channel`, `channel`, `level`, `redis`, `dummy`
|
- `TYPE`: **persistable-channel**: General queue type, currently support: `persistable-channel`, `channel`, `level`, `redis`, `dummy`
|
||||||
- `DATADIR`: **queues/**: Base DataDir for storing persistent and level queues. `DATADIR` for inidividual queues can be set in `queue.name` sections but will default to `DATADIR/`**`name`**.
|
- `DATADIR`: **queues/**: Base DataDir for storing persistent and level queues. `DATADIR` for individual queues can be set in `queue.name` sections but will default to `DATADIR/`**`name`**.
|
||||||
- `LENGTH`: **20**: Maximal queue size before channel queues block
|
- `LENGTH`: **20**: Maximal queue size before channel queues block
|
||||||
- `BATCH_LENGTH`: **20**: Batch data before passing to the handler
|
- `BATCH_LENGTH`: **20**: Batch data before passing to the handler
|
||||||
- `CONN_STR`: **addrs=127.0.0.1:6379 db=0**: Connection string for the redis queue type.
|
- `CONN_STR`: **redis://127.0.0.1:6379/0**: Connection string for the redis queue type. Options can be set using query params. Similarly LevelDB options can also be set using: **leveldb://relative/path?option=value** or **leveldb:///absolute/path?option=value**
|
||||||
- `QUEUE_NAME`: **_queue**: The suffix for default redis queue name. Individual queues will default to **`name`**`QUEUE_NAME` but can be overriden in the specific `queue.name` section.
|
- `QUEUE_NAME`: **_queue**: The suffix for default redis and disk queue name. Individual queues will default to **`name`**`QUEUE_NAME` but can be overriden in the specific `queue.name` section.
|
||||||
- `SET_NAME`: **_unique**: The suffix that will added to the default redis
|
- `SET_NAME`: **_unique**: The suffix that will be added to the default redis and disk queue `set` name for unique queues. Individual queues will default to
|
||||||
set name for unique queues. Individual queues will default to
|
**`name`**`QUEUE_NAME`_`SET_NAME`_ but can be overridden in the specific `queue.name` section.
|
||||||
**`name`**`QUEUE_NAME`_`SET_NAME`_ but can be overridden in the specific
|
|
||||||
`queue.name` section.
|
|
||||||
- `WRAP_IF_NECESSARY`: **true**: Will wrap queues with a timeoutable queue if the selected queue is not ready to be created - (Only relevant for the level queue.)
|
- `WRAP_IF_NECESSARY`: **true**: Will wrap queues with a timeoutable queue if the selected queue is not ready to be created - (Only relevant for the level queue.)
|
||||||
- `MAX_ATTEMPTS`: **10**: Maximum number of attempts to create the wrapped queue
|
- `MAX_ATTEMPTS`: **10**: Maximum number of attempts to create the wrapped queue
|
||||||
- `TIMEOUT`: **GRACEFUL_HAMMER_TIME + 30s**: Timeout the creation of the wrapped queue if it takes longer than this to create.
|
- `TIMEOUT`: **GRACEFUL_HAMMER_TIME + 30s**: Timeout the creation of the wrapped queue if it takes longer than this to create.
|
||||||
|
@ -459,7 +457,7 @@ set name for unique queues. Individual queues will default to
|
||||||
- `ADAPTER`: **memory**: Cache engine adapter, either `memory`, `redis`, or `memcache`.
|
- `ADAPTER`: **memory**: Cache engine adapter, either `memory`, `redis`, or `memcache`.
|
||||||
- `INTERVAL`: **60**: Garbage Collection interval (sec), for memory cache only.
|
- `INTERVAL`: **60**: Garbage Collection interval (sec), for memory cache only.
|
||||||
- `HOST`: **\<empty\>**: Connection string for `redis` and `memcache`.
|
- `HOST`: **\<empty\>**: Connection string for `redis` and `memcache`.
|
||||||
- Redis: `network=tcp,addr=127.0.0.1:6379,password=macaron,db=0,pool_size=100,idle_timeout=180`
|
- Redis: `redis://:macaron@127.0.0.1:6379/0?pool_size=100&idle_timeout=180s`
|
||||||
- Memcache: `127.0.0.1:9090;127.0.0.1:9091`
|
- Memcache: `127.0.0.1:9090;127.0.0.1:9091`
|
||||||
- `ITEM_TTL`: **16h**: Time to keep items in cache if not used, Setting it to 0 disables caching.
|
- `ITEM_TTL`: **16h**: Time to keep items in cache if not used, Setting it to 0 disables caching.
|
||||||
|
|
||||||
|
@ -708,7 +706,7 @@ Task queue configuration has been moved to `queue.task`. However, the below conf
|
||||||
|
|
||||||
- `QUEUE_TYPE`: **channel**: Task queue type, could be `channel` or `redis`.
|
- `QUEUE_TYPE`: **channel**: Task queue type, could be `channel` or `redis`.
|
||||||
- `QUEUE_LENGTH`: **1000**: Task queue length, available only when `QUEUE_TYPE` is `channel`.
|
- `QUEUE_LENGTH`: **1000**: Task queue length, available only when `QUEUE_TYPE` is `channel`.
|
||||||
- `QUEUE_CONN_STR`: **addrs=127.0.0.1:6379 db=0**: Task queue connection string, available only when `QUEUE_TYPE` is `redis`. If redis needs a password, use `addrs=127.0.0.1:6379 password=123 db=0`.
|
- `QUEUE_CONN_STR`: **redis://127.0.0.1:6379/0**: Task queue connection string, available only when `QUEUE_TYPE` is `redis`. If redis needs a password, use `redis://123@127.0.0.1:6379/0`.
|
||||||
|
|
||||||
## Migrations (`migrations`)
|
## Migrations (`migrations`)
|
||||||
|
|
||||||
|
|
3
go.mod
3
go.mod
|
@ -38,7 +38,7 @@ require (
|
||||||
github.com/go-enry/go-enry/v2 v2.5.2
|
github.com/go-enry/go-enry/v2 v2.5.2
|
||||||
github.com/go-git/go-billy/v5 v5.0.0
|
github.com/go-git/go-billy/v5 v5.0.0
|
||||||
github.com/go-git/go-git/v5 v5.1.0
|
github.com/go-git/go-git/v5 v5.1.0
|
||||||
github.com/go-redis/redis v6.15.2+incompatible
|
github.com/go-redis/redis/v7 v7.4.0
|
||||||
github.com/go-sql-driver/mysql v1.5.0
|
github.com/go-sql-driver/mysql v1.5.0
|
||||||
github.com/go-swagger/go-swagger v0.25.0
|
github.com/go-swagger/go-swagger v0.25.0
|
||||||
github.com/go-testfixtures/testfixtures/v3 v3.4.0
|
github.com/go-testfixtures/testfixtures/v3 v3.4.0
|
||||||
|
@ -88,6 +88,7 @@ require (
|
||||||
github.com/shurcooL/httpfs v0.0.0-20190527155220-6a4d4a70508b // indirect
|
github.com/shurcooL/httpfs v0.0.0-20190527155220-6a4d4a70508b // indirect
|
||||||
github.com/shurcooL/vfsgen v0.0.0-20181202132449-6a9ea43bcacd
|
github.com/shurcooL/vfsgen v0.0.0-20181202132449-6a9ea43bcacd
|
||||||
github.com/stretchr/testify v1.6.1
|
github.com/stretchr/testify v1.6.1
|
||||||
|
github.com/syndtr/goleveldb v1.0.0
|
||||||
github.com/tecbot/gorocksdb v0.0.0-20181010114359-8752a9433481 // indirect
|
github.com/tecbot/gorocksdb v0.0.0-20181010114359-8752a9433481 // indirect
|
||||||
github.com/tinylib/msgp v1.1.2 // indirect
|
github.com/tinylib/msgp v1.1.2 // indirect
|
||||||
github.com/tstranex/u2f v1.0.0
|
github.com/tstranex/u2f v1.0.0
|
||||||
|
|
7
go.sum
7
go.sum
|
@ -342,6 +342,8 @@ github.com/go-openapi/validate v0.19.10 h1:tG3SZ5DC5KF4cyt7nqLVcQXGj5A7mpaYkAcNP
|
||||||
github.com/go-openapi/validate v0.19.10/go.mod h1:RKEZTUWDkxKQxN2jDT7ZnZi2bhZlbNMAuKvKB+IaGx8=
|
github.com/go-openapi/validate v0.19.10/go.mod h1:RKEZTUWDkxKQxN2jDT7ZnZi2bhZlbNMAuKvKB+IaGx8=
|
||||||
github.com/go-redis/redis v6.15.2+incompatible h1:9SpNVG76gr6InJGxoZ6IuuxaCOQwDAhzyXg+Bs+0Sb4=
|
github.com/go-redis/redis v6.15.2+incompatible h1:9SpNVG76gr6InJGxoZ6IuuxaCOQwDAhzyXg+Bs+0Sb4=
|
||||||
github.com/go-redis/redis v6.15.2+incompatible/go.mod h1:NAIEuMOZ/fxfXJIrKDQDz8wamY7mA7PouImQ2Jvg6kA=
|
github.com/go-redis/redis v6.15.2+incompatible/go.mod h1:NAIEuMOZ/fxfXJIrKDQDz8wamY7mA7PouImQ2Jvg6kA=
|
||||||
|
github.com/go-redis/redis/v7 v7.4.0 h1:7obg6wUoj05T0EpY0o8B59S9w5yeMWql7sw2kwNW1x4=
|
||||||
|
github.com/go-redis/redis/v7 v7.4.0/go.mod h1:JDNMw23GTyLNC4GZu9njt15ctBQVn7xjRfnwdHj/Dcg=
|
||||||
github.com/go-sql-driver/mysql v1.4.1 h1:g24URVg0OFbNUTx9qqY1IRZ9D9z3iPyi5zKhQZpNwpA=
|
github.com/go-sql-driver/mysql v1.4.1 h1:g24URVg0OFbNUTx9qqY1IRZ9D9z3iPyi5zKhQZpNwpA=
|
||||||
github.com/go-sql-driver/mysql v1.4.1/go.mod h1:zAC/RDZ24gD3HViQzih4MyKcchzm+sOG5ZlKdlhCg5w=
|
github.com/go-sql-driver/mysql v1.4.1/go.mod h1:zAC/RDZ24gD3HViQzih4MyKcchzm+sOG5ZlKdlhCg5w=
|
||||||
github.com/go-sql-driver/mysql v1.5.0 h1:ozyZYNQW3x3HtqT1jira07DN2PArx2v7/mN66gGcHOs=
|
github.com/go-sql-driver/mysql v1.5.0 h1:ozyZYNQW3x3HtqT1jira07DN2PArx2v7/mN66gGcHOs=
|
||||||
|
@ -730,9 +732,13 @@ github.com/onsi/ginkgo v1.6.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+W
|
||||||
github.com/onsi/ginkgo v1.7.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE=
|
github.com/onsi/ginkgo v1.7.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE=
|
||||||
github.com/onsi/ginkgo v1.8.0 h1:VkHVNpR4iVnU8XQR6DBm8BqYjN7CRzw+xKUbVVbbW9w=
|
github.com/onsi/ginkgo v1.8.0 h1:VkHVNpR4iVnU8XQR6DBm8BqYjN7CRzw+xKUbVVbbW9w=
|
||||||
github.com/onsi/ginkgo v1.8.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE=
|
github.com/onsi/ginkgo v1.8.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE=
|
||||||
|
github.com/onsi/ginkgo v1.10.1 h1:q/mM8GF/n0shIN8SaAZ0V+jnLPzen6WIVZdiwrRlMlo=
|
||||||
|
github.com/onsi/ginkgo v1.10.1/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE=
|
||||||
github.com/onsi/gomega v1.4.3/go.mod h1:ex+gbHU/CVuBBDIJjb2X0qEXbFg53c61hWP/1CpauHY=
|
github.com/onsi/gomega v1.4.3/go.mod h1:ex+gbHU/CVuBBDIJjb2X0qEXbFg53c61hWP/1CpauHY=
|
||||||
github.com/onsi/gomega v1.5.0 h1:izbySO9zDPmjJ8rDjLvkA2zJHIo+HkYXHnf7eN7SSyo=
|
github.com/onsi/gomega v1.5.0 h1:izbySO9zDPmjJ8rDjLvkA2zJHIo+HkYXHnf7eN7SSyo=
|
||||||
github.com/onsi/gomega v1.5.0/go.mod h1:ex+gbHU/CVuBBDIJjb2X0qEXbFg53c61hWP/1CpauHY=
|
github.com/onsi/gomega v1.5.0/go.mod h1:ex+gbHU/CVuBBDIJjb2X0qEXbFg53c61hWP/1CpauHY=
|
||||||
|
github.com/onsi/gomega v1.7.0 h1:XPnZz8VVBHjVsy1vzJmRwIcSwiUO+JFfrv/xGiigmME=
|
||||||
|
github.com/onsi/gomega v1.7.0/go.mod h1:ex+gbHU/CVuBBDIJjb2X0qEXbFg53c61hWP/1CpauHY=
|
||||||
github.com/opentracing/opentracing-go v1.1.0/go.mod h1:UkNAQd3GIcIGf0SeVgPpRdFStlNbqXla1AfSYxPUl2o=
|
github.com/opentracing/opentracing-go v1.1.0/go.mod h1:UkNAQd3GIcIGf0SeVgPpRdFStlNbqXla1AfSYxPUl2o=
|
||||||
github.com/openzipkin/zipkin-go v0.1.6/go.mod h1:QgAqvLzwWbR/WpD4A3cGpPtJrZXNIiJc5AZX7/PBEpw=
|
github.com/openzipkin/zipkin-go v0.1.6/go.mod h1:QgAqvLzwWbR/WpD4A3cGpPtJrZXNIiJc5AZX7/PBEpw=
|
||||||
github.com/pascaldekloe/goe v0.0.0-20180627143212-57f6aae5913c/go.mod h1:lzWF7FIEvWOWxwDKqyGYQf6ZUaNfKdP144TG7ZOy1lc=
|
github.com/pascaldekloe/goe v0.0.0-20180627143212-57f6aae5913c/go.mod h1:lzWF7FIEvWOWxwDKqyGYQf6ZUaNfKdP144TG7ZOy1lc=
|
||||||
|
@ -1014,6 +1020,7 @@ golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLL
|
||||||
golang.org/x/net v0.0.0-20190724013045-ca1201d0de80/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
|
golang.org/x/net v0.0.0-20190724013045-ca1201d0de80/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
|
||||||
golang.org/x/net v0.0.0-20190813141303-74dc4d7220e7/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
|
golang.org/x/net v0.0.0-20190813141303-74dc4d7220e7/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
|
||||||
golang.org/x/net v0.0.0-20190827160401-ba9fcec4b297/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
|
golang.org/x/net v0.0.0-20190827160401-ba9fcec4b297/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
|
||||||
|
golang.org/x/net v0.0.0-20190923162816-aa69164e4478/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
|
||||||
golang.org/x/net v0.0.0-20200202094626-16171245cfb2/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
|
golang.org/x/net v0.0.0-20200202094626-16171245cfb2/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
|
||||||
golang.org/x/net v0.0.0-20200226121028-0de0cce0169b/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
|
golang.org/x/net v0.0.0-20200226121028-0de0cce0169b/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
|
||||||
golang.org/x/net v0.0.0-20200301022130-244492dfa37a/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
|
golang.org/x/net v0.0.0-20200301022130-244492dfa37a/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
|
||||||
|
|
|
@ -13,7 +13,6 @@ import (
|
||||||
mc "gitea.com/macaron/cache"
|
mc "gitea.com/macaron/cache"
|
||||||
|
|
||||||
_ "gitea.com/macaron/cache/memcache" // memcache plugin for cache
|
_ "gitea.com/macaron/cache/memcache" // memcache plugin for cache
|
||||||
_ "gitea.com/macaron/cache/redis"
|
|
||||||
)
|
)
|
||||||
|
|
||||||
var (
|
var (
|
||||||
|
|
|
@ -1,35 +1,23 @@
|
||||||
// Copyright 2013 Beego Authors
|
// Copyright 2020 The Gitea Authors. All rights reserved.
|
||||||
// Copyright 2014 The Macaron Authors
|
// Use of this source code is governed by a MIT-style
|
||||||
//
|
// license that can be found in the LICENSE file.
|
||||||
// Licensed under the Apache License, Version 2.0 (the "License"): you may
|
|
||||||
// not use this file except in compliance with the License. You may obtain
|
|
||||||
// a copy of the License at
|
|
||||||
//
|
|
||||||
// http://www.apache.org/licenses/LICENSE-2.0
|
|
||||||
//
|
|
||||||
// Unless required by applicable law or agreed to in writing, software
|
|
||||||
// distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
|
||||||
// WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
|
||||||
// License for the specific language governing permissions and limitations
|
|
||||||
// under the License.
|
|
||||||
|
|
||||||
package cache
|
package cache
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"fmt"
|
"fmt"
|
||||||
"strings"
|
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
"github.com/go-redis/redis"
|
"code.gitea.io/gitea/modules/nosql"
|
||||||
"github.com/unknwon/com"
|
|
||||||
"gopkg.in/ini.v1"
|
|
||||||
|
|
||||||
"gitea.com/macaron/cache"
|
"gitea.com/macaron/cache"
|
||||||
|
"github.com/go-redis/redis/v7"
|
||||||
|
"github.com/unknwon/com"
|
||||||
)
|
)
|
||||||
|
|
||||||
// RedisCacher represents a redis cache adapter implementation.
|
// RedisCacher represents a redis cache adapter implementation.
|
||||||
type RedisCacher struct {
|
type RedisCacher struct {
|
||||||
c *redis.Client
|
c redis.UniversalClient
|
||||||
prefix string
|
prefix string
|
||||||
hsetName string
|
hsetName string
|
||||||
occupyMode bool
|
occupyMode bool
|
||||||
|
@ -112,7 +100,7 @@ func (c *RedisCacher) IsExist(key string) bool {
|
||||||
// Flush deletes all cached data.
|
// Flush deletes all cached data.
|
||||||
func (c *RedisCacher) Flush() error {
|
func (c *RedisCacher) Flush() error {
|
||||||
if c.occupyMode {
|
if c.occupyMode {
|
||||||
return c.c.FlushDb().Err()
|
return c.c.FlushDB().Err()
|
||||||
}
|
}
|
||||||
|
|
||||||
keys, err := c.c.HKeys(c.hsetName).Result()
|
keys, err := c.c.HKeys(c.hsetName).Result()
|
||||||
|
@ -131,46 +119,20 @@ func (c *RedisCacher) StartAndGC(opts cache.Options) error {
|
||||||
c.hsetName = "MacaronCache"
|
c.hsetName = "MacaronCache"
|
||||||
c.occupyMode = opts.OccupyMode
|
c.occupyMode = opts.OccupyMode
|
||||||
|
|
||||||
cfg, err := ini.Load([]byte(strings.Replace(opts.AdapterConfig, ",", "\n", -1)))
|
uri := nosql.ToRedisURI(opts.AdapterConfig)
|
||||||
if err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
|
|
||||||
opt := &redis.Options{
|
c.c = nosql.GetManager().GetRedisClient(uri.String())
|
||||||
Network: "tcp",
|
|
||||||
}
|
for k, v := range uri.Query() {
|
||||||
for k, v := range cfg.Section("").KeysHash() {
|
|
||||||
switch k {
|
switch k {
|
||||||
case "network":
|
|
||||||
opt.Network = v
|
|
||||||
case "addr":
|
|
||||||
opt.Addr = v
|
|
||||||
case "password":
|
|
||||||
opt.Password = v
|
|
||||||
case "db":
|
|
||||||
opt.DB = com.StrTo(v).MustInt()
|
|
||||||
case "pool_size":
|
|
||||||
opt.PoolSize = com.StrTo(v).MustInt()
|
|
||||||
case "idle_timeout":
|
|
||||||
opt.IdleTimeout, err = time.ParseDuration(v + "s")
|
|
||||||
if err != nil {
|
|
||||||
return fmt.Errorf("error parsing idle timeout: %v", err)
|
|
||||||
}
|
|
||||||
case "hset_name":
|
case "hset_name":
|
||||||
c.hsetName = v
|
c.hsetName = v[0]
|
||||||
case "prefix":
|
case "prefix":
|
||||||
c.prefix = v
|
c.prefix = v[0]
|
||||||
default:
|
|
||||||
return fmt.Errorf("session/redis: unsupported option '%s'", k)
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
c.c = redis.NewClient(opt)
|
return c.c.Ping().Err()
|
||||||
if err = c.c.Ping().Err(); err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
|
|
||||||
return nil
|
|
||||||
}
|
}
|
||||||
|
|
||||||
func init() {
|
func init() {
|
|
@ -0,0 +1,25 @@
|
||||||
|
// Copyright 2020 The Gitea Authors. All rights reserved.
|
||||||
|
// Use of this source code is governed by a MIT-style
|
||||||
|
// license that can be found in the LICENSE file.
|
||||||
|
|
||||||
|
package nosql
|
||||||
|
|
||||||
|
import "net/url"
|
||||||
|
|
||||||
|
// ToLevelDBURI converts old style connections to a LevelDBURI
|
||||||
|
//
|
||||||
|
// A LevelDBURI matches the pattern:
|
||||||
|
//
|
||||||
|
// leveldb://path[?[option=value]*]
|
||||||
|
//
|
||||||
|
// We have previously just provided the path but this prevent other options
|
||||||
|
func ToLevelDBURI(connection string) *url.URL {
|
||||||
|
uri, err := url.Parse(connection)
|
||||||
|
if err == nil && uri.Scheme == "leveldb" {
|
||||||
|
return uri
|
||||||
|
}
|
||||||
|
uri, _ = url.Parse("leveldb://common")
|
||||||
|
uri.Host = ""
|
||||||
|
uri.Path = connection
|
||||||
|
return uri
|
||||||
|
}
|
|
@ -0,0 +1,71 @@
|
||||||
|
// Copyright 2020 The Gitea Authors. All rights reserved.
|
||||||
|
// Use of this source code is governed by a MIT-style
|
||||||
|
// license that can be found in the LICENSE file.
|
||||||
|
|
||||||
|
package nosql
|
||||||
|
|
||||||
|
import (
|
||||||
|
"strconv"
|
||||||
|
"sync"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/go-redis/redis/v7"
|
||||||
|
"github.com/syndtr/goleveldb/leveldb"
|
||||||
|
)
|
||||||
|
|
||||||
|
var manager *Manager
|
||||||
|
|
||||||
|
// Manager is the nosql connection manager
|
||||||
|
type Manager struct {
|
||||||
|
mutex sync.Mutex
|
||||||
|
|
||||||
|
RedisConnections map[string]*redisClientHolder
|
||||||
|
LevelDBConnections map[string]*levelDBHolder
|
||||||
|
}
|
||||||
|
|
||||||
|
type redisClientHolder struct {
|
||||||
|
redis.UniversalClient
|
||||||
|
name []string
|
||||||
|
count int64
|
||||||
|
}
|
||||||
|
|
||||||
|
func (r *redisClientHolder) Close() error {
|
||||||
|
return manager.CloseRedisClient(r.name[0])
|
||||||
|
}
|
||||||
|
|
||||||
|
type levelDBHolder struct {
|
||||||
|
name []string
|
||||||
|
count int64
|
||||||
|
db *leveldb.DB
|
||||||
|
}
|
||||||
|
|
||||||
|
func init() {
|
||||||
|
_ = GetManager()
|
||||||
|
}
|
||||||
|
|
||||||
|
// GetManager returns a Manager and initializes one as singleton is there's none yet
|
||||||
|
func GetManager() *Manager {
|
||||||
|
if manager == nil {
|
||||||
|
manager = &Manager{
|
||||||
|
RedisConnections: make(map[string]*redisClientHolder),
|
||||||
|
LevelDBConnections: make(map[string]*levelDBHolder),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return manager
|
||||||
|
}
|
||||||
|
|
||||||
|
func valToTimeDuration(vs []string) (result time.Duration) {
|
||||||
|
var err error
|
||||||
|
for _, v := range vs {
|
||||||
|
result, err = time.ParseDuration(v)
|
||||||
|
if err != nil {
|
||||||
|
var val int
|
||||||
|
val, err = strconv.Atoi(v)
|
||||||
|
result = time.Duration(val)
|
||||||
|
}
|
||||||
|
if err == nil {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return
|
||||||
|
}
|
|
@ -0,0 +1,151 @@
|
||||||
|
// Copyright 2020 The Gitea Authors. All rights reserved.
|
||||||
|
// Use of this source code is governed by a MIT-style
|
||||||
|
// license that can be found in the LICENSE file.
|
||||||
|
|
||||||
|
package nosql
|
||||||
|
|
||||||
|
import (
|
||||||
|
"path"
|
||||||
|
"strconv"
|
||||||
|
"strings"
|
||||||
|
|
||||||
|
"github.com/syndtr/goleveldb/leveldb"
|
||||||
|
"github.com/syndtr/goleveldb/leveldb/errors"
|
||||||
|
"github.com/syndtr/goleveldb/leveldb/opt"
|
||||||
|
)
|
||||||
|
|
||||||
|
// CloseLevelDB closes a levelDB
|
||||||
|
func (m *Manager) CloseLevelDB(connection string) error {
|
||||||
|
m.mutex.Lock()
|
||||||
|
defer m.mutex.Unlock()
|
||||||
|
db, ok := m.LevelDBConnections[connection]
|
||||||
|
if !ok {
|
||||||
|
connection = ToLevelDBURI(connection).String()
|
||||||
|
db, ok = m.LevelDBConnections[connection]
|
||||||
|
}
|
||||||
|
if !ok {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
db.count--
|
||||||
|
if db.count > 0 {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, name := range db.name {
|
||||||
|
delete(m.LevelDBConnections, name)
|
||||||
|
}
|
||||||
|
return db.db.Close()
|
||||||
|
}
|
||||||
|
|
||||||
|
// GetLevelDB gets a levelDB for a particular connection
|
||||||
|
func (m *Manager) GetLevelDB(connection string) (*leveldb.DB, error) {
|
||||||
|
m.mutex.Lock()
|
||||||
|
defer m.mutex.Unlock()
|
||||||
|
db, ok := m.LevelDBConnections[connection]
|
||||||
|
if ok {
|
||||||
|
db.count++
|
||||||
|
|
||||||
|
return db.db, nil
|
||||||
|
}
|
||||||
|
dataDir := connection
|
||||||
|
uri := ToLevelDBURI(connection)
|
||||||
|
db = &levelDBHolder{
|
||||||
|
name: []string{connection, uri.String()},
|
||||||
|
}
|
||||||
|
|
||||||
|
dataDir = path.Join(uri.Host, uri.Path)
|
||||||
|
opts := &opt.Options{}
|
||||||
|
for k, v := range uri.Query() {
|
||||||
|
switch replacer.Replace(strings.ToLower(k)) {
|
||||||
|
case "blockcachecapacity":
|
||||||
|
opts.BlockCacheCapacity, _ = strconv.Atoi(v[0])
|
||||||
|
case "blockcacheevictremoved":
|
||||||
|
opts.BlockCacheEvictRemoved, _ = strconv.ParseBool(v[0])
|
||||||
|
case "blockrestartinterval":
|
||||||
|
opts.BlockRestartInterval, _ = strconv.Atoi(v[0])
|
||||||
|
case "blocksize":
|
||||||
|
opts.BlockSize, _ = strconv.Atoi(v[0])
|
||||||
|
case "compactionexpandlimitfactor":
|
||||||
|
opts.CompactionExpandLimitFactor, _ = strconv.Atoi(v[0])
|
||||||
|
case "compactiongpoverlapsfactor":
|
||||||
|
opts.CompactionGPOverlapsFactor, _ = strconv.Atoi(v[0])
|
||||||
|
case "compactionl0trigger":
|
||||||
|
opts.CompactionL0Trigger, _ = strconv.Atoi(v[0])
|
||||||
|
case "compactionsourcelimitfactor":
|
||||||
|
opts.CompactionSourceLimitFactor, _ = strconv.Atoi(v[0])
|
||||||
|
case "compactiontablesize":
|
||||||
|
opts.CompactionTableSize, _ = strconv.Atoi(v[0])
|
||||||
|
case "compactiontablesizemultiplier":
|
||||||
|
opts.CompactionTableSizeMultiplier, _ = strconv.ParseFloat(v[0], 64)
|
||||||
|
case "compactiontablesizemultiplierperlevel":
|
||||||
|
for _, val := range v {
|
||||||
|
f, _ := strconv.ParseFloat(val, 64)
|
||||||
|
opts.CompactionTableSizeMultiplierPerLevel = append(opts.CompactionTableSizeMultiplierPerLevel, f)
|
||||||
|
}
|
||||||
|
case "compactiontotalsize":
|
||||||
|
opts.CompactionTotalSize, _ = strconv.Atoi(v[0])
|
||||||
|
case "compactiontotalsizemultiplier":
|
||||||
|
opts.CompactionTotalSizeMultiplier, _ = strconv.ParseFloat(v[0], 64)
|
||||||
|
case "compactiontotalsizemultiplierperlevel":
|
||||||
|
for _, val := range v {
|
||||||
|
f, _ := strconv.ParseFloat(val, 64)
|
||||||
|
opts.CompactionTotalSizeMultiplierPerLevel = append(opts.CompactionTotalSizeMultiplierPerLevel, f)
|
||||||
|
}
|
||||||
|
case "compression":
|
||||||
|
val, _ := strconv.Atoi(v[0])
|
||||||
|
opts.Compression = opt.Compression(val)
|
||||||
|
case "disablebufferpool":
|
||||||
|
opts.DisableBufferPool, _ = strconv.ParseBool(v[0])
|
||||||
|
case "disableblockcache":
|
||||||
|
opts.DisableBlockCache, _ = strconv.ParseBool(v[0])
|
||||||
|
case "disablecompactionbackoff":
|
||||||
|
opts.DisableCompactionBackoff, _ = strconv.ParseBool(v[0])
|
||||||
|
case "disablelargebatchtransaction":
|
||||||
|
opts.DisableLargeBatchTransaction, _ = strconv.ParseBool(v[0])
|
||||||
|
case "errorifexist":
|
||||||
|
opts.ErrorIfExist, _ = strconv.ParseBool(v[0])
|
||||||
|
case "errorifmissing":
|
||||||
|
opts.ErrorIfMissing, _ = strconv.ParseBool(v[0])
|
||||||
|
case "iteratorsamplingrate":
|
||||||
|
opts.IteratorSamplingRate, _ = strconv.Atoi(v[0])
|
||||||
|
case "nosync":
|
||||||
|
opts.NoSync, _ = strconv.ParseBool(v[0])
|
||||||
|
case "nowritemerge":
|
||||||
|
opts.NoWriteMerge, _ = strconv.ParseBool(v[0])
|
||||||
|
case "openfilescachecapacity":
|
||||||
|
opts.OpenFilesCacheCapacity, _ = strconv.Atoi(v[0])
|
||||||
|
case "readonly":
|
||||||
|
opts.ReadOnly, _ = strconv.ParseBool(v[0])
|
||||||
|
case "strict":
|
||||||
|
val, _ := strconv.Atoi(v[0])
|
||||||
|
opts.Strict = opt.Strict(val)
|
||||||
|
case "writebuffer":
|
||||||
|
opts.WriteBuffer, _ = strconv.Atoi(v[0])
|
||||||
|
case "writel0pausetrigger":
|
||||||
|
opts.WriteL0PauseTrigger, _ = strconv.Atoi(v[0])
|
||||||
|
case "writel0slowdowntrigger":
|
||||||
|
opts.WriteL0SlowdownTrigger, _ = strconv.Atoi(v[0])
|
||||||
|
case "clientname":
|
||||||
|
db.name = append(db.name, v[0])
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
var err error
|
||||||
|
db.db, err = leveldb.OpenFile(dataDir, opts)
|
||||||
|
if err != nil {
|
||||||
|
if !errors.IsCorrupted(err) {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
db.db, err = leveldb.RecoverFile(dataDir, opts)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, name := range db.name {
|
||||||
|
m.LevelDBConnections[name] = db
|
||||||
|
}
|
||||||
|
db.count++
|
||||||
|
return db.db, nil
|
||||||
|
}
|
|
@ -0,0 +1,205 @@
|
||||||
|
// Copyright 2020 The Gitea Authors. All rights reserved.
|
||||||
|
// Use of this source code is governed by a MIT-style
|
||||||
|
// license that can be found in the LICENSE file.
|
||||||
|
|
||||||
|
package nosql
|
||||||
|
|
||||||
|
import (
|
||||||
|
"crypto/tls"
|
||||||
|
"path"
|
||||||
|
"strconv"
|
||||||
|
"strings"
|
||||||
|
|
||||||
|
"github.com/go-redis/redis/v7"
|
||||||
|
)
|
||||||
|
|
||||||
|
var replacer = strings.NewReplacer("_", "", "-", "")
|
||||||
|
|
||||||
|
// CloseRedisClient closes a redis client
|
||||||
|
func (m *Manager) CloseRedisClient(connection string) error {
|
||||||
|
m.mutex.Lock()
|
||||||
|
defer m.mutex.Unlock()
|
||||||
|
client, ok := m.RedisConnections[connection]
|
||||||
|
if !ok {
|
||||||
|
connection = ToRedisURI(connection).String()
|
||||||
|
client, ok = m.RedisConnections[connection]
|
||||||
|
}
|
||||||
|
if !ok {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
client.count--
|
||||||
|
if client.count > 0 {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, name := range client.name {
|
||||||
|
delete(m.RedisConnections, name)
|
||||||
|
}
|
||||||
|
return client.UniversalClient.Close()
|
||||||
|
}
|
||||||
|
|
||||||
|
// GetRedisClient gets a redis client for a particular connection
|
||||||
|
func (m *Manager) GetRedisClient(connection string) redis.UniversalClient {
|
||||||
|
m.mutex.Lock()
|
||||||
|
defer m.mutex.Unlock()
|
||||||
|
client, ok := m.RedisConnections[connection]
|
||||||
|
if ok {
|
||||||
|
client.count++
|
||||||
|
return client
|
||||||
|
}
|
||||||
|
|
||||||
|
uri := ToRedisURI(connection)
|
||||||
|
client, ok = m.RedisConnections[uri.String()]
|
||||||
|
if ok {
|
||||||
|
client.count++
|
||||||
|
return client
|
||||||
|
}
|
||||||
|
client = &redisClientHolder{
|
||||||
|
name: []string{connection, uri.String()},
|
||||||
|
}
|
||||||
|
|
||||||
|
opts := &redis.UniversalOptions{}
|
||||||
|
tlsConfig := &tls.Config{}
|
||||||
|
|
||||||
|
// Handle username/password
|
||||||
|
if password, ok := uri.User.Password(); ok {
|
||||||
|
opts.Password = password
|
||||||
|
// Username does not appear to be handled by redis.Options
|
||||||
|
opts.Username = uri.User.Username()
|
||||||
|
} else if uri.User.Username() != "" {
|
||||||
|
// assume this is the password
|
||||||
|
opts.Password = uri.User.Username()
|
||||||
|
}
|
||||||
|
|
||||||
|
// Now handle the uri query sets
|
||||||
|
for k, v := range uri.Query() {
|
||||||
|
switch replacer.Replace(strings.ToLower(k)) {
|
||||||
|
case "addr":
|
||||||
|
opts.Addrs = append(opts.Addrs, v...)
|
||||||
|
case "addrs":
|
||||||
|
opts.Addrs = append(opts.Addrs, strings.Split(v[0], ",")...)
|
||||||
|
case "username":
|
||||||
|
opts.Username = v[0]
|
||||||
|
case "password":
|
||||||
|
opts.Password = v[0]
|
||||||
|
case "database":
|
||||||
|
fallthrough
|
||||||
|
case "db":
|
||||||
|
opts.DB, _ = strconv.Atoi(v[0])
|
||||||
|
case "maxretries":
|
||||||
|
opts.MaxRetries, _ = strconv.Atoi(v[0])
|
||||||
|
case "minretrybackoff":
|
||||||
|
opts.MinRetryBackoff = valToTimeDuration(v)
|
||||||
|
case "maxretrybackoff":
|
||||||
|
opts.MaxRetryBackoff = valToTimeDuration(v)
|
||||||
|
case "timeout":
|
||||||
|
timeout := valToTimeDuration(v)
|
||||||
|
if timeout != 0 {
|
||||||
|
if opts.DialTimeout == 0 {
|
||||||
|
opts.DialTimeout = timeout
|
||||||
|
}
|
||||||
|
if opts.ReadTimeout == 0 {
|
||||||
|
opts.ReadTimeout = timeout
|
||||||
|
}
|
||||||
|
}
|
||||||
|
case "dialtimeout":
|
||||||
|
opts.DialTimeout = valToTimeDuration(v)
|
||||||
|
case "readtimeout":
|
||||||
|
opts.ReadTimeout = valToTimeDuration(v)
|
||||||
|
case "writetimeout":
|
||||||
|
opts.WriteTimeout = valToTimeDuration(v)
|
||||||
|
case "poolsize":
|
||||||
|
opts.PoolSize, _ = strconv.Atoi(v[0])
|
||||||
|
case "minidleconns":
|
||||||
|
opts.MinIdleConns, _ = strconv.Atoi(v[0])
|
||||||
|
case "pooltimeout":
|
||||||
|
opts.PoolTimeout = valToTimeDuration(v)
|
||||||
|
case "idletimeout":
|
||||||
|
opts.IdleTimeout = valToTimeDuration(v)
|
||||||
|
case "idlecheckfrequency":
|
||||||
|
opts.IdleCheckFrequency = valToTimeDuration(v)
|
||||||
|
case "maxredirects":
|
||||||
|
opts.MaxRedirects, _ = strconv.Atoi(v[0])
|
||||||
|
case "readonly":
|
||||||
|
opts.ReadOnly, _ = strconv.ParseBool(v[0])
|
||||||
|
case "routebylatency":
|
||||||
|
opts.RouteByLatency, _ = strconv.ParseBool(v[0])
|
||||||
|
case "routerandomly":
|
||||||
|
opts.RouteRandomly, _ = strconv.ParseBool(v[0])
|
||||||
|
case "sentinelmasterid":
|
||||||
|
fallthrough
|
||||||
|
case "mastername":
|
||||||
|
opts.MasterName = v[0]
|
||||||
|
case "skipverify":
|
||||||
|
fallthrough
|
||||||
|
case "insecureskipverify":
|
||||||
|
insecureSkipVerify, _ := strconv.ParseBool(v[0])
|
||||||
|
tlsConfig.InsecureSkipVerify = insecureSkipVerify
|
||||||
|
case "clientname":
|
||||||
|
client.name = append(client.name, v[0])
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
switch uri.Scheme {
|
||||||
|
case "redis+sentinels":
|
||||||
|
fallthrough
|
||||||
|
case "rediss+sentinel":
|
||||||
|
opts.TLSConfig = tlsConfig
|
||||||
|
fallthrough
|
||||||
|
case "redis+sentinel":
|
||||||
|
if uri.Host != "" {
|
||||||
|
opts.Addrs = append(opts.Addrs, strings.Split(uri.Host, ",")...)
|
||||||
|
}
|
||||||
|
if uri.Path != "" {
|
||||||
|
if db, err := strconv.Atoi(uri.Path); err == nil {
|
||||||
|
opts.DB = db
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
client.UniversalClient = redis.NewFailoverClient(opts.Failover())
|
||||||
|
case "redis+clusters":
|
||||||
|
fallthrough
|
||||||
|
case "rediss+cluster":
|
||||||
|
opts.TLSConfig = tlsConfig
|
||||||
|
fallthrough
|
||||||
|
case "redis+cluster":
|
||||||
|
if uri.Host != "" {
|
||||||
|
opts.Addrs = append(opts.Addrs, strings.Split(uri.Host, ",")...)
|
||||||
|
}
|
||||||
|
if uri.Path != "" {
|
||||||
|
if db, err := strconv.Atoi(uri.Path); err == nil {
|
||||||
|
opts.DB = db
|
||||||
|
}
|
||||||
|
}
|
||||||
|
client.UniversalClient = redis.NewClusterClient(opts.Cluster())
|
||||||
|
case "redis+socket":
|
||||||
|
simpleOpts := opts.Simple()
|
||||||
|
simpleOpts.Network = "unix"
|
||||||
|
simpleOpts.Addr = path.Join(uri.Host, uri.Path)
|
||||||
|
client.UniversalClient = redis.NewClient(simpleOpts)
|
||||||
|
case "rediss":
|
||||||
|
opts.TLSConfig = tlsConfig
|
||||||
|
fallthrough
|
||||||
|
case "redis":
|
||||||
|
if uri.Host != "" {
|
||||||
|
opts.Addrs = append(opts.Addrs, strings.Split(uri.Host, ",")...)
|
||||||
|
}
|
||||||
|
if uri.Path != "" {
|
||||||
|
if db, err := strconv.Atoi(uri.Path); err == nil {
|
||||||
|
opts.DB = db
|
||||||
|
}
|
||||||
|
}
|
||||||
|
client.UniversalClient = redis.NewClient(opts.Simple())
|
||||||
|
default:
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, name := range client.name {
|
||||||
|
m.RedisConnections[name] = client
|
||||||
|
}
|
||||||
|
|
||||||
|
client.count++
|
||||||
|
|
||||||
|
return client
|
||||||
|
}
|
|
@ -0,0 +1,102 @@
|
||||||
|
// Copyright 2020 The Gitea Authors. All rights reserved.
|
||||||
|
// Use of this source code is governed by a MIT-style
|
||||||
|
// license that can be found in the LICENSE file.
|
||||||
|
|
||||||
|
package nosql
|
||||||
|
|
||||||
|
import (
|
||||||
|
"net/url"
|
||||||
|
"strconv"
|
||||||
|
"strings"
|
||||||
|
)
|
||||||
|
|
||||||
|
// The file contains common redis connection functions
|
||||||
|
|
||||||
|
// ToRedisURI converts old style connections to a RedisURI
|
||||||
|
//
|
||||||
|
// A RedisURI matches the pattern:
|
||||||
|
//
|
||||||
|
// redis://[username:password@]host[:port][/database][?[option=value]*]
|
||||||
|
// rediss://[username:password@]host[:port][/database][?[option=value]*]
|
||||||
|
// redis+socket://[username:password@]path[/database][?[option=value]*]
|
||||||
|
// redis+sentinel://[password@]host1 [: port1][, host2 [:port2]][, hostN [:portN]][/ database][?[option=value]*]
|
||||||
|
// redis+cluster://[password@]host1 [: port1][, host2 [:port2]][, hostN [:portN]][/ database][?[option=value]*]
|
||||||
|
//
|
||||||
|
// We have previously used a URI like:
|
||||||
|
// addrs=127.0.0.1:6379 db=0
|
||||||
|
// network=tcp,addr=127.0.0.1:6379,password=macaron,db=0,pool_size=100,idle_timeout=180
|
||||||
|
//
|
||||||
|
// We need to convert this old style to the new style
|
||||||
|
func ToRedisURI(connection string) *url.URL {
|
||||||
|
uri, err := url.Parse(connection)
|
||||||
|
if err == nil && strings.HasPrefix(uri.Scheme, "redis") {
|
||||||
|
// OK we're going to assume that this is a reasonable redis URI
|
||||||
|
return uri
|
||||||
|
}
|
||||||
|
|
||||||
|
// Let's set a nice default
|
||||||
|
uri, _ = url.Parse("redis://127.0.0.1:6379/0")
|
||||||
|
network := "tcp"
|
||||||
|
query := uri.Query()
|
||||||
|
|
||||||
|
// OK so there are two types: Space delimited and Comma delimited
|
||||||
|
// Let's assume that we have a space delimited string - as this is the most common
|
||||||
|
fields := strings.Fields(connection)
|
||||||
|
if len(fields) == 1 {
|
||||||
|
// It's a comma delimited string, then...
|
||||||
|
fields = strings.Split(connection, ",")
|
||||||
|
|
||||||
|
}
|
||||||
|
for _, f := range fields {
|
||||||
|
items := strings.SplitN(f, "=", 2)
|
||||||
|
if len(items) < 2 {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
switch strings.ToLower(items[0]) {
|
||||||
|
case "network":
|
||||||
|
if items[1] == "unix" {
|
||||||
|
uri.Scheme = "redis+socket"
|
||||||
|
}
|
||||||
|
network = items[1]
|
||||||
|
case "addrs":
|
||||||
|
uri.Host = items[1]
|
||||||
|
// now we need to handle the clustering
|
||||||
|
if strings.Contains(items[1], ",") && network == "tcp" {
|
||||||
|
uri.Scheme = "redis+cluster"
|
||||||
|
}
|
||||||
|
case "addr":
|
||||||
|
uri.Host = items[1]
|
||||||
|
case "password":
|
||||||
|
uri.User = url.UserPassword(uri.User.Username(), items[1])
|
||||||
|
case "username":
|
||||||
|
password, set := uri.User.Password()
|
||||||
|
if !set {
|
||||||
|
uri.User = url.User(items[1])
|
||||||
|
} else {
|
||||||
|
uri.User = url.UserPassword(items[1], password)
|
||||||
|
}
|
||||||
|
case "db":
|
||||||
|
uri.Path = "/" + items[1]
|
||||||
|
case "idle_timeout":
|
||||||
|
_, err := strconv.Atoi(items[1])
|
||||||
|
if err == nil {
|
||||||
|
query.Add("idle_timeout", items[1]+"s")
|
||||||
|
} else {
|
||||||
|
query.Add("idle_timeout", items[1])
|
||||||
|
}
|
||||||
|
default:
|
||||||
|
// Other options become query params
|
||||||
|
query.Add(items[0], items[1])
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Finally we need to fix up the Host if we have a unix port
|
||||||
|
if uri.Scheme == "redis+socket" {
|
||||||
|
query.Set("db", uri.Path)
|
||||||
|
uri.Path = uri.Host
|
||||||
|
uri.Host = ""
|
||||||
|
}
|
||||||
|
uri.RawQuery = query.Encode()
|
||||||
|
|
||||||
|
return uri
|
||||||
|
}
|
|
@ -0,0 +1,35 @@
|
||||||
|
// Copyright 2020 The Gitea Authors. All rights reserved.
|
||||||
|
// Use of this source code is governed by a MIT-style
|
||||||
|
// license that can be found in the LICENSE file.
|
||||||
|
|
||||||
|
package nosql
|
||||||
|
|
||||||
|
import (
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestToRedisURI(t *testing.T) {
|
||||||
|
tests := []struct {
|
||||||
|
name string
|
||||||
|
connection string
|
||||||
|
want string
|
||||||
|
}{
|
||||||
|
{
|
||||||
|
name: "old_default",
|
||||||
|
connection: "addrs=127.0.0.1:6379 db=0",
|
||||||
|
want: "redis://127.0.0.1:6379/0",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "old_macaron_session_default",
|
||||||
|
connection: "network=tcp,addr=127.0.0.1:6379,password=macaron,db=0,pool_size=100,idle_timeout=180",
|
||||||
|
want: "redis://:macaron@127.0.0.1:6379/0?idle_timeout=180s&pool_size=100",
|
||||||
|
},
|
||||||
|
}
|
||||||
|
for _, tt := range tests {
|
||||||
|
t.Run(tt.name, func(t *testing.T) {
|
||||||
|
if got := ToRedisURI(tt.connection); got == nil || got.String() != tt.want {
|
||||||
|
t.Errorf(`ToRedisURI(%q) = %s, want %s`, tt.connection, got.String(), tt.want)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
|
@ -5,6 +5,8 @@
|
||||||
package queue
|
package queue
|
||||||
|
|
||||||
import (
|
import (
|
||||||
|
"code.gitea.io/gitea/modules/nosql"
|
||||||
|
|
||||||
"gitea.com/lunny/levelqueue"
|
"gitea.com/lunny/levelqueue"
|
||||||
)
|
)
|
||||||
|
|
||||||
|
@ -15,6 +17,8 @@ const LevelQueueType Type = "level"
|
||||||
type LevelQueueConfiguration struct {
|
type LevelQueueConfiguration struct {
|
||||||
ByteFIFOQueueConfiguration
|
ByteFIFOQueueConfiguration
|
||||||
DataDir string
|
DataDir string
|
||||||
|
ConnectionString string
|
||||||
|
QueueName string
|
||||||
}
|
}
|
||||||
|
|
||||||
// LevelQueue implements a disk library queue
|
// LevelQueue implements a disk library queue
|
||||||
|
@ -30,7 +34,11 @@ func NewLevelQueue(handle HandlerFunc, cfg, exemplar interface{}) (Queue, error)
|
||||||
}
|
}
|
||||||
config := configInterface.(LevelQueueConfiguration)
|
config := configInterface.(LevelQueueConfiguration)
|
||||||
|
|
||||||
byteFIFO, err := NewLevelQueueByteFIFO(config.DataDir)
|
if len(config.ConnectionString) == 0 {
|
||||||
|
config.ConnectionString = config.DataDir
|
||||||
|
}
|
||||||
|
|
||||||
|
byteFIFO, err := NewLevelQueueByteFIFO(config.ConnectionString, config.QueueName)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
|
@ -52,16 +60,23 @@ var _ (ByteFIFO) = &LevelQueueByteFIFO{}
|
||||||
// LevelQueueByteFIFO represents a ByteFIFO formed from a LevelQueue
|
// LevelQueueByteFIFO represents a ByteFIFO formed from a LevelQueue
|
||||||
type LevelQueueByteFIFO struct {
|
type LevelQueueByteFIFO struct {
|
||||||
internal *levelqueue.Queue
|
internal *levelqueue.Queue
|
||||||
|
connection string
|
||||||
}
|
}
|
||||||
|
|
||||||
// NewLevelQueueByteFIFO creates a ByteFIFO formed from a LevelQueue
|
// NewLevelQueueByteFIFO creates a ByteFIFO formed from a LevelQueue
|
||||||
func NewLevelQueueByteFIFO(dataDir string) (*LevelQueueByteFIFO, error) {
|
func NewLevelQueueByteFIFO(connection, prefix string) (*LevelQueueByteFIFO, error) {
|
||||||
internal, err := levelqueue.Open(dataDir)
|
db, err := nosql.GetManager().GetLevelDB(connection)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
|
||||||
|
internal, err := levelqueue.NewQueue(db, []byte(prefix), false)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
|
|
||||||
return &LevelQueueByteFIFO{
|
return &LevelQueueByteFIFO{
|
||||||
|
connection: connection,
|
||||||
internal: internal,
|
internal: internal,
|
||||||
}, nil
|
}, nil
|
||||||
}
|
}
|
||||||
|
@ -87,7 +102,9 @@ func (fifo *LevelQueueByteFIFO) Pop() ([]byte, error) {
|
||||||
|
|
||||||
// Close this fifo
|
// Close this fifo
|
||||||
func (fifo *LevelQueueByteFIFO) Close() error {
|
func (fifo *LevelQueueByteFIFO) Close() error {
|
||||||
return fifo.internal.Close()
|
err := fifo.internal.Close()
|
||||||
|
_ = nosql.GetManager().CloseLevelDB(fifo.connection)
|
||||||
|
return err
|
||||||
}
|
}
|
||||||
|
|
||||||
// Len returns the length of the fifo
|
// Len returns the length of the fifo
|
||||||
|
|
|
@ -5,12 +5,10 @@
|
||||||
package queue
|
package queue
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"errors"
|
|
||||||
"strings"
|
|
||||||
|
|
||||||
"code.gitea.io/gitea/modules/log"
|
"code.gitea.io/gitea/modules/log"
|
||||||
|
"code.gitea.io/gitea/modules/nosql"
|
||||||
|
|
||||||
"github.com/go-redis/redis"
|
"github.com/go-redis/redis/v7"
|
||||||
)
|
)
|
||||||
|
|
||||||
// RedisQueueType is the type for redis queue
|
// RedisQueueType is the type for redis queue
|
||||||
|
@ -75,10 +73,7 @@ type RedisByteFIFO struct {
|
||||||
|
|
||||||
// RedisByteFIFOConfiguration is the configuration for the RedisByteFIFO
|
// RedisByteFIFOConfiguration is the configuration for the RedisByteFIFO
|
||||||
type RedisByteFIFOConfiguration struct {
|
type RedisByteFIFOConfiguration struct {
|
||||||
Network string
|
ConnectionString string
|
||||||
Addresses string
|
|
||||||
Password string
|
|
||||||
DBIndex int
|
|
||||||
QueueName string
|
QueueName string
|
||||||
}
|
}
|
||||||
|
|
||||||
|
@ -87,21 +82,7 @@ func NewRedisByteFIFO(config RedisByteFIFOConfiguration) (*RedisByteFIFO, error)
|
||||||
fifo := &RedisByteFIFO{
|
fifo := &RedisByteFIFO{
|
||||||
queueName: config.QueueName,
|
queueName: config.QueueName,
|
||||||
}
|
}
|
||||||
dbs := strings.Split(config.Addresses, ",")
|
fifo.client = nosql.GetManager().GetRedisClient(config.ConnectionString)
|
||||||
if len(dbs) == 0 {
|
|
||||||
return nil, errors.New("no redis host specified")
|
|
||||||
} else if len(dbs) == 1 {
|
|
||||||
fifo.client = redis.NewClient(&redis.Options{
|
|
||||||
Network: config.Network,
|
|
||||||
Addr: strings.TrimSpace(dbs[0]), // use default Addr
|
|
||||||
Password: config.Password, // no password set
|
|
||||||
DB: config.DBIndex, // use default DB
|
|
||||||
})
|
|
||||||
} else {
|
|
||||||
fifo.client = redis.NewClusterClient(&redis.ClusterOptions{
|
|
||||||
Addrs: dbs,
|
|
||||||
})
|
|
||||||
}
|
|
||||||
if err := fifo.client.Ping().Err(); err != nil {
|
if err := fifo.client.Ping().Err(); err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
|
|
|
@ -5,6 +5,8 @@
|
||||||
package queue
|
package queue
|
||||||
|
|
||||||
import (
|
import (
|
||||||
|
"code.gitea.io/gitea/modules/nosql"
|
||||||
|
|
||||||
"gitea.com/lunny/levelqueue"
|
"gitea.com/lunny/levelqueue"
|
||||||
)
|
)
|
||||||
|
|
||||||
|
@ -15,6 +17,8 @@ const LevelUniqueQueueType Type = "unique-level"
|
||||||
type LevelUniqueQueueConfiguration struct {
|
type LevelUniqueQueueConfiguration struct {
|
||||||
ByteFIFOQueueConfiguration
|
ByteFIFOQueueConfiguration
|
||||||
DataDir string
|
DataDir string
|
||||||
|
ConnectionString string
|
||||||
|
QueueName string
|
||||||
}
|
}
|
||||||
|
|
||||||
// LevelUniqueQueue implements a disk library queue
|
// LevelUniqueQueue implements a disk library queue
|
||||||
|
@ -34,7 +38,11 @@ func NewLevelUniqueQueue(handle HandlerFunc, cfg, exemplar interface{}) (Queue,
|
||||||
}
|
}
|
||||||
config := configInterface.(LevelUniqueQueueConfiguration)
|
config := configInterface.(LevelUniqueQueueConfiguration)
|
||||||
|
|
||||||
byteFIFO, err := NewLevelUniqueQueueByteFIFO(config.DataDir)
|
if len(config.ConnectionString) == 0 {
|
||||||
|
config.ConnectionString = config.DataDir
|
||||||
|
}
|
||||||
|
|
||||||
|
byteFIFO, err := NewLevelUniqueQueueByteFIFO(config.ConnectionString, config.QueueName)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
|
@ -56,16 +64,23 @@ var _ (UniqueByteFIFO) = &LevelUniqueQueueByteFIFO{}
|
||||||
// LevelUniqueQueueByteFIFO represents a ByteFIFO formed from a LevelUniqueQueue
|
// LevelUniqueQueueByteFIFO represents a ByteFIFO formed from a LevelUniqueQueue
|
||||||
type LevelUniqueQueueByteFIFO struct {
|
type LevelUniqueQueueByteFIFO struct {
|
||||||
internal *levelqueue.UniqueQueue
|
internal *levelqueue.UniqueQueue
|
||||||
|
connection string
|
||||||
}
|
}
|
||||||
|
|
||||||
// NewLevelUniqueQueueByteFIFO creates a new ByteFIFO formed from a LevelUniqueQueue
|
// NewLevelUniqueQueueByteFIFO creates a new ByteFIFO formed from a LevelUniqueQueue
|
||||||
func NewLevelUniqueQueueByteFIFO(dataDir string) (*LevelUniqueQueueByteFIFO, error) {
|
func NewLevelUniqueQueueByteFIFO(connection, prefix string) (*LevelUniqueQueueByteFIFO, error) {
|
||||||
internal, err := levelqueue.OpenUnique(dataDir)
|
db, err := nosql.GetManager().GetLevelDB(connection)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
|
||||||
|
internal, err := levelqueue.NewUniqueQueue(db, []byte(prefix), []byte(prefix+"-unique"), false)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
|
|
||||||
return &LevelUniqueQueueByteFIFO{
|
return &LevelUniqueQueueByteFIFO{
|
||||||
|
connection: connection,
|
||||||
internal: internal,
|
internal: internal,
|
||||||
}, nil
|
}, nil
|
||||||
}
|
}
|
||||||
|
@ -96,7 +111,9 @@ func (fifo *LevelUniqueQueueByteFIFO) Has(data []byte) (bool, error) {
|
||||||
|
|
||||||
// Close this fifo
|
// Close this fifo
|
||||||
func (fifo *LevelUniqueQueueByteFIFO) Close() error {
|
func (fifo *LevelUniqueQueueByteFIFO) Close() error {
|
||||||
return fifo.internal.Close()
|
err := fifo.internal.Close()
|
||||||
|
_ = nosql.GetManager().CloseLevelDB(fifo.connection)
|
||||||
|
return err
|
||||||
}
|
}
|
||||||
|
|
||||||
func init() {
|
func init() {
|
||||||
|
|
|
@ -4,7 +4,7 @@
|
||||||
|
|
||||||
package queue
|
package queue
|
||||||
|
|
||||||
import "github.com/go-redis/redis"
|
import "github.com/go-redis/redis/v7"
|
||||||
|
|
||||||
// RedisUniqueQueueType is the type for redis queue
|
// RedisUniqueQueueType is the type for redis queue
|
||||||
const RedisUniqueQueueType Type = "unique-redis"
|
const RedisUniqueQueueType Type = "unique-redis"
|
||||||
|
|
|
@ -1,5 +1,6 @@
|
||||||
// Copyright 2013 Beego Authors
|
// Copyright 2013 Beego Authors
|
||||||
// Copyright 2014 The Macaron Authors
|
// Copyright 2014 The Macaron Authors
|
||||||
|
// Copyright 2020 The Gitea Authors. All rights reserved.
|
||||||
//
|
//
|
||||||
// Licensed under the Apache License, Version 2.0 (the "License"): you may
|
// Licensed under the Apache License, Version 2.0 (the "License"): you may
|
||||||
// not use this file except in compliance with the License. You may obtain
|
// not use this file except in compliance with the License. You may obtain
|
||||||
|
@ -17,19 +18,18 @@ package session
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"fmt"
|
"fmt"
|
||||||
"strings"
|
|
||||||
"sync"
|
"sync"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
|
"code.gitea.io/gitea/modules/nosql"
|
||||||
|
|
||||||
"gitea.com/macaron/session"
|
"gitea.com/macaron/session"
|
||||||
"github.com/go-redis/redis"
|
"github.com/go-redis/redis/v7"
|
||||||
"github.com/unknwon/com"
|
|
||||||
"gopkg.in/ini.v1"
|
|
||||||
)
|
)
|
||||||
|
|
||||||
// RedisStore represents a redis session store implementation.
|
// RedisStore represents a redis session store implementation.
|
||||||
type RedisStore struct {
|
type RedisStore struct {
|
||||||
c *redis.Client
|
c redis.UniversalClient
|
||||||
prefix, sid string
|
prefix, sid string
|
||||||
duration time.Duration
|
duration time.Duration
|
||||||
lock sync.RWMutex
|
lock sync.RWMutex
|
||||||
|
@ -37,7 +37,7 @@ type RedisStore struct {
|
||||||
}
|
}
|
||||||
|
|
||||||
// NewRedisStore creates and returns a redis session store.
|
// NewRedisStore creates and returns a redis session store.
|
||||||
func NewRedisStore(c *redis.Client, prefix, sid string, dur time.Duration, kv map[interface{}]interface{}) *RedisStore {
|
func NewRedisStore(c redis.UniversalClient, prefix, sid string, dur time.Duration, kv map[interface{}]interface{}) *RedisStore {
|
||||||
return &RedisStore{
|
return &RedisStore{
|
||||||
c: c,
|
c: c,
|
||||||
prefix: prefix,
|
prefix: prefix,
|
||||||
|
@ -104,7 +104,7 @@ func (s *RedisStore) Flush() error {
|
||||||
|
|
||||||
// RedisProvider represents a redis session provider implementation.
|
// RedisProvider represents a redis session provider implementation.
|
||||||
type RedisProvider struct {
|
type RedisProvider struct {
|
||||||
c *redis.Client
|
c redis.UniversalClient
|
||||||
duration time.Duration
|
duration time.Duration
|
||||||
prefix string
|
prefix string
|
||||||
}
|
}
|
||||||
|
@ -117,39 +117,16 @@ func (p *RedisProvider) Init(maxlifetime int64, configs string) (err error) {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
|
|
||||||
cfg, err := ini.Load([]byte(strings.Replace(configs, ",", "\n", -1)))
|
uri := nosql.ToRedisURI(configs)
|
||||||
if err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
|
|
||||||
opt := &redis.Options{
|
for k, v := range uri.Query() {
|
||||||
Network: "tcp",
|
|
||||||
}
|
|
||||||
for k, v := range cfg.Section("").KeysHash() {
|
|
||||||
switch k {
|
switch k {
|
||||||
case "network":
|
|
||||||
opt.Network = v
|
|
||||||
case "addr":
|
|
||||||
opt.Addr = v
|
|
||||||
case "password":
|
|
||||||
opt.Password = v
|
|
||||||
case "db":
|
|
||||||
opt.DB = com.StrTo(v).MustInt()
|
|
||||||
case "pool_size":
|
|
||||||
opt.PoolSize = com.StrTo(v).MustInt()
|
|
||||||
case "idle_timeout":
|
|
||||||
opt.IdleTimeout, err = time.ParseDuration(v + "s")
|
|
||||||
if err != nil {
|
|
||||||
return fmt.Errorf("error parsing idle timeout: %v", err)
|
|
||||||
}
|
|
||||||
case "prefix":
|
case "prefix":
|
||||||
p.prefix = v
|
p.prefix = v[0]
|
||||||
default:
|
|
||||||
return fmt.Errorf("session/redis: unsupported option '%s'", k)
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
p.c = redis.NewClient(opt)
|
p.c = nosql.GetManager().GetRedisClient(uri.String())
|
||||||
return p.c.Ping().Err()
|
return p.c.Ping().Err()
|
||||||
}
|
}
|
||||||
|
|
||||||
|
@ -228,11 +205,11 @@ func (p *RedisProvider) Regenerate(oldsid, sid string) (_ session.RawStore, err
|
||||||
|
|
||||||
// Count counts and returns number of sessions.
|
// Count counts and returns number of sessions.
|
||||||
func (p *RedisProvider) Count() int {
|
func (p *RedisProvider) Count() int {
|
||||||
return int(p.c.DbSize().Val())
|
return int(p.c.DBSize().Val())
|
||||||
}
|
}
|
||||||
|
|
||||||
// GC calls GC to clean expired sessions.
|
// GC calls GC to clean expired sessions.
|
||||||
func (_ *RedisProvider) GC() {}
|
func (*RedisProvider) GC() {}
|
||||||
|
|
||||||
func init() {
|
func init() {
|
||||||
session.Register("redis", &RedisProvider{})
|
session.Register("redis", &RedisProvider{})
|
|
@ -15,7 +15,6 @@ import (
|
||||||
mysql "gitea.com/macaron/session/mysql"
|
mysql "gitea.com/macaron/session/mysql"
|
||||||
nodb "gitea.com/macaron/session/nodb"
|
nodb "gitea.com/macaron/session/nodb"
|
||||||
postgres "gitea.com/macaron/session/postgres"
|
postgres "gitea.com/macaron/session/postgres"
|
||||||
redis "gitea.com/macaron/session/redis"
|
|
||||||
)
|
)
|
||||||
|
|
||||||
// VirtualSessionProvider represents a shadowed session provider implementation.
|
// VirtualSessionProvider represents a shadowed session provider implementation.
|
||||||
|
@ -40,7 +39,7 @@ func (o *VirtualSessionProvider) Init(gclifetime int64, config string) error {
|
||||||
case "file":
|
case "file":
|
||||||
o.provider = &session.FileProvider{}
|
o.provider = &session.FileProvider{}
|
||||||
case "redis":
|
case "redis":
|
||||||
o.provider = &redis.RedisProvider{}
|
o.provider = &RedisProvider{}
|
||||||
case "mysql":
|
case "mysql":
|
||||||
o.provider = &mysql.MysqlProvider{}
|
o.provider = &mysql.MysqlProvider{}
|
||||||
case "postgres":
|
case "postgres":
|
||||||
|
|
|
@ -1 +0,0 @@
|
||||||
ignore
|
|
|
@ -1 +0,0 @@
|
||||||
ignore
|
|
|
@ -1,19 +0,0 @@
|
||||||
sudo: false
|
|
||||||
language: go
|
|
||||||
|
|
||||||
services:
|
|
||||||
- redis-server
|
|
||||||
|
|
||||||
go:
|
|
||||||
- 1.9.x
|
|
||||||
- 1.10.x
|
|
||||||
- 1.11.x
|
|
||||||
- tip
|
|
||||||
|
|
||||||
matrix:
|
|
||||||
allow_failures:
|
|
||||||
- go: tip
|
|
||||||
|
|
||||||
install:
|
|
||||||
- go get github.com/onsi/ginkgo
|
|
||||||
- go get github.com/onsi/gomega
|
|
|
@ -1,25 +0,0 @@
|
||||||
# Changelog
|
|
||||||
|
|
||||||
## Unreleased
|
|
||||||
|
|
||||||
- Cluster and Ring pipelines process commands for each node in its own goroutine.
|
|
||||||
|
|
||||||
## 6.14
|
|
||||||
|
|
||||||
- Added Options.MinIdleConns.
|
|
||||||
- Added Options.MaxConnAge.
|
|
||||||
- PoolStats.FreeConns is renamed to PoolStats.IdleConns.
|
|
||||||
- Add Client.Do to simplify creating custom commands.
|
|
||||||
- Add Cmd.String, Cmd.Int, Cmd.Int64, Cmd.Uint64, Cmd.Float64, and Cmd.Bool helpers.
|
|
||||||
- Lower memory usage.
|
|
||||||
|
|
||||||
## v6.13
|
|
||||||
|
|
||||||
- Ring got new options called `HashReplicas` and `Hash`. It is recommended to set `HashReplicas = 1000` for better keys distribution between shards.
|
|
||||||
- Cluster client was optimized to use much less memory when reloading cluster state.
|
|
||||||
- PubSub.ReceiveMessage is re-worked to not use ReceiveTimeout so it does not lose data when timeout occurres. In most cases it is recommended to use PubSub.Channel instead.
|
|
||||||
- Dialer.KeepAlive is set to 5 minutes by default.
|
|
||||||
|
|
||||||
## v6.12
|
|
||||||
|
|
||||||
- ClusterClient got new option called `ClusterSlots` which allows to build cluster of normal Redis Servers that don't have cluster mode enabled. See https://godoc.org/github.com/go-redis/redis#example-NewClusterClient--ManualSetup
|
|
|
@ -1,89 +0,0 @@
|
||||||
package internal
|
|
||||||
|
|
||||||
import (
|
|
||||||
"io"
|
|
||||||
"net"
|
|
||||||
"strings"
|
|
||||||
|
|
||||||
"github.com/go-redis/redis/internal/proto"
|
|
||||||
)
|
|
||||||
|
|
||||||
func IsRetryableError(err error, retryTimeout bool) bool {
|
|
||||||
if err == nil {
|
|
||||||
return false
|
|
||||||
}
|
|
||||||
if err == io.EOF {
|
|
||||||
return true
|
|
||||||
}
|
|
||||||
if netErr, ok := err.(net.Error); ok {
|
|
||||||
if netErr.Timeout() {
|
|
||||||
return retryTimeout
|
|
||||||
}
|
|
||||||
return true
|
|
||||||
}
|
|
||||||
s := err.Error()
|
|
||||||
if s == "ERR max number of clients reached" {
|
|
||||||
return true
|
|
||||||
}
|
|
||||||
if strings.HasPrefix(s, "LOADING ") {
|
|
||||||
return true
|
|
||||||
}
|
|
||||||
if strings.HasPrefix(s, "READONLY ") {
|
|
||||||
return true
|
|
||||||
}
|
|
||||||
if strings.HasPrefix(s, "CLUSTERDOWN ") {
|
|
||||||
return true
|
|
||||||
}
|
|
||||||
return false
|
|
||||||
}
|
|
||||||
|
|
||||||
func IsRedisError(err error) bool {
|
|
||||||
_, ok := err.(proto.RedisError)
|
|
||||||
return ok
|
|
||||||
}
|
|
||||||
|
|
||||||
func IsBadConn(err error, allowTimeout bool) bool {
|
|
||||||
if err == nil {
|
|
||||||
return false
|
|
||||||
}
|
|
||||||
if IsRedisError(err) {
|
|
||||||
// #790
|
|
||||||
return IsReadOnlyError(err)
|
|
||||||
}
|
|
||||||
if allowTimeout {
|
|
||||||
if netErr, ok := err.(net.Error); ok && netErr.Timeout() {
|
|
||||||
return false
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return true
|
|
||||||
}
|
|
||||||
|
|
||||||
func IsMovedError(err error) (moved bool, ask bool, addr string) {
|
|
||||||
if !IsRedisError(err) {
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
s := err.Error()
|
|
||||||
if strings.HasPrefix(s, "MOVED ") {
|
|
||||||
moved = true
|
|
||||||
} else if strings.HasPrefix(s, "ASK ") {
|
|
||||||
ask = true
|
|
||||||
} else {
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
ind := strings.LastIndex(s, " ")
|
|
||||||
if ind == -1 {
|
|
||||||
return false, false, ""
|
|
||||||
}
|
|
||||||
addr = s[ind+1:]
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
func IsLoadingError(err error) bool {
|
|
||||||
return strings.HasPrefix(err.Error(), "LOADING ")
|
|
||||||
}
|
|
||||||
|
|
||||||
func IsReadOnlyError(err error) bool {
|
|
||||||
return strings.HasPrefix(err.Error(), "READONLY ")
|
|
||||||
}
|
|
|
@ -1,15 +0,0 @@
|
||||||
package internal
|
|
||||||
|
|
||||||
import (
|
|
||||||
"fmt"
|
|
||||||
"log"
|
|
||||||
)
|
|
||||||
|
|
||||||
var Logger *log.Logger
|
|
||||||
|
|
||||||
func Logf(s string, args ...interface{}) {
|
|
||||||
if Logger == nil {
|
|
||||||
return
|
|
||||||
}
|
|
||||||
Logger.Output(2, fmt.Sprintf(s, args...))
|
|
||||||
}
|
|
|
@ -1,93 +0,0 @@
|
||||||
package pool
|
|
||||||
|
|
||||||
import (
|
|
||||||
"net"
|
|
||||||
"sync/atomic"
|
|
||||||
"time"
|
|
||||||
|
|
||||||
"github.com/go-redis/redis/internal/proto"
|
|
||||||
)
|
|
||||||
|
|
||||||
var noDeadline = time.Time{}
|
|
||||||
|
|
||||||
type Conn struct {
|
|
||||||
netConn net.Conn
|
|
||||||
|
|
||||||
rd *proto.Reader
|
|
||||||
rdLocked bool
|
|
||||||
wr *proto.Writer
|
|
||||||
|
|
||||||
InitedAt time.Time
|
|
||||||
pooled bool
|
|
||||||
usedAt atomic.Value
|
|
||||||
}
|
|
||||||
|
|
||||||
func NewConn(netConn net.Conn) *Conn {
|
|
||||||
cn := &Conn{
|
|
||||||
netConn: netConn,
|
|
||||||
}
|
|
||||||
cn.rd = proto.NewReader(netConn)
|
|
||||||
cn.wr = proto.NewWriter(netConn)
|
|
||||||
cn.SetUsedAt(time.Now())
|
|
||||||
return cn
|
|
||||||
}
|
|
||||||
|
|
||||||
func (cn *Conn) UsedAt() time.Time {
|
|
||||||
return cn.usedAt.Load().(time.Time)
|
|
||||||
}
|
|
||||||
|
|
||||||
func (cn *Conn) SetUsedAt(tm time.Time) {
|
|
||||||
cn.usedAt.Store(tm)
|
|
||||||
}
|
|
||||||
|
|
||||||
func (cn *Conn) SetNetConn(netConn net.Conn) {
|
|
||||||
cn.netConn = netConn
|
|
||||||
cn.rd.Reset(netConn)
|
|
||||||
cn.wr.Reset(netConn)
|
|
||||||
}
|
|
||||||
|
|
||||||
func (cn *Conn) setReadTimeout(timeout time.Duration) error {
|
|
||||||
now := time.Now()
|
|
||||||
cn.SetUsedAt(now)
|
|
||||||
if timeout > 0 {
|
|
||||||
return cn.netConn.SetReadDeadline(now.Add(timeout))
|
|
||||||
}
|
|
||||||
return cn.netConn.SetReadDeadline(noDeadline)
|
|
||||||
}
|
|
||||||
|
|
||||||
func (cn *Conn) setWriteTimeout(timeout time.Duration) error {
|
|
||||||
now := time.Now()
|
|
||||||
cn.SetUsedAt(now)
|
|
||||||
if timeout > 0 {
|
|
||||||
return cn.netConn.SetWriteDeadline(now.Add(timeout))
|
|
||||||
}
|
|
||||||
return cn.netConn.SetWriteDeadline(noDeadline)
|
|
||||||
}
|
|
||||||
|
|
||||||
func (cn *Conn) Write(b []byte) (int, error) {
|
|
||||||
return cn.netConn.Write(b)
|
|
||||||
}
|
|
||||||
|
|
||||||
func (cn *Conn) RemoteAddr() net.Addr {
|
|
||||||
return cn.netConn.RemoteAddr()
|
|
||||||
}
|
|
||||||
|
|
||||||
func (cn *Conn) WithReader(timeout time.Duration, fn func(rd *proto.Reader) error) error {
|
|
||||||
_ = cn.setReadTimeout(timeout)
|
|
||||||
return fn(cn.rd)
|
|
||||||
}
|
|
||||||
|
|
||||||
func (cn *Conn) WithWriter(timeout time.Duration, fn func(wr *proto.Writer) error) error {
|
|
||||||
_ = cn.setWriteTimeout(timeout)
|
|
||||||
|
|
||||||
firstErr := fn(cn.wr)
|
|
||||||
err := cn.wr.Flush()
|
|
||||||
if err != nil && firstErr == nil {
|
|
||||||
firstErr = err
|
|
||||||
}
|
|
||||||
return firstErr
|
|
||||||
}
|
|
||||||
|
|
||||||
func (cn *Conn) Close() error {
|
|
||||||
return cn.netConn.Close()
|
|
||||||
}
|
|
|
@ -1,53 +0,0 @@
|
||||||
package pool
|
|
||||||
|
|
||||||
type SingleConnPool struct {
|
|
||||||
cn *Conn
|
|
||||||
}
|
|
||||||
|
|
||||||
var _ Pooler = (*SingleConnPool)(nil)
|
|
||||||
|
|
||||||
func NewSingleConnPool(cn *Conn) *SingleConnPool {
|
|
||||||
return &SingleConnPool{
|
|
||||||
cn: cn,
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func (p *SingleConnPool) NewConn() (*Conn, error) {
|
|
||||||
panic("not implemented")
|
|
||||||
}
|
|
||||||
|
|
||||||
func (p *SingleConnPool) CloseConn(*Conn) error {
|
|
||||||
panic("not implemented")
|
|
||||||
}
|
|
||||||
|
|
||||||
func (p *SingleConnPool) Get() (*Conn, error) {
|
|
||||||
return p.cn, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func (p *SingleConnPool) Put(cn *Conn) {
|
|
||||||
if p.cn != cn {
|
|
||||||
panic("p.cn != cn")
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func (p *SingleConnPool) Remove(cn *Conn) {
|
|
||||||
if p.cn != cn {
|
|
||||||
panic("p.cn != cn")
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func (p *SingleConnPool) Len() int {
|
|
||||||
return 1
|
|
||||||
}
|
|
||||||
|
|
||||||
func (p *SingleConnPool) IdleLen() int {
|
|
||||||
return 0
|
|
||||||
}
|
|
||||||
|
|
||||||
func (p *SingleConnPool) Stats() *Stats {
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func (p *SingleConnPool) Close() error {
|
|
||||||
return nil
|
|
||||||
}
|
|
|
@ -1,29 +0,0 @@
|
||||||
package internal
|
|
||||||
|
|
||||||
import "github.com/go-redis/redis/internal/util"
|
|
||||||
|
|
||||||
func ToLower(s string) string {
|
|
||||||
if isLower(s) {
|
|
||||||
return s
|
|
||||||
}
|
|
||||||
|
|
||||||
b := make([]byte, len(s))
|
|
||||||
for i := range b {
|
|
||||||
c := s[i]
|
|
||||||
if c >= 'A' && c <= 'Z' {
|
|
||||||
c += 'a' - 'A'
|
|
||||||
}
|
|
||||||
b[i] = c
|
|
||||||
}
|
|
||||||
return util.BytesToString(b)
|
|
||||||
}
|
|
||||||
|
|
||||||
func isLower(s string) bool {
|
|
||||||
for i := 0; i < len(s); i++ {
|
|
||||||
c := s[i]
|
|
||||||
if c >= 'A' && c <= 'Z' {
|
|
||||||
return false
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return true
|
|
||||||
}
|
|
|
@ -1,580 +0,0 @@
|
||||||
package redis
|
|
||||||
|
|
||||||
import (
|
|
||||||
"context"
|
|
||||||
"fmt"
|
|
||||||
"log"
|
|
||||||
"os"
|
|
||||||
"time"
|
|
||||||
|
|
||||||
"github.com/go-redis/redis/internal"
|
|
||||||
"github.com/go-redis/redis/internal/pool"
|
|
||||||
"github.com/go-redis/redis/internal/proto"
|
|
||||||
)
|
|
||||||
|
|
||||||
// Nil reply Redis returns when key does not exist.
|
|
||||||
const Nil = proto.Nil
|
|
||||||
|
|
||||||
func init() {
|
|
||||||
SetLogger(log.New(os.Stderr, "redis: ", log.LstdFlags|log.Lshortfile))
|
|
||||||
}
|
|
||||||
|
|
||||||
func SetLogger(logger *log.Logger) {
|
|
||||||
internal.Logger = logger
|
|
||||||
}
|
|
||||||
|
|
||||||
type baseClient struct {
|
|
||||||
opt *Options
|
|
||||||
connPool pool.Pooler
|
|
||||||
limiter Limiter
|
|
||||||
|
|
||||||
process func(Cmder) error
|
|
||||||
processPipeline func([]Cmder) error
|
|
||||||
processTxPipeline func([]Cmder) error
|
|
||||||
|
|
||||||
onClose func() error // hook called when client is closed
|
|
||||||
}
|
|
||||||
|
|
||||||
func (c *baseClient) init() {
|
|
||||||
c.process = c.defaultProcess
|
|
||||||
c.processPipeline = c.defaultProcessPipeline
|
|
||||||
c.processTxPipeline = c.defaultProcessTxPipeline
|
|
||||||
}
|
|
||||||
|
|
||||||
func (c *baseClient) String() string {
|
|
||||||
return fmt.Sprintf("Redis<%s db:%d>", c.getAddr(), c.opt.DB)
|
|
||||||
}
|
|
||||||
|
|
||||||
func (c *baseClient) newConn() (*pool.Conn, error) {
|
|
||||||
cn, err := c.connPool.NewConn()
|
|
||||||
if err != nil {
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
|
|
||||||
if cn.InitedAt.IsZero() {
|
|
||||||
if err := c.initConn(cn); err != nil {
|
|
||||||
_ = c.connPool.CloseConn(cn)
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
return cn, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func (c *baseClient) getConn() (*pool.Conn, error) {
|
|
||||||
if c.limiter != nil {
|
|
||||||
err := c.limiter.Allow()
|
|
||||||
if err != nil {
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
cn, err := c._getConn()
|
|
||||||
if err != nil {
|
|
||||||
if c.limiter != nil {
|
|
||||||
c.limiter.ReportResult(err)
|
|
||||||
}
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
return cn, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func (c *baseClient) _getConn() (*pool.Conn, error) {
|
|
||||||
cn, err := c.connPool.Get()
|
|
||||||
if err != nil {
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
|
|
||||||
if cn.InitedAt.IsZero() {
|
|
||||||
err := c.initConn(cn)
|
|
||||||
if err != nil {
|
|
||||||
c.connPool.Remove(cn)
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
return cn, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func (c *baseClient) releaseConn(cn *pool.Conn, err error) {
|
|
||||||
if c.limiter != nil {
|
|
||||||
c.limiter.ReportResult(err)
|
|
||||||
}
|
|
||||||
|
|
||||||
if internal.IsBadConn(err, false) {
|
|
||||||
c.connPool.Remove(cn)
|
|
||||||
} else {
|
|
||||||
c.connPool.Put(cn)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func (c *baseClient) releaseConnStrict(cn *pool.Conn, err error) {
|
|
||||||
if c.limiter != nil {
|
|
||||||
c.limiter.ReportResult(err)
|
|
||||||
}
|
|
||||||
|
|
||||||
if err == nil || internal.IsRedisError(err) {
|
|
||||||
c.connPool.Put(cn)
|
|
||||||
} else {
|
|
||||||
c.connPool.Remove(cn)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func (c *baseClient) initConn(cn *pool.Conn) error {
|
|
||||||
cn.InitedAt = time.Now()
|
|
||||||
|
|
||||||
if c.opt.Password == "" &&
|
|
||||||
c.opt.DB == 0 &&
|
|
||||||
!c.opt.readOnly &&
|
|
||||||
c.opt.OnConnect == nil {
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
|
|
||||||
conn := newConn(c.opt, cn)
|
|
||||||
_, err := conn.Pipelined(func(pipe Pipeliner) error {
|
|
||||||
if c.opt.Password != "" {
|
|
||||||
pipe.Auth(c.opt.Password)
|
|
||||||
}
|
|
||||||
|
|
||||||
if c.opt.DB > 0 {
|
|
||||||
pipe.Select(c.opt.DB)
|
|
||||||
}
|
|
||||||
|
|
||||||
if c.opt.readOnly {
|
|
||||||
pipe.ReadOnly()
|
|
||||||
}
|
|
||||||
|
|
||||||
return nil
|
|
||||||
})
|
|
||||||
if err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
|
|
||||||
if c.opt.OnConnect != nil {
|
|
||||||
return c.opt.OnConnect(conn)
|
|
||||||
}
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
|
|
||||||
// Do creates a Cmd from the args and processes the cmd.
|
|
||||||
func (c *baseClient) Do(args ...interface{}) *Cmd {
|
|
||||||
cmd := NewCmd(args...)
|
|
||||||
_ = c.Process(cmd)
|
|
||||||
return cmd
|
|
||||||
}
|
|
||||||
|
|
||||||
// WrapProcess wraps function that processes Redis commands.
|
|
||||||
func (c *baseClient) WrapProcess(
|
|
||||||
fn func(oldProcess func(cmd Cmder) error) func(cmd Cmder) error,
|
|
||||||
) {
|
|
||||||
c.process = fn(c.process)
|
|
||||||
}
|
|
||||||
|
|
||||||
func (c *baseClient) Process(cmd Cmder) error {
|
|
||||||
return c.process(cmd)
|
|
||||||
}
|
|
||||||
|
|
||||||
func (c *baseClient) defaultProcess(cmd Cmder) error {
|
|
||||||
for attempt := 0; attempt <= c.opt.MaxRetries; attempt++ {
|
|
||||||
if attempt > 0 {
|
|
||||||
time.Sleep(c.retryBackoff(attempt))
|
|
||||||
}
|
|
||||||
|
|
||||||
cn, err := c.getConn()
|
|
||||||
if err != nil {
|
|
||||||
cmd.setErr(err)
|
|
||||||
if internal.IsRetryableError(err, true) {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
|
|
||||||
err = cn.WithWriter(c.opt.WriteTimeout, func(wr *proto.Writer) error {
|
|
||||||
return writeCmd(wr, cmd)
|
|
||||||
})
|
|
||||||
if err != nil {
|
|
||||||
c.releaseConn(cn, err)
|
|
||||||
cmd.setErr(err)
|
|
||||||
if internal.IsRetryableError(err, true) {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
|
|
||||||
err = cn.WithReader(c.cmdTimeout(cmd), func(rd *proto.Reader) error {
|
|
||||||
return cmd.readReply(rd)
|
|
||||||
})
|
|
||||||
c.releaseConn(cn, err)
|
|
||||||
if err != nil && internal.IsRetryableError(err, cmd.readTimeout() == nil) {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
|
|
||||||
return cmd.Err()
|
|
||||||
}
|
|
||||||
|
|
||||||
func (c *baseClient) retryBackoff(attempt int) time.Duration {
|
|
||||||
return internal.RetryBackoff(attempt, c.opt.MinRetryBackoff, c.opt.MaxRetryBackoff)
|
|
||||||
}
|
|
||||||
|
|
||||||
func (c *baseClient) cmdTimeout(cmd Cmder) time.Duration {
|
|
||||||
if timeout := cmd.readTimeout(); timeout != nil {
|
|
||||||
t := *timeout
|
|
||||||
if t == 0 {
|
|
||||||
return 0
|
|
||||||
}
|
|
||||||
return t + 10*time.Second
|
|
||||||
}
|
|
||||||
return c.opt.ReadTimeout
|
|
||||||
}
|
|
||||||
|
|
||||||
// Close closes the client, releasing any open resources.
|
|
||||||
//
|
|
||||||
// It is rare to Close a Client, as the Client is meant to be
|
|
||||||
// long-lived and shared between many goroutines.
|
|
||||||
func (c *baseClient) Close() error {
|
|
||||||
var firstErr error
|
|
||||||
if c.onClose != nil {
|
|
||||||
if err := c.onClose(); err != nil && firstErr == nil {
|
|
||||||
firstErr = err
|
|
||||||
}
|
|
||||||
}
|
|
||||||
if err := c.connPool.Close(); err != nil && firstErr == nil {
|
|
||||||
firstErr = err
|
|
||||||
}
|
|
||||||
return firstErr
|
|
||||||
}
|
|
||||||
|
|
||||||
func (c *baseClient) getAddr() string {
|
|
||||||
return c.opt.Addr
|
|
||||||
}
|
|
||||||
|
|
||||||
func (c *baseClient) WrapProcessPipeline(
|
|
||||||
fn func(oldProcess func([]Cmder) error) func([]Cmder) error,
|
|
||||||
) {
|
|
||||||
c.processPipeline = fn(c.processPipeline)
|
|
||||||
c.processTxPipeline = fn(c.processTxPipeline)
|
|
||||||
}
|
|
||||||
|
|
||||||
func (c *baseClient) defaultProcessPipeline(cmds []Cmder) error {
|
|
||||||
return c.generalProcessPipeline(cmds, c.pipelineProcessCmds)
|
|
||||||
}
|
|
||||||
|
|
||||||
func (c *baseClient) defaultProcessTxPipeline(cmds []Cmder) error {
|
|
||||||
return c.generalProcessPipeline(cmds, c.txPipelineProcessCmds)
|
|
||||||
}
|
|
||||||
|
|
||||||
type pipelineProcessor func(*pool.Conn, []Cmder) (bool, error)
|
|
||||||
|
|
||||||
func (c *baseClient) generalProcessPipeline(cmds []Cmder, p pipelineProcessor) error {
|
|
||||||
for attempt := 0; attempt <= c.opt.MaxRetries; attempt++ {
|
|
||||||
if attempt > 0 {
|
|
||||||
time.Sleep(c.retryBackoff(attempt))
|
|
||||||
}
|
|
||||||
|
|
||||||
cn, err := c.getConn()
|
|
||||||
if err != nil {
|
|
||||||
setCmdsErr(cmds, err)
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
|
|
||||||
canRetry, err := p(cn, cmds)
|
|
||||||
c.releaseConnStrict(cn, err)
|
|
||||||
|
|
||||||
if !canRetry || !internal.IsRetryableError(err, true) {
|
|
||||||
break
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return cmdsFirstErr(cmds)
|
|
||||||
}
|
|
||||||
|
|
||||||
func (c *baseClient) pipelineProcessCmds(cn *pool.Conn, cmds []Cmder) (bool, error) {
|
|
||||||
err := cn.WithWriter(c.opt.WriteTimeout, func(wr *proto.Writer) error {
|
|
||||||
return writeCmd(wr, cmds...)
|
|
||||||
})
|
|
||||||
if err != nil {
|
|
||||||
setCmdsErr(cmds, err)
|
|
||||||
return true, err
|
|
||||||
}
|
|
||||||
|
|
||||||
err = cn.WithReader(c.opt.ReadTimeout, func(rd *proto.Reader) error {
|
|
||||||
return pipelineReadCmds(rd, cmds)
|
|
||||||
})
|
|
||||||
return true, err
|
|
||||||
}
|
|
||||||
|
|
||||||
func pipelineReadCmds(rd *proto.Reader, cmds []Cmder) error {
|
|
||||||
for _, cmd := range cmds {
|
|
||||||
err := cmd.readReply(rd)
|
|
||||||
if err != nil && !internal.IsRedisError(err) {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func (c *baseClient) txPipelineProcessCmds(cn *pool.Conn, cmds []Cmder) (bool, error) {
|
|
||||||
err := cn.WithWriter(c.opt.WriteTimeout, func(wr *proto.Writer) error {
|
|
||||||
return txPipelineWriteMulti(wr, cmds)
|
|
||||||
})
|
|
||||||
if err != nil {
|
|
||||||
setCmdsErr(cmds, err)
|
|
||||||
return true, err
|
|
||||||
}
|
|
||||||
|
|
||||||
err = cn.WithReader(c.opt.ReadTimeout, func(rd *proto.Reader) error {
|
|
||||||
err := txPipelineReadQueued(rd, cmds)
|
|
||||||
if err != nil {
|
|
||||||
setCmdsErr(cmds, err)
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
return pipelineReadCmds(rd, cmds)
|
|
||||||
})
|
|
||||||
return false, err
|
|
||||||
}
|
|
||||||
|
|
||||||
func txPipelineWriteMulti(wr *proto.Writer, cmds []Cmder) error {
|
|
||||||
multiExec := make([]Cmder, 0, len(cmds)+2)
|
|
||||||
multiExec = append(multiExec, NewStatusCmd("MULTI"))
|
|
||||||
multiExec = append(multiExec, cmds...)
|
|
||||||
multiExec = append(multiExec, NewSliceCmd("EXEC"))
|
|
||||||
return writeCmd(wr, multiExec...)
|
|
||||||
}
|
|
||||||
|
|
||||||
func txPipelineReadQueued(rd *proto.Reader, cmds []Cmder) error {
|
|
||||||
// Parse queued replies.
|
|
||||||
var statusCmd StatusCmd
|
|
||||||
err := statusCmd.readReply(rd)
|
|
||||||
if err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
|
|
||||||
for range cmds {
|
|
||||||
err = statusCmd.readReply(rd)
|
|
||||||
if err != nil && !internal.IsRedisError(err) {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// Parse number of replies.
|
|
||||||
line, err := rd.ReadLine()
|
|
||||||
if err != nil {
|
|
||||||
if err == Nil {
|
|
||||||
err = TxFailedErr
|
|
||||||
}
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
|
|
||||||
switch line[0] {
|
|
||||||
case proto.ErrorReply:
|
|
||||||
return proto.ParseErrorReply(line)
|
|
||||||
case proto.ArrayReply:
|
|
||||||
// ok
|
|
||||||
default:
|
|
||||||
err := fmt.Errorf("redis: expected '*', but got line %q", line)
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
|
|
||||||
//------------------------------------------------------------------------------
|
|
||||||
|
|
||||||
// Client is a Redis client representing a pool of zero or more
|
|
||||||
// underlying connections. It's safe for concurrent use by multiple
|
|
||||||
// goroutines.
|
|
||||||
type Client struct {
|
|
||||||
baseClient
|
|
||||||
cmdable
|
|
||||||
|
|
||||||
ctx context.Context
|
|
||||||
}
|
|
||||||
|
|
||||||
// NewClient returns a client to the Redis Server specified by Options.
|
|
||||||
func NewClient(opt *Options) *Client {
|
|
||||||
opt.init()
|
|
||||||
|
|
||||||
c := Client{
|
|
||||||
baseClient: baseClient{
|
|
||||||
opt: opt,
|
|
||||||
connPool: newConnPool(opt),
|
|
||||||
},
|
|
||||||
}
|
|
||||||
c.baseClient.init()
|
|
||||||
c.init()
|
|
||||||
|
|
||||||
return &c
|
|
||||||
}
|
|
||||||
|
|
||||||
func (c *Client) init() {
|
|
||||||
c.cmdable.setProcessor(c.Process)
|
|
||||||
}
|
|
||||||
|
|
||||||
func (c *Client) Context() context.Context {
|
|
||||||
if c.ctx != nil {
|
|
||||||
return c.ctx
|
|
||||||
}
|
|
||||||
return context.Background()
|
|
||||||
}
|
|
||||||
|
|
||||||
func (c *Client) WithContext(ctx context.Context) *Client {
|
|
||||||
if ctx == nil {
|
|
||||||
panic("nil context")
|
|
||||||
}
|
|
||||||
c2 := c.clone()
|
|
||||||
c2.ctx = ctx
|
|
||||||
return c2
|
|
||||||
}
|
|
||||||
|
|
||||||
func (c *Client) clone() *Client {
|
|
||||||
cp := *c
|
|
||||||
cp.init()
|
|
||||||
return &cp
|
|
||||||
}
|
|
||||||
|
|
||||||
// Options returns read-only Options that were used to create the client.
|
|
||||||
func (c *Client) Options() *Options {
|
|
||||||
return c.opt
|
|
||||||
}
|
|
||||||
|
|
||||||
func (c *Client) SetLimiter(l Limiter) *Client {
|
|
||||||
c.limiter = l
|
|
||||||
return c
|
|
||||||
}
|
|
||||||
|
|
||||||
type PoolStats pool.Stats
|
|
||||||
|
|
||||||
// PoolStats returns connection pool stats.
|
|
||||||
func (c *Client) PoolStats() *PoolStats {
|
|
||||||
stats := c.connPool.Stats()
|
|
||||||
return (*PoolStats)(stats)
|
|
||||||
}
|
|
||||||
|
|
||||||
func (c *Client) Pipelined(fn func(Pipeliner) error) ([]Cmder, error) {
|
|
||||||
return c.Pipeline().Pipelined(fn)
|
|
||||||
}
|
|
||||||
|
|
||||||
func (c *Client) Pipeline() Pipeliner {
|
|
||||||
pipe := Pipeline{
|
|
||||||
exec: c.processPipeline,
|
|
||||||
}
|
|
||||||
pipe.statefulCmdable.setProcessor(pipe.Process)
|
|
||||||
return &pipe
|
|
||||||
}
|
|
||||||
|
|
||||||
func (c *Client) TxPipelined(fn func(Pipeliner) error) ([]Cmder, error) {
|
|
||||||
return c.TxPipeline().Pipelined(fn)
|
|
||||||
}
|
|
||||||
|
|
||||||
// TxPipeline acts like Pipeline, but wraps queued commands with MULTI/EXEC.
|
|
||||||
func (c *Client) TxPipeline() Pipeliner {
|
|
||||||
pipe := Pipeline{
|
|
||||||
exec: c.processTxPipeline,
|
|
||||||
}
|
|
||||||
pipe.statefulCmdable.setProcessor(pipe.Process)
|
|
||||||
return &pipe
|
|
||||||
}
|
|
||||||
|
|
||||||
func (c *Client) pubSub() *PubSub {
|
|
||||||
pubsub := &PubSub{
|
|
||||||
opt: c.opt,
|
|
||||||
|
|
||||||
newConn: func(channels []string) (*pool.Conn, error) {
|
|
||||||
return c.newConn()
|
|
||||||
},
|
|
||||||
closeConn: c.connPool.CloseConn,
|
|
||||||
}
|
|
||||||
pubsub.init()
|
|
||||||
return pubsub
|
|
||||||
}
|
|
||||||
|
|
||||||
// Subscribe subscribes the client to the specified channels.
|
|
||||||
// Channels can be omitted to create empty subscription.
|
|
||||||
// Note that this method does not wait on a response from Redis, so the
|
|
||||||
// subscription may not be active immediately. To force the connection to wait,
|
|
||||||
// you may call the Receive() method on the returned *PubSub like so:
|
|
||||||
//
|
|
||||||
// sub := client.Subscribe(queryResp)
|
|
||||||
// iface, err := sub.Receive()
|
|
||||||
// if err != nil {
|
|
||||||
// // handle error
|
|
||||||
// }
|
|
||||||
//
|
|
||||||
// // Should be *Subscription, but others are possible if other actions have been
|
|
||||||
// // taken on sub since it was created.
|
|
||||||
// switch iface.(type) {
|
|
||||||
// case *Subscription:
|
|
||||||
// // subscribe succeeded
|
|
||||||
// case *Message:
|
|
||||||
// // received first message
|
|
||||||
// case *Pong:
|
|
||||||
// // pong received
|
|
||||||
// default:
|
|
||||||
// // handle error
|
|
||||||
// }
|
|
||||||
//
|
|
||||||
// ch := sub.Channel()
|
|
||||||
func (c *Client) Subscribe(channels ...string) *PubSub {
|
|
||||||
pubsub := c.pubSub()
|
|
||||||
if len(channels) > 0 {
|
|
||||||
_ = pubsub.Subscribe(channels...)
|
|
||||||
}
|
|
||||||
return pubsub
|
|
||||||
}
|
|
||||||
|
|
||||||
// PSubscribe subscribes the client to the given patterns.
|
|
||||||
// Patterns can be omitted to create empty subscription.
|
|
||||||
func (c *Client) PSubscribe(channels ...string) *PubSub {
|
|
||||||
pubsub := c.pubSub()
|
|
||||||
if len(channels) > 0 {
|
|
||||||
_ = pubsub.PSubscribe(channels...)
|
|
||||||
}
|
|
||||||
return pubsub
|
|
||||||
}
|
|
||||||
|
|
||||||
//------------------------------------------------------------------------------
|
|
||||||
|
|
||||||
// Conn is like Client, but its pool contains single connection.
|
|
||||||
type Conn struct {
|
|
||||||
baseClient
|
|
||||||
statefulCmdable
|
|
||||||
}
|
|
||||||
|
|
||||||
func newConn(opt *Options, cn *pool.Conn) *Conn {
|
|
||||||
c := Conn{
|
|
||||||
baseClient: baseClient{
|
|
||||||
opt: opt,
|
|
||||||
connPool: pool.NewSingleConnPool(cn),
|
|
||||||
},
|
|
||||||
}
|
|
||||||
c.baseClient.init()
|
|
||||||
c.statefulCmdable.setProcessor(c.Process)
|
|
||||||
return &c
|
|
||||||
}
|
|
||||||
|
|
||||||
func (c *Conn) Pipelined(fn func(Pipeliner) error) ([]Cmder, error) {
|
|
||||||
return c.Pipeline().Pipelined(fn)
|
|
||||||
}
|
|
||||||
|
|
||||||
func (c *Conn) Pipeline() Pipeliner {
|
|
||||||
pipe := Pipeline{
|
|
||||||
exec: c.processPipeline,
|
|
||||||
}
|
|
||||||
pipe.statefulCmdable.setProcessor(pipe.Process)
|
|
||||||
return &pipe
|
|
||||||
}
|
|
||||||
|
|
||||||
func (c *Conn) TxPipelined(fn func(Pipeliner) error) ([]Cmder, error) {
|
|
||||||
return c.TxPipeline().Pipelined(fn)
|
|
||||||
}
|
|
||||||
|
|
||||||
// TxPipeline acts like Pipeline, but wraps queued commands with MULTI/EXEC.
|
|
||||||
func (c *Conn) TxPipeline() Pipeliner {
|
|
||||||
pipe := Pipeline{
|
|
||||||
exec: c.processTxPipeline,
|
|
||||||
}
|
|
||||||
pipe.statefulCmdable.setProcessor(pipe.Process)
|
|
||||||
return &pipe
|
|
||||||
}
|
|
|
@ -0,0 +1,15 @@
|
||||||
|
run:
|
||||||
|
concurrency: 8
|
||||||
|
deadline: 5m
|
||||||
|
tests: false
|
||||||
|
linters:
|
||||||
|
enable-all: true
|
||||||
|
disable:
|
||||||
|
- funlen
|
||||||
|
- gochecknoglobals
|
||||||
|
- gocognit
|
||||||
|
- goconst
|
||||||
|
- godox
|
||||||
|
- gosec
|
||||||
|
- maligned
|
||||||
|
- wsl
|
|
@ -0,0 +1,22 @@
|
||||||
|
dist: xenial
|
||||||
|
language: go
|
||||||
|
|
||||||
|
services:
|
||||||
|
- redis-server
|
||||||
|
|
||||||
|
go:
|
||||||
|
- 1.12.x
|
||||||
|
- 1.13.x
|
||||||
|
- tip
|
||||||
|
|
||||||
|
matrix:
|
||||||
|
allow_failures:
|
||||||
|
- go: tip
|
||||||
|
|
||||||
|
env:
|
||||||
|
- GO111MODULE=on
|
||||||
|
|
||||||
|
go_import_path: github.com/go-redis/redis
|
||||||
|
|
||||||
|
before_install:
|
||||||
|
- curl -sfL https://install.goreleaser.com/github.com/golangci/golangci-lint.sh | sh -s -- -b $(go env GOPATH)/bin v1.21.0
|
|
@ -0,0 +1,46 @@
|
||||||
|
# Changelog
|
||||||
|
|
||||||
|
## v7.2
|
||||||
|
|
||||||
|
- Existing `HMSet` is renamed to `HSet` and old deprecated `HMSet` is restored for Redis 3 users.
|
||||||
|
|
||||||
|
## v7.1
|
||||||
|
|
||||||
|
- Existing `Cmd.String` is renamed to `Cmd.Text`. New `Cmd.String` implements `fmt.Stringer` interface.
|
||||||
|
|
||||||
|
## v7
|
||||||
|
|
||||||
|
- *Important*. Tx.Pipeline now returns a non-transactional pipeline. Use Tx.TxPipeline for a transactional pipeline.
|
||||||
|
- WrapProcess is replaced with more convenient AddHook that has access to context.Context.
|
||||||
|
- WithContext now can not be used to create a shallow copy of the client.
|
||||||
|
- New methods ProcessContext, DoContext, and ExecContext.
|
||||||
|
- Client respects Context.Deadline when setting net.Conn deadline.
|
||||||
|
- Client listens on Context.Done while waiting for a connection from the pool and returns an error when context context is cancelled.
|
||||||
|
- Add PubSub.ChannelWithSubscriptions that sends `*Subscription` in addition to `*Message` to allow detecting reconnections.
|
||||||
|
- `time.Time` is now marshalled in RFC3339 format. `rdb.Get("foo").Time()` helper is added to parse the time.
|
||||||
|
- `SetLimiter` is removed and added `Options.Limiter` instead.
|
||||||
|
- `HMSet` is deprecated as of Redis v4.
|
||||||
|
|
||||||
|
## v6.15
|
||||||
|
|
||||||
|
- Cluster and Ring pipelines process commands for each node in its own goroutine.
|
||||||
|
|
||||||
|
## 6.14
|
||||||
|
|
||||||
|
- Added Options.MinIdleConns.
|
||||||
|
- Added Options.MaxConnAge.
|
||||||
|
- PoolStats.FreeConns is renamed to PoolStats.IdleConns.
|
||||||
|
- Add Client.Do to simplify creating custom commands.
|
||||||
|
- Add Cmd.String, Cmd.Int, Cmd.Int64, Cmd.Uint64, Cmd.Float64, and Cmd.Bool helpers.
|
||||||
|
- Lower memory usage.
|
||||||
|
|
||||||
|
## v6.13
|
||||||
|
|
||||||
|
- Ring got new options called `HashReplicas` and `Hash`. It is recommended to set `HashReplicas = 1000` for better keys distribution between shards.
|
||||||
|
- Cluster client was optimized to use much less memory when reloading cluster state.
|
||||||
|
- PubSub.ReceiveMessage is re-worked to not use ReceiveTimeout so it does not lose data when timeout occurres. In most cases it is recommended to use PubSub.Channel instead.
|
||||||
|
- Dialer.KeepAlive is set to 5 minutes by default.
|
||||||
|
|
||||||
|
## v6.12
|
||||||
|
|
||||||
|
- ClusterClient got new option called `ClusterSlots` which allows to build cluster of normal Redis Servers that don't have cluster mode enabled. See https://godoc.org/github.com/go-redis/redis#example-NewClusterClient--ManualSetup
|
|
@ -1,10 +1,9 @@
|
||||||
all: testdeps
|
all: testdeps
|
||||||
go test ./...
|
go test ./...
|
||||||
go test ./... -short -race
|
go test ./... -short -race
|
||||||
|
go test ./... -run=NONE -bench=. -benchmem
|
||||||
env GOOS=linux GOARCH=386 go test ./...
|
env GOOS=linux GOARCH=386 go test ./...
|
||||||
go vet
|
golangci-lint run
|
||||||
go get github.com/gordonklaus/ineffassign
|
|
||||||
ineffassign .
|
|
||||||
|
|
||||||
testdeps: testdata/redis/src/redis-server
|
testdeps: testdata/redis/src/redis-server
|
||||||
|
|
||||||
|
@ -15,8 +14,7 @@ bench: testdeps
|
||||||
|
|
||||||
testdata/redis:
|
testdata/redis:
|
||||||
mkdir -p $@
|
mkdir -p $@
|
||||||
wget -qO- https://github.com/antirez/redis/archive/5.0.tar.gz | tar xvz --strip-components=1 -C $@
|
wget -qO- http://download.redis.io/redis-stable.tar.gz | tar xvz --strip-components=1 -C $@
|
||||||
|
|
||||||
testdata/redis/src/redis-server: testdata/redis
|
testdata/redis/src/redis-server: testdata/redis
|
||||||
sed -i.bak 's/libjemalloc.a/libjemalloc.a -lrt/g' $</src/Makefile
|
|
||||||
cd $< && make all
|
cd $< && make all
|
|
@ -9,7 +9,7 @@ Supports:
|
||||||
- Redis 3 commands except QUIT, MONITOR, SLOWLOG and SYNC.
|
- Redis 3 commands except QUIT, MONITOR, SLOWLOG and SYNC.
|
||||||
- Automatic connection pooling with [circuit breaker](https://en.wikipedia.org/wiki/Circuit_breaker_design_pattern) support.
|
- Automatic connection pooling with [circuit breaker](https://en.wikipedia.org/wiki/Circuit_breaker_design_pattern) support.
|
||||||
- [Pub/Sub](https://godoc.org/github.com/go-redis/redis#PubSub).
|
- [Pub/Sub](https://godoc.org/github.com/go-redis/redis#PubSub).
|
||||||
- [Transactions](https://godoc.org/github.com/go-redis/redis#Multi).
|
- [Transactions](https://godoc.org/github.com/go-redis/redis#example-Client-TxPipeline).
|
||||||
- [Pipeline](https://godoc.org/github.com/go-redis/redis#example-Client-Pipeline) and [TxPipeline](https://godoc.org/github.com/go-redis/redis#example-Client-TxPipeline).
|
- [Pipeline](https://godoc.org/github.com/go-redis/redis#example-Client-Pipeline) and [TxPipeline](https://godoc.org/github.com/go-redis/redis#example-Client-TxPipeline).
|
||||||
- [Scripting](https://godoc.org/github.com/go-redis/redis#Script).
|
- [Scripting](https://godoc.org/github.com/go-redis/redis#Script).
|
||||||
- [Timeouts](https://godoc.org/github.com/go-redis/redis#Options).
|
- [Timeouts](https://godoc.org/github.com/go-redis/redis#Options).
|
||||||
|
@ -20,28 +20,29 @@ Supports:
|
||||||
- [Instrumentation](https://godoc.org/github.com/go-redis/redis#ex-package--Instrumentation).
|
- [Instrumentation](https://godoc.org/github.com/go-redis/redis#ex-package--Instrumentation).
|
||||||
- [Cache friendly](https://github.com/go-redis/cache).
|
- [Cache friendly](https://github.com/go-redis/cache).
|
||||||
- [Rate limiting](https://github.com/go-redis/redis_rate).
|
- [Rate limiting](https://github.com/go-redis/redis_rate).
|
||||||
- [Distributed Locks](https://github.com/bsm/redis-lock).
|
- [Distributed Locks](https://github.com/bsm/redislock).
|
||||||
|
|
||||||
API docs: https://godoc.org/github.com/go-redis/redis.
|
API docs: https://godoc.org/github.com/go-redis/redis.
|
||||||
Examples: https://godoc.org/github.com/go-redis/redis#pkg-examples.
|
Examples: https://godoc.org/github.com/go-redis/redis#pkg-examples.
|
||||||
|
|
||||||
## Installation
|
## Installation
|
||||||
|
|
||||||
Install:
|
go-redis requires a Go version with [Modules](https://github.com/golang/go/wiki/Modules) support and uses import versioning. So please make sure to initialize a Go module before installing go-redis:
|
||||||
|
|
||||||
```shell
|
``` shell
|
||||||
go get -u github.com/go-redis/redis
|
go mod init github.com/my/repo
|
||||||
|
go get github.com/go-redis/redis/v7
|
||||||
```
|
```
|
||||||
|
|
||||||
Import:
|
Import:
|
||||||
|
|
||||||
```go
|
``` go
|
||||||
import "github.com/go-redis/redis"
|
import "github.com/go-redis/redis/v7"
|
||||||
```
|
```
|
||||||
|
|
||||||
## Quickstart
|
## Quickstart
|
||||||
|
|
||||||
```go
|
``` go
|
||||||
func ExampleNewClient() {
|
func ExampleNewClient() {
|
||||||
client := redis.NewClient(&redis.Options{
|
client := redis.NewClient(&redis.Options{
|
||||||
Addr: "localhost:6379",
|
Addr: "localhost:6379",
|
||||||
|
@ -55,6 +56,11 @@ func ExampleNewClient() {
|
||||||
}
|
}
|
||||||
|
|
||||||
func ExampleClient() {
|
func ExampleClient() {
|
||||||
|
client := redis.NewClient(&redis.Options{
|
||||||
|
Addr: "localhost:6379",
|
||||||
|
Password: "", // no password set
|
||||||
|
DB: 0, // use default DB
|
||||||
|
})
|
||||||
err := client.Set("key", "value", 0).Err()
|
err := client.Set("key", "value", 0).Err()
|
||||||
if err != nil {
|
if err != nil {
|
||||||
panic(err)
|
panic(err)
|
||||||
|
@ -87,15 +93,15 @@ Please go through [examples](https://godoc.org/github.com/go-redis/redis#pkg-exa
|
||||||
|
|
||||||
Some corner cases:
|
Some corner cases:
|
||||||
|
|
||||||
```go
|
``` go
|
||||||
// SET key value EX 10 NX
|
// SET key value EX 10 NX
|
||||||
set, err := client.SetNX("key", "value", 10*time.Second).Result()
|
set, err := client.SetNX("key", "value", 10*time.Second).Result()
|
||||||
|
|
||||||
// SORT list LIMIT 0 2 ASC
|
// SORT list LIMIT 0 2 ASC
|
||||||
vals, err := client.Sort("list", redis.Sort{Offset: 0, Count: 2, Order: "ASC"}).Result()
|
vals, err := client.Sort("list", &redis.Sort{Offset: 0, Count: 2, Order: "ASC"}).Result()
|
||||||
|
|
||||||
// ZRANGEBYSCORE zset -inf +inf WITHSCORES LIMIT 0 2
|
// ZRANGEBYSCORE zset -inf +inf WITHSCORES LIMIT 0 2
|
||||||
vals, err := client.ZRangeByScoreWithScores("zset", redis.ZRangeBy{
|
vals, err := client.ZRangeByScoreWithScores("zset", &redis.ZRangeBy{
|
||||||
Min: "-inf",
|
Min: "-inf",
|
||||||
Max: "+inf",
|
Max: "+inf",
|
||||||
Offset: 0,
|
Offset: 0,
|
||||||
|
@ -103,44 +109,20 @@ vals, err := client.ZRangeByScoreWithScores("zset", redis.ZRangeBy{
|
||||||
}).Result()
|
}).Result()
|
||||||
|
|
||||||
// ZINTERSTORE out 2 zset1 zset2 WEIGHTS 2 3 AGGREGATE SUM
|
// ZINTERSTORE out 2 zset1 zset2 WEIGHTS 2 3 AGGREGATE SUM
|
||||||
vals, err := client.ZInterStore("out", redis.ZStore{Weights: []int64{2, 3}}, "zset1", "zset2").Result()
|
vals, err := client.ZInterStore("out", &redis.ZStore{
|
||||||
|
Keys: []string{"zset1", "zset2"},
|
||||||
|
Weights: []int64{2, 3}
|
||||||
|
}).Result()
|
||||||
|
|
||||||
// EVAL "return {KEYS[1],ARGV[1]}" 1 "key" "hello"
|
// EVAL "return {KEYS[1],ARGV[1]}" 1 "key" "hello"
|
||||||
vals, err := client.Eval("return {KEYS[1],ARGV[1]}", []string{"key"}, "hello").Result()
|
vals, err := client.Eval("return {KEYS[1],ARGV[1]}", []string{"key"}, "hello").Result()
|
||||||
```
|
|
||||||
|
|
||||||
## Benchmark
|
// custom command
|
||||||
|
res, err := client.Do("set", "key", "value").Result()
|
||||||
go-redis vs redigo:
|
|
||||||
|
|
||||||
```
|
|
||||||
BenchmarkSetGoRedis10Conns64Bytes-4 200000 7621 ns/op 210 B/op 6 allocs/op
|
|
||||||
BenchmarkSetGoRedis100Conns64Bytes-4 200000 7554 ns/op 210 B/op 6 allocs/op
|
|
||||||
BenchmarkSetGoRedis10Conns1KB-4 200000 7697 ns/op 210 B/op 6 allocs/op
|
|
||||||
BenchmarkSetGoRedis100Conns1KB-4 200000 7688 ns/op 210 B/op 6 allocs/op
|
|
||||||
BenchmarkSetGoRedis10Conns10KB-4 200000 9214 ns/op 210 B/op 6 allocs/op
|
|
||||||
BenchmarkSetGoRedis100Conns10KB-4 200000 9181 ns/op 210 B/op 6 allocs/op
|
|
||||||
BenchmarkSetGoRedis10Conns1MB-4 2000 583242 ns/op 2337 B/op 6 allocs/op
|
|
||||||
BenchmarkSetGoRedis100Conns1MB-4 2000 583089 ns/op 2338 B/op 6 allocs/op
|
|
||||||
BenchmarkSetRedigo10Conns64Bytes-4 200000 7576 ns/op 208 B/op 7 allocs/op
|
|
||||||
BenchmarkSetRedigo100Conns64Bytes-4 200000 7782 ns/op 208 B/op 7 allocs/op
|
|
||||||
BenchmarkSetRedigo10Conns1KB-4 200000 7958 ns/op 208 B/op 7 allocs/op
|
|
||||||
BenchmarkSetRedigo100Conns1KB-4 200000 7725 ns/op 208 B/op 7 allocs/op
|
|
||||||
BenchmarkSetRedigo10Conns10KB-4 100000 18442 ns/op 208 B/op 7 allocs/op
|
|
||||||
BenchmarkSetRedigo100Conns10KB-4 100000 18818 ns/op 208 B/op 7 allocs/op
|
|
||||||
BenchmarkSetRedigo10Conns1MB-4 2000 668829 ns/op 226 B/op 7 allocs/op
|
|
||||||
BenchmarkSetRedigo100Conns1MB-4 2000 679542 ns/op 226 B/op 7 allocs/op
|
|
||||||
```
|
|
||||||
|
|
||||||
Redis Cluster:
|
|
||||||
|
|
||||||
```
|
|
||||||
BenchmarkRedisPing-4 200000 6983 ns/op 116 B/op 4 allocs/op
|
|
||||||
BenchmarkRedisClusterPing-4 100000 11535 ns/op 117 B/op 4 allocs/op
|
|
||||||
```
|
```
|
||||||
|
|
||||||
## See also
|
## See also
|
||||||
|
|
||||||
- [Golang PostgreSQL ORM](https://github.com/go-pg/pg)
|
- [Golang PostgreSQL ORM](https://github.com/go-pg/pg)
|
||||||
- [Golang msgpack](https://github.com/vmihailenco/msgpack)
|
- [Golang msgpack](https://github.com/vmihailenco/msgpack)
|
||||||
- [Golang message task queue](https://github.com/go-msgqueue/msgqueue)
|
- [Golang message task queue](https://github.com/vmihailenco/taskq)
|
840
vendor/github.com/go-redis/redis/cluster.go → vendor/github.com/go-redis/redis/v7/cluster.go
generated
vendored
840
vendor/github.com/go-redis/redis/cluster.go → vendor/github.com/go-redis/redis/v7/cluster.go
generated
vendored
File diff suppressed because it is too large
Load Diff
|
@ -14,7 +14,7 @@ func (c *ClusterClient) DBSize() *IntCmd {
|
||||||
return nil
|
return nil
|
||||||
})
|
})
|
||||||
if err != nil {
|
if err != nil {
|
||||||
cmd.setErr(err)
|
cmd.SetErr(err)
|
||||||
return cmd
|
return cmd
|
||||||
}
|
}
|
||||||
cmd.val = size
|
cmd.val = size
|
844
vendor/github.com/go-redis/redis/command.go → vendor/github.com/go-redis/redis/v7/command.go
generated
vendored
844
vendor/github.com/go-redis/redis/command.go → vendor/github.com/go-redis/redis/v7/command.go
generated
vendored
File diff suppressed because it is too large
Load Diff
1202
vendor/github.com/go-redis/redis/commands.go → vendor/github.com/go-redis/redis/v7/commands.go
generated
vendored
1202
vendor/github.com/go-redis/redis/commands.go → vendor/github.com/go-redis/redis/v7/commands.go
generated
vendored
File diff suppressed because it is too large
Load Diff
0
vendor/github.com/go-redis/redis/doc.go → vendor/github.com/go-redis/redis/v7/doc.go
generated
vendored
0
vendor/github.com/go-redis/redis/doc.go → vendor/github.com/go-redis/redis/v7/doc.go
generated
vendored
|
@ -0,0 +1,108 @@
|
||||||
|
package redis
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"io"
|
||||||
|
"net"
|
||||||
|
"strings"
|
||||||
|
|
||||||
|
"github.com/go-redis/redis/v7/internal/pool"
|
||||||
|
"github.com/go-redis/redis/v7/internal/proto"
|
||||||
|
)
|
||||||
|
|
||||||
|
var ErrClosed = pool.ErrClosed
|
||||||
|
|
||||||
|
type Error interface {
|
||||||
|
error
|
||||||
|
|
||||||
|
// RedisError is a no-op function but
|
||||||
|
// serves to distinguish types that are Redis
|
||||||
|
// errors from ordinary errors: a type is a
|
||||||
|
// Redis error if it has a RedisError method.
|
||||||
|
RedisError()
|
||||||
|
}
|
||||||
|
|
||||||
|
var _ Error = proto.RedisError("")
|
||||||
|
|
||||||
|
func isRetryableError(err error, retryTimeout bool) bool {
|
||||||
|
switch err {
|
||||||
|
case nil, context.Canceled, context.DeadlineExceeded:
|
||||||
|
return false
|
||||||
|
case io.EOF:
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
if netErr, ok := err.(net.Error); ok {
|
||||||
|
if netErr.Timeout() {
|
||||||
|
return retryTimeout
|
||||||
|
}
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
|
||||||
|
s := err.Error()
|
||||||
|
if s == "ERR max number of clients reached" {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
if strings.HasPrefix(s, "LOADING ") {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
if strings.HasPrefix(s, "READONLY ") {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
if strings.HasPrefix(s, "CLUSTERDOWN ") {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
|
||||||
|
func isRedisError(err error) bool {
|
||||||
|
_, ok := err.(proto.RedisError)
|
||||||
|
return ok
|
||||||
|
}
|
||||||
|
|
||||||
|
func isBadConn(err error, allowTimeout bool) bool {
|
||||||
|
if err == nil {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
if isRedisError(err) {
|
||||||
|
// Close connections in read only state in case domain addr is used
|
||||||
|
// and domain resolves to a different Redis Server. See #790.
|
||||||
|
return isReadOnlyError(err)
|
||||||
|
}
|
||||||
|
if allowTimeout {
|
||||||
|
if netErr, ok := err.(net.Error); ok && netErr.Timeout() {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
|
||||||
|
func isMovedError(err error) (moved bool, ask bool, addr string) {
|
||||||
|
if !isRedisError(err) {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
s := err.Error()
|
||||||
|
switch {
|
||||||
|
case strings.HasPrefix(s, "MOVED "):
|
||||||
|
moved = true
|
||||||
|
case strings.HasPrefix(s, "ASK "):
|
||||||
|
ask = true
|
||||||
|
default:
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
ind := strings.LastIndex(s, " ")
|
||||||
|
if ind == -1 {
|
||||||
|
return false, false, ""
|
||||||
|
}
|
||||||
|
addr = s[ind+1:]
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
func isLoadingError(err error) bool {
|
||||||
|
return strings.HasPrefix(err.Error(), "LOADING ")
|
||||||
|
}
|
||||||
|
|
||||||
|
func isReadOnlyError(err error) bool {
|
||||||
|
return strings.HasPrefix(err.Error(), "READONLY ")
|
||||||
|
}
|
|
@ -0,0 +1,15 @@
|
||||||
|
module github.com/go-redis/redis/v7
|
||||||
|
|
||||||
|
require (
|
||||||
|
github.com/golang/protobuf v1.3.2 // indirect
|
||||||
|
github.com/kr/pretty v0.1.0 // indirect
|
||||||
|
github.com/onsi/ginkgo v1.10.1
|
||||||
|
github.com/onsi/gomega v1.7.0
|
||||||
|
golang.org/x/net v0.0.0-20190923162816-aa69164e4478 // indirect
|
||||||
|
golang.org/x/sys v0.0.0-20191010194322-b09406accb47 // indirect
|
||||||
|
golang.org/x/text v0.3.2 // indirect
|
||||||
|
gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15 // indirect
|
||||||
|
gopkg.in/yaml.v2 v2.2.4 // indirect
|
||||||
|
)
|
||||||
|
|
||||||
|
go 1.11
|
|
@ -0,0 +1,47 @@
|
||||||
|
github.com/fsnotify/fsnotify v1.4.7 h1:IXs+QLmnXW2CcXuY+8Mzv/fWEsPGWxqefPtCP5CnV9I=
|
||||||
|
github.com/fsnotify/fsnotify v1.4.7/go.mod h1:jwhsz4b93w/PPRr/qN1Yymfu8t87LnFCMoQvtojpjFo=
|
||||||
|
github.com/golang/protobuf v1.2.0 h1:P3YflyNX/ehuJFLhxviNdFxQPkGK5cDcApsge1SqnvM=
|
||||||
|
github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
|
||||||
|
github.com/golang/protobuf v1.3.2 h1:6nsPYzhq5kReh6QImI3k5qWzO4PEbvbIW2cwSfR/6xs=
|
||||||
|
github.com/golang/protobuf v1.3.2/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
|
||||||
|
github.com/hpcloud/tail v1.0.0 h1:nfCOvKYfkgYP8hkirhJocXT2+zOD8yUNjXaWfTlyFKI=
|
||||||
|
github.com/hpcloud/tail v1.0.0/go.mod h1:ab1qPbhIpdTxEkNHXyeSf5vhxWSCs/tWer42PpOxQnU=
|
||||||
|
github.com/kr/pretty v0.1.0 h1:L/CwN0zerZDmRFUapSPitk6f+Q3+0za1rQkzVuMiMFI=
|
||||||
|
github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=
|
||||||
|
github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
|
||||||
|
github.com/kr/text v0.1.0 h1:45sCR5RtlFHMR4UwH9sdQ5TC8v0qDQCHnXt+kaKSTVE=
|
||||||
|
github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
|
||||||
|
github.com/onsi/ginkgo v1.6.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE=
|
||||||
|
github.com/onsi/ginkgo v1.10.1 h1:q/mM8GF/n0shIN8SaAZ0V+jnLPzen6WIVZdiwrRlMlo=
|
||||||
|
github.com/onsi/ginkgo v1.10.1/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE=
|
||||||
|
github.com/onsi/gomega v1.7.0 h1:XPnZz8VVBHjVsy1vzJmRwIcSwiUO+JFfrv/xGiigmME=
|
||||||
|
github.com/onsi/gomega v1.7.0/go.mod h1:ex+gbHU/CVuBBDIJjb2X0qEXbFg53c61hWP/1CpauHY=
|
||||||
|
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
|
||||||
|
golang.org/x/net v0.0.0-20180906233101-161cd47e91fd h1:nTDtHvHSdCn1m6ITfMRqtOd/9+7a3s8RBNOZ3eYZzJA=
|
||||||
|
golang.org/x/net v0.0.0-20180906233101-161cd47e91fd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
|
||||||
|
golang.org/x/net v0.0.0-20190923162816-aa69164e4478 h1:l5EDrHhldLYb3ZRHDUhXF7Om7MvYXnkV9/iQNo1lX6g=
|
||||||
|
golang.org/x/net v0.0.0-20190923162816-aa69164e4478/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
|
||||||
|
golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f h1:wMNYb4v58l5UBM7MYRLPG6ZhfOqbKu7X5eyFl8ZhKvA=
|
||||||
|
golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
|
||||||
|
golang.org/x/sys v0.0.0-20180909124046-d0be0721c37e h1:o3PsSEY8E4eXWkXrIP9YJALUkVZqzHJT5DOasTyn8Vs=
|
||||||
|
golang.org/x/sys v0.0.0-20180909124046-d0be0721c37e/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
|
||||||
|
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
|
||||||
|
golang.org/x/sys v0.0.0-20191010194322-b09406accb47 h1:/XfQ9z7ib8eEJX2hdgFTZJ/ntt0swNk5oYBziWeTCvY=
|
||||||
|
golang.org/x/sys v0.0.0-20191010194322-b09406accb47/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
|
||||||
|
golang.org/x/text v0.3.0 h1:g61tztE5qeGQ89tm6NTjjM9VPIm088od1l6aSorWRWg=
|
||||||
|
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
|
||||||
|
golang.org/x/text v0.3.2 h1:tW2bmiBqwgJj/UpqtC8EpXEZVYOwU0yG4iWbprSVAcs=
|
||||||
|
golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk=
|
||||||
|
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
|
||||||
|
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
|
||||||
|
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
|
||||||
|
gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15 h1:YR8cESwS4TdDjEe65xsg0ogRM/Nc3DYOhEAlW+xobZo=
|
||||||
|
gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
|
||||||
|
gopkg.in/fsnotify.v1 v1.4.7 h1:xOHLXZwVvI9hhs+cLKq5+I5onOuwQLhQwiu63xxlHs4=
|
||||||
|
gopkg.in/fsnotify.v1 v1.4.7/go.mod h1:Tz8NjZHkW78fSQdbUxIjBTcgA1z1m8ZHf0WmKUhAMys=
|
||||||
|
gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7 h1:uRGJdciOHaEIrze2W8Q3AKkepLTh2hOroT7a+7czfdQ=
|
||||||
|
gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7/go.mod h1:dt/ZhP58zS4L8KSrWDmTeBkI65Dw0HsyUHuEVlX15mw=
|
||||||
|
gopkg.in/yaml.v2 v2.2.1 h1:mUhvW9EsL+naU5Q3cakzfE91YhliOondGd6ZrsDBHQE=
|
||||||
|
gopkg.in/yaml.v2 v2.2.1/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
|
||||||
|
gopkg.in/yaml.v2 v2.2.4 h1:/eiJrUcujPVeJ3xlSWaiNi3uSVmDGBK1pDHUHAnao1I=
|
||||||
|
gopkg.in/yaml.v2 v2.2.4/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
|
|
@ -0,0 +1,8 @@
|
||||||
|
package internal
|
||||||
|
|
||||||
|
import (
|
||||||
|
"log"
|
||||||
|
"os"
|
||||||
|
)
|
||||||
|
|
||||||
|
var Logger = log.New(os.Stderr, "redis: ", log.LstdFlags|log.Lshortfile)
|
|
@ -0,0 +1,118 @@
|
||||||
|
package pool
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"net"
|
||||||
|
"sync/atomic"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/go-redis/redis/v7/internal/proto"
|
||||||
|
)
|
||||||
|
|
||||||
|
var noDeadline = time.Time{}
|
||||||
|
|
||||||
|
type Conn struct {
|
||||||
|
netConn net.Conn
|
||||||
|
|
||||||
|
rd *proto.Reader
|
||||||
|
wr *proto.Writer
|
||||||
|
|
||||||
|
Inited bool
|
||||||
|
pooled bool
|
||||||
|
createdAt time.Time
|
||||||
|
usedAt int64 // atomic
|
||||||
|
}
|
||||||
|
|
||||||
|
func NewConn(netConn net.Conn) *Conn {
|
||||||
|
cn := &Conn{
|
||||||
|
netConn: netConn,
|
||||||
|
createdAt: time.Now(),
|
||||||
|
}
|
||||||
|
cn.rd = proto.NewReader(netConn)
|
||||||
|
cn.wr = proto.NewWriter(netConn)
|
||||||
|
cn.SetUsedAt(time.Now())
|
||||||
|
return cn
|
||||||
|
}
|
||||||
|
|
||||||
|
func (cn *Conn) UsedAt() time.Time {
|
||||||
|
unix := atomic.LoadInt64(&cn.usedAt)
|
||||||
|
return time.Unix(unix, 0)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (cn *Conn) SetUsedAt(tm time.Time) {
|
||||||
|
atomic.StoreInt64(&cn.usedAt, tm.Unix())
|
||||||
|
}
|
||||||
|
|
||||||
|
func (cn *Conn) SetNetConn(netConn net.Conn) {
|
||||||
|
cn.netConn = netConn
|
||||||
|
cn.rd.Reset(netConn)
|
||||||
|
cn.wr.Reset(netConn)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (cn *Conn) Write(b []byte) (int, error) {
|
||||||
|
return cn.netConn.Write(b)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (cn *Conn) RemoteAddr() net.Addr {
|
||||||
|
return cn.netConn.RemoteAddr()
|
||||||
|
}
|
||||||
|
|
||||||
|
func (cn *Conn) WithReader(ctx context.Context, timeout time.Duration, fn func(rd *proto.Reader) error) error {
|
||||||
|
err := cn.netConn.SetReadDeadline(cn.deadline(ctx, timeout))
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
return fn(cn.rd)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (cn *Conn) WithWriter(
|
||||||
|
ctx context.Context, timeout time.Duration, fn func(wr *proto.Writer) error,
|
||||||
|
) error {
|
||||||
|
err := cn.netConn.SetWriteDeadline(cn.deadline(ctx, timeout))
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
if cn.wr.Buffered() > 0 {
|
||||||
|
cn.wr.Reset(cn.netConn)
|
||||||
|
}
|
||||||
|
|
||||||
|
err = fn(cn.wr)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
return cn.wr.Flush()
|
||||||
|
}
|
||||||
|
|
||||||
|
func (cn *Conn) Close() error {
|
||||||
|
return cn.netConn.Close()
|
||||||
|
}
|
||||||
|
|
||||||
|
func (cn *Conn) deadline(ctx context.Context, timeout time.Duration) time.Time {
|
||||||
|
tm := time.Now()
|
||||||
|
cn.SetUsedAt(tm)
|
||||||
|
|
||||||
|
if timeout > 0 {
|
||||||
|
tm = tm.Add(timeout)
|
||||||
|
}
|
||||||
|
|
||||||
|
if ctx != nil {
|
||||||
|
deadline, ok := ctx.Deadline()
|
||||||
|
if ok {
|
||||||
|
if timeout == 0 {
|
||||||
|
return deadline
|
||||||
|
}
|
||||||
|
if deadline.Before(tm) {
|
||||||
|
return deadline
|
||||||
|
}
|
||||||
|
return tm
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if timeout > 0 {
|
||||||
|
return tm
|
||||||
|
}
|
||||||
|
|
||||||
|
return noDeadline
|
||||||
|
}
|
|
@ -1,13 +1,14 @@
|
||||||
package pool
|
package pool
|
||||||
|
|
||||||
import (
|
import (
|
||||||
|
"context"
|
||||||
"errors"
|
"errors"
|
||||||
"net"
|
"net"
|
||||||
"sync"
|
"sync"
|
||||||
"sync/atomic"
|
"sync/atomic"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
"github.com/go-redis/redis/internal"
|
"github.com/go-redis/redis/v7/internal"
|
||||||
)
|
)
|
||||||
|
|
||||||
var ErrClosed = errors.New("redis: client is closed")
|
var ErrClosed = errors.New("redis: client is closed")
|
||||||
|
@ -33,12 +34,12 @@ type Stats struct {
|
||||||
}
|
}
|
||||||
|
|
||||||
type Pooler interface {
|
type Pooler interface {
|
||||||
NewConn() (*Conn, error)
|
NewConn(context.Context) (*Conn, error)
|
||||||
CloseConn(*Conn) error
|
CloseConn(*Conn) error
|
||||||
|
|
||||||
Get() (*Conn, error)
|
Get(context.Context) (*Conn, error)
|
||||||
Put(*Conn)
|
Put(*Conn)
|
||||||
Remove(*Conn)
|
Remove(*Conn, error)
|
||||||
|
|
||||||
Len() int
|
Len() int
|
||||||
IdleLen() int
|
IdleLen() int
|
||||||
|
@ -48,7 +49,7 @@ type Pooler interface {
|
||||||
}
|
}
|
||||||
|
|
||||||
type Options struct {
|
type Options struct {
|
||||||
Dialer func() (net.Conn, error)
|
Dialer func(context.Context) (net.Conn, error)
|
||||||
OnClose func(*Conn) error
|
OnClose func(*Conn) error
|
||||||
|
|
||||||
PoolSize int
|
PoolSize int
|
||||||
|
@ -78,6 +79,7 @@ type ConnPool struct {
|
||||||
stats Stats
|
stats Stats
|
||||||
|
|
||||||
_closed uint32 // atomic
|
_closed uint32 // atomic
|
||||||
|
closedCh chan struct{}
|
||||||
}
|
}
|
||||||
|
|
||||||
var _ Pooler = (*ConnPool)(nil)
|
var _ Pooler = (*ConnPool)(nil)
|
||||||
|
@ -89,11 +91,12 @@ func NewConnPool(opt *Options) *ConnPool {
|
||||||
queue: make(chan struct{}, opt.PoolSize),
|
queue: make(chan struct{}, opt.PoolSize),
|
||||||
conns: make([]*Conn, 0, opt.PoolSize),
|
conns: make([]*Conn, 0, opt.PoolSize),
|
||||||
idleConns: make([]*Conn, 0, opt.PoolSize),
|
idleConns: make([]*Conn, 0, opt.PoolSize),
|
||||||
|
closedCh: make(chan struct{}),
|
||||||
}
|
}
|
||||||
|
|
||||||
for i := 0; i < opt.MinIdleConns; i++ {
|
p.connsMu.Lock()
|
||||||
p.checkMinIdleConns()
|
p.checkMinIdleConns()
|
||||||
}
|
p.connsMu.Unlock()
|
||||||
|
|
||||||
if opt.IdleTimeout > 0 && opt.IdleCheckFrequency > 0 {
|
if opt.IdleTimeout > 0 && opt.IdleCheckFrequency > 0 {
|
||||||
go p.reaper(opt.IdleCheckFrequency)
|
go p.reaper(opt.IdleCheckFrequency)
|
||||||
|
@ -106,31 +109,40 @@ func (p *ConnPool) checkMinIdleConns() {
|
||||||
if p.opt.MinIdleConns == 0 {
|
if p.opt.MinIdleConns == 0 {
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
if p.poolSize < p.opt.PoolSize && p.idleConnsLen < p.opt.MinIdleConns {
|
for p.poolSize < p.opt.PoolSize && p.idleConnsLen < p.opt.MinIdleConns {
|
||||||
p.poolSize++
|
p.poolSize++
|
||||||
p.idleConnsLen++
|
p.idleConnsLen++
|
||||||
go p.addIdleConn()
|
go func() {
|
||||||
|
err := p.addIdleConn()
|
||||||
|
if err != nil {
|
||||||
|
p.connsMu.Lock()
|
||||||
|
p.poolSize--
|
||||||
|
p.idleConnsLen--
|
||||||
|
p.connsMu.Unlock()
|
||||||
|
}
|
||||||
|
}()
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func (p *ConnPool) addIdleConn() {
|
func (p *ConnPool) addIdleConn() error {
|
||||||
cn, err := p.newConn(true)
|
cn, err := p.dialConn(context.TODO(), true)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return
|
return err
|
||||||
}
|
}
|
||||||
|
|
||||||
p.connsMu.Lock()
|
p.connsMu.Lock()
|
||||||
p.conns = append(p.conns, cn)
|
p.conns = append(p.conns, cn)
|
||||||
p.idleConns = append(p.idleConns, cn)
|
p.idleConns = append(p.idleConns, cn)
|
||||||
p.connsMu.Unlock()
|
p.connsMu.Unlock()
|
||||||
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
func (p *ConnPool) NewConn() (*Conn, error) {
|
func (p *ConnPool) NewConn(ctx context.Context) (*Conn, error) {
|
||||||
return p._NewConn(false)
|
return p.newConn(ctx, false)
|
||||||
}
|
}
|
||||||
|
|
||||||
func (p *ConnPool) _NewConn(pooled bool) (*Conn, error) {
|
func (p *ConnPool) newConn(ctx context.Context, pooled bool) (*Conn, error) {
|
||||||
cn, err := p.newConn(pooled)
|
cn, err := p.dialConn(ctx, pooled)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
|
@ -138,17 +150,18 @@ func (p *ConnPool) _NewConn(pooled bool) (*Conn, error) {
|
||||||
p.connsMu.Lock()
|
p.connsMu.Lock()
|
||||||
p.conns = append(p.conns, cn)
|
p.conns = append(p.conns, cn)
|
||||||
if pooled {
|
if pooled {
|
||||||
if p.poolSize < p.opt.PoolSize {
|
// If pool is full remove the cn on next Put.
|
||||||
p.poolSize++
|
if p.poolSize >= p.opt.PoolSize {
|
||||||
} else {
|
|
||||||
cn.pooled = false
|
cn.pooled = false
|
||||||
|
} else {
|
||||||
|
p.poolSize++
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
p.connsMu.Unlock()
|
p.connsMu.Unlock()
|
||||||
return cn, nil
|
return cn, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
func (p *ConnPool) newConn(pooled bool) (*Conn, error) {
|
func (p *ConnPool) dialConn(ctx context.Context, pooled bool) (*Conn, error) {
|
||||||
if p.closed() {
|
if p.closed() {
|
||||||
return nil, ErrClosed
|
return nil, ErrClosed
|
||||||
}
|
}
|
||||||
|
@ -157,7 +170,7 @@ func (p *ConnPool) newConn(pooled bool) (*Conn, error) {
|
||||||
return nil, p.getLastDialError()
|
return nil, p.getLastDialError()
|
||||||
}
|
}
|
||||||
|
|
||||||
netConn, err := p.opt.Dialer()
|
netConn, err := p.opt.Dialer(ctx)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
p.setLastDialError(err)
|
p.setLastDialError(err)
|
||||||
if atomic.AddUint32(&p.dialErrorsNum, 1) == uint32(p.opt.PoolSize) {
|
if atomic.AddUint32(&p.dialErrorsNum, 1) == uint32(p.opt.PoolSize) {
|
||||||
|
@ -177,7 +190,7 @@ func (p *ConnPool) tryDial() {
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
|
||||||
conn, err := p.opt.Dialer()
|
conn, err := p.opt.Dialer(context.Background())
|
||||||
if err != nil {
|
if err != nil {
|
||||||
p.setLastDialError(err)
|
p.setLastDialError(err)
|
||||||
time.Sleep(time.Second)
|
time.Sleep(time.Second)
|
||||||
|
@ -204,12 +217,12 @@ func (p *ConnPool) getLastDialError() error {
|
||||||
}
|
}
|
||||||
|
|
||||||
// Get returns existed connection from the pool or creates a new one.
|
// Get returns existed connection from the pool or creates a new one.
|
||||||
func (p *ConnPool) Get() (*Conn, error) {
|
func (p *ConnPool) Get(ctx context.Context) (*Conn, error) {
|
||||||
if p.closed() {
|
if p.closed() {
|
||||||
return nil, ErrClosed
|
return nil, ErrClosed
|
||||||
}
|
}
|
||||||
|
|
||||||
err := p.waitTurn()
|
err := p.waitTurn(ctx)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
|
@ -234,7 +247,7 @@ func (p *ConnPool) Get() (*Conn, error) {
|
||||||
|
|
||||||
atomic.AddUint32(&p.stats.Misses, 1)
|
atomic.AddUint32(&p.stats.Misses, 1)
|
||||||
|
|
||||||
newcn, err := p._NewConn(true)
|
newcn, err := p.newConn(ctx, true)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
p.freeTurn()
|
p.freeTurn()
|
||||||
return nil, err
|
return nil, err
|
||||||
|
@ -247,15 +260,29 @@ func (p *ConnPool) getTurn() {
|
||||||
p.queue <- struct{}{}
|
p.queue <- struct{}{}
|
||||||
}
|
}
|
||||||
|
|
||||||
func (p *ConnPool) waitTurn() error {
|
func (p *ConnPool) waitTurn(ctx context.Context) error {
|
||||||
|
select {
|
||||||
|
case <-ctx.Done():
|
||||||
|
return ctx.Err()
|
||||||
|
default:
|
||||||
|
}
|
||||||
|
|
||||||
select {
|
select {
|
||||||
case p.queue <- struct{}{}:
|
case p.queue <- struct{}{}:
|
||||||
return nil
|
return nil
|
||||||
default:
|
default:
|
||||||
|
}
|
||||||
|
|
||||||
timer := timers.Get().(*time.Timer)
|
timer := timers.Get().(*time.Timer)
|
||||||
timer.Reset(p.opt.PoolTimeout)
|
timer.Reset(p.opt.PoolTimeout)
|
||||||
|
|
||||||
select {
|
select {
|
||||||
|
case <-ctx.Done():
|
||||||
|
if !timer.Stop() {
|
||||||
|
<-timer.C
|
||||||
|
}
|
||||||
|
timers.Put(timer)
|
||||||
|
return ctx.Err()
|
||||||
case p.queue <- struct{}{}:
|
case p.queue <- struct{}{}:
|
||||||
if !timer.Stop() {
|
if !timer.Stop() {
|
||||||
<-timer.C
|
<-timer.C
|
||||||
|
@ -267,7 +294,6 @@ func (p *ConnPool) waitTurn() error {
|
||||||
atomic.AddUint32(&p.stats.Timeouts, 1)
|
atomic.AddUint32(&p.stats.Timeouts, 1)
|
||||||
return ErrPoolTimeout
|
return ErrPoolTimeout
|
||||||
}
|
}
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
func (p *ConnPool) freeTurn() {
|
func (p *ConnPool) freeTurn() {
|
||||||
|
@ -288,8 +314,14 @@ func (p *ConnPool) popIdle() *Conn {
|
||||||
}
|
}
|
||||||
|
|
||||||
func (p *ConnPool) Put(cn *Conn) {
|
func (p *ConnPool) Put(cn *Conn) {
|
||||||
|
if cn.rd.Buffered() > 0 {
|
||||||
|
internal.Logger.Printf("Conn has unread data")
|
||||||
|
p.Remove(cn, BadConnError{})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
if !cn.pooled {
|
if !cn.pooled {
|
||||||
p.Remove(cn)
|
p.Remove(cn, nil)
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
|
||||||
|
@ -300,19 +332,24 @@ func (p *ConnPool) Put(cn *Conn) {
|
||||||
p.freeTurn()
|
p.freeTurn()
|
||||||
}
|
}
|
||||||
|
|
||||||
func (p *ConnPool) Remove(cn *Conn) {
|
func (p *ConnPool) Remove(cn *Conn, reason error) {
|
||||||
p.removeConn(cn)
|
p.removeConnWithLock(cn)
|
||||||
p.freeTurn()
|
p.freeTurn()
|
||||||
_ = p.closeConn(cn)
|
_ = p.closeConn(cn)
|
||||||
}
|
}
|
||||||
|
|
||||||
func (p *ConnPool) CloseConn(cn *Conn) error {
|
func (p *ConnPool) CloseConn(cn *Conn) error {
|
||||||
p.removeConn(cn)
|
p.removeConnWithLock(cn)
|
||||||
return p.closeConn(cn)
|
return p.closeConn(cn)
|
||||||
}
|
}
|
||||||
|
|
||||||
func (p *ConnPool) removeConn(cn *Conn) {
|
func (p *ConnPool) removeConnWithLock(cn *Conn) {
|
||||||
p.connsMu.Lock()
|
p.connsMu.Lock()
|
||||||
|
p.removeConn(cn)
|
||||||
|
p.connsMu.Unlock()
|
||||||
|
}
|
||||||
|
|
||||||
|
func (p *ConnPool) removeConn(cn *Conn) {
|
||||||
for i, c := range p.conns {
|
for i, c := range p.conns {
|
||||||
if c == cn {
|
if c == cn {
|
||||||
p.conns = append(p.conns[:i], p.conns[i+1:]...)
|
p.conns = append(p.conns[:i], p.conns[i+1:]...)
|
||||||
|
@ -320,10 +357,9 @@ func (p *ConnPool) removeConn(cn *Conn) {
|
||||||
p.poolSize--
|
p.poolSize--
|
||||||
p.checkMinIdleConns()
|
p.checkMinIdleConns()
|
||||||
}
|
}
|
||||||
break
|
return
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
p.connsMu.Unlock()
|
|
||||||
}
|
}
|
||||||
|
|
||||||
func (p *ConnPool) closeConn(cn *Conn) error {
|
func (p *ConnPool) closeConn(cn *Conn) error {
|
||||||
|
@ -384,6 +420,7 @@ func (p *ConnPool) Close() error {
|
||||||
if !atomic.CompareAndSwapUint32(&p._closed, 0, 1) {
|
if !atomic.CompareAndSwapUint32(&p._closed, 0, 1) {
|
||||||
return ErrClosed
|
return ErrClosed
|
||||||
}
|
}
|
||||||
|
close(p.closedCh)
|
||||||
|
|
||||||
var firstErr error
|
var firstErr error
|
||||||
p.connsMu.Lock()
|
p.connsMu.Lock()
|
||||||
|
@ -401,6 +438,51 @@ func (p *ConnPool) Close() error {
|
||||||
return firstErr
|
return firstErr
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func (p *ConnPool) reaper(frequency time.Duration) {
|
||||||
|
ticker := time.NewTicker(frequency)
|
||||||
|
defer ticker.Stop()
|
||||||
|
|
||||||
|
for {
|
||||||
|
select {
|
||||||
|
case <-ticker.C:
|
||||||
|
// It is possible that ticker and closedCh arrive together,
|
||||||
|
// and select pseudo-randomly pick ticker case, we double
|
||||||
|
// check here to prevent being executed after closed.
|
||||||
|
if p.closed() {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
_, err := p.ReapStaleConns()
|
||||||
|
if err != nil {
|
||||||
|
internal.Logger.Printf("ReapStaleConns failed: %s", err)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
case <-p.closedCh:
|
||||||
|
return
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func (p *ConnPool) ReapStaleConns() (int, error) {
|
||||||
|
var n int
|
||||||
|
for {
|
||||||
|
p.getTurn()
|
||||||
|
|
||||||
|
p.connsMu.Lock()
|
||||||
|
cn := p.reapStaleConn()
|
||||||
|
p.connsMu.Unlock()
|
||||||
|
p.freeTurn()
|
||||||
|
|
||||||
|
if cn != nil {
|
||||||
|
_ = p.closeConn(cn)
|
||||||
|
n++
|
||||||
|
} else {
|
||||||
|
break
|
||||||
|
}
|
||||||
|
}
|
||||||
|
atomic.AddUint32(&p.stats.StaleConns, uint32(n))
|
||||||
|
return n, nil
|
||||||
|
}
|
||||||
|
|
||||||
func (p *ConnPool) reapStaleConn() *Conn {
|
func (p *ConnPool) reapStaleConn() *Conn {
|
||||||
if len(p.idleConns) == 0 {
|
if len(p.idleConns) == 0 {
|
||||||
return nil
|
return nil
|
||||||
|
@ -413,52 +495,11 @@ func (p *ConnPool) reapStaleConn() *Conn {
|
||||||
|
|
||||||
p.idleConns = append(p.idleConns[:0], p.idleConns[1:]...)
|
p.idleConns = append(p.idleConns[:0], p.idleConns[1:]...)
|
||||||
p.idleConnsLen--
|
p.idleConnsLen--
|
||||||
|
p.removeConn(cn)
|
||||||
|
|
||||||
return cn
|
return cn
|
||||||
}
|
}
|
||||||
|
|
||||||
func (p *ConnPool) ReapStaleConns() (int, error) {
|
|
||||||
var n int
|
|
||||||
for {
|
|
||||||
p.getTurn()
|
|
||||||
|
|
||||||
p.connsMu.Lock()
|
|
||||||
cn := p.reapStaleConn()
|
|
||||||
p.connsMu.Unlock()
|
|
||||||
|
|
||||||
if cn != nil {
|
|
||||||
p.removeConn(cn)
|
|
||||||
}
|
|
||||||
|
|
||||||
p.freeTurn()
|
|
||||||
|
|
||||||
if cn != nil {
|
|
||||||
p.closeConn(cn)
|
|
||||||
n++
|
|
||||||
} else {
|
|
||||||
break
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return n, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func (p *ConnPool) reaper(frequency time.Duration) {
|
|
||||||
ticker := time.NewTicker(frequency)
|
|
||||||
defer ticker.Stop()
|
|
||||||
|
|
||||||
for range ticker.C {
|
|
||||||
if p.closed() {
|
|
||||||
break
|
|
||||||
}
|
|
||||||
n, err := p.ReapStaleConns()
|
|
||||||
if err != nil {
|
|
||||||
internal.Logf("ReapStaleConns failed: %s", err)
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
atomic.AddUint32(&p.stats.StaleConns, uint32(n))
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func (p *ConnPool) isStaleConn(cn *Conn) bool {
|
func (p *ConnPool) isStaleConn(cn *Conn) bool {
|
||||||
if p.opt.IdleTimeout == 0 && p.opt.MaxConnAge == 0 {
|
if p.opt.IdleTimeout == 0 && p.opt.MaxConnAge == 0 {
|
||||||
return false
|
return false
|
||||||
|
@ -468,7 +509,7 @@ func (p *ConnPool) isStaleConn(cn *Conn) bool {
|
||||||
if p.opt.IdleTimeout > 0 && now.Sub(cn.UsedAt()) >= p.opt.IdleTimeout {
|
if p.opt.IdleTimeout > 0 && now.Sub(cn.UsedAt()) >= p.opt.IdleTimeout {
|
||||||
return true
|
return true
|
||||||
}
|
}
|
||||||
if p.opt.MaxConnAge > 0 && now.Sub(cn.InitedAt) >= p.opt.MaxConnAge {
|
if p.opt.MaxConnAge > 0 && now.Sub(cn.createdAt) >= p.opt.MaxConnAge {
|
||||||
return true
|
return true
|
||||||
}
|
}
|
||||||
|
|
|
@ -0,0 +1,208 @@
|
||||||
|
package pool
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"fmt"
|
||||||
|
"sync/atomic"
|
||||||
|
)
|
||||||
|
|
||||||
|
const (
|
||||||
|
stateDefault = 0
|
||||||
|
stateInited = 1
|
||||||
|
stateClosed = 2
|
||||||
|
)
|
||||||
|
|
||||||
|
type BadConnError struct {
|
||||||
|
wrapped error
|
||||||
|
}
|
||||||
|
|
||||||
|
var _ error = (*BadConnError)(nil)
|
||||||
|
|
||||||
|
func (e BadConnError) Error() string {
|
||||||
|
s := "redis: Conn is in a bad state"
|
||||||
|
if e.wrapped != nil {
|
||||||
|
s += ": " + e.wrapped.Error()
|
||||||
|
}
|
||||||
|
return s
|
||||||
|
}
|
||||||
|
|
||||||
|
func (e BadConnError) Unwrap() error {
|
||||||
|
return e.wrapped
|
||||||
|
}
|
||||||
|
|
||||||
|
type SingleConnPool struct {
|
||||||
|
pool Pooler
|
||||||
|
level int32 // atomic
|
||||||
|
|
||||||
|
state uint32 // atomic
|
||||||
|
ch chan *Conn
|
||||||
|
|
||||||
|
_badConnError atomic.Value
|
||||||
|
}
|
||||||
|
|
||||||
|
var _ Pooler = (*SingleConnPool)(nil)
|
||||||
|
|
||||||
|
func NewSingleConnPool(pool Pooler) *SingleConnPool {
|
||||||
|
p, ok := pool.(*SingleConnPool)
|
||||||
|
if !ok {
|
||||||
|
p = &SingleConnPool{
|
||||||
|
pool: pool,
|
||||||
|
ch: make(chan *Conn, 1),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
atomic.AddInt32(&p.level, 1)
|
||||||
|
return p
|
||||||
|
}
|
||||||
|
|
||||||
|
func (p *SingleConnPool) SetConn(cn *Conn) {
|
||||||
|
if atomic.CompareAndSwapUint32(&p.state, stateDefault, stateInited) {
|
||||||
|
p.ch <- cn
|
||||||
|
} else {
|
||||||
|
panic("not reached")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func (p *SingleConnPool) NewConn(ctx context.Context) (*Conn, error) {
|
||||||
|
return p.pool.NewConn(ctx)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (p *SingleConnPool) CloseConn(cn *Conn) error {
|
||||||
|
return p.pool.CloseConn(cn)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (p *SingleConnPool) Get(ctx context.Context) (*Conn, error) {
|
||||||
|
// In worst case this races with Close which is not a very common operation.
|
||||||
|
for i := 0; i < 1000; i++ {
|
||||||
|
switch atomic.LoadUint32(&p.state) {
|
||||||
|
case stateDefault:
|
||||||
|
cn, err := p.pool.Get(ctx)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
if atomic.CompareAndSwapUint32(&p.state, stateDefault, stateInited) {
|
||||||
|
return cn, nil
|
||||||
|
}
|
||||||
|
p.pool.Remove(cn, ErrClosed)
|
||||||
|
case stateInited:
|
||||||
|
if err := p.badConnError(); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
cn, ok := <-p.ch
|
||||||
|
if !ok {
|
||||||
|
return nil, ErrClosed
|
||||||
|
}
|
||||||
|
return cn, nil
|
||||||
|
case stateClosed:
|
||||||
|
return nil, ErrClosed
|
||||||
|
default:
|
||||||
|
panic("not reached")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return nil, fmt.Errorf("redis: SingleConnPool.Get: infinite loop")
|
||||||
|
}
|
||||||
|
|
||||||
|
func (p *SingleConnPool) Put(cn *Conn) {
|
||||||
|
defer func() {
|
||||||
|
if recover() != nil {
|
||||||
|
p.freeConn(cn)
|
||||||
|
}
|
||||||
|
}()
|
||||||
|
p.ch <- cn
|
||||||
|
}
|
||||||
|
|
||||||
|
func (p *SingleConnPool) freeConn(cn *Conn) {
|
||||||
|
if err := p.badConnError(); err != nil {
|
||||||
|
p.pool.Remove(cn, err)
|
||||||
|
} else {
|
||||||
|
p.pool.Put(cn)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func (p *SingleConnPool) Remove(cn *Conn, reason error) {
|
||||||
|
defer func() {
|
||||||
|
if recover() != nil {
|
||||||
|
p.pool.Remove(cn, ErrClosed)
|
||||||
|
}
|
||||||
|
}()
|
||||||
|
p._badConnError.Store(BadConnError{wrapped: reason})
|
||||||
|
p.ch <- cn
|
||||||
|
}
|
||||||
|
|
||||||
|
func (p *SingleConnPool) Len() int {
|
||||||
|
switch atomic.LoadUint32(&p.state) {
|
||||||
|
case stateDefault:
|
||||||
|
return 0
|
||||||
|
case stateInited:
|
||||||
|
return 1
|
||||||
|
case stateClosed:
|
||||||
|
return 0
|
||||||
|
default:
|
||||||
|
panic("not reached")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func (p *SingleConnPool) IdleLen() int {
|
||||||
|
return len(p.ch)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (p *SingleConnPool) Stats() *Stats {
|
||||||
|
return &Stats{}
|
||||||
|
}
|
||||||
|
|
||||||
|
func (p *SingleConnPool) Close() error {
|
||||||
|
level := atomic.AddInt32(&p.level, -1)
|
||||||
|
if level > 0 {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
for i := 0; i < 1000; i++ {
|
||||||
|
state := atomic.LoadUint32(&p.state)
|
||||||
|
if state == stateClosed {
|
||||||
|
return ErrClosed
|
||||||
|
}
|
||||||
|
if atomic.CompareAndSwapUint32(&p.state, state, stateClosed) {
|
||||||
|
close(p.ch)
|
||||||
|
cn, ok := <-p.ch
|
||||||
|
if ok {
|
||||||
|
p.freeConn(cn)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return fmt.Errorf("redis: SingleConnPool.Close: infinite loop")
|
||||||
|
}
|
||||||
|
|
||||||
|
func (p *SingleConnPool) Reset() error {
|
||||||
|
if p.badConnError() == nil {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
select {
|
||||||
|
case cn, ok := <-p.ch:
|
||||||
|
if !ok {
|
||||||
|
return ErrClosed
|
||||||
|
}
|
||||||
|
p.pool.Remove(cn, ErrClosed)
|
||||||
|
p._badConnError.Store(BadConnError{wrapped: nil})
|
||||||
|
default:
|
||||||
|
return fmt.Errorf("redis: SingleConnPool does not have a Conn")
|
||||||
|
}
|
||||||
|
|
||||||
|
if !atomic.CompareAndSwapUint32(&p.state, stateInited, stateDefault) {
|
||||||
|
state := atomic.LoadUint32(&p.state)
|
||||||
|
return fmt.Errorf("redis: invalid SingleConnPool state: %d", state)
|
||||||
|
}
|
||||||
|
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (p *SingleConnPool) badConnError() error {
|
||||||
|
if v := p._badConnError.Load(); v != nil {
|
||||||
|
err := v.(BadConnError)
|
||||||
|
if err.wrapped != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
|
@ -1,6 +1,9 @@
|
||||||
package pool
|
package pool
|
||||||
|
|
||||||
import "sync"
|
import (
|
||||||
|
"context"
|
||||||
|
"sync"
|
||||||
|
)
|
||||||
|
|
||||||
type StickyConnPool struct {
|
type StickyConnPool struct {
|
||||||
pool *ConnPool
|
pool *ConnPool
|
||||||
|
@ -20,7 +23,7 @@ func NewStickyConnPool(pool *ConnPool, reusable bool) *StickyConnPool {
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func (p *StickyConnPool) NewConn() (*Conn, error) {
|
func (p *StickyConnPool) NewConn(context.Context) (*Conn, error) {
|
||||||
panic("not implemented")
|
panic("not implemented")
|
||||||
}
|
}
|
||||||
|
|
||||||
|
@ -28,7 +31,7 @@ func (p *StickyConnPool) CloseConn(*Conn) error {
|
||||||
panic("not implemented")
|
panic("not implemented")
|
||||||
}
|
}
|
||||||
|
|
||||||
func (p *StickyConnPool) Get() (*Conn, error) {
|
func (p *StickyConnPool) Get(ctx context.Context) (*Conn, error) {
|
||||||
p.mu.Lock()
|
p.mu.Lock()
|
||||||
defer p.mu.Unlock()
|
defer p.mu.Unlock()
|
||||||
|
|
||||||
|
@ -39,7 +42,7 @@ func (p *StickyConnPool) Get() (*Conn, error) {
|
||||||
return p.cn, nil
|
return p.cn, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
cn, err := p.pool.Get()
|
cn, err := p.pool.Get(ctx)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
|
@ -55,13 +58,13 @@ func (p *StickyConnPool) putUpstream() {
|
||||||
|
|
||||||
func (p *StickyConnPool) Put(cn *Conn) {}
|
func (p *StickyConnPool) Put(cn *Conn) {}
|
||||||
|
|
||||||
func (p *StickyConnPool) removeUpstream() {
|
func (p *StickyConnPool) removeUpstream(reason error) {
|
||||||
p.pool.Remove(p.cn)
|
p.pool.Remove(p.cn, reason)
|
||||||
p.cn = nil
|
p.cn = nil
|
||||||
}
|
}
|
||||||
|
|
||||||
func (p *StickyConnPool) Remove(cn *Conn) {
|
func (p *StickyConnPool) Remove(cn *Conn, reason error) {
|
||||||
p.removeUpstream()
|
p.removeUpstream(reason)
|
||||||
}
|
}
|
||||||
|
|
||||||
func (p *StickyConnPool) Len() int {
|
func (p *StickyConnPool) Len() int {
|
||||||
|
@ -101,7 +104,7 @@ func (p *StickyConnPool) Close() error {
|
||||||
if p.reusable {
|
if p.reusable {
|
||||||
p.putUpstream()
|
p.putUpstream()
|
||||||
} else {
|
} else {
|
||||||
p.removeUpstream()
|
p.removeUpstream(ErrClosed)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
|
@ -4,9 +4,8 @@ import (
|
||||||
"bufio"
|
"bufio"
|
||||||
"fmt"
|
"fmt"
|
||||||
"io"
|
"io"
|
||||||
"strconv"
|
|
||||||
|
|
||||||
"github.com/go-redis/redis/internal/util"
|
"github.com/go-redis/redis/v7/internal/util"
|
||||||
)
|
)
|
||||||
|
|
||||||
const (
|
const (
|
||||||
|
@ -25,6 +24,8 @@ type RedisError string
|
||||||
|
|
||||||
func (e RedisError) Error() string { return string(e) }
|
func (e RedisError) Error() string { return string(e) }
|
||||||
|
|
||||||
|
func (RedisError) RedisError() {}
|
||||||
|
|
||||||
//------------------------------------------------------------------------------
|
//------------------------------------------------------------------------------
|
||||||
|
|
||||||
type MultiBulkParse func(*Reader, int64) (interface{}, error)
|
type MultiBulkParse func(*Reader, int64) (interface{}, error)
|
||||||
|
@ -41,27 +42,44 @@ func NewReader(rd io.Reader) *Reader {
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func (r *Reader) Buffered() int {
|
||||||
|
return r.rd.Buffered()
|
||||||
|
}
|
||||||
|
|
||||||
|
func (r *Reader) Peek(n int) ([]byte, error) {
|
||||||
|
return r.rd.Peek(n)
|
||||||
|
}
|
||||||
|
|
||||||
func (r *Reader) Reset(rd io.Reader) {
|
func (r *Reader) Reset(rd io.Reader) {
|
||||||
r.rd.Reset(rd)
|
r.rd.Reset(rd)
|
||||||
}
|
}
|
||||||
|
|
||||||
func (r *Reader) ReadLine() ([]byte, error) {
|
func (r *Reader) ReadLine() ([]byte, error) {
|
||||||
line, isPrefix, err := r.rd.ReadLine()
|
line, err := r.readLine()
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
if isPrefix {
|
|
||||||
return nil, bufio.ErrBufferFull
|
|
||||||
}
|
|
||||||
if len(line) == 0 {
|
|
||||||
return nil, fmt.Errorf("redis: reply is empty")
|
|
||||||
}
|
|
||||||
if isNilReply(line) {
|
if isNilReply(line) {
|
||||||
return nil, Nil
|
return nil, Nil
|
||||||
}
|
}
|
||||||
return line, nil
|
return line, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// readLine that returns an error if:
|
||||||
|
// - there is a pending read error;
|
||||||
|
// - or line does not end with \r\n.
|
||||||
|
func (r *Reader) readLine() ([]byte, error) {
|
||||||
|
b, err := r.rd.ReadSlice('\n')
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
if len(b) <= 2 || b[len(b)-1] != '\n' || b[len(b)-2] != '\r' {
|
||||||
|
return nil, fmt.Errorf("redis: invalid reply: %q", b)
|
||||||
|
}
|
||||||
|
b = b[:len(b)-2]
|
||||||
|
return b, nil
|
||||||
|
}
|
||||||
|
|
||||||
func (r *Reader) ReadReply(m MultiBulkParse) (interface{}, error) {
|
func (r *Reader) ReadReply(m MultiBulkParse) (interface{}, error) {
|
||||||
line, err := r.ReadLine()
|
line, err := r.ReadLine()
|
||||||
if err != nil {
|
if err != nil {
|
||||||
|
@ -82,6 +100,10 @@ func (r *Reader) ReadReply(m MultiBulkParse) (interface{}, error) {
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
|
if m == nil {
|
||||||
|
err := fmt.Errorf("redis: got %.100q, but multi bulk parser is nil", line)
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
return m(r, n)
|
return m(r, n)
|
||||||
}
|
}
|
||||||
return nil, fmt.Errorf("redis: can't parse %.100q", line)
|
return nil, fmt.Errorf("redis: can't parse %.100q", line)
|
||||||
|
@ -126,7 +148,7 @@ func (r *Reader) readStringReply(line []byte) (string, error) {
|
||||||
return "", Nil
|
return "", Nil
|
||||||
}
|
}
|
||||||
|
|
||||||
replyLen, err := strconv.Atoi(string(line[1:]))
|
replyLen, err := util.Atoi(line[1:])
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return "", err
|
return "", err
|
||||||
}
|
}
|
||||||
|
@ -251,7 +273,7 @@ func (r *Reader) _readTmpBytesReply(line []byte) ([]byte, error) {
|
||||||
return nil, Nil
|
return nil, Nil
|
||||||
}
|
}
|
||||||
|
|
||||||
replyLen, err := strconv.Atoi(string(line[1:]))
|
replyLen, err := util.Atoi(line[1:])
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
|
@ -266,10 +288,12 @@ func (r *Reader) _readTmpBytesReply(line []byte) ([]byte, error) {
|
||||||
}
|
}
|
||||||
|
|
||||||
func (r *Reader) buf(n int) []byte {
|
func (r *Reader) buf(n int) []byte {
|
||||||
if d := n - cap(r._buf); d > 0 {
|
if n <= cap(r._buf) {
|
||||||
r._buf = append(r._buf, make([]byte, d)...)
|
|
||||||
}
|
|
||||||
return r._buf[:n]
|
return r._buf[:n]
|
||||||
|
}
|
||||||
|
d := n - cap(r._buf)
|
||||||
|
r._buf = append(r._buf, make([]byte, d)...)
|
||||||
|
return r._buf
|
||||||
}
|
}
|
||||||
|
|
||||||
func isNilReply(b []byte) bool {
|
func isNilReply(b []byte) bool {
|
|
@ -5,7 +5,7 @@ import (
|
||||||
"fmt"
|
"fmt"
|
||||||
"reflect"
|
"reflect"
|
||||||
|
|
||||||
"github.com/go-redis/redis/internal/util"
|
"github.com/go-redis/redis/v7/internal/util"
|
||||||
)
|
)
|
||||||
|
|
||||||
func Scan(b []byte, v interface{}) error {
|
func Scan(b []byte, v interface{}) error {
|
|
@ -6,8 +6,9 @@ import (
|
||||||
"fmt"
|
"fmt"
|
||||||
"io"
|
"io"
|
||||||
"strconv"
|
"strconv"
|
||||||
|
"time"
|
||||||
|
|
||||||
"github.com/go-redis/redis/internal/util"
|
"github.com/go-redis/redis/v7/internal/util"
|
||||||
)
|
)
|
||||||
|
|
||||||
type Writer struct {
|
type Writer struct {
|
||||||
|
@ -89,9 +90,10 @@ func (w *Writer) writeArg(v interface{}) error {
|
||||||
case bool:
|
case bool:
|
||||||
if v {
|
if v {
|
||||||
return w.int(1)
|
return w.int(1)
|
||||||
} else {
|
|
||||||
return w.int(0)
|
|
||||||
}
|
}
|
||||||
|
return w.int(0)
|
||||||
|
case time.Time:
|
||||||
|
return w.string(v.Format(time.RFC3339Nano))
|
||||||
case encoding.BinaryMarshaler:
|
case encoding.BinaryMarshaler:
|
||||||
b, err := v.MarshalBinary()
|
b, err := v.MarshalBinary()
|
||||||
if err != nil {
|
if err != nil {
|
||||||
|
@ -150,6 +152,10 @@ func (w *Writer) crlf() error {
|
||||||
return w.wr.WriteByte('\n')
|
return w.wr.WriteByte('\n')
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func (w *Writer) Buffered() int {
|
||||||
|
return w.wr.Buffered()
|
||||||
|
}
|
||||||
|
|
||||||
func (w *Writer) Reset(wr io.Writer) {
|
func (w *Writer) Reset(wr io.Writer) {
|
||||||
w.wr.Reset(wr)
|
w.wr.Reset(wr)
|
||||||
}
|
}
|
|
@ -0,0 +1,56 @@
|
||||||
|
package internal
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/go-redis/redis/v7/internal/util"
|
||||||
|
)
|
||||||
|
|
||||||
|
func Sleep(ctx context.Context, dur time.Duration) error {
|
||||||
|
t := time.NewTimer(dur)
|
||||||
|
defer t.Stop()
|
||||||
|
|
||||||
|
select {
|
||||||
|
case <-t.C:
|
||||||
|
return nil
|
||||||
|
case <-ctx.Done():
|
||||||
|
return ctx.Err()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func ToLower(s string) string {
|
||||||
|
if isLower(s) {
|
||||||
|
return s
|
||||||
|
}
|
||||||
|
|
||||||
|
b := make([]byte, len(s))
|
||||||
|
for i := range b {
|
||||||
|
c := s[i]
|
||||||
|
if c >= 'A' && c <= 'Z' {
|
||||||
|
c += 'a' - 'A'
|
||||||
|
}
|
||||||
|
b[i] = c
|
||||||
|
}
|
||||||
|
return util.BytesToString(b)
|
||||||
|
}
|
||||||
|
|
||||||
|
func isLower(s string) bool {
|
||||||
|
for i := 0; i < len(s); i++ {
|
||||||
|
c := s[i]
|
||||||
|
if c >= 'A' && c <= 'Z' {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
|
||||||
|
func Unwrap(err error) error {
|
||||||
|
u, ok := err.(interface {
|
||||||
|
Unwrap() error
|
||||||
|
})
|
||||||
|
if !ok {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
return u.Unwrap()
|
||||||
|
}
|
|
@ -1,6 +1,8 @@
|
||||||
package redis
|
package redis
|
||||||
|
|
||||||
import "sync"
|
import (
|
||||||
|
"sync"
|
||||||
|
)
|
||||||
|
|
||||||
// ScanIterator is used to incrementally iterate over a collection of elements.
|
// ScanIterator is used to incrementally iterate over a collection of elements.
|
||||||
// It's safe for concurrent use by multiple goroutines.
|
// It's safe for concurrent use by multiple goroutines.
|
||||||
|
@ -41,10 +43,10 @@ func (it *ScanIterator) Next() bool {
|
||||||
}
|
}
|
||||||
|
|
||||||
// Fetch next page.
|
// Fetch next page.
|
||||||
if it.cmd._args[0] == "scan" {
|
if it.cmd.args[0] == "scan" {
|
||||||
it.cmd._args[1] = it.cmd.cursor
|
it.cmd.args[1] = it.cmd.cursor
|
||||||
} else {
|
} else {
|
||||||
it.cmd._args[2] = it.cmd.cursor
|
it.cmd.args[2] = it.cmd.cursor
|
||||||
}
|
}
|
||||||
|
|
||||||
err := it.cmd.process(it.cmd)
|
err := it.cmd.process(it.cmd)
|
|
@ -1,6 +1,7 @@
|
||||||
package redis
|
package redis
|
||||||
|
|
||||||
import (
|
import (
|
||||||
|
"context"
|
||||||
"crypto/tls"
|
"crypto/tls"
|
||||||
"errors"
|
"errors"
|
||||||
"fmt"
|
"fmt"
|
||||||
|
@ -11,17 +12,17 @@ import (
|
||||||
"strings"
|
"strings"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
"github.com/go-redis/redis/internal/pool"
|
"github.com/go-redis/redis/v7/internal/pool"
|
||||||
)
|
)
|
||||||
|
|
||||||
// Limiter is the interface of a rate limiter or a circuit breaker.
|
// Limiter is the interface of a rate limiter or a circuit breaker.
|
||||||
type Limiter interface {
|
type Limiter interface {
|
||||||
// Allow returns a nil if operation is allowed or an error otherwise.
|
// Allow returns nil if operation is allowed or an error otherwise.
|
||||||
// If operation is allowed client must report the result of operation
|
// If operation is allowed client must ReportResult of the operation
|
||||||
// whether is a success or a failure.
|
// whether it is a success or a failure.
|
||||||
Allow() error
|
Allow() error
|
||||||
// ReportResult reports the result of previously allowed operation.
|
// ReportResult reports the result of the previously allowed operation.
|
||||||
// nil indicates a success, non-nil error indicates a failure.
|
// nil indicates a success, non-nil error usually indicates a failure.
|
||||||
ReportResult(result error)
|
ReportResult(result error)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
@ -34,13 +35,18 @@ type Options struct {
|
||||||
|
|
||||||
// Dialer creates new network connection and has priority over
|
// Dialer creates new network connection and has priority over
|
||||||
// Network and Addr options.
|
// Network and Addr options.
|
||||||
Dialer func() (net.Conn, error)
|
Dialer func(ctx context.Context, network, addr string) (net.Conn, error)
|
||||||
|
|
||||||
// Hook that is called when new connection is established.
|
// Hook that is called when new connection is established.
|
||||||
OnConnect func(*Conn) error
|
OnConnect func(*Conn) error
|
||||||
|
|
||||||
|
// Use the specified Username to authenticate the current connection with one of the connections defined in the ACL
|
||||||
|
// list when connecting to a Redis 6.0 instance, or greater, that is using the Redis ACL system.
|
||||||
|
Username string
|
||||||
|
|
||||||
// Optional password. Must match the password specified in the
|
// Optional password. Must match the password specified in the
|
||||||
// requirepass server configuration option.
|
// requirepass server configuration option (if connecting to a Redis 5.0 instance, or lower),
|
||||||
|
// or the User Password when connecting to a Redis 6.0 instance, or greater, that is using the Redis ACL system.
|
||||||
Password string
|
Password string
|
||||||
// Database to be selected after connecting to the server.
|
// Database to be selected after connecting to the server.
|
||||||
DB int
|
DB int
|
||||||
|
@ -95,26 +101,32 @@ type Options struct {
|
||||||
|
|
||||||
// TLS Config to use. When set TLS will be negotiated.
|
// TLS Config to use. When set TLS will be negotiated.
|
||||||
TLSConfig *tls.Config
|
TLSConfig *tls.Config
|
||||||
|
|
||||||
|
// Limiter interface used to implemented circuit breaker or rate limiter.
|
||||||
|
Limiter Limiter
|
||||||
}
|
}
|
||||||
|
|
||||||
func (opt *Options) init() {
|
func (opt *Options) init() {
|
||||||
if opt.Network == "" {
|
|
||||||
opt.Network = "tcp"
|
|
||||||
}
|
|
||||||
if opt.Addr == "" {
|
if opt.Addr == "" {
|
||||||
opt.Addr = "localhost:6379"
|
opt.Addr = "localhost:6379"
|
||||||
}
|
}
|
||||||
|
if opt.Network == "" {
|
||||||
|
if strings.HasPrefix(opt.Addr, "/") {
|
||||||
|
opt.Network = "unix"
|
||||||
|
} else {
|
||||||
|
opt.Network = "tcp"
|
||||||
|
}
|
||||||
|
}
|
||||||
if opt.Dialer == nil {
|
if opt.Dialer == nil {
|
||||||
opt.Dialer = func() (net.Conn, error) {
|
opt.Dialer = func(ctx context.Context, network, addr string) (net.Conn, error) {
|
||||||
netDialer := &net.Dialer{
|
netDialer := &net.Dialer{
|
||||||
Timeout: opt.DialTimeout,
|
Timeout: opt.DialTimeout,
|
||||||
KeepAlive: 5 * time.Minute,
|
KeepAlive: 5 * time.Minute,
|
||||||
}
|
}
|
||||||
if opt.TLSConfig == nil {
|
if opt.TLSConfig == nil {
|
||||||
return netDialer.Dial(opt.Network, opt.Addr)
|
return netDialer.DialContext(ctx, network, addr)
|
||||||
} else {
|
|
||||||
return tls.DialWithDialer(netDialer, opt.Network, opt.Addr, opt.TLSConfig)
|
|
||||||
}
|
}
|
||||||
|
return tls.DialWithDialer(netDialer, network, addr, opt.TLSConfig)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
if opt.PoolSize == 0 {
|
if opt.PoolSize == 0 {
|
||||||
|
@ -145,6 +157,9 @@ func (opt *Options) init() {
|
||||||
opt.IdleCheckFrequency = time.Minute
|
opt.IdleCheckFrequency = time.Minute
|
||||||
}
|
}
|
||||||
|
|
||||||
|
if opt.MaxRetries == -1 {
|
||||||
|
opt.MaxRetries = 0
|
||||||
|
}
|
||||||
switch opt.MinRetryBackoff {
|
switch opt.MinRetryBackoff {
|
||||||
case -1:
|
case -1:
|
||||||
opt.MinRetryBackoff = 0
|
opt.MinRetryBackoff = 0
|
||||||
|
@ -159,6 +174,11 @@ func (opt *Options) init() {
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func (opt *Options) clone() *Options {
|
||||||
|
clone := *opt
|
||||||
|
return &clone
|
||||||
|
}
|
||||||
|
|
||||||
// ParseURL parses an URL into Options that can be used to connect to Redis.
|
// ParseURL parses an URL into Options that can be used to connect to Redis.
|
||||||
func ParseURL(redisURL string) (*Options, error) {
|
func ParseURL(redisURL string) (*Options, error) {
|
||||||
o := &Options{Network: "tcp"}
|
o := &Options{Network: "tcp"}
|
||||||
|
@ -172,6 +192,7 @@ func ParseURL(redisURL string) (*Options, error) {
|
||||||
}
|
}
|
||||||
|
|
||||||
if u.User != nil {
|
if u.User != nil {
|
||||||
|
o.Username = u.User.Username()
|
||||||
if p, ok := u.User.Password(); ok {
|
if p, ok := u.User.Password(); ok {
|
||||||
o.Password = p
|
o.Password = p
|
||||||
}
|
}
|
||||||
|
@ -215,7 +236,9 @@ func ParseURL(redisURL string) (*Options, error) {
|
||||||
|
|
||||||
func newConnPool(opt *Options) *pool.ConnPool {
|
func newConnPool(opt *Options) *pool.ConnPool {
|
||||||
return pool.NewConnPool(&pool.Options{
|
return pool.NewConnPool(&pool.Options{
|
||||||
Dialer: opt.Dialer,
|
Dialer: func(ctx context.Context) (net.Conn, error) {
|
||||||
|
return opt.Dialer(ctx, opt.Network, opt.Addr)
|
||||||
|
},
|
||||||
PoolSize: opt.PoolSize,
|
PoolSize: opt.PoolSize,
|
||||||
MinIdleConns: opt.MinIdleConns,
|
MinIdleConns: opt.MinIdleConns,
|
||||||
MaxConnAge: opt.MaxConnAge,
|
MaxConnAge: opt.MaxConnAge,
|
|
@ -1,13 +1,27 @@
|
||||||
package redis
|
package redis
|
||||||
|
|
||||||
import (
|
import (
|
||||||
|
"context"
|
||||||
"sync"
|
"sync"
|
||||||
|
|
||||||
"github.com/go-redis/redis/internal/pool"
|
"github.com/go-redis/redis/v7/internal/pool"
|
||||||
)
|
)
|
||||||
|
|
||||||
type pipelineExecer func([]Cmder) error
|
type pipelineExecer func(context.Context, []Cmder) error
|
||||||
|
|
||||||
|
// Pipeliner is an mechanism to realise Redis Pipeline technique.
|
||||||
|
//
|
||||||
|
// Pipelining is a technique to extremely speed up processing by packing
|
||||||
|
// operations to batches, send them at once to Redis and read a replies in a
|
||||||
|
// singe step.
|
||||||
|
// See https://redis.io/topics/pipelining
|
||||||
|
//
|
||||||
|
// Pay attention, that Pipeline is not a transaction, so you can get unexpected
|
||||||
|
// results in case of big pipelines and small read/write timeouts.
|
||||||
|
// Redis client has retransmission logic in case of timeouts, pipeline
|
||||||
|
// can be retransmitted and commands can be executed more then once.
|
||||||
|
// To avoid this: it is good idea to use reasonable bigger read/write timeouts
|
||||||
|
// depends of your batch size and/or use TxPipeline.
|
||||||
type Pipeliner interface {
|
type Pipeliner interface {
|
||||||
StatefulCmdable
|
StatefulCmdable
|
||||||
Do(args ...interface{}) *Cmd
|
Do(args ...interface{}) *Cmd
|
||||||
|
@ -15,6 +29,7 @@ type Pipeliner interface {
|
||||||
Close() error
|
Close() error
|
||||||
Discard() error
|
Discard() error
|
||||||
Exec() ([]Cmder, error)
|
Exec() ([]Cmder, error)
|
||||||
|
ExecContext(ctx context.Context) ([]Cmder, error)
|
||||||
}
|
}
|
||||||
|
|
||||||
var _ Pipeliner = (*Pipeline)(nil)
|
var _ Pipeliner = (*Pipeline)(nil)
|
||||||
|
@ -23,8 +38,10 @@ var _ Pipeliner = (*Pipeline)(nil)
|
||||||
// http://redis.io/topics/pipelining. It's safe for concurrent use
|
// http://redis.io/topics/pipelining. It's safe for concurrent use
|
||||||
// by multiple goroutines.
|
// by multiple goroutines.
|
||||||
type Pipeline struct {
|
type Pipeline struct {
|
||||||
|
cmdable
|
||||||
statefulCmdable
|
statefulCmdable
|
||||||
|
|
||||||
|
ctx context.Context
|
||||||
exec pipelineExecer
|
exec pipelineExecer
|
||||||
|
|
||||||
mu sync.Mutex
|
mu sync.Mutex
|
||||||
|
@ -32,6 +49,11 @@ type Pipeline struct {
|
||||||
closed bool
|
closed bool
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func (c *Pipeline) init() {
|
||||||
|
c.cmdable = c.Process
|
||||||
|
c.statefulCmdable = c.Process
|
||||||
|
}
|
||||||
|
|
||||||
func (c *Pipeline) Do(args ...interface{}) *Cmd {
|
func (c *Pipeline) Do(args ...interface{}) *Cmd {
|
||||||
cmd := NewCmd(args...)
|
cmd := NewCmd(args...)
|
||||||
_ = c.Process(cmd)
|
_ = c.Process(cmd)
|
||||||
|
@ -49,7 +71,7 @@ func (c *Pipeline) Process(cmd Cmder) error {
|
||||||
// Close closes the pipeline, releasing any open resources.
|
// Close closes the pipeline, releasing any open resources.
|
||||||
func (c *Pipeline) Close() error {
|
func (c *Pipeline) Close() error {
|
||||||
c.mu.Lock()
|
c.mu.Lock()
|
||||||
c.discard()
|
_ = c.discard()
|
||||||
c.closed = true
|
c.closed = true
|
||||||
c.mu.Unlock()
|
c.mu.Unlock()
|
||||||
return nil
|
return nil
|
||||||
|
@ -77,6 +99,10 @@ func (c *Pipeline) discard() error {
|
||||||
// Exec always returns list of commands and error of the first failed
|
// Exec always returns list of commands and error of the first failed
|
||||||
// command if any.
|
// command if any.
|
||||||
func (c *Pipeline) Exec() ([]Cmder, error) {
|
func (c *Pipeline) Exec() ([]Cmder, error) {
|
||||||
|
return c.ExecContext(c.ctx)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *Pipeline) ExecContext(ctx context.Context) ([]Cmder, error) {
|
||||||
c.mu.Lock()
|
c.mu.Lock()
|
||||||
defer c.mu.Unlock()
|
defer c.mu.Unlock()
|
||||||
|
|
||||||
|
@ -91,10 +117,10 @@ func (c *Pipeline) Exec() ([]Cmder, error) {
|
||||||
cmds := c.cmds
|
cmds := c.cmds
|
||||||
c.cmds = nil
|
c.cmds = nil
|
||||||
|
|
||||||
return cmds, c.exec(cmds)
|
return cmds, c.exec(ctx, cmds)
|
||||||
}
|
}
|
||||||
|
|
||||||
func (c *Pipeline) pipelined(fn func(Pipeliner) error) ([]Cmder, error) {
|
func (c *Pipeline) Pipelined(fn func(Pipeliner) error) ([]Cmder, error) {
|
||||||
if err := fn(c); err != nil {
|
if err := fn(c); err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
|
@ -103,16 +129,12 @@ func (c *Pipeline) pipelined(fn func(Pipeliner) error) ([]Cmder, error) {
|
||||||
return cmds, err
|
return cmds, err
|
||||||
}
|
}
|
||||||
|
|
||||||
func (c *Pipeline) Pipelined(fn func(Pipeliner) error) ([]Cmder, error) {
|
|
||||||
return c.pipelined(fn)
|
|
||||||
}
|
|
||||||
|
|
||||||
func (c *Pipeline) Pipeline() Pipeliner {
|
func (c *Pipeline) Pipeline() Pipeliner {
|
||||||
return c
|
return c
|
||||||
}
|
}
|
||||||
|
|
||||||
func (c *Pipeline) TxPipelined(fn func(Pipeliner) error) ([]Cmder, error) {
|
func (c *Pipeline) TxPipelined(fn func(Pipeliner) error) ([]Cmder, error) {
|
||||||
return c.pipelined(fn)
|
return c.Pipelined(fn)
|
||||||
}
|
}
|
||||||
|
|
||||||
func (c *Pipeline) TxPipeline() Pipeliner {
|
func (c *Pipeline) TxPipeline() Pipeliner {
|
292
vendor/github.com/go-redis/redis/pubsub.go → vendor/github.com/go-redis/redis/v7/pubsub.go
generated
vendored
292
vendor/github.com/go-redis/redis/pubsub.go → vendor/github.com/go-redis/redis/v7/pubsub.go
generated
vendored
|
@ -1,19 +1,23 @@
|
||||||
package redis
|
package redis
|
||||||
|
|
||||||
import (
|
import (
|
||||||
|
"context"
|
||||||
"errors"
|
"errors"
|
||||||
"fmt"
|
"fmt"
|
||||||
|
"strings"
|
||||||
"sync"
|
"sync"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
"github.com/go-redis/redis/internal"
|
"github.com/go-redis/redis/v7/internal"
|
||||||
"github.com/go-redis/redis/internal/pool"
|
"github.com/go-redis/redis/v7/internal/pool"
|
||||||
"github.com/go-redis/redis/internal/proto"
|
"github.com/go-redis/redis/v7/internal/proto"
|
||||||
)
|
)
|
||||||
|
|
||||||
|
const pingTimeout = 30 * time.Second
|
||||||
|
|
||||||
var errPingTimeout = errors.New("redis: ping timeout")
|
var errPingTimeout = errors.New("redis: ping timeout")
|
||||||
|
|
||||||
// PubSub implements Pub/Sub commands bas described in
|
// PubSub implements Pub/Sub commands as described in
|
||||||
// http://redis.io/topics/pubsub. Message receiving is NOT safe
|
// http://redis.io/topics/pubsub. Message receiving is NOT safe
|
||||||
// for concurrent use by multiple goroutines.
|
// for concurrent use by multiple goroutines.
|
||||||
//
|
//
|
||||||
|
@ -29,28 +33,36 @@ type PubSub struct {
|
||||||
cn *pool.Conn
|
cn *pool.Conn
|
||||||
channels map[string]struct{}
|
channels map[string]struct{}
|
||||||
patterns map[string]struct{}
|
patterns map[string]struct{}
|
||||||
|
|
||||||
closed bool
|
closed bool
|
||||||
exit chan struct{}
|
exit chan struct{}
|
||||||
|
|
||||||
cmd *Cmd
|
cmd *Cmd
|
||||||
|
|
||||||
chOnce sync.Once
|
chOnce sync.Once
|
||||||
ch chan *Message
|
msgCh chan *Message
|
||||||
|
allCh chan interface{}
|
||||||
ping chan struct{}
|
ping chan struct{}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func (c *PubSub) String() string {
|
||||||
|
channels := mapKeys(c.channels)
|
||||||
|
channels = append(channels, mapKeys(c.patterns)...)
|
||||||
|
return fmt.Sprintf("PubSub(%s)", strings.Join(channels, ", "))
|
||||||
|
}
|
||||||
|
|
||||||
func (c *PubSub) init() {
|
func (c *PubSub) init() {
|
||||||
c.exit = make(chan struct{})
|
c.exit = make(chan struct{})
|
||||||
}
|
}
|
||||||
|
|
||||||
func (c *PubSub) conn() (*pool.Conn, error) {
|
func (c *PubSub) connWithLock() (*pool.Conn, error) {
|
||||||
c.mu.Lock()
|
c.mu.Lock()
|
||||||
cn, err := c._conn(nil)
|
cn, err := c.conn(nil)
|
||||||
c.mu.Unlock()
|
c.mu.Unlock()
|
||||||
return cn, err
|
return cn, err
|
||||||
}
|
}
|
||||||
|
|
||||||
func (c *PubSub) _conn(newChannels []string) (*pool.Conn, error) {
|
func (c *PubSub) conn(newChannels []string) (*pool.Conn, error) {
|
||||||
if c.closed {
|
if c.closed {
|
||||||
return nil, pool.ErrClosed
|
return nil, pool.ErrClosed
|
||||||
}
|
}
|
||||||
|
@ -75,8 +87,8 @@ func (c *PubSub) _conn(newChannels []string) (*pool.Conn, error) {
|
||||||
return cn, nil
|
return cn, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
func (c *PubSub) writeCmd(cn *pool.Conn, cmd Cmder) error {
|
func (c *PubSub) writeCmd(ctx context.Context, cn *pool.Conn, cmd Cmder) error {
|
||||||
return cn.WithWriter(c.opt.WriteTimeout, func(wr *proto.Writer) error {
|
return cn.WithWriter(ctx, c.opt.WriteTimeout, func(wr *proto.Writer) error {
|
||||||
return writeCmd(wr, cmd)
|
return writeCmd(wr, cmd)
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
@ -85,10 +97,7 @@ func (c *PubSub) resubscribe(cn *pool.Conn) error {
|
||||||
var firstErr error
|
var firstErr error
|
||||||
|
|
||||||
if len(c.channels) > 0 {
|
if len(c.channels) > 0 {
|
||||||
err := c._subscribe(cn, "subscribe", mapKeys(c.channels))
|
firstErr = c._subscribe(cn, "subscribe", mapKeys(c.channels))
|
||||||
if err != nil && firstErr == nil {
|
|
||||||
firstErr = err
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
if len(c.patterns) > 0 {
|
if len(c.patterns) > 0 {
|
||||||
|
@ -120,35 +129,35 @@ func (c *PubSub) _subscribe(
|
||||||
args = append(args, channel)
|
args = append(args, channel)
|
||||||
}
|
}
|
||||||
cmd := NewSliceCmd(args...)
|
cmd := NewSliceCmd(args...)
|
||||||
return c.writeCmd(cn, cmd)
|
return c.writeCmd(context.TODO(), cn, cmd)
|
||||||
}
|
}
|
||||||
|
|
||||||
func (c *PubSub) releaseConn(cn *pool.Conn, err error, allowTimeout bool) {
|
func (c *PubSub) releaseConnWithLock(cn *pool.Conn, err error, allowTimeout bool) {
|
||||||
c.mu.Lock()
|
c.mu.Lock()
|
||||||
c._releaseConn(cn, err, allowTimeout)
|
c.releaseConn(cn, err, allowTimeout)
|
||||||
c.mu.Unlock()
|
c.mu.Unlock()
|
||||||
}
|
}
|
||||||
|
|
||||||
func (c *PubSub) _releaseConn(cn *pool.Conn, err error, allowTimeout bool) {
|
func (c *PubSub) releaseConn(cn *pool.Conn, err error, allowTimeout bool) {
|
||||||
if c.cn != cn {
|
if c.cn != cn {
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
if internal.IsBadConn(err, allowTimeout) {
|
if isBadConn(err, allowTimeout) {
|
||||||
c._reconnect(err)
|
c.reconnect(err)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func (c *PubSub) _reconnect(reason error) {
|
func (c *PubSub) reconnect(reason error) {
|
||||||
_ = c._closeTheCn(reason)
|
_ = c.closeTheCn(reason)
|
||||||
_, _ = c._conn(nil)
|
_, _ = c.conn(nil)
|
||||||
}
|
}
|
||||||
|
|
||||||
func (c *PubSub) _closeTheCn(reason error) error {
|
func (c *PubSub) closeTheCn(reason error) error {
|
||||||
if c.cn == nil {
|
if c.cn == nil {
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
if !c.closed {
|
if !c.closed {
|
||||||
internal.Logf("redis: discarding bad PubSub connection: %s", reason)
|
internal.Logger.Printf("redis: discarding bad PubSub connection: %s", reason)
|
||||||
}
|
}
|
||||||
err := c.closeConn(c.cn)
|
err := c.closeConn(c.cn)
|
||||||
c.cn = nil
|
c.cn = nil
|
||||||
|
@ -165,8 +174,7 @@ func (c *PubSub) Close() error {
|
||||||
c.closed = true
|
c.closed = true
|
||||||
close(c.exit)
|
close(c.exit)
|
||||||
|
|
||||||
err := c._closeTheCn(pool.ErrClosed)
|
return c.closeTheCn(pool.ErrClosed)
|
||||||
return err
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// Subscribe the client to the specified channels. It returns
|
// Subscribe the client to the specified channels. It returns
|
||||||
|
@ -228,13 +236,13 @@ func (c *PubSub) PUnsubscribe(patterns ...string) error {
|
||||||
}
|
}
|
||||||
|
|
||||||
func (c *PubSub) subscribe(redisCmd string, channels ...string) error {
|
func (c *PubSub) subscribe(redisCmd string, channels ...string) error {
|
||||||
cn, err := c._conn(channels)
|
cn, err := c.conn(channels)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
|
|
||||||
err = c._subscribe(cn, redisCmd, channels)
|
err = c._subscribe(cn, redisCmd, channels)
|
||||||
c._releaseConn(cn, err, false)
|
c.releaseConn(cn, err, false)
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
|
|
||||||
|
@ -245,13 +253,13 @@ func (c *PubSub) Ping(payload ...string) error {
|
||||||
}
|
}
|
||||||
cmd := NewCmd(args...)
|
cmd := NewCmd(args...)
|
||||||
|
|
||||||
cn, err := c.conn()
|
cn, err := c.connWithLock()
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
|
|
||||||
err = c.writeCmd(cn, cmd)
|
err = c.writeCmd(context.TODO(), cn, cmd)
|
||||||
c.releaseConn(cn, err, false)
|
c.releaseConnWithLock(cn, err, false)
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
|
|
||||||
|
@ -301,9 +309,11 @@ func (c *PubSub) newMessage(reply interface{}) (interface{}, error) {
|
||||||
case []interface{}:
|
case []interface{}:
|
||||||
switch kind := reply[0].(string); kind {
|
switch kind := reply[0].(string); kind {
|
||||||
case "subscribe", "unsubscribe", "psubscribe", "punsubscribe":
|
case "subscribe", "unsubscribe", "psubscribe", "punsubscribe":
|
||||||
|
// Can be nil in case of "unsubscribe".
|
||||||
|
channel, _ := reply[1].(string)
|
||||||
return &Subscription{
|
return &Subscription{
|
||||||
Kind: kind,
|
Kind: kind,
|
||||||
Channel: reply[1].(string),
|
Channel: channel,
|
||||||
Count: int(reply[2].(int64)),
|
Count: int(reply[2].(int64)),
|
||||||
}, nil
|
}, nil
|
||||||
case "message":
|
case "message":
|
||||||
|
@ -337,16 +347,16 @@ func (c *PubSub) ReceiveTimeout(timeout time.Duration) (interface{}, error) {
|
||||||
c.cmd = NewCmd()
|
c.cmd = NewCmd()
|
||||||
}
|
}
|
||||||
|
|
||||||
cn, err := c.conn()
|
cn, err := c.connWithLock()
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
|
|
||||||
err = cn.WithReader(timeout, func(rd *proto.Reader) error {
|
err = cn.WithReader(context.TODO(), timeout, func(rd *proto.Reader) error {
|
||||||
return c.cmd.readReply(rd)
|
return c.cmd.readReply(rd)
|
||||||
})
|
})
|
||||||
|
|
||||||
c.releaseConn(cn, err, timeout > 0)
|
c.releaseConnWithLock(cn, err, timeout > 0)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
|
@ -386,63 +396,64 @@ func (c *PubSub) ReceiveMessage() (*Message, error) {
|
||||||
}
|
}
|
||||||
|
|
||||||
// Channel returns a Go channel for concurrently receiving messages.
|
// Channel returns a Go channel for concurrently receiving messages.
|
||||||
// It periodically sends Ping messages to test connection health.
|
// The channel is closed together with the PubSub. If the Go channel
|
||||||
// The channel is closed with PubSub. Receive* APIs can not be used
|
// is blocked full for 30 seconds the message is dropped.
|
||||||
// after channel is created.
|
// Receive* APIs can not be used after channel is created.
|
||||||
|
//
|
||||||
|
// go-redis periodically sends ping messages to test connection health
|
||||||
|
// and re-subscribes if ping can not not received for 30 seconds.
|
||||||
func (c *PubSub) Channel() <-chan *Message {
|
func (c *PubSub) Channel() <-chan *Message {
|
||||||
c.chOnce.Do(c.initChannel)
|
return c.ChannelSize(100)
|
||||||
return c.ch
|
|
||||||
}
|
}
|
||||||
|
|
||||||
func (c *PubSub) initChannel() {
|
// ChannelSize is like Channel, but creates a Go channel
|
||||||
c.ch = make(chan *Message, 100)
|
// with specified buffer size.
|
||||||
c.ping = make(chan struct{}, 10)
|
func (c *PubSub) ChannelSize(size int) <-chan *Message {
|
||||||
|
c.chOnce.Do(func() {
|
||||||
|
c.initPing()
|
||||||
|
c.initMsgChan(size)
|
||||||
|
})
|
||||||
|
if c.msgCh == nil {
|
||||||
|
err := fmt.Errorf("redis: Channel can't be called after ChannelWithSubscriptions")
|
||||||
|
panic(err)
|
||||||
|
}
|
||||||
|
if cap(c.msgCh) != size {
|
||||||
|
err := fmt.Errorf("redis: PubSub.Channel size can not be changed once created")
|
||||||
|
panic(err)
|
||||||
|
}
|
||||||
|
return c.msgCh
|
||||||
|
}
|
||||||
|
|
||||||
|
// ChannelWithSubscriptions is like Channel, but message type can be either
|
||||||
|
// *Subscription or *Message. Subscription messages can be used to detect
|
||||||
|
// reconnections.
|
||||||
|
//
|
||||||
|
// ChannelWithSubscriptions can not be used together with Channel or ChannelSize.
|
||||||
|
func (c *PubSub) ChannelWithSubscriptions(size int) <-chan interface{} {
|
||||||
|
c.chOnce.Do(func() {
|
||||||
|
c.initPing()
|
||||||
|
c.initAllChan(size)
|
||||||
|
})
|
||||||
|
if c.allCh == nil {
|
||||||
|
err := fmt.Errorf("redis: ChannelWithSubscriptions can't be called after Channel")
|
||||||
|
panic(err)
|
||||||
|
}
|
||||||
|
if cap(c.allCh) != size {
|
||||||
|
err := fmt.Errorf("redis: PubSub.Channel size can not be changed once created")
|
||||||
|
panic(err)
|
||||||
|
}
|
||||||
|
return c.allCh
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *PubSub) initPing() {
|
||||||
|
c.ping = make(chan struct{}, 1)
|
||||||
go func() {
|
go func() {
|
||||||
var errCount int
|
timer := time.NewTimer(pingTimeout)
|
||||||
for {
|
|
||||||
msg, err := c.Receive()
|
|
||||||
if err != nil {
|
|
||||||
if err == pool.ErrClosed {
|
|
||||||
close(c.ch)
|
|
||||||
return
|
|
||||||
}
|
|
||||||
if errCount > 0 {
|
|
||||||
time.Sleep(c.retryBackoff(errCount))
|
|
||||||
}
|
|
||||||
errCount++
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
errCount = 0
|
|
||||||
|
|
||||||
// Any message is as good as a ping.
|
|
||||||
select {
|
|
||||||
case c.ping <- struct{}{}:
|
|
||||||
default:
|
|
||||||
}
|
|
||||||
|
|
||||||
switch msg := msg.(type) {
|
|
||||||
case *Subscription:
|
|
||||||
// Ignore.
|
|
||||||
case *Pong:
|
|
||||||
// Ignore.
|
|
||||||
case *Message:
|
|
||||||
c.ch <- msg
|
|
||||||
default:
|
|
||||||
internal.Logf("redis: unknown message: %T", msg)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}()
|
|
||||||
|
|
||||||
go func() {
|
|
||||||
const timeout = 5 * time.Second
|
|
||||||
|
|
||||||
timer := time.NewTimer(timeout)
|
|
||||||
timer.Stop()
|
timer.Stop()
|
||||||
|
|
||||||
healthy := true
|
healthy := true
|
||||||
for {
|
for {
|
||||||
timer.Reset(timeout)
|
timer.Reset(pingTimeout)
|
||||||
select {
|
select {
|
||||||
case <-c.ping:
|
case <-c.ping:
|
||||||
healthy = true
|
healthy = true
|
||||||
|
@ -458,7 +469,8 @@ func (c *PubSub) initChannel() {
|
||||||
pingErr = errPingTimeout
|
pingErr = errPingTimeout
|
||||||
}
|
}
|
||||||
c.mu.Lock()
|
c.mu.Lock()
|
||||||
c._reconnect(pingErr)
|
c.reconnect(pingErr)
|
||||||
|
healthy = true
|
||||||
c.mu.Unlock()
|
c.mu.Unlock()
|
||||||
}
|
}
|
||||||
case <-c.exit:
|
case <-c.exit:
|
||||||
|
@ -468,6 +480,116 @@ func (c *PubSub) initChannel() {
|
||||||
}()
|
}()
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// initMsgChan must be in sync with initAllChan.
|
||||||
|
func (c *PubSub) initMsgChan(size int) {
|
||||||
|
c.msgCh = make(chan *Message, size)
|
||||||
|
go func() {
|
||||||
|
timer := time.NewTimer(pingTimeout)
|
||||||
|
timer.Stop()
|
||||||
|
|
||||||
|
var errCount int
|
||||||
|
for {
|
||||||
|
msg, err := c.Receive()
|
||||||
|
if err != nil {
|
||||||
|
if err == pool.ErrClosed {
|
||||||
|
close(c.msgCh)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if errCount > 0 {
|
||||||
|
time.Sleep(c.retryBackoff(errCount))
|
||||||
|
}
|
||||||
|
errCount++
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
errCount = 0
|
||||||
|
|
||||||
|
// Any message is as good as a ping.
|
||||||
|
select {
|
||||||
|
case c.ping <- struct{}{}:
|
||||||
|
default:
|
||||||
|
}
|
||||||
|
|
||||||
|
switch msg := msg.(type) {
|
||||||
|
case *Subscription:
|
||||||
|
// Ignore.
|
||||||
|
case *Pong:
|
||||||
|
// Ignore.
|
||||||
|
case *Message:
|
||||||
|
timer.Reset(pingTimeout)
|
||||||
|
select {
|
||||||
|
case c.msgCh <- msg:
|
||||||
|
if !timer.Stop() {
|
||||||
|
<-timer.C
|
||||||
|
}
|
||||||
|
case <-timer.C:
|
||||||
|
internal.Logger.Printf(
|
||||||
|
"redis: %s channel is full for %s (message is dropped)", c, pingTimeout)
|
||||||
|
}
|
||||||
|
default:
|
||||||
|
internal.Logger.Printf("redis: unknown message type: %T", msg)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}()
|
||||||
|
}
|
||||||
|
|
||||||
|
// initAllChan must be in sync with initMsgChan.
|
||||||
|
func (c *PubSub) initAllChan(size int) {
|
||||||
|
c.allCh = make(chan interface{}, size)
|
||||||
|
go func() {
|
||||||
|
timer := time.NewTimer(pingTimeout)
|
||||||
|
timer.Stop()
|
||||||
|
|
||||||
|
var errCount int
|
||||||
|
for {
|
||||||
|
msg, err := c.Receive()
|
||||||
|
if err != nil {
|
||||||
|
if err == pool.ErrClosed {
|
||||||
|
close(c.allCh)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if errCount > 0 {
|
||||||
|
time.Sleep(c.retryBackoff(errCount))
|
||||||
|
}
|
||||||
|
errCount++
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
errCount = 0
|
||||||
|
|
||||||
|
// Any message is as good as a ping.
|
||||||
|
select {
|
||||||
|
case c.ping <- struct{}{}:
|
||||||
|
default:
|
||||||
|
}
|
||||||
|
|
||||||
|
switch msg := msg.(type) {
|
||||||
|
case *Subscription:
|
||||||
|
c.sendMessage(msg, timer)
|
||||||
|
case *Pong:
|
||||||
|
// Ignore.
|
||||||
|
case *Message:
|
||||||
|
c.sendMessage(msg, timer)
|
||||||
|
default:
|
||||||
|
internal.Logger.Printf("redis: unknown message type: %T", msg)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}()
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *PubSub) sendMessage(msg interface{}, timer *time.Timer) {
|
||||||
|
timer.Reset(pingTimeout)
|
||||||
|
select {
|
||||||
|
case c.allCh <- msg:
|
||||||
|
if !timer.Stop() {
|
||||||
|
<-timer.C
|
||||||
|
}
|
||||||
|
case <-timer.C:
|
||||||
|
internal.Logger.Printf(
|
||||||
|
"redis: %s channel is full for %s (message is dropped)", c, pingTimeout)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func (c *PubSub) retryBackoff(attempt int) time.Duration {
|
func (c *PubSub) retryBackoff(attempt int) time.Duration {
|
||||||
return internal.RetryBackoff(attempt, c.opt.MinRetryBackoff, c.opt.MaxRetryBackoff)
|
return internal.RetryBackoff(attempt, c.opt.MinRetryBackoff, c.opt.MaxRetryBackoff)
|
||||||
}
|
}
|
|
@ -0,0 +1,758 @@
|
||||||
|
package redis
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"fmt"
|
||||||
|
"log"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/go-redis/redis/v7/internal"
|
||||||
|
"github.com/go-redis/redis/v7/internal/pool"
|
||||||
|
"github.com/go-redis/redis/v7/internal/proto"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Nil reply returned by Redis when key does not exist.
|
||||||
|
const Nil = proto.Nil
|
||||||
|
|
||||||
|
func SetLogger(logger *log.Logger) {
|
||||||
|
internal.Logger = logger
|
||||||
|
}
|
||||||
|
|
||||||
|
//------------------------------------------------------------------------------
|
||||||
|
|
||||||
|
type Hook interface {
|
||||||
|
BeforeProcess(ctx context.Context, cmd Cmder) (context.Context, error)
|
||||||
|
AfterProcess(ctx context.Context, cmd Cmder) error
|
||||||
|
|
||||||
|
BeforeProcessPipeline(ctx context.Context, cmds []Cmder) (context.Context, error)
|
||||||
|
AfterProcessPipeline(ctx context.Context, cmds []Cmder) error
|
||||||
|
}
|
||||||
|
|
||||||
|
type hooks struct {
|
||||||
|
hooks []Hook
|
||||||
|
}
|
||||||
|
|
||||||
|
func (hs *hooks) lock() {
|
||||||
|
hs.hooks = hs.hooks[:len(hs.hooks):len(hs.hooks)]
|
||||||
|
}
|
||||||
|
|
||||||
|
func (hs hooks) clone() hooks {
|
||||||
|
clone := hs
|
||||||
|
clone.lock()
|
||||||
|
return clone
|
||||||
|
}
|
||||||
|
|
||||||
|
func (hs *hooks) AddHook(hook Hook) {
|
||||||
|
hs.hooks = append(hs.hooks, hook)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (hs hooks) process(
|
||||||
|
ctx context.Context, cmd Cmder, fn func(context.Context, Cmder) error,
|
||||||
|
) error {
|
||||||
|
ctx, err := hs.beforeProcess(ctx, cmd)
|
||||||
|
if err != nil {
|
||||||
|
cmd.SetErr(err)
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
cmdErr := fn(ctx, cmd)
|
||||||
|
|
||||||
|
if err := hs.afterProcess(ctx, cmd); err != nil {
|
||||||
|
cmd.SetErr(err)
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
return cmdErr
|
||||||
|
}
|
||||||
|
|
||||||
|
func (hs hooks) beforeProcess(ctx context.Context, cmd Cmder) (context.Context, error) {
|
||||||
|
for _, h := range hs.hooks {
|
||||||
|
var err error
|
||||||
|
ctx, err = h.BeforeProcess(ctx, cmd)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return ctx, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (hs hooks) afterProcess(ctx context.Context, cmd Cmder) error {
|
||||||
|
var firstErr error
|
||||||
|
for _, h := range hs.hooks {
|
||||||
|
err := h.AfterProcess(ctx, cmd)
|
||||||
|
if err != nil && firstErr == nil {
|
||||||
|
firstErr = err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return firstErr
|
||||||
|
}
|
||||||
|
|
||||||
|
func (hs hooks) processPipeline(
|
||||||
|
ctx context.Context, cmds []Cmder, fn func(context.Context, []Cmder) error,
|
||||||
|
) error {
|
||||||
|
ctx, err := hs.beforeProcessPipeline(ctx, cmds)
|
||||||
|
if err != nil {
|
||||||
|
setCmdsErr(cmds, err)
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
cmdsErr := fn(ctx, cmds)
|
||||||
|
|
||||||
|
if err := hs.afterProcessPipeline(ctx, cmds); err != nil {
|
||||||
|
setCmdsErr(cmds, err)
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
return cmdsErr
|
||||||
|
}
|
||||||
|
|
||||||
|
func (hs hooks) beforeProcessPipeline(ctx context.Context, cmds []Cmder) (context.Context, error) {
|
||||||
|
for _, h := range hs.hooks {
|
||||||
|
var err error
|
||||||
|
ctx, err = h.BeforeProcessPipeline(ctx, cmds)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return ctx, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (hs hooks) afterProcessPipeline(ctx context.Context, cmds []Cmder) error {
|
||||||
|
var firstErr error
|
||||||
|
for _, h := range hs.hooks {
|
||||||
|
err := h.AfterProcessPipeline(ctx, cmds)
|
||||||
|
if err != nil && firstErr == nil {
|
||||||
|
firstErr = err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return firstErr
|
||||||
|
}
|
||||||
|
|
||||||
|
func (hs hooks) processTxPipeline(
|
||||||
|
ctx context.Context, cmds []Cmder, fn func(context.Context, []Cmder) error,
|
||||||
|
) error {
|
||||||
|
cmds = wrapMultiExec(cmds)
|
||||||
|
return hs.processPipeline(ctx, cmds, fn)
|
||||||
|
}
|
||||||
|
|
||||||
|
//------------------------------------------------------------------------------
|
||||||
|
|
||||||
|
type baseClient struct {
|
||||||
|
opt *Options
|
||||||
|
connPool pool.Pooler
|
||||||
|
|
||||||
|
onClose func() error // hook called when client is closed
|
||||||
|
}
|
||||||
|
|
||||||
|
func newBaseClient(opt *Options, connPool pool.Pooler) *baseClient {
|
||||||
|
return &baseClient{
|
||||||
|
opt: opt,
|
||||||
|
connPool: connPool,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *baseClient) clone() *baseClient {
|
||||||
|
clone := *c
|
||||||
|
return &clone
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *baseClient) withTimeout(timeout time.Duration) *baseClient {
|
||||||
|
opt := c.opt.clone()
|
||||||
|
opt.ReadTimeout = timeout
|
||||||
|
opt.WriteTimeout = timeout
|
||||||
|
|
||||||
|
clone := c.clone()
|
||||||
|
clone.opt = opt
|
||||||
|
|
||||||
|
return clone
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *baseClient) String() string {
|
||||||
|
return fmt.Sprintf("Redis<%s db:%d>", c.getAddr(), c.opt.DB)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *baseClient) newConn(ctx context.Context) (*pool.Conn, error) {
|
||||||
|
cn, err := c.connPool.NewConn(ctx)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
|
||||||
|
err = c.initConn(ctx, cn)
|
||||||
|
if err != nil {
|
||||||
|
_ = c.connPool.CloseConn(cn)
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
|
||||||
|
return cn, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *baseClient) getConn(ctx context.Context) (*pool.Conn, error) {
|
||||||
|
if c.opt.Limiter != nil {
|
||||||
|
err := c.opt.Limiter.Allow()
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
cn, err := c._getConn(ctx)
|
||||||
|
if err != nil {
|
||||||
|
if c.opt.Limiter != nil {
|
||||||
|
c.opt.Limiter.ReportResult(err)
|
||||||
|
}
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
return cn, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *baseClient) _getConn(ctx context.Context) (*pool.Conn, error) {
|
||||||
|
cn, err := c.connPool.Get(ctx)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
|
||||||
|
err = c.initConn(ctx, cn)
|
||||||
|
if err != nil {
|
||||||
|
c.connPool.Remove(cn, err)
|
||||||
|
if err := internal.Unwrap(err); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
|
||||||
|
return cn, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *baseClient) initConn(ctx context.Context, cn *pool.Conn) error {
|
||||||
|
if cn.Inited {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
cn.Inited = true
|
||||||
|
|
||||||
|
if c.opt.Password == "" &&
|
||||||
|
c.opt.DB == 0 &&
|
||||||
|
!c.opt.readOnly &&
|
||||||
|
c.opt.OnConnect == nil {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
connPool := pool.NewSingleConnPool(nil)
|
||||||
|
connPool.SetConn(cn)
|
||||||
|
conn := newConn(ctx, c.opt, connPool)
|
||||||
|
|
||||||
|
_, err := conn.Pipelined(func(pipe Pipeliner) error {
|
||||||
|
if c.opt.Password != "" {
|
||||||
|
if c.opt.Username != "" {
|
||||||
|
pipe.AuthACL(c.opt.Username, c.opt.Password)
|
||||||
|
} else {
|
||||||
|
pipe.Auth(c.opt.Password)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if c.opt.DB > 0 {
|
||||||
|
pipe.Select(c.opt.DB)
|
||||||
|
}
|
||||||
|
|
||||||
|
if c.opt.readOnly {
|
||||||
|
pipe.ReadOnly()
|
||||||
|
}
|
||||||
|
|
||||||
|
return nil
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
if c.opt.OnConnect != nil {
|
||||||
|
return c.opt.OnConnect(conn)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *baseClient) releaseConn(cn *pool.Conn, err error) {
|
||||||
|
if c.opt.Limiter != nil {
|
||||||
|
c.opt.Limiter.ReportResult(err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if isBadConn(err, false) {
|
||||||
|
c.connPool.Remove(cn, err)
|
||||||
|
} else {
|
||||||
|
c.connPool.Put(cn)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *baseClient) withConn(
|
||||||
|
ctx context.Context, fn func(context.Context, *pool.Conn) error,
|
||||||
|
) error {
|
||||||
|
cn, err := c.getConn(ctx)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
defer func() {
|
||||||
|
c.releaseConn(cn, err)
|
||||||
|
}()
|
||||||
|
|
||||||
|
err = fn(ctx, cn)
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *baseClient) process(ctx context.Context, cmd Cmder) error {
|
||||||
|
err := c._process(ctx, cmd)
|
||||||
|
if err != nil {
|
||||||
|
cmd.SetErr(err)
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *baseClient) _process(ctx context.Context, cmd Cmder) error {
|
||||||
|
var lastErr error
|
||||||
|
for attempt := 0; attempt <= c.opt.MaxRetries; attempt++ {
|
||||||
|
if attempt > 0 {
|
||||||
|
if err := internal.Sleep(ctx, c.retryBackoff(attempt)); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
retryTimeout := true
|
||||||
|
lastErr = c.withConn(ctx, func(ctx context.Context, cn *pool.Conn) error {
|
||||||
|
err := cn.WithWriter(ctx, c.opt.WriteTimeout, func(wr *proto.Writer) error {
|
||||||
|
return writeCmd(wr, cmd)
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
err = cn.WithReader(ctx, c.cmdTimeout(cmd), cmd.readReply)
|
||||||
|
if err != nil {
|
||||||
|
retryTimeout = cmd.readTimeout() == nil
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
return nil
|
||||||
|
})
|
||||||
|
if lastErr == nil || !isRetryableError(lastErr, retryTimeout) {
|
||||||
|
return lastErr
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return lastErr
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *baseClient) retryBackoff(attempt int) time.Duration {
|
||||||
|
return internal.RetryBackoff(attempt, c.opt.MinRetryBackoff, c.opt.MaxRetryBackoff)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *baseClient) cmdTimeout(cmd Cmder) time.Duration {
|
||||||
|
if timeout := cmd.readTimeout(); timeout != nil {
|
||||||
|
t := *timeout
|
||||||
|
if t == 0 {
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
return t + 10*time.Second
|
||||||
|
}
|
||||||
|
return c.opt.ReadTimeout
|
||||||
|
}
|
||||||
|
|
||||||
|
// Close closes the client, releasing any open resources.
|
||||||
|
//
|
||||||
|
// It is rare to Close a Client, as the Client is meant to be
|
||||||
|
// long-lived and shared between many goroutines.
|
||||||
|
func (c *baseClient) Close() error {
|
||||||
|
var firstErr error
|
||||||
|
if c.onClose != nil {
|
||||||
|
if err := c.onClose(); err != nil {
|
||||||
|
firstErr = err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if err := c.connPool.Close(); err != nil && firstErr == nil {
|
||||||
|
firstErr = err
|
||||||
|
}
|
||||||
|
return firstErr
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *baseClient) getAddr() string {
|
||||||
|
return c.opt.Addr
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *baseClient) processPipeline(ctx context.Context, cmds []Cmder) error {
|
||||||
|
return c.generalProcessPipeline(ctx, cmds, c.pipelineProcessCmds)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *baseClient) processTxPipeline(ctx context.Context, cmds []Cmder) error {
|
||||||
|
return c.generalProcessPipeline(ctx, cmds, c.txPipelineProcessCmds)
|
||||||
|
}
|
||||||
|
|
||||||
|
type pipelineProcessor func(context.Context, *pool.Conn, []Cmder) (bool, error)
|
||||||
|
|
||||||
|
func (c *baseClient) generalProcessPipeline(
|
||||||
|
ctx context.Context, cmds []Cmder, p pipelineProcessor,
|
||||||
|
) error {
|
||||||
|
err := c._generalProcessPipeline(ctx, cmds, p)
|
||||||
|
if err != nil {
|
||||||
|
setCmdsErr(cmds, err)
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
return cmdsFirstErr(cmds)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *baseClient) _generalProcessPipeline(
|
||||||
|
ctx context.Context, cmds []Cmder, p pipelineProcessor,
|
||||||
|
) error {
|
||||||
|
var lastErr error
|
||||||
|
for attempt := 0; attempt <= c.opt.MaxRetries; attempt++ {
|
||||||
|
if attempt > 0 {
|
||||||
|
if err := internal.Sleep(ctx, c.retryBackoff(attempt)); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
var canRetry bool
|
||||||
|
lastErr = c.withConn(ctx, func(ctx context.Context, cn *pool.Conn) error {
|
||||||
|
var err error
|
||||||
|
canRetry, err = p(ctx, cn, cmds)
|
||||||
|
return err
|
||||||
|
})
|
||||||
|
if lastErr == nil || !canRetry || !isRetryableError(lastErr, true) {
|
||||||
|
return lastErr
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return lastErr
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *baseClient) pipelineProcessCmds(
|
||||||
|
ctx context.Context, cn *pool.Conn, cmds []Cmder,
|
||||||
|
) (bool, error) {
|
||||||
|
err := cn.WithWriter(ctx, c.opt.WriteTimeout, func(wr *proto.Writer) error {
|
||||||
|
return writeCmds(wr, cmds)
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
return true, err
|
||||||
|
}
|
||||||
|
|
||||||
|
err = cn.WithReader(ctx, c.opt.ReadTimeout, func(rd *proto.Reader) error {
|
||||||
|
return pipelineReadCmds(rd, cmds)
|
||||||
|
})
|
||||||
|
return true, err
|
||||||
|
}
|
||||||
|
|
||||||
|
func pipelineReadCmds(rd *proto.Reader, cmds []Cmder) error {
|
||||||
|
for _, cmd := range cmds {
|
||||||
|
err := cmd.readReply(rd)
|
||||||
|
if err != nil && !isRedisError(err) {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *baseClient) txPipelineProcessCmds(
|
||||||
|
ctx context.Context, cn *pool.Conn, cmds []Cmder,
|
||||||
|
) (bool, error) {
|
||||||
|
err := cn.WithWriter(ctx, c.opt.WriteTimeout, func(wr *proto.Writer) error {
|
||||||
|
return writeCmds(wr, cmds)
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
return true, err
|
||||||
|
}
|
||||||
|
|
||||||
|
err = cn.WithReader(ctx, c.opt.ReadTimeout, func(rd *proto.Reader) error {
|
||||||
|
statusCmd := cmds[0].(*StatusCmd)
|
||||||
|
// Trim multi and exec.
|
||||||
|
cmds = cmds[1 : len(cmds)-1]
|
||||||
|
|
||||||
|
err := txPipelineReadQueued(rd, statusCmd, cmds)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
return pipelineReadCmds(rd, cmds)
|
||||||
|
})
|
||||||
|
return false, err
|
||||||
|
}
|
||||||
|
|
||||||
|
func wrapMultiExec(cmds []Cmder) []Cmder {
|
||||||
|
if len(cmds) == 0 {
|
||||||
|
panic("not reached")
|
||||||
|
}
|
||||||
|
cmds = append(cmds, make([]Cmder, 2)...)
|
||||||
|
copy(cmds[1:], cmds[:len(cmds)-2])
|
||||||
|
cmds[0] = NewStatusCmd("multi")
|
||||||
|
cmds[len(cmds)-1] = NewSliceCmd("exec")
|
||||||
|
return cmds
|
||||||
|
}
|
||||||
|
|
||||||
|
func txPipelineReadQueued(rd *proto.Reader, statusCmd *StatusCmd, cmds []Cmder) error {
|
||||||
|
// Parse queued replies.
|
||||||
|
if err := statusCmd.readReply(rd); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
for range cmds {
|
||||||
|
if err := statusCmd.readReply(rd); err != nil && !isRedisError(err) {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Parse number of replies.
|
||||||
|
line, err := rd.ReadLine()
|
||||||
|
if err != nil {
|
||||||
|
if err == Nil {
|
||||||
|
err = TxFailedErr
|
||||||
|
}
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
switch line[0] {
|
||||||
|
case proto.ErrorReply:
|
||||||
|
return proto.ParseErrorReply(line)
|
||||||
|
case proto.ArrayReply:
|
||||||
|
// ok
|
||||||
|
default:
|
||||||
|
err := fmt.Errorf("redis: expected '*', but got line %q", line)
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
//------------------------------------------------------------------------------
|
||||||
|
|
||||||
|
// Client is a Redis client representing a pool of zero or more
|
||||||
|
// underlying connections. It's safe for concurrent use by multiple
|
||||||
|
// goroutines.
|
||||||
|
type Client struct {
|
||||||
|
*baseClient
|
||||||
|
cmdable
|
||||||
|
hooks
|
||||||
|
ctx context.Context
|
||||||
|
}
|
||||||
|
|
||||||
|
// NewClient returns a client to the Redis Server specified by Options.
|
||||||
|
func NewClient(opt *Options) *Client {
|
||||||
|
opt.init()
|
||||||
|
|
||||||
|
c := Client{
|
||||||
|
baseClient: newBaseClient(opt, newConnPool(opt)),
|
||||||
|
ctx: context.Background(),
|
||||||
|
}
|
||||||
|
c.cmdable = c.Process
|
||||||
|
|
||||||
|
return &c
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *Client) clone() *Client {
|
||||||
|
clone := *c
|
||||||
|
clone.cmdable = clone.Process
|
||||||
|
clone.hooks.lock()
|
||||||
|
return &clone
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *Client) WithTimeout(timeout time.Duration) *Client {
|
||||||
|
clone := c.clone()
|
||||||
|
clone.baseClient = c.baseClient.withTimeout(timeout)
|
||||||
|
return clone
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *Client) Context() context.Context {
|
||||||
|
return c.ctx
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *Client) WithContext(ctx context.Context) *Client {
|
||||||
|
if ctx == nil {
|
||||||
|
panic("nil context")
|
||||||
|
}
|
||||||
|
clone := c.clone()
|
||||||
|
clone.ctx = ctx
|
||||||
|
return clone
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *Client) Conn() *Conn {
|
||||||
|
return newConn(c.ctx, c.opt, pool.NewSingleConnPool(c.connPool))
|
||||||
|
}
|
||||||
|
|
||||||
|
// Do creates a Cmd from the args and processes the cmd.
|
||||||
|
func (c *Client) Do(args ...interface{}) *Cmd {
|
||||||
|
return c.DoContext(c.ctx, args...)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *Client) DoContext(ctx context.Context, args ...interface{}) *Cmd {
|
||||||
|
cmd := NewCmd(args...)
|
||||||
|
_ = c.ProcessContext(ctx, cmd)
|
||||||
|
return cmd
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *Client) Process(cmd Cmder) error {
|
||||||
|
return c.ProcessContext(c.ctx, cmd)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *Client) ProcessContext(ctx context.Context, cmd Cmder) error {
|
||||||
|
return c.hooks.process(ctx, cmd, c.baseClient.process)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *Client) processPipeline(ctx context.Context, cmds []Cmder) error {
|
||||||
|
return c.hooks.processPipeline(ctx, cmds, c.baseClient.processPipeline)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *Client) processTxPipeline(ctx context.Context, cmds []Cmder) error {
|
||||||
|
return c.hooks.processTxPipeline(ctx, cmds, c.baseClient.processTxPipeline)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Options returns read-only Options that were used to create the client.
|
||||||
|
func (c *Client) Options() *Options {
|
||||||
|
return c.opt
|
||||||
|
}
|
||||||
|
|
||||||
|
type PoolStats pool.Stats
|
||||||
|
|
||||||
|
// PoolStats returns connection pool stats.
|
||||||
|
func (c *Client) PoolStats() *PoolStats {
|
||||||
|
stats := c.connPool.Stats()
|
||||||
|
return (*PoolStats)(stats)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *Client) Pipelined(fn func(Pipeliner) error) ([]Cmder, error) {
|
||||||
|
return c.Pipeline().Pipelined(fn)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *Client) Pipeline() Pipeliner {
|
||||||
|
pipe := Pipeline{
|
||||||
|
ctx: c.ctx,
|
||||||
|
exec: c.processPipeline,
|
||||||
|
}
|
||||||
|
pipe.init()
|
||||||
|
return &pipe
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *Client) TxPipelined(fn func(Pipeliner) error) ([]Cmder, error) {
|
||||||
|
return c.TxPipeline().Pipelined(fn)
|
||||||
|
}
|
||||||
|
|
||||||
|
// TxPipeline acts like Pipeline, but wraps queued commands with MULTI/EXEC.
|
||||||
|
func (c *Client) TxPipeline() Pipeliner {
|
||||||
|
pipe := Pipeline{
|
||||||
|
ctx: c.ctx,
|
||||||
|
exec: c.processTxPipeline,
|
||||||
|
}
|
||||||
|
pipe.init()
|
||||||
|
return &pipe
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *Client) pubSub() *PubSub {
|
||||||
|
pubsub := &PubSub{
|
||||||
|
opt: c.opt,
|
||||||
|
|
||||||
|
newConn: func(channels []string) (*pool.Conn, error) {
|
||||||
|
return c.newConn(context.TODO())
|
||||||
|
},
|
||||||
|
closeConn: c.connPool.CloseConn,
|
||||||
|
}
|
||||||
|
pubsub.init()
|
||||||
|
return pubsub
|
||||||
|
}
|
||||||
|
|
||||||
|
// Subscribe subscribes the client to the specified channels.
|
||||||
|
// Channels can be omitted to create empty subscription.
|
||||||
|
// Note that this method does not wait on a response from Redis, so the
|
||||||
|
// subscription may not be active immediately. To force the connection to wait,
|
||||||
|
// you may call the Receive() method on the returned *PubSub like so:
|
||||||
|
//
|
||||||
|
// sub := client.Subscribe(queryResp)
|
||||||
|
// iface, err := sub.Receive()
|
||||||
|
// if err != nil {
|
||||||
|
// // handle error
|
||||||
|
// }
|
||||||
|
//
|
||||||
|
// // Should be *Subscription, but others are possible if other actions have been
|
||||||
|
// // taken on sub since it was created.
|
||||||
|
// switch iface.(type) {
|
||||||
|
// case *Subscription:
|
||||||
|
// // subscribe succeeded
|
||||||
|
// case *Message:
|
||||||
|
// // received first message
|
||||||
|
// case *Pong:
|
||||||
|
// // pong received
|
||||||
|
// default:
|
||||||
|
// // handle error
|
||||||
|
// }
|
||||||
|
//
|
||||||
|
// ch := sub.Channel()
|
||||||
|
func (c *Client) Subscribe(channels ...string) *PubSub {
|
||||||
|
pubsub := c.pubSub()
|
||||||
|
if len(channels) > 0 {
|
||||||
|
_ = pubsub.Subscribe(channels...)
|
||||||
|
}
|
||||||
|
return pubsub
|
||||||
|
}
|
||||||
|
|
||||||
|
// PSubscribe subscribes the client to the given patterns.
|
||||||
|
// Patterns can be omitted to create empty subscription.
|
||||||
|
func (c *Client) PSubscribe(channels ...string) *PubSub {
|
||||||
|
pubsub := c.pubSub()
|
||||||
|
if len(channels) > 0 {
|
||||||
|
_ = pubsub.PSubscribe(channels...)
|
||||||
|
}
|
||||||
|
return pubsub
|
||||||
|
}
|
||||||
|
|
||||||
|
//------------------------------------------------------------------------------
|
||||||
|
|
||||||
|
type conn struct {
|
||||||
|
baseClient
|
||||||
|
cmdable
|
||||||
|
statefulCmdable
|
||||||
|
}
|
||||||
|
|
||||||
|
// Conn is like Client, but its pool contains single connection.
|
||||||
|
type Conn struct {
|
||||||
|
*conn
|
||||||
|
ctx context.Context
|
||||||
|
}
|
||||||
|
|
||||||
|
func newConn(ctx context.Context, opt *Options, connPool pool.Pooler) *Conn {
|
||||||
|
c := Conn{
|
||||||
|
conn: &conn{
|
||||||
|
baseClient: baseClient{
|
||||||
|
opt: opt,
|
||||||
|
connPool: connPool,
|
||||||
|
},
|
||||||
|
},
|
||||||
|
ctx: ctx,
|
||||||
|
}
|
||||||
|
c.cmdable = c.Process
|
||||||
|
c.statefulCmdable = c.Process
|
||||||
|
return &c
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *Conn) Process(cmd Cmder) error {
|
||||||
|
return c.ProcessContext(c.ctx, cmd)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *Conn) ProcessContext(ctx context.Context, cmd Cmder) error {
|
||||||
|
return c.baseClient.process(ctx, cmd)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *Conn) Pipelined(fn func(Pipeliner) error) ([]Cmder, error) {
|
||||||
|
return c.Pipeline().Pipelined(fn)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *Conn) Pipeline() Pipeliner {
|
||||||
|
pipe := Pipeline{
|
||||||
|
ctx: c.ctx,
|
||||||
|
exec: c.processPipeline,
|
||||||
|
}
|
||||||
|
pipe.init()
|
||||||
|
return &pipe
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *Conn) TxPipelined(fn func(Pipeliner) error) ([]Cmder, error) {
|
||||||
|
return c.TxPipeline().Pipelined(fn)
|
||||||
|
}
|
||||||
|
|
||||||
|
// TxPipeline acts like Pipeline, but wraps queued commands with MULTI/EXEC.
|
||||||
|
func (c *Conn) TxPipeline() Pipeliner {
|
||||||
|
pipe := Pipeline{
|
||||||
|
ctx: c.ctx,
|
||||||
|
exec: c.processTxPipeline,
|
||||||
|
}
|
||||||
|
pipe.init()
|
||||||
|
return &pipe
|
||||||
|
}
|
|
@ -6,7 +6,7 @@ import "time"
|
||||||
func NewCmdResult(val interface{}, err error) *Cmd {
|
func NewCmdResult(val interface{}, err error) *Cmd {
|
||||||
var cmd Cmd
|
var cmd Cmd
|
||||||
cmd.val = val
|
cmd.val = val
|
||||||
cmd.setErr(err)
|
cmd.SetErr(err)
|
||||||
return &cmd
|
return &cmd
|
||||||
}
|
}
|
||||||
|
|
||||||
|
@ -14,7 +14,7 @@ func NewCmdResult(val interface{}, err error) *Cmd {
|
||||||
func NewSliceResult(val []interface{}, err error) *SliceCmd {
|
func NewSliceResult(val []interface{}, err error) *SliceCmd {
|
||||||
var cmd SliceCmd
|
var cmd SliceCmd
|
||||||
cmd.val = val
|
cmd.val = val
|
||||||
cmd.setErr(err)
|
cmd.SetErr(err)
|
||||||
return &cmd
|
return &cmd
|
||||||
}
|
}
|
||||||
|
|
||||||
|
@ -22,7 +22,7 @@ func NewSliceResult(val []interface{}, err error) *SliceCmd {
|
||||||
func NewStatusResult(val string, err error) *StatusCmd {
|
func NewStatusResult(val string, err error) *StatusCmd {
|
||||||
var cmd StatusCmd
|
var cmd StatusCmd
|
||||||
cmd.val = val
|
cmd.val = val
|
||||||
cmd.setErr(err)
|
cmd.SetErr(err)
|
||||||
return &cmd
|
return &cmd
|
||||||
}
|
}
|
||||||
|
|
||||||
|
@ -30,7 +30,7 @@ func NewStatusResult(val string, err error) *StatusCmd {
|
||||||
func NewIntResult(val int64, err error) *IntCmd {
|
func NewIntResult(val int64, err error) *IntCmd {
|
||||||
var cmd IntCmd
|
var cmd IntCmd
|
||||||
cmd.val = val
|
cmd.val = val
|
||||||
cmd.setErr(err)
|
cmd.SetErr(err)
|
||||||
return &cmd
|
return &cmd
|
||||||
}
|
}
|
||||||
|
|
||||||
|
@ -38,7 +38,7 @@ func NewIntResult(val int64, err error) *IntCmd {
|
||||||
func NewDurationResult(val time.Duration, err error) *DurationCmd {
|
func NewDurationResult(val time.Duration, err error) *DurationCmd {
|
||||||
var cmd DurationCmd
|
var cmd DurationCmd
|
||||||
cmd.val = val
|
cmd.val = val
|
||||||
cmd.setErr(err)
|
cmd.SetErr(err)
|
||||||
return &cmd
|
return &cmd
|
||||||
}
|
}
|
||||||
|
|
||||||
|
@ -46,7 +46,7 @@ func NewDurationResult(val time.Duration, err error) *DurationCmd {
|
||||||
func NewBoolResult(val bool, err error) *BoolCmd {
|
func NewBoolResult(val bool, err error) *BoolCmd {
|
||||||
var cmd BoolCmd
|
var cmd BoolCmd
|
||||||
cmd.val = val
|
cmd.val = val
|
||||||
cmd.setErr(err)
|
cmd.SetErr(err)
|
||||||
return &cmd
|
return &cmd
|
||||||
}
|
}
|
||||||
|
|
||||||
|
@ -54,7 +54,7 @@ func NewBoolResult(val bool, err error) *BoolCmd {
|
||||||
func NewStringResult(val string, err error) *StringCmd {
|
func NewStringResult(val string, err error) *StringCmd {
|
||||||
var cmd StringCmd
|
var cmd StringCmd
|
||||||
cmd.val = val
|
cmd.val = val
|
||||||
cmd.setErr(err)
|
cmd.SetErr(err)
|
||||||
return &cmd
|
return &cmd
|
||||||
}
|
}
|
||||||
|
|
||||||
|
@ -62,7 +62,7 @@ func NewStringResult(val string, err error) *StringCmd {
|
||||||
func NewFloatResult(val float64, err error) *FloatCmd {
|
func NewFloatResult(val float64, err error) *FloatCmd {
|
||||||
var cmd FloatCmd
|
var cmd FloatCmd
|
||||||
cmd.val = val
|
cmd.val = val
|
||||||
cmd.setErr(err)
|
cmd.SetErr(err)
|
||||||
return &cmd
|
return &cmd
|
||||||
}
|
}
|
||||||
|
|
||||||
|
@ -70,7 +70,7 @@ func NewFloatResult(val float64, err error) *FloatCmd {
|
||||||
func NewStringSliceResult(val []string, err error) *StringSliceCmd {
|
func NewStringSliceResult(val []string, err error) *StringSliceCmd {
|
||||||
var cmd StringSliceCmd
|
var cmd StringSliceCmd
|
||||||
cmd.val = val
|
cmd.val = val
|
||||||
cmd.setErr(err)
|
cmd.SetErr(err)
|
||||||
return &cmd
|
return &cmd
|
||||||
}
|
}
|
||||||
|
|
||||||
|
@ -78,7 +78,7 @@ func NewStringSliceResult(val []string, err error) *StringSliceCmd {
|
||||||
func NewBoolSliceResult(val []bool, err error) *BoolSliceCmd {
|
func NewBoolSliceResult(val []bool, err error) *BoolSliceCmd {
|
||||||
var cmd BoolSliceCmd
|
var cmd BoolSliceCmd
|
||||||
cmd.val = val
|
cmd.val = val
|
||||||
cmd.setErr(err)
|
cmd.SetErr(err)
|
||||||
return &cmd
|
return &cmd
|
||||||
}
|
}
|
||||||
|
|
||||||
|
@ -86,7 +86,7 @@ func NewBoolSliceResult(val []bool, err error) *BoolSliceCmd {
|
||||||
func NewStringStringMapResult(val map[string]string, err error) *StringStringMapCmd {
|
func NewStringStringMapResult(val map[string]string, err error) *StringStringMapCmd {
|
||||||
var cmd StringStringMapCmd
|
var cmd StringStringMapCmd
|
||||||
cmd.val = val
|
cmd.val = val
|
||||||
cmd.setErr(err)
|
cmd.SetErr(err)
|
||||||
return &cmd
|
return &cmd
|
||||||
}
|
}
|
||||||
|
|
||||||
|
@ -94,7 +94,15 @@ func NewStringStringMapResult(val map[string]string, err error) *StringStringMap
|
||||||
func NewStringIntMapCmdResult(val map[string]int64, err error) *StringIntMapCmd {
|
func NewStringIntMapCmdResult(val map[string]int64, err error) *StringIntMapCmd {
|
||||||
var cmd StringIntMapCmd
|
var cmd StringIntMapCmd
|
||||||
cmd.val = val
|
cmd.val = val
|
||||||
cmd.setErr(err)
|
cmd.SetErr(err)
|
||||||
|
return &cmd
|
||||||
|
}
|
||||||
|
|
||||||
|
// NewTimeCmdResult returns a TimeCmd initialised with val and err for testing
|
||||||
|
func NewTimeCmdResult(val time.Time, err error) *TimeCmd {
|
||||||
|
var cmd TimeCmd
|
||||||
|
cmd.val = val
|
||||||
|
cmd.SetErr(err)
|
||||||
return &cmd
|
return &cmd
|
||||||
}
|
}
|
||||||
|
|
||||||
|
@ -102,7 +110,15 @@ func NewStringIntMapCmdResult(val map[string]int64, err error) *StringIntMapCmd
|
||||||
func NewZSliceCmdResult(val []Z, err error) *ZSliceCmd {
|
func NewZSliceCmdResult(val []Z, err error) *ZSliceCmd {
|
||||||
var cmd ZSliceCmd
|
var cmd ZSliceCmd
|
||||||
cmd.val = val
|
cmd.val = val
|
||||||
cmd.setErr(err)
|
cmd.SetErr(err)
|
||||||
|
return &cmd
|
||||||
|
}
|
||||||
|
|
||||||
|
// NewZWithKeyCmdResult returns a NewZWithKeyCmd initialised with val and err for testing
|
||||||
|
func NewZWithKeyCmdResult(val *ZWithKey, err error) *ZWithKeyCmd {
|
||||||
|
var cmd ZWithKeyCmd
|
||||||
|
cmd.val = val
|
||||||
|
cmd.SetErr(err)
|
||||||
return &cmd
|
return &cmd
|
||||||
}
|
}
|
||||||
|
|
||||||
|
@ -111,7 +127,7 @@ func NewScanCmdResult(keys []string, cursor uint64, err error) *ScanCmd {
|
||||||
var cmd ScanCmd
|
var cmd ScanCmd
|
||||||
cmd.page = keys
|
cmd.page = keys
|
||||||
cmd.cursor = cursor
|
cmd.cursor = cursor
|
||||||
cmd.setErr(err)
|
cmd.SetErr(err)
|
||||||
return &cmd
|
return &cmd
|
||||||
}
|
}
|
||||||
|
|
||||||
|
@ -119,7 +135,7 @@ func NewScanCmdResult(keys []string, cursor uint64, err error) *ScanCmd {
|
||||||
func NewClusterSlotsCmdResult(val []ClusterSlot, err error) *ClusterSlotsCmd {
|
func NewClusterSlotsCmdResult(val []ClusterSlot, err error) *ClusterSlotsCmd {
|
||||||
var cmd ClusterSlotsCmd
|
var cmd ClusterSlotsCmd
|
||||||
cmd.val = val
|
cmd.val = val
|
||||||
cmd.setErr(err)
|
cmd.SetErr(err)
|
||||||
return &cmd
|
return &cmd
|
||||||
}
|
}
|
||||||
|
|
||||||
|
@ -127,7 +143,15 @@ func NewClusterSlotsCmdResult(val []ClusterSlot, err error) *ClusterSlotsCmd {
|
||||||
func NewGeoLocationCmdResult(val []GeoLocation, err error) *GeoLocationCmd {
|
func NewGeoLocationCmdResult(val []GeoLocation, err error) *GeoLocationCmd {
|
||||||
var cmd GeoLocationCmd
|
var cmd GeoLocationCmd
|
||||||
cmd.locations = val
|
cmd.locations = val
|
||||||
cmd.setErr(err)
|
cmd.SetErr(err)
|
||||||
|
return &cmd
|
||||||
|
}
|
||||||
|
|
||||||
|
// NewGeoPosCmdResult returns a GeoPosCmd initialised with val and err for testing
|
||||||
|
func NewGeoPosCmdResult(val []*GeoPos, err error) *GeoPosCmd {
|
||||||
|
var cmd GeoPosCmd
|
||||||
|
cmd.val = val
|
||||||
|
cmd.SetErr(err)
|
||||||
return &cmd
|
return &cmd
|
||||||
}
|
}
|
||||||
|
|
||||||
|
@ -135,6 +159,22 @@ func NewGeoLocationCmdResult(val []GeoLocation, err error) *GeoLocationCmd {
|
||||||
func NewCommandsInfoCmdResult(val map[string]*CommandInfo, err error) *CommandsInfoCmd {
|
func NewCommandsInfoCmdResult(val map[string]*CommandInfo, err error) *CommandsInfoCmd {
|
||||||
var cmd CommandsInfoCmd
|
var cmd CommandsInfoCmd
|
||||||
cmd.val = val
|
cmd.val = val
|
||||||
cmd.setErr(err)
|
cmd.SetErr(err)
|
||||||
|
return &cmd
|
||||||
|
}
|
||||||
|
|
||||||
|
// NewXMessageSliceCmdResult returns a XMessageSliceCmd initialised with val and err for testing
|
||||||
|
func NewXMessageSliceCmdResult(val []XMessage, err error) *XMessageSliceCmd {
|
||||||
|
var cmd XMessageSliceCmd
|
||||||
|
cmd.val = val
|
||||||
|
cmd.SetErr(err)
|
||||||
|
return &cmd
|
||||||
|
}
|
||||||
|
|
||||||
|
// NewXStreamSliceCmdResult returns a XStreamSliceCmd initialised with val and err for testing
|
||||||
|
func NewXStreamSliceCmdResult(val []XStream, err error) *XStreamSliceCmd {
|
||||||
|
var cmd XStreamSliceCmd
|
||||||
|
cmd.val = val
|
||||||
|
cmd.SetErr(err)
|
||||||
return &cmd
|
return &cmd
|
||||||
}
|
}
|
302
vendor/github.com/go-redis/redis/ring.go → vendor/github.com/go-redis/redis/v7/ring.go
generated
vendored
302
vendor/github.com/go-redis/redis/ring.go → vendor/github.com/go-redis/redis/v7/ring.go
generated
vendored
|
@ -10,10 +10,10 @@ import (
|
||||||
"sync/atomic"
|
"sync/atomic"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
"github.com/go-redis/redis/internal"
|
"github.com/go-redis/redis/v7/internal"
|
||||||
"github.com/go-redis/redis/internal/consistenthash"
|
"github.com/go-redis/redis/v7/internal/consistenthash"
|
||||||
"github.com/go-redis/redis/internal/hashtag"
|
"github.com/go-redis/redis/v7/internal/hashtag"
|
||||||
"github.com/go-redis/redis/internal/pool"
|
"github.com/go-redis/redis/v7/internal/pool"
|
||||||
)
|
)
|
||||||
|
|
||||||
// Hash is type of hash function used in consistent hash.
|
// Hash is type of hash function used in consistent hash.
|
||||||
|
@ -27,6 +27,10 @@ type RingOptions struct {
|
||||||
// Map of name => host:port addresses of ring shards.
|
// Map of name => host:port addresses of ring shards.
|
||||||
Addrs map[string]string
|
Addrs map[string]string
|
||||||
|
|
||||||
|
// Map of name => password of ring shards, to allow different shards to have
|
||||||
|
// different passwords. It will be ignored if the Password field is set.
|
||||||
|
Passwords map[string]string
|
||||||
|
|
||||||
// Frequency of PING commands sent to check shards availability.
|
// Frequency of PING commands sent to check shards availability.
|
||||||
// Shard is considered down after 3 subsequent failed checks.
|
// Shard is considered down after 3 subsequent failed checks.
|
||||||
HeartbeatFrequency time.Duration
|
HeartbeatFrequency time.Duration
|
||||||
|
@ -52,6 +56,12 @@ type RingOptions struct {
|
||||||
// See https://arxiv.org/abs/1406.2294 for reference
|
// See https://arxiv.org/abs/1406.2294 for reference
|
||||||
HashReplicas int
|
HashReplicas int
|
||||||
|
|
||||||
|
// NewClient creates a shard client with provided name and options.
|
||||||
|
NewClient func(name string, opt *Options) *Client
|
||||||
|
|
||||||
|
// Optional hook that is called when a new shard is created.
|
||||||
|
OnNewShard func(*Client)
|
||||||
|
|
||||||
// Following options are copied from Options struct.
|
// Following options are copied from Options struct.
|
||||||
|
|
||||||
OnConnect func(*Conn) error
|
OnConnect func(*Conn) error
|
||||||
|
@ -98,12 +108,12 @@ func (opt *RingOptions) init() {
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func (opt *RingOptions) clientOptions() *Options {
|
func (opt *RingOptions) clientOptions(shard string) *Options {
|
||||||
return &Options{
|
return &Options{
|
||||||
OnConnect: opt.OnConnect,
|
OnConnect: opt.OnConnect,
|
||||||
|
|
||||||
DB: opt.DB,
|
DB: opt.DB,
|
||||||
Password: opt.Password,
|
Password: opt.getPassword(shard),
|
||||||
|
|
||||||
DialTimeout: opt.DialTimeout,
|
DialTimeout: opt.DialTimeout,
|
||||||
ReadTimeout: opt.ReadTimeout,
|
ReadTimeout: opt.ReadTimeout,
|
||||||
|
@ -118,6 +128,13 @@ func (opt *RingOptions) clientOptions() *Options {
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func (opt *RingOptions) getPassword(shard string) string {
|
||||||
|
if opt.Password == "" {
|
||||||
|
return opt.Passwords[shard]
|
||||||
|
}
|
||||||
|
return opt.Password
|
||||||
|
}
|
||||||
|
|
||||||
//------------------------------------------------------------------------------
|
//------------------------------------------------------------------------------
|
||||||
|
|
||||||
type ringShard struct {
|
type ringShard struct {
|
||||||
|
@ -260,7 +277,7 @@ func (c *ringShards) Heartbeat(frequency time.Duration) {
|
||||||
for _, shard := range shards {
|
for _, shard := range shards {
|
||||||
err := shard.Client.Ping().Err()
|
err := shard.Client.Ping().Err()
|
||||||
if shard.Vote(err == nil || err == pool.ErrPoolTimeout) {
|
if shard.Vote(err == nil || err == pool.ErrPoolTimeout) {
|
||||||
internal.Logf("ring shard state changed: %s", shard)
|
internal.Logger.Printf("ring shard state changed: %s", shard)
|
||||||
rebalance = true
|
rebalance = true
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
@ -273,9 +290,13 @@ func (c *ringShards) Heartbeat(frequency time.Duration) {
|
||||||
|
|
||||||
// rebalance removes dead shards from the Ring.
|
// rebalance removes dead shards from the Ring.
|
||||||
func (c *ringShards) rebalance() {
|
func (c *ringShards) rebalance() {
|
||||||
|
c.mu.RLock()
|
||||||
|
shards := c.shards
|
||||||
|
c.mu.RUnlock()
|
||||||
|
|
||||||
hash := newConsistentHash(c.opt)
|
hash := newConsistentHash(c.opt)
|
||||||
var shardsNum int
|
var shardsNum int
|
||||||
for name, shard := range c.shards {
|
for name, shard := range shards {
|
||||||
if shard.IsUp() {
|
if shard.IsUp() {
|
||||||
hash.Add(name)
|
hash.Add(name)
|
||||||
shardsNum++
|
shardsNum++
|
||||||
|
@ -319,6 +340,12 @@ func (c *ringShards) Close() error {
|
||||||
|
|
||||||
//------------------------------------------------------------------------------
|
//------------------------------------------------------------------------------
|
||||||
|
|
||||||
|
type ring struct {
|
||||||
|
opt *RingOptions
|
||||||
|
shards *ringShards
|
||||||
|
cmdsInfoCache *cmdsInfoCache //nolint:structcheck
|
||||||
|
}
|
||||||
|
|
||||||
// Ring is a Redis client that uses consistent hashing to distribute
|
// Ring is a Redis client that uses consistent hashing to distribute
|
||||||
// keys across multiple Redis servers (shards). It's safe for
|
// keys across multiple Redis servers (shards). It's safe for
|
||||||
// concurrent use by multiple goroutines.
|
// concurrent use by multiple goroutines.
|
||||||
|
@ -334,61 +361,82 @@ func (c *ringShards) Close() error {
|
||||||
// and can tolerate losing data when one of the servers dies.
|
// and can tolerate losing data when one of the servers dies.
|
||||||
// Otherwise you should use Redis Cluster.
|
// Otherwise you should use Redis Cluster.
|
||||||
type Ring struct {
|
type Ring struct {
|
||||||
|
*ring
|
||||||
cmdable
|
cmdable
|
||||||
|
hooks
|
||||||
ctx context.Context
|
ctx context.Context
|
||||||
|
|
||||||
opt *RingOptions
|
|
||||||
shards *ringShards
|
|
||||||
cmdsInfoCache *cmdsInfoCache
|
|
||||||
|
|
||||||
process func(Cmder) error
|
|
||||||
processPipeline func([]Cmder) error
|
|
||||||
}
|
}
|
||||||
|
|
||||||
func NewRing(opt *RingOptions) *Ring {
|
func NewRing(opt *RingOptions) *Ring {
|
||||||
opt.init()
|
opt.init()
|
||||||
|
|
||||||
ring := &Ring{
|
ring := Ring{
|
||||||
|
ring: &ring{
|
||||||
opt: opt,
|
opt: opt,
|
||||||
shards: newRingShards(opt),
|
shards: newRingShards(opt),
|
||||||
|
},
|
||||||
|
ctx: context.Background(),
|
||||||
}
|
}
|
||||||
ring.cmdsInfoCache = newCmdsInfoCache(ring.cmdsInfo)
|
ring.cmdsInfoCache = newCmdsInfoCache(ring.cmdsInfo)
|
||||||
|
ring.cmdable = ring.Process
|
||||||
ring.process = ring.defaultProcess
|
|
||||||
ring.processPipeline = ring.defaultProcessPipeline
|
|
||||||
ring.cmdable.setProcessor(ring.Process)
|
|
||||||
|
|
||||||
for name, addr := range opt.Addrs {
|
for name, addr := range opt.Addrs {
|
||||||
clopt := opt.clientOptions()
|
shard := newRingShard(opt, name, addr)
|
||||||
clopt.Addr = addr
|
ring.shards.Add(name, shard)
|
||||||
ring.shards.Add(name, NewClient(clopt))
|
|
||||||
}
|
}
|
||||||
|
|
||||||
go ring.shards.Heartbeat(opt.HeartbeatFrequency)
|
go ring.shards.Heartbeat(opt.HeartbeatFrequency)
|
||||||
|
|
||||||
return ring
|
return &ring
|
||||||
|
}
|
||||||
|
|
||||||
|
func newRingShard(opt *RingOptions, name, addr string) *Client {
|
||||||
|
clopt := opt.clientOptions(name)
|
||||||
|
clopt.Addr = addr
|
||||||
|
var shard *Client
|
||||||
|
if opt.NewClient != nil {
|
||||||
|
shard = opt.NewClient(name, clopt)
|
||||||
|
} else {
|
||||||
|
shard = NewClient(clopt)
|
||||||
|
}
|
||||||
|
if opt.OnNewShard != nil {
|
||||||
|
opt.OnNewShard(shard)
|
||||||
|
}
|
||||||
|
return shard
|
||||||
}
|
}
|
||||||
|
|
||||||
func (c *Ring) Context() context.Context {
|
func (c *Ring) Context() context.Context {
|
||||||
if c.ctx != nil {
|
|
||||||
return c.ctx
|
return c.ctx
|
||||||
}
|
|
||||||
return context.Background()
|
|
||||||
}
|
}
|
||||||
|
|
||||||
func (c *Ring) WithContext(ctx context.Context) *Ring {
|
func (c *Ring) WithContext(ctx context.Context) *Ring {
|
||||||
if ctx == nil {
|
if ctx == nil {
|
||||||
panic("nil context")
|
panic("nil context")
|
||||||
}
|
}
|
||||||
c2 := c.copy()
|
clone := *c
|
||||||
c2.ctx = ctx
|
clone.cmdable = clone.Process
|
||||||
return c2
|
clone.hooks.lock()
|
||||||
|
clone.ctx = ctx
|
||||||
|
return &clone
|
||||||
}
|
}
|
||||||
|
|
||||||
func (c *Ring) copy() *Ring {
|
// Do creates a Cmd from the args and processes the cmd.
|
||||||
cp := *c
|
func (c *Ring) Do(args ...interface{}) *Cmd {
|
||||||
return &cp
|
return c.DoContext(c.ctx, args...)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *Ring) DoContext(ctx context.Context, args ...interface{}) *Cmd {
|
||||||
|
cmd := NewCmd(args...)
|
||||||
|
_ = c.ProcessContext(ctx, cmd)
|
||||||
|
return cmd
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *Ring) Process(cmd Cmder) error {
|
||||||
|
return c.ProcessContext(c.ctx, cmd)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *Ring) ProcessContext(ctx context.Context, cmd Cmder) error {
|
||||||
|
return c.hooks.process(ctx, cmd, c.process)
|
||||||
}
|
}
|
||||||
|
|
||||||
// Options returns read-only Options that were used to create the client.
|
// Options returns read-only Options that were used to create the client.
|
||||||
|
@ -428,7 +476,7 @@ func (c *Ring) Subscribe(channels ...string) *PubSub {
|
||||||
|
|
||||||
shard, err := c.shards.GetByKey(channels[0])
|
shard, err := c.shards.GetByKey(channels[0])
|
||||||
if err != nil {
|
if err != nil {
|
||||||
// TODO: return PubSub with sticky error
|
//TODO: return PubSub with sticky error
|
||||||
panic(err)
|
panic(err)
|
||||||
}
|
}
|
||||||
return shard.Client.Subscribe(channels...)
|
return shard.Client.Subscribe(channels...)
|
||||||
|
@ -442,7 +490,7 @@ func (c *Ring) PSubscribe(channels ...string) *PubSub {
|
||||||
|
|
||||||
shard, err := c.shards.GetByKey(channels[0])
|
shard, err := c.shards.GetByKey(channels[0])
|
||||||
if err != nil {
|
if err != nil {
|
||||||
// TODO: return PubSub with sticky error
|
//TODO: return PubSub with sticky error
|
||||||
panic(err)
|
panic(err)
|
||||||
}
|
}
|
||||||
return shard.Client.PSubscribe(channels...)
|
return shard.Client.PSubscribe(channels...)
|
||||||
|
@ -503,7 +551,7 @@ func (c *Ring) cmdInfo(name string) *CommandInfo {
|
||||||
}
|
}
|
||||||
info := cmdsInfo[name]
|
info := cmdsInfo[name]
|
||||||
if info == nil {
|
if info == nil {
|
||||||
internal.Logf("info for cmd=%s not found", name)
|
internal.Logger.Printf("info for cmd=%s not found", name)
|
||||||
}
|
}
|
||||||
return info
|
return info
|
||||||
}
|
}
|
||||||
|
@ -518,65 +566,78 @@ func (c *Ring) cmdShard(cmd Cmder) (*ringShard, error) {
|
||||||
return c.shards.GetByKey(firstKey)
|
return c.shards.GetByKey(firstKey)
|
||||||
}
|
}
|
||||||
|
|
||||||
// Do creates a Cmd from the args and processes the cmd.
|
func (c *Ring) process(ctx context.Context, cmd Cmder) error {
|
||||||
func (c *Ring) Do(args ...interface{}) *Cmd {
|
err := c._process(ctx, cmd)
|
||||||
cmd := NewCmd(args...)
|
if err != nil {
|
||||||
c.Process(cmd)
|
cmd.SetErr(err)
|
||||||
return cmd
|
return err
|
||||||
|
}
|
||||||
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
func (c *Ring) WrapProcess(
|
func (c *Ring) _process(ctx context.Context, cmd Cmder) error {
|
||||||
fn func(oldProcess func(cmd Cmder) error) func(cmd Cmder) error,
|
var lastErr error
|
||||||
) {
|
|
||||||
c.process = fn(c.process)
|
|
||||||
}
|
|
||||||
|
|
||||||
func (c *Ring) Process(cmd Cmder) error {
|
|
||||||
return c.process(cmd)
|
|
||||||
}
|
|
||||||
|
|
||||||
func (c *Ring) defaultProcess(cmd Cmder) error {
|
|
||||||
for attempt := 0; attempt <= c.opt.MaxRetries; attempt++ {
|
for attempt := 0; attempt <= c.opt.MaxRetries; attempt++ {
|
||||||
if attempt > 0 {
|
if attempt > 0 {
|
||||||
time.Sleep(c.retryBackoff(attempt))
|
if err := internal.Sleep(ctx, c.retryBackoff(attempt)); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
shard, err := c.cmdShard(cmd)
|
shard, err := c.cmdShard(cmd)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
cmd.setErr(err)
|
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
|
|
||||||
err = shard.Client.Process(cmd)
|
lastErr = shard.Client.ProcessContext(ctx, cmd)
|
||||||
if err == nil {
|
if lastErr == nil || !isRetryableError(lastErr, cmd.readTimeout() == nil) {
|
||||||
return nil
|
return lastErr
|
||||||
}
|
|
||||||
if !internal.IsRetryableError(err, cmd.readTimeout() == nil) {
|
|
||||||
return err
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
return cmd.Err()
|
return lastErr
|
||||||
}
|
|
||||||
|
|
||||||
func (c *Ring) Pipeline() Pipeliner {
|
|
||||||
pipe := Pipeline{
|
|
||||||
exec: c.processPipeline,
|
|
||||||
}
|
|
||||||
pipe.cmdable.setProcessor(pipe.Process)
|
|
||||||
return &pipe
|
|
||||||
}
|
}
|
||||||
|
|
||||||
func (c *Ring) Pipelined(fn func(Pipeliner) error) ([]Cmder, error) {
|
func (c *Ring) Pipelined(fn func(Pipeliner) error) ([]Cmder, error) {
|
||||||
return c.Pipeline().Pipelined(fn)
|
return c.Pipeline().Pipelined(fn)
|
||||||
}
|
}
|
||||||
|
|
||||||
func (c *Ring) WrapProcessPipeline(
|
func (c *Ring) Pipeline() Pipeliner {
|
||||||
fn func(oldProcess func([]Cmder) error) func([]Cmder) error,
|
pipe := Pipeline{
|
||||||
) {
|
ctx: c.ctx,
|
||||||
c.processPipeline = fn(c.processPipeline)
|
exec: c.processPipeline,
|
||||||
|
}
|
||||||
|
pipe.init()
|
||||||
|
return &pipe
|
||||||
}
|
}
|
||||||
|
|
||||||
func (c *Ring) defaultProcessPipeline(cmds []Cmder) error {
|
func (c *Ring) processPipeline(ctx context.Context, cmds []Cmder) error {
|
||||||
|
return c.hooks.processPipeline(ctx, cmds, func(ctx context.Context, cmds []Cmder) error {
|
||||||
|
return c.generalProcessPipeline(ctx, cmds, false)
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *Ring) TxPipelined(fn func(Pipeliner) error) ([]Cmder, error) {
|
||||||
|
return c.TxPipeline().Pipelined(fn)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *Ring) TxPipeline() Pipeliner {
|
||||||
|
pipe := Pipeline{
|
||||||
|
ctx: c.ctx,
|
||||||
|
exec: c.processTxPipeline,
|
||||||
|
}
|
||||||
|
pipe.init()
|
||||||
|
return &pipe
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *Ring) processTxPipeline(ctx context.Context, cmds []Cmder) error {
|
||||||
|
return c.hooks.processPipeline(ctx, cmds, func(ctx context.Context, cmds []Cmder) error {
|
||||||
|
return c.generalProcessPipeline(ctx, cmds, true)
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *Ring) generalProcessPipeline(
|
||||||
|
ctx context.Context, cmds []Cmder, tx bool,
|
||||||
|
) error {
|
||||||
cmdsMap := make(map[string][]Cmder)
|
cmdsMap := make(map[string][]Cmder)
|
||||||
for _, cmd := range cmds {
|
for _, cmd := range cmds {
|
||||||
cmdInfo := c.cmdInfo(cmd.Name())
|
cmdInfo := c.cmdInfo(cmd.Name())
|
||||||
|
@ -587,62 +648,36 @@ func (c *Ring) defaultProcessPipeline(cmds []Cmder) error {
|
||||||
cmdsMap[hash] = append(cmdsMap[hash], cmd)
|
cmdsMap[hash] = append(cmdsMap[hash], cmd)
|
||||||
}
|
}
|
||||||
|
|
||||||
for attempt := 0; attempt <= c.opt.MaxRetries; attempt++ {
|
|
||||||
if attempt > 0 {
|
|
||||||
time.Sleep(c.retryBackoff(attempt))
|
|
||||||
}
|
|
||||||
|
|
||||||
var mu sync.Mutex
|
|
||||||
var failedCmdsMap map[string][]Cmder
|
|
||||||
var wg sync.WaitGroup
|
var wg sync.WaitGroup
|
||||||
|
|
||||||
for hash, cmds := range cmdsMap {
|
for hash, cmds := range cmdsMap {
|
||||||
wg.Add(1)
|
wg.Add(1)
|
||||||
go func(hash string, cmds []Cmder) {
|
go func(hash string, cmds []Cmder) {
|
||||||
defer wg.Done()
|
defer wg.Done()
|
||||||
|
|
||||||
shard, err := c.shards.GetByHash(hash)
|
_ = c.processShardPipeline(ctx, hash, cmds, tx)
|
||||||
if err != nil {
|
|
||||||
setCmdsErr(cmds, err)
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
cn, err := shard.Client.getConn()
|
|
||||||
if err != nil {
|
|
||||||
setCmdsErr(cmds, err)
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
canRetry, err := shard.Client.pipelineProcessCmds(cn, cmds)
|
|
||||||
shard.Client.releaseConnStrict(cn, err)
|
|
||||||
|
|
||||||
if canRetry && internal.IsRetryableError(err, true) {
|
|
||||||
mu.Lock()
|
|
||||||
if failedCmdsMap == nil {
|
|
||||||
failedCmdsMap = make(map[string][]Cmder)
|
|
||||||
}
|
|
||||||
failedCmdsMap[hash] = cmds
|
|
||||||
mu.Unlock()
|
|
||||||
}
|
|
||||||
}(hash, cmds)
|
}(hash, cmds)
|
||||||
}
|
}
|
||||||
|
|
||||||
wg.Wait()
|
wg.Wait()
|
||||||
if len(failedCmdsMap) == 0 {
|
|
||||||
break
|
|
||||||
}
|
|
||||||
cmdsMap = failedCmdsMap
|
|
||||||
}
|
|
||||||
|
|
||||||
return cmdsFirstErr(cmds)
|
return cmdsFirstErr(cmds)
|
||||||
}
|
}
|
||||||
|
|
||||||
func (c *Ring) TxPipeline() Pipeliner {
|
func (c *Ring) processShardPipeline(
|
||||||
panic("not implemented")
|
ctx context.Context, hash string, cmds []Cmder, tx bool,
|
||||||
}
|
) error {
|
||||||
|
//TODO: retry?
|
||||||
|
shard, err := c.shards.GetByHash(hash)
|
||||||
|
if err != nil {
|
||||||
|
setCmdsErr(cmds, err)
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
func (c *Ring) TxPipelined(fn func(Pipeliner) error) ([]Cmder, error) {
|
if tx {
|
||||||
panic("not implemented")
|
err = shard.Client.processTxPipeline(ctx, cmds)
|
||||||
|
} else {
|
||||||
|
err = shard.Client.processPipeline(ctx, cmds)
|
||||||
|
}
|
||||||
|
return err
|
||||||
}
|
}
|
||||||
|
|
||||||
// Close closes the ring client, releasing any open resources.
|
// Close closes the ring client, releasing any open resources.
|
||||||
|
@ -653,6 +688,39 @@ func (c *Ring) Close() error {
|
||||||
return c.shards.Close()
|
return c.shards.Close()
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func (c *Ring) Watch(fn func(*Tx) error, keys ...string) error {
|
||||||
|
if len(keys) == 0 {
|
||||||
|
return fmt.Errorf("redis: Watch requires at least one key")
|
||||||
|
}
|
||||||
|
|
||||||
|
var shards []*ringShard
|
||||||
|
for _, key := range keys {
|
||||||
|
if key != "" {
|
||||||
|
shard, err := c.shards.GetByKey(hashtag.Key(key))
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
shards = append(shards, shard)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if len(shards) == 0 {
|
||||||
|
return fmt.Errorf("redis: Watch requires at least one shard")
|
||||||
|
}
|
||||||
|
|
||||||
|
if len(shards) > 1 {
|
||||||
|
for _, shard := range shards[1:] {
|
||||||
|
if shard.Client != shards[0].Client {
|
||||||
|
err := fmt.Errorf("redis: Watch requires all keys to be in the same shard")
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return shards[0].Client.Watch(fn, keys...)
|
||||||
|
}
|
||||||
|
|
||||||
func newConsistentHash(opt *RingOptions) *consistenthash.Map {
|
func newConsistentHash(opt *RingOptions) *consistenthash.Map {
|
||||||
return consistenthash.New(opt.HashReplicas, consistenthash.Hash(opt.Hash))
|
return consistenthash.New(opt.HashReplicas, consistenthash.Hash(opt.Hash))
|
||||||
}
|
}
|
|
@ -24,7 +24,7 @@ type Script struct {
|
||||||
|
|
||||||
func NewScript(src string) *Script {
|
func NewScript(src string) *Script {
|
||||||
h := sha1.New()
|
h := sha1.New()
|
||||||
io.WriteString(h, src)
|
_, _ = io.WriteString(h, src)
|
||||||
return &Script{
|
return &Script{
|
||||||
src: src,
|
src: src,
|
||||||
hash: hex.EncodeToString(h.Sum(nil)),
|
hash: hex.EncodeToString(h.Sum(nil)),
|
|
@ -1,6 +1,7 @@
|
||||||
package redis
|
package redis
|
||||||
|
|
||||||
import (
|
import (
|
||||||
|
"context"
|
||||||
"crypto/tls"
|
"crypto/tls"
|
||||||
"errors"
|
"errors"
|
||||||
"net"
|
"net"
|
||||||
|
@ -8,8 +9,8 @@ import (
|
||||||
"sync"
|
"sync"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
"github.com/go-redis/redis/internal"
|
"github.com/go-redis/redis/v7/internal"
|
||||||
"github.com/go-redis/redis/internal/pool"
|
"github.com/go-redis/redis/v7/internal/pool"
|
||||||
)
|
)
|
||||||
|
|
||||||
//------------------------------------------------------------------------------
|
//------------------------------------------------------------------------------
|
||||||
|
@ -21,11 +22,15 @@ type FailoverOptions struct {
|
||||||
MasterName string
|
MasterName string
|
||||||
// A seed list of host:port addresses of sentinel nodes.
|
// A seed list of host:port addresses of sentinel nodes.
|
||||||
SentinelAddrs []string
|
SentinelAddrs []string
|
||||||
|
SentinelUsername string
|
||||||
|
SentinelPassword string
|
||||||
|
|
||||||
// Following options are copied from Options struct.
|
// Following options are copied from Options struct.
|
||||||
|
|
||||||
|
Dialer func(ctx context.Context, network, addr string) (net.Conn, error)
|
||||||
OnConnect func(*Conn) error
|
OnConnect func(*Conn) error
|
||||||
|
|
||||||
|
Username string
|
||||||
Password string
|
Password string
|
||||||
DB int
|
DB int
|
||||||
|
|
||||||
|
@ -50,13 +55,16 @@ type FailoverOptions struct {
|
||||||
func (opt *FailoverOptions) options() *Options {
|
func (opt *FailoverOptions) options() *Options {
|
||||||
return &Options{
|
return &Options{
|
||||||
Addr: "FailoverClient",
|
Addr: "FailoverClient",
|
||||||
|
Dialer: opt.Dialer,
|
||||||
OnConnect: opt.OnConnect,
|
OnConnect: opt.OnConnect,
|
||||||
|
|
||||||
DB: opt.DB,
|
DB: opt.DB,
|
||||||
|
Username: opt.Username,
|
||||||
Password: opt.Password,
|
Password: opt.Password,
|
||||||
|
|
||||||
MaxRetries: opt.MaxRetries,
|
MaxRetries: opt.MaxRetries,
|
||||||
|
MinRetryBackoff: opt.MinRetryBackoff,
|
||||||
|
MaxRetryBackoff: opt.MaxRetryBackoff,
|
||||||
|
|
||||||
DialTimeout: opt.DialTimeout,
|
DialTimeout: opt.DialTimeout,
|
||||||
ReadTimeout: opt.ReadTimeout,
|
ReadTimeout: opt.ReadTimeout,
|
||||||
|
@ -66,6 +74,8 @@ func (opt *FailoverOptions) options() *Options {
|
||||||
PoolTimeout: opt.PoolTimeout,
|
PoolTimeout: opt.PoolTimeout,
|
||||||
IdleTimeout: opt.IdleTimeout,
|
IdleTimeout: opt.IdleTimeout,
|
||||||
IdleCheckFrequency: opt.IdleCheckFrequency,
|
IdleCheckFrequency: opt.IdleCheckFrequency,
|
||||||
|
MinIdleConns: opt.MinIdleConns,
|
||||||
|
MaxConnAge: opt.MaxConnAge,
|
||||||
|
|
||||||
TLSConfig: opt.TLSConfig,
|
TLSConfig: opt.TLSConfig,
|
||||||
}
|
}
|
||||||
|
@ -81,22 +91,18 @@ func NewFailoverClient(failoverOpt *FailoverOptions) *Client {
|
||||||
failover := &sentinelFailover{
|
failover := &sentinelFailover{
|
||||||
masterName: failoverOpt.MasterName,
|
masterName: failoverOpt.MasterName,
|
||||||
sentinelAddrs: failoverOpt.SentinelAddrs,
|
sentinelAddrs: failoverOpt.SentinelAddrs,
|
||||||
|
username: failoverOpt.SentinelUsername,
|
||||||
|
password: failoverOpt.SentinelPassword,
|
||||||
|
|
||||||
opt: opt,
|
opt: opt,
|
||||||
}
|
}
|
||||||
|
|
||||||
c := Client{
|
c := Client{
|
||||||
baseClient: baseClient{
|
baseClient: newBaseClient(opt, failover.Pool()),
|
||||||
opt: opt,
|
ctx: context.Background(),
|
||||||
connPool: failover.Pool(),
|
|
||||||
|
|
||||||
onClose: func() error {
|
|
||||||
return failover.Close()
|
|
||||||
},
|
|
||||||
},
|
|
||||||
}
|
}
|
||||||
c.baseClient.init()
|
c.cmdable = c.Process
|
||||||
c.cmdable.setProcessor(c.Process)
|
c.onClose = failover.Close
|
||||||
|
|
||||||
return &c
|
return &c
|
||||||
}
|
}
|
||||||
|
@ -104,27 +110,49 @@ func NewFailoverClient(failoverOpt *FailoverOptions) *Client {
|
||||||
//------------------------------------------------------------------------------
|
//------------------------------------------------------------------------------
|
||||||
|
|
||||||
type SentinelClient struct {
|
type SentinelClient struct {
|
||||||
baseClient
|
*baseClient
|
||||||
|
ctx context.Context
|
||||||
}
|
}
|
||||||
|
|
||||||
func NewSentinelClient(opt *Options) *SentinelClient {
|
func NewSentinelClient(opt *Options) *SentinelClient {
|
||||||
opt.init()
|
opt.init()
|
||||||
c := &SentinelClient{
|
c := &SentinelClient{
|
||||||
baseClient: baseClient{
|
baseClient: &baseClient{
|
||||||
opt: opt,
|
opt: opt,
|
||||||
connPool: newConnPool(opt),
|
connPool: newConnPool(opt),
|
||||||
},
|
},
|
||||||
|
ctx: context.Background(),
|
||||||
}
|
}
|
||||||
c.baseClient.init()
|
|
||||||
return c
|
return c
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func (c *SentinelClient) Context() context.Context {
|
||||||
|
return c.ctx
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *SentinelClient) WithContext(ctx context.Context) *SentinelClient {
|
||||||
|
if ctx == nil {
|
||||||
|
panic("nil context")
|
||||||
|
}
|
||||||
|
clone := *c
|
||||||
|
clone.ctx = ctx
|
||||||
|
return &clone
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *SentinelClient) Process(cmd Cmder) error {
|
||||||
|
return c.ProcessContext(c.ctx, cmd)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *SentinelClient) ProcessContext(ctx context.Context, cmd Cmder) error {
|
||||||
|
return c.baseClient.process(ctx, cmd)
|
||||||
|
}
|
||||||
|
|
||||||
func (c *SentinelClient) pubSub() *PubSub {
|
func (c *SentinelClient) pubSub() *PubSub {
|
||||||
pubsub := &PubSub{
|
pubsub := &PubSub{
|
||||||
opt: c.opt,
|
opt: c.opt,
|
||||||
|
|
||||||
newConn: func(channels []string) (*pool.Conn, error) {
|
newConn: func(channels []string) (*pool.Conn, error) {
|
||||||
return c.newConn()
|
return c.newConn(context.TODO())
|
||||||
},
|
},
|
||||||
closeConn: c.connPool.CloseConn,
|
closeConn: c.connPool.CloseConn,
|
||||||
}
|
}
|
||||||
|
@ -132,6 +160,14 @@ func (c *SentinelClient) pubSub() *PubSub {
|
||||||
return pubsub
|
return pubsub
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Ping is used to test if a connection is still alive, or to
|
||||||
|
// measure latency.
|
||||||
|
func (c *SentinelClient) Ping() *StringCmd {
|
||||||
|
cmd := NewStringCmd("ping")
|
||||||
|
_ = c.Process(cmd)
|
||||||
|
return cmd
|
||||||
|
}
|
||||||
|
|
||||||
// Subscribe subscribes the client to the specified channels.
|
// Subscribe subscribes the client to the specified channels.
|
||||||
// Channels can be omitted to create empty subscription.
|
// Channels can be omitted to create empty subscription.
|
||||||
func (c *SentinelClient) Subscribe(channels ...string) *PubSub {
|
func (c *SentinelClient) Subscribe(channels ...string) *PubSub {
|
||||||
|
@ -154,13 +190,13 @@ func (c *SentinelClient) PSubscribe(channels ...string) *PubSub {
|
||||||
|
|
||||||
func (c *SentinelClient) GetMasterAddrByName(name string) *StringSliceCmd {
|
func (c *SentinelClient) GetMasterAddrByName(name string) *StringSliceCmd {
|
||||||
cmd := NewStringSliceCmd("sentinel", "get-master-addr-by-name", name)
|
cmd := NewStringSliceCmd("sentinel", "get-master-addr-by-name", name)
|
||||||
c.Process(cmd)
|
_ = c.Process(cmd)
|
||||||
return cmd
|
return cmd
|
||||||
}
|
}
|
||||||
|
|
||||||
func (c *SentinelClient) Sentinels(name string) *SliceCmd {
|
func (c *SentinelClient) Sentinels(name string) *SliceCmd {
|
||||||
cmd := NewSliceCmd("sentinel", "sentinels", name)
|
cmd := NewSliceCmd("sentinel", "sentinels", name)
|
||||||
c.Process(cmd)
|
_ = c.Process(cmd)
|
||||||
return cmd
|
return cmd
|
||||||
}
|
}
|
||||||
|
|
||||||
|
@ -168,7 +204,7 @@ func (c *SentinelClient) Sentinels(name string) *SliceCmd {
|
||||||
// asking for agreement to other Sentinels.
|
// asking for agreement to other Sentinels.
|
||||||
func (c *SentinelClient) Failover(name string) *StatusCmd {
|
func (c *SentinelClient) Failover(name string) *StatusCmd {
|
||||||
cmd := NewStatusCmd("sentinel", "failover", name)
|
cmd := NewStatusCmd("sentinel", "failover", name)
|
||||||
c.Process(cmd)
|
_ = c.Process(cmd)
|
||||||
return cmd
|
return cmd
|
||||||
}
|
}
|
||||||
|
|
||||||
|
@ -178,7 +214,70 @@ func (c *SentinelClient) Failover(name string) *StatusCmd {
|
||||||
// already discovered and associated with the master.
|
// already discovered and associated with the master.
|
||||||
func (c *SentinelClient) Reset(pattern string) *IntCmd {
|
func (c *SentinelClient) Reset(pattern string) *IntCmd {
|
||||||
cmd := NewIntCmd("sentinel", "reset", pattern)
|
cmd := NewIntCmd("sentinel", "reset", pattern)
|
||||||
c.Process(cmd)
|
_ = c.Process(cmd)
|
||||||
|
return cmd
|
||||||
|
}
|
||||||
|
|
||||||
|
// FlushConfig forces Sentinel to rewrite its configuration on disk, including
|
||||||
|
// the current Sentinel state.
|
||||||
|
func (c *SentinelClient) FlushConfig() *StatusCmd {
|
||||||
|
cmd := NewStatusCmd("sentinel", "flushconfig")
|
||||||
|
_ = c.Process(cmd)
|
||||||
|
return cmd
|
||||||
|
}
|
||||||
|
|
||||||
|
// Master shows the state and info of the specified master.
|
||||||
|
func (c *SentinelClient) Master(name string) *StringStringMapCmd {
|
||||||
|
cmd := NewStringStringMapCmd("sentinel", "master", name)
|
||||||
|
_ = c.Process(cmd)
|
||||||
|
return cmd
|
||||||
|
}
|
||||||
|
|
||||||
|
// Masters shows a list of monitored masters and their state.
|
||||||
|
func (c *SentinelClient) Masters() *SliceCmd {
|
||||||
|
cmd := NewSliceCmd("sentinel", "masters")
|
||||||
|
_ = c.Process(cmd)
|
||||||
|
return cmd
|
||||||
|
}
|
||||||
|
|
||||||
|
// Slaves shows a list of slaves for the specified master and their state.
|
||||||
|
func (c *SentinelClient) Slaves(name string) *SliceCmd {
|
||||||
|
cmd := NewSliceCmd("sentinel", "slaves", name)
|
||||||
|
_ = c.Process(cmd)
|
||||||
|
return cmd
|
||||||
|
}
|
||||||
|
|
||||||
|
// CkQuorum checks if the current Sentinel configuration is able to reach the
|
||||||
|
// quorum needed to failover a master, and the majority needed to authorize the
|
||||||
|
// failover. This command should be used in monitoring systems to check if a
|
||||||
|
// Sentinel deployment is ok.
|
||||||
|
func (c *SentinelClient) CkQuorum(name string) *StringCmd {
|
||||||
|
cmd := NewStringCmd("sentinel", "ckquorum", name)
|
||||||
|
_ = c.Process(cmd)
|
||||||
|
return cmd
|
||||||
|
}
|
||||||
|
|
||||||
|
// Monitor tells the Sentinel to start monitoring a new master with the specified
|
||||||
|
// name, ip, port, and quorum.
|
||||||
|
func (c *SentinelClient) Monitor(name, ip, port, quorum string) *StringCmd {
|
||||||
|
cmd := NewStringCmd("sentinel", "monitor", name, ip, port, quorum)
|
||||||
|
_ = c.Process(cmd)
|
||||||
|
return cmd
|
||||||
|
}
|
||||||
|
|
||||||
|
// Set is used in order to change configuration parameters of a specific master.
|
||||||
|
func (c *SentinelClient) Set(name, option, value string) *StringCmd {
|
||||||
|
cmd := NewStringCmd("sentinel", "set", name, option, value)
|
||||||
|
_ = c.Process(cmd)
|
||||||
|
return cmd
|
||||||
|
}
|
||||||
|
|
||||||
|
// Remove is used in order to remove the specified master: the master will no
|
||||||
|
// longer be monitored, and will totally be removed from the internal state of
|
||||||
|
// the Sentinel.
|
||||||
|
func (c *SentinelClient) Remove(name string) *StringCmd {
|
||||||
|
cmd := NewStringCmd("sentinel", "remove", name)
|
||||||
|
_ = c.Process(cmd)
|
||||||
return cmd
|
return cmd
|
||||||
}
|
}
|
||||||
|
|
||||||
|
@ -186,6 +285,8 @@ type sentinelFailover struct {
|
||||||
sentinelAddrs []string
|
sentinelAddrs []string
|
||||||
|
|
||||||
opt *Options
|
opt *Options
|
||||||
|
username string
|
||||||
|
password string
|
||||||
|
|
||||||
pool *pool.ConnPool
|
pool *pool.ConnPool
|
||||||
poolOnce sync.Once
|
poolOnce sync.Once
|
||||||
|
@ -206,19 +307,36 @@ func (c *sentinelFailover) Close() error {
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func (c *sentinelFailover) closeSentinel() error {
|
||||||
|
firstErr := c.pubsub.Close()
|
||||||
|
c.pubsub = nil
|
||||||
|
|
||||||
|
err := c.sentinel.Close()
|
||||||
|
if err != nil && firstErr == nil {
|
||||||
|
firstErr = err
|
||||||
|
}
|
||||||
|
c.sentinel = nil
|
||||||
|
|
||||||
|
return firstErr
|
||||||
|
}
|
||||||
|
|
||||||
func (c *sentinelFailover) Pool() *pool.ConnPool {
|
func (c *sentinelFailover) Pool() *pool.ConnPool {
|
||||||
c.poolOnce.Do(func() {
|
c.poolOnce.Do(func() {
|
||||||
c.opt.Dialer = c.dial
|
opt := *c.opt
|
||||||
c.pool = newConnPool(c.opt)
|
opt.Dialer = c.dial
|
||||||
|
c.pool = newConnPool(&opt)
|
||||||
})
|
})
|
||||||
return c.pool
|
return c.pool
|
||||||
}
|
}
|
||||||
|
|
||||||
func (c *sentinelFailover) dial() (net.Conn, error) {
|
func (c *sentinelFailover) dial(ctx context.Context, network, _ string) (net.Conn, error) {
|
||||||
addr, err := c.MasterAddr()
|
addr, err := c.MasterAddr()
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
|
if c.opt.Dialer != nil {
|
||||||
|
return c.opt.Dialer(ctx, network, addr)
|
||||||
|
}
|
||||||
return net.DialTimeout("tcp", addr, c.opt.DialTimeout)
|
return net.DialTimeout("tcp", addr, c.opt.DialTimeout)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
@ -232,17 +350,35 @@ func (c *sentinelFailover) MasterAddr() (string, error) {
|
||||||
}
|
}
|
||||||
|
|
||||||
func (c *sentinelFailover) masterAddr() (string, error) {
|
func (c *sentinelFailover) masterAddr() (string, error) {
|
||||||
addr := c.getMasterAddr()
|
c.mu.RLock()
|
||||||
|
sentinel := c.sentinel
|
||||||
|
c.mu.RUnlock()
|
||||||
|
|
||||||
|
if sentinel != nil {
|
||||||
|
addr := c.getMasterAddr(sentinel)
|
||||||
if addr != "" {
|
if addr != "" {
|
||||||
return addr, nil
|
return addr, nil
|
||||||
}
|
}
|
||||||
|
}
|
||||||
|
|
||||||
c.mu.Lock()
|
c.mu.Lock()
|
||||||
defer c.mu.Unlock()
|
defer c.mu.Unlock()
|
||||||
|
|
||||||
|
if c.sentinel != nil {
|
||||||
|
addr := c.getMasterAddr(c.sentinel)
|
||||||
|
if addr != "" {
|
||||||
|
return addr, nil
|
||||||
|
}
|
||||||
|
_ = c.closeSentinel()
|
||||||
|
}
|
||||||
|
|
||||||
for i, sentinelAddr := range c.sentinelAddrs {
|
for i, sentinelAddr := range c.sentinelAddrs {
|
||||||
sentinel := NewSentinelClient(&Options{
|
sentinel := NewSentinelClient(&Options{
|
||||||
Addr: sentinelAddr,
|
Addr: sentinelAddr,
|
||||||
|
Dialer: c.opt.Dialer,
|
||||||
|
|
||||||
|
Username: c.username,
|
||||||
|
Password: c.password,
|
||||||
|
|
||||||
MaxRetries: c.opt.MaxRetries,
|
MaxRetries: c.opt.MaxRetries,
|
||||||
|
|
||||||
|
@ -260,7 +396,7 @@ func (c *sentinelFailover) masterAddr() (string, error) {
|
||||||
|
|
||||||
masterAddr, err := sentinel.GetMasterAddrByName(c.masterName).Result()
|
masterAddr, err := sentinel.GetMasterAddrByName(c.masterName).Result()
|
||||||
if err != nil {
|
if err != nil {
|
||||||
internal.Logf("sentinel: GetMasterAddrByName master=%q failed: %s",
|
internal.Logger.Printf("sentinel: GetMasterAddrByName master=%q failed: %s",
|
||||||
c.masterName, err)
|
c.masterName, err)
|
||||||
_ = sentinel.Close()
|
_ = sentinel.Close()
|
||||||
continue
|
continue
|
||||||
|
@ -277,27 +413,13 @@ func (c *sentinelFailover) masterAddr() (string, error) {
|
||||||
return "", errors.New("redis: all sentinels are unreachable")
|
return "", errors.New("redis: all sentinels are unreachable")
|
||||||
}
|
}
|
||||||
|
|
||||||
func (c *sentinelFailover) getMasterAddr() string {
|
func (c *sentinelFailover) getMasterAddr(sentinel *SentinelClient) string {
|
||||||
c.mu.RLock()
|
|
||||||
sentinel := c.sentinel
|
|
||||||
c.mu.RUnlock()
|
|
||||||
|
|
||||||
if sentinel == nil {
|
|
||||||
return ""
|
|
||||||
}
|
|
||||||
|
|
||||||
addr, err := sentinel.GetMasterAddrByName(c.masterName).Result()
|
addr, err := sentinel.GetMasterAddrByName(c.masterName).Result()
|
||||||
if err != nil {
|
if err != nil {
|
||||||
internal.Logf("sentinel: GetMasterAddrByName name=%q failed: %s",
|
internal.Logger.Printf("sentinel: GetMasterAddrByName name=%q failed: %s",
|
||||||
c.masterName, err)
|
c.masterName, err)
|
||||||
c.mu.Lock()
|
|
||||||
if c.sentinel == sentinel {
|
|
||||||
c.closeSentinel()
|
|
||||||
}
|
|
||||||
c.mu.Unlock()
|
|
||||||
return ""
|
return ""
|
||||||
}
|
}
|
||||||
|
|
||||||
return net.JoinHostPort(addr[0], addr[1])
|
return net.JoinHostPort(addr[0], addr[1])
|
||||||
}
|
}
|
||||||
|
|
||||||
|
@ -312,7 +434,11 @@ func (c *sentinelFailover) switchMaster(addr string) {
|
||||||
c.mu.Lock()
|
c.mu.Lock()
|
||||||
defer c.mu.Unlock()
|
defer c.mu.Unlock()
|
||||||
|
|
||||||
internal.Logf("sentinel: new master=%q addr=%q",
|
if c._masterAddr == addr {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
internal.Logger.Printf("sentinel: new master=%q addr=%q",
|
||||||
c.masterName, addr)
|
c.masterName, addr)
|
||||||
_ = c.Pool().Filter(func(cn *pool.Conn) bool {
|
_ = c.Pool().Filter(func(cn *pool.Conn) bool {
|
||||||
return cn.RemoteAddr().String() != addr
|
return cn.RemoteAddr().String() != addr
|
||||||
|
@ -321,35 +447,20 @@ func (c *sentinelFailover) switchMaster(addr string) {
|
||||||
}
|
}
|
||||||
|
|
||||||
func (c *sentinelFailover) setSentinel(sentinel *SentinelClient) {
|
func (c *sentinelFailover) setSentinel(sentinel *SentinelClient) {
|
||||||
c.discoverSentinels(sentinel)
|
if c.sentinel != nil {
|
||||||
|
panic("not reached")
|
||||||
|
}
|
||||||
c.sentinel = sentinel
|
c.sentinel = sentinel
|
||||||
|
c.discoverSentinels()
|
||||||
|
|
||||||
c.pubsub = sentinel.Subscribe("+switch-master")
|
c.pubsub = sentinel.Subscribe("+switch-master")
|
||||||
go c.listen(c.pubsub)
|
go c.listen(c.pubsub)
|
||||||
}
|
}
|
||||||
|
|
||||||
func (c *sentinelFailover) closeSentinel() error {
|
func (c *sentinelFailover) discoverSentinels() {
|
||||||
var firstErr error
|
sentinels, err := c.sentinel.Sentinels(c.masterName).Result()
|
||||||
|
|
||||||
err := c.pubsub.Close()
|
|
||||||
if err != nil && firstErr == err {
|
|
||||||
firstErr = err
|
|
||||||
}
|
|
||||||
c.pubsub = nil
|
|
||||||
|
|
||||||
err = c.sentinel.Close()
|
|
||||||
if err != nil && firstErr == err {
|
|
||||||
firstErr = err
|
|
||||||
}
|
|
||||||
c.sentinel = nil
|
|
||||||
|
|
||||||
return firstErr
|
|
||||||
}
|
|
||||||
|
|
||||||
func (c *sentinelFailover) discoverSentinels(sentinel *SentinelClient) {
|
|
||||||
sentinels, err := sentinel.Sentinels(c.masterName).Result()
|
|
||||||
if err != nil {
|
if err != nil {
|
||||||
internal.Logf("sentinel: Sentinels master=%q failed: %s", c.masterName, err)
|
internal.Logger.Printf("sentinel: Sentinels master=%q failed: %s", c.masterName, err)
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
for _, sentinel := range sentinels {
|
for _, sentinel := range sentinels {
|
||||||
|
@ -359,7 +470,7 @@ func (c *sentinelFailover) discoverSentinels(sentinel *SentinelClient) {
|
||||||
if key == "name" {
|
if key == "name" {
|
||||||
sentinelAddr := vals[i+1].(string)
|
sentinelAddr := vals[i+1].(string)
|
||||||
if !contains(c.sentinelAddrs, sentinelAddr) {
|
if !contains(c.sentinelAddrs, sentinelAddr) {
|
||||||
internal.Logf("sentinel: discovered new sentinel=%q for master=%q",
|
internal.Logger.Printf("sentinel: discovered new sentinel=%q for master=%q",
|
||||||
sentinelAddr, c.masterName)
|
sentinelAddr, c.masterName)
|
||||||
c.sentinelAddrs = append(c.sentinelAddrs, sentinelAddr)
|
c.sentinelAddrs = append(c.sentinelAddrs, sentinelAddr)
|
||||||
}
|
}
|
||||||
|
@ -376,11 +487,10 @@ func (c *sentinelFailover) listen(pubsub *PubSub) {
|
||||||
break
|
break
|
||||||
}
|
}
|
||||||
|
|
||||||
switch msg.Channel {
|
if msg.Channel == "+switch-master" {
|
||||||
case "+switch-master":
|
|
||||||
parts := strings.Split(msg.Payload, " ")
|
parts := strings.Split(msg.Payload, " ")
|
||||||
if parts[0] != c.masterName {
|
if parts[0] != c.masterName {
|
||||||
internal.Logf("sentinel: ignore addr for master=%q", parts[0])
|
internal.Logger.Printf("sentinel: ignore addr for master=%q", parts[0])
|
||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
addr := net.JoinHostPort(parts[3], parts[4])
|
addr := net.JoinHostPort(parts[3], parts[4])
|
91
vendor/github.com/go-redis/redis/tx.go → vendor/github.com/go-redis/redis/v7/tx.go
generated
vendored
91
vendor/github.com/go-redis/redis/tx.go → vendor/github.com/go-redis/redis/v7/tx.go
generated
vendored
|
@ -1,8 +1,10 @@
|
||||||
package redis
|
package redis
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"github.com/go-redis/redis/internal/pool"
|
"context"
|
||||||
"github.com/go-redis/redis/internal/proto"
|
|
||||||
|
"github.com/go-redis/redis/v7/internal/pool"
|
||||||
|
"github.com/go-redis/redis/v7/internal/proto"
|
||||||
)
|
)
|
||||||
|
|
||||||
// TxFailedErr transaction redis failed.
|
// TxFailedErr transaction redis failed.
|
||||||
|
@ -13,28 +15,64 @@ const TxFailedErr = proto.RedisError("redis: transaction failed")
|
||||||
// by multiple goroutines, because Exec resets list of watched keys.
|
// by multiple goroutines, because Exec resets list of watched keys.
|
||||||
// If you don't need WATCH it is better to use Pipeline.
|
// If you don't need WATCH it is better to use Pipeline.
|
||||||
type Tx struct {
|
type Tx struct {
|
||||||
statefulCmdable
|
|
||||||
baseClient
|
baseClient
|
||||||
|
cmdable
|
||||||
|
statefulCmdable
|
||||||
|
hooks
|
||||||
|
ctx context.Context
|
||||||
}
|
}
|
||||||
|
|
||||||
func (c *Client) newTx() *Tx {
|
func (c *Client) newTx(ctx context.Context) *Tx {
|
||||||
tx := Tx{
|
tx := Tx{
|
||||||
baseClient: baseClient{
|
baseClient: baseClient{
|
||||||
opt: c.opt,
|
opt: c.opt,
|
||||||
connPool: pool.NewStickyConnPool(c.connPool.(*pool.ConnPool), true),
|
connPool: pool.NewStickyConnPool(c.connPool.(*pool.ConnPool), true),
|
||||||
},
|
},
|
||||||
|
hooks: c.hooks.clone(),
|
||||||
|
ctx: ctx,
|
||||||
}
|
}
|
||||||
tx.baseClient.init()
|
tx.init()
|
||||||
tx.statefulCmdable.setProcessor(tx.Process)
|
|
||||||
return &tx
|
return &tx
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func (c *Tx) init() {
|
||||||
|
c.cmdable = c.Process
|
||||||
|
c.statefulCmdable = c.Process
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *Tx) Context() context.Context {
|
||||||
|
return c.ctx
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *Tx) WithContext(ctx context.Context) *Tx {
|
||||||
|
if ctx == nil {
|
||||||
|
panic("nil context")
|
||||||
|
}
|
||||||
|
clone := *c
|
||||||
|
clone.init()
|
||||||
|
clone.hooks.lock()
|
||||||
|
clone.ctx = ctx
|
||||||
|
return &clone
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *Tx) Process(cmd Cmder) error {
|
||||||
|
return c.ProcessContext(c.ctx, cmd)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *Tx) ProcessContext(ctx context.Context, cmd Cmder) error {
|
||||||
|
return c.hooks.process(ctx, cmd, c.baseClient.process)
|
||||||
|
}
|
||||||
|
|
||||||
// Watch prepares a transaction and marks the keys to be watched
|
// Watch prepares a transaction and marks the keys to be watched
|
||||||
// for conditional execution if there are any keys.
|
// for conditional execution if there are any keys.
|
||||||
//
|
//
|
||||||
// The transaction is automatically closed when fn exits.
|
// The transaction is automatically closed when fn exits.
|
||||||
func (c *Client) Watch(fn func(*Tx) error, keys ...string) error {
|
func (c *Client) Watch(fn func(*Tx) error, keys ...string) error {
|
||||||
tx := c.newTx()
|
return c.WatchContext(c.ctx, fn, keys...)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *Client) WatchContext(ctx context.Context, fn func(*Tx) error, keys ...string) error {
|
||||||
|
tx := c.newTx(ctx)
|
||||||
if len(keys) > 0 {
|
if len(keys) > 0 {
|
||||||
if err := tx.Watch(keys...).Err(); err != nil {
|
if err := tx.Watch(keys...).Err(); err != nil {
|
||||||
_ = tx.Close()
|
_ = tx.Close()
|
||||||
|
@ -62,7 +100,7 @@ func (c *Tx) Watch(keys ...string) *StatusCmd {
|
||||||
args[1+i] = key
|
args[1+i] = key
|
||||||
}
|
}
|
||||||
cmd := NewStatusCmd(args...)
|
cmd := NewStatusCmd(args...)
|
||||||
c.Process(cmd)
|
_ = c.Process(cmd)
|
||||||
return cmd
|
return cmd
|
||||||
}
|
}
|
||||||
|
|
||||||
|
@ -74,20 +112,29 @@ func (c *Tx) Unwatch(keys ...string) *StatusCmd {
|
||||||
args[1+i] = key
|
args[1+i] = key
|
||||||
}
|
}
|
||||||
cmd := NewStatusCmd(args...)
|
cmd := NewStatusCmd(args...)
|
||||||
c.Process(cmd)
|
_ = c.Process(cmd)
|
||||||
return cmd
|
return cmd
|
||||||
}
|
}
|
||||||
|
|
||||||
// Pipeline creates a new pipeline. It is more convenient to use Pipelined.
|
// Pipeline creates a pipeline. Usually it is more convenient to use Pipelined.
|
||||||
func (c *Tx) Pipeline() Pipeliner {
|
func (c *Tx) Pipeline() Pipeliner {
|
||||||
pipe := Pipeline{
|
pipe := Pipeline{
|
||||||
exec: c.processTxPipeline,
|
ctx: c.ctx,
|
||||||
|
exec: func(ctx context.Context, cmds []Cmder) error {
|
||||||
|
return c.hooks.processPipeline(ctx, cmds, c.baseClient.processPipeline)
|
||||||
|
},
|
||||||
}
|
}
|
||||||
pipe.statefulCmdable.setProcessor(pipe.Process)
|
pipe.init()
|
||||||
return &pipe
|
return &pipe
|
||||||
}
|
}
|
||||||
|
|
||||||
// Pipelined executes commands queued in the fn in a transaction.
|
// Pipelined executes commands queued in the fn outside of the transaction.
|
||||||
|
// Use TxPipelined if you need transactional behavior.
|
||||||
|
func (c *Tx) Pipelined(fn func(Pipeliner) error) ([]Cmder, error) {
|
||||||
|
return c.Pipeline().Pipelined(fn)
|
||||||
|
}
|
||||||
|
|
||||||
|
// TxPipelined executes commands queued in the fn in the transaction.
|
||||||
//
|
//
|
||||||
// When using WATCH, EXEC will execute commands only if the watched keys
|
// When using WATCH, EXEC will execute commands only if the watched keys
|
||||||
// were not modified, allowing for a check-and-set mechanism.
|
// were not modified, allowing for a check-and-set mechanism.
|
||||||
|
@ -95,16 +142,18 @@ func (c *Tx) Pipeline() Pipeliner {
|
||||||
// Exec always returns list of commands. If transaction fails
|
// Exec always returns list of commands. If transaction fails
|
||||||
// TxFailedErr is returned. Otherwise Exec returns an error of the first
|
// TxFailedErr is returned. Otherwise Exec returns an error of the first
|
||||||
// failed command or nil.
|
// failed command or nil.
|
||||||
func (c *Tx) Pipelined(fn func(Pipeliner) error) ([]Cmder, error) {
|
|
||||||
return c.Pipeline().Pipelined(fn)
|
|
||||||
}
|
|
||||||
|
|
||||||
// TxPipelined is an alias for Pipelined.
|
|
||||||
func (c *Tx) TxPipelined(fn func(Pipeliner) error) ([]Cmder, error) {
|
func (c *Tx) TxPipelined(fn func(Pipeliner) error) ([]Cmder, error) {
|
||||||
return c.Pipelined(fn)
|
return c.TxPipeline().Pipelined(fn)
|
||||||
}
|
}
|
||||||
|
|
||||||
// TxPipeline is an alias for Pipeline.
|
// TxPipeline creates a pipeline. Usually it is more convenient to use TxPipelined.
|
||||||
func (c *Tx) TxPipeline() Pipeliner {
|
func (c *Tx) TxPipeline() Pipeliner {
|
||||||
return c.Pipeline()
|
pipe := Pipeline{
|
||||||
|
ctx: c.ctx,
|
||||||
|
exec: func(ctx context.Context, cmds []Cmder) error {
|
||||||
|
return c.hooks.processTxPipeline(ctx, cmds, c.baseClient.processTxPipeline)
|
||||||
|
},
|
||||||
|
}
|
||||||
|
pipe.init()
|
||||||
|
return &pipe
|
||||||
}
|
}
|
|
@ -1,7 +1,9 @@
|
||||||
package redis
|
package redis
|
||||||
|
|
||||||
import (
|
import (
|
||||||
|
"context"
|
||||||
"crypto/tls"
|
"crypto/tls"
|
||||||
|
"net"
|
||||||
"time"
|
"time"
|
||||||
)
|
)
|
||||||
|
|
||||||
|
@ -18,7 +20,9 @@ type UniversalOptions struct {
|
||||||
|
|
||||||
// Common options.
|
// Common options.
|
||||||
|
|
||||||
|
Dialer func(ctx context.Context, network, addr string) (net.Conn, error)
|
||||||
OnConnect func(*Conn) error
|
OnConnect func(*Conn) error
|
||||||
|
Username string
|
||||||
Password string
|
Password string
|
||||||
MaxRetries int
|
MaxRetries int
|
||||||
MinRetryBackoff time.Duration
|
MinRetryBackoff time.Duration
|
||||||
|
@ -46,15 +50,18 @@ type UniversalOptions struct {
|
||||||
MasterName string
|
MasterName string
|
||||||
}
|
}
|
||||||
|
|
||||||
func (o *UniversalOptions) cluster() *ClusterOptions {
|
// Cluster returns cluster options created from the universal options.
|
||||||
|
func (o *UniversalOptions) Cluster() *ClusterOptions {
|
||||||
if len(o.Addrs) == 0 {
|
if len(o.Addrs) == 0 {
|
||||||
o.Addrs = []string{"127.0.0.1:6379"}
|
o.Addrs = []string{"127.0.0.1:6379"}
|
||||||
}
|
}
|
||||||
|
|
||||||
return &ClusterOptions{
|
return &ClusterOptions{
|
||||||
Addrs: o.Addrs,
|
Addrs: o.Addrs,
|
||||||
|
Dialer: o.Dialer,
|
||||||
OnConnect: o.OnConnect,
|
OnConnect: o.OnConnect,
|
||||||
|
|
||||||
|
Username: o.Username,
|
||||||
Password: o.Password,
|
Password: o.Password,
|
||||||
|
|
||||||
MaxRedirects: o.MaxRedirects,
|
MaxRedirects: o.MaxRedirects,
|
||||||
|
@ -80,7 +87,8 @@ func (o *UniversalOptions) cluster() *ClusterOptions {
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func (o *UniversalOptions) failover() *FailoverOptions {
|
// Failover returns failover options created from the universal options.
|
||||||
|
func (o *UniversalOptions) Failover() *FailoverOptions {
|
||||||
if len(o.Addrs) == 0 {
|
if len(o.Addrs) == 0 {
|
||||||
o.Addrs = []string{"127.0.0.1:26379"}
|
o.Addrs = []string{"127.0.0.1:26379"}
|
||||||
}
|
}
|
||||||
|
@ -88,9 +96,12 @@ func (o *UniversalOptions) failover() *FailoverOptions {
|
||||||
return &FailoverOptions{
|
return &FailoverOptions{
|
||||||
SentinelAddrs: o.Addrs,
|
SentinelAddrs: o.Addrs,
|
||||||
MasterName: o.MasterName,
|
MasterName: o.MasterName,
|
||||||
|
|
||||||
|
Dialer: o.Dialer,
|
||||||
OnConnect: o.OnConnect,
|
OnConnect: o.OnConnect,
|
||||||
|
|
||||||
DB: o.DB,
|
DB: o.DB,
|
||||||
|
Username: o.Username,
|
||||||
Password: o.Password,
|
Password: o.Password,
|
||||||
|
|
||||||
MaxRetries: o.MaxRetries,
|
MaxRetries: o.MaxRetries,
|
||||||
|
@ -112,7 +123,8 @@ func (o *UniversalOptions) failover() *FailoverOptions {
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func (o *UniversalOptions) simple() *Options {
|
// Simple returns basic options created from the universal options.
|
||||||
|
func (o *UniversalOptions) Simple() *Options {
|
||||||
addr := "127.0.0.1:6379"
|
addr := "127.0.0.1:6379"
|
||||||
if len(o.Addrs) > 0 {
|
if len(o.Addrs) > 0 {
|
||||||
addr = o.Addrs[0]
|
addr = o.Addrs[0]
|
||||||
|
@ -120,9 +132,11 @@ func (o *UniversalOptions) simple() *Options {
|
||||||
|
|
||||||
return &Options{
|
return &Options{
|
||||||
Addr: addr,
|
Addr: addr,
|
||||||
|
Dialer: o.Dialer,
|
||||||
OnConnect: o.OnConnect,
|
OnConnect: o.OnConnect,
|
||||||
|
|
||||||
DB: o.DB,
|
DB: o.DB,
|
||||||
|
Username: o.Username,
|
||||||
Password: o.Password,
|
Password: o.Password,
|
||||||
|
|
||||||
MaxRetries: o.MaxRetries,
|
MaxRetries: o.MaxRetries,
|
||||||
|
@ -147,14 +161,18 @@ func (o *UniversalOptions) simple() *Options {
|
||||||
// --------------------------------------------------------------------
|
// --------------------------------------------------------------------
|
||||||
|
|
||||||
// UniversalClient is an abstract client which - based on the provided options -
|
// UniversalClient is an abstract client which - based on the provided options -
|
||||||
// can connect to either clusters, or sentinel-backed failover instances or simple
|
// can connect to either clusters, or sentinel-backed failover instances
|
||||||
// single-instance servers. This can be useful for testing cluster-specific
|
// or simple single-instance servers. This can be useful for testing
|
||||||
// applications locally.
|
// cluster-specific applications locally.
|
||||||
type UniversalClient interface {
|
type UniversalClient interface {
|
||||||
Cmdable
|
Cmdable
|
||||||
|
Context() context.Context
|
||||||
|
AddHook(Hook)
|
||||||
Watch(fn func(*Tx) error, keys ...string) error
|
Watch(fn func(*Tx) error, keys ...string) error
|
||||||
|
Do(args ...interface{}) *Cmd
|
||||||
|
DoContext(ctx context.Context, args ...interface{}) *Cmd
|
||||||
Process(cmd Cmder) error
|
Process(cmd Cmder) error
|
||||||
WrapProcess(fn func(oldProcess func(cmd Cmder) error) func(cmd Cmder) error)
|
ProcessContext(ctx context.Context, cmd Cmder) error
|
||||||
Subscribe(channels ...string) *PubSub
|
Subscribe(channels ...string) *PubSub
|
||||||
PSubscribe(channels ...string) *PubSub
|
PSubscribe(channels ...string) *PubSub
|
||||||
Close() error
|
Close() error
|
||||||
|
@ -162,6 +180,7 @@ type UniversalClient interface {
|
||||||
|
|
||||||
var _ UniversalClient = (*Client)(nil)
|
var _ UniversalClient = (*Client)(nil)
|
||||||
var _ UniversalClient = (*ClusterClient)(nil)
|
var _ UniversalClient = (*ClusterClient)(nil)
|
||||||
|
var _ UniversalClient = (*Ring)(nil)
|
||||||
|
|
||||||
// NewUniversalClient returns a new multi client. The type of client returned depends
|
// NewUniversalClient returns a new multi client. The type of client returned depends
|
||||||
// on the following three conditions:
|
// on the following three conditions:
|
||||||
|
@ -171,9 +190,9 @@ var _ UniversalClient = (*ClusterClient)(nil)
|
||||||
// 3. otherwise, a single-node redis Client will be returned.
|
// 3. otherwise, a single-node redis Client will be returned.
|
||||||
func NewUniversalClient(opts *UniversalOptions) UniversalClient {
|
func NewUniversalClient(opts *UniversalOptions) UniversalClient {
|
||||||
if opts.MasterName != "" {
|
if opts.MasterName != "" {
|
||||||
return NewFailoverClient(opts.failover())
|
return NewFailoverClient(opts.Failover())
|
||||||
} else if len(opts.Addrs) > 1 {
|
} else if len(opts.Addrs) > 1 {
|
||||||
return NewClusterClient(opts.cluster())
|
return NewClusterClient(opts.Cluster())
|
||||||
}
|
}
|
||||||
return NewClient(opts.simple())
|
return NewClient(opts.Simple())
|
||||||
}
|
}
|
|
@ -14,7 +14,6 @@ gitea.com/macaron/binding
|
||||||
## explicit
|
## explicit
|
||||||
gitea.com/macaron/cache
|
gitea.com/macaron/cache
|
||||||
gitea.com/macaron/cache/memcache
|
gitea.com/macaron/cache/memcache
|
||||||
gitea.com/macaron/cache/redis
|
|
||||||
# gitea.com/macaron/captcha v0.0.0-20190822015246-daa973478bae
|
# gitea.com/macaron/captcha v0.0.0-20190822015246-daa973478bae
|
||||||
## explicit
|
## explicit
|
||||||
gitea.com/macaron/captcha
|
gitea.com/macaron/captcha
|
||||||
|
@ -44,7 +43,6 @@ gitea.com/macaron/session/memcache
|
||||||
gitea.com/macaron/session/mysql
|
gitea.com/macaron/session/mysql
|
||||||
gitea.com/macaron/session/nodb
|
gitea.com/macaron/session/nodb
|
||||||
gitea.com/macaron/session/postgres
|
gitea.com/macaron/session/postgres
|
||||||
gitea.com/macaron/session/redis
|
|
||||||
# gitea.com/macaron/toolbox v0.0.0-20190822013122-05ff0fc766b7
|
# gitea.com/macaron/toolbox v0.0.0-20190822013122-05ff0fc766b7
|
||||||
## explicit
|
## explicit
|
||||||
gitea.com/macaron/toolbox
|
gitea.com/macaron/toolbox
|
||||||
|
@ -347,15 +345,15 @@ github.com/go-openapi/strfmt
|
||||||
github.com/go-openapi/swag
|
github.com/go-openapi/swag
|
||||||
# github.com/go-openapi/validate v0.19.10
|
# github.com/go-openapi/validate v0.19.10
|
||||||
github.com/go-openapi/validate
|
github.com/go-openapi/validate
|
||||||
# github.com/go-redis/redis v6.15.2+incompatible
|
# github.com/go-redis/redis/v7 v7.4.0
|
||||||
## explicit
|
## explicit
|
||||||
github.com/go-redis/redis
|
github.com/go-redis/redis/v7
|
||||||
github.com/go-redis/redis/internal
|
github.com/go-redis/redis/v7/internal
|
||||||
github.com/go-redis/redis/internal/consistenthash
|
github.com/go-redis/redis/v7/internal/consistenthash
|
||||||
github.com/go-redis/redis/internal/hashtag
|
github.com/go-redis/redis/v7/internal/hashtag
|
||||||
github.com/go-redis/redis/internal/pool
|
github.com/go-redis/redis/v7/internal/pool
|
||||||
github.com/go-redis/redis/internal/proto
|
github.com/go-redis/redis/v7/internal/proto
|
||||||
github.com/go-redis/redis/internal/util
|
github.com/go-redis/redis/v7/internal/util
|
||||||
# github.com/go-sql-driver/mysql v1.5.0
|
# github.com/go-sql-driver/mysql v1.5.0
|
||||||
## explicit
|
## explicit
|
||||||
github.com/go-sql-driver/mysql
|
github.com/go-sql-driver/mysql
|
||||||
|
@ -692,6 +690,7 @@ github.com/stretchr/testify/require
|
||||||
# github.com/subosito/gotenv v1.2.0
|
# github.com/subosito/gotenv v1.2.0
|
||||||
github.com/subosito/gotenv
|
github.com/subosito/gotenv
|
||||||
# github.com/syndtr/goleveldb v1.0.0
|
# github.com/syndtr/goleveldb v1.0.0
|
||||||
|
## explicit
|
||||||
github.com/syndtr/goleveldb/leveldb
|
github.com/syndtr/goleveldb/leveldb
|
||||||
github.com/syndtr/goleveldb/leveldb/cache
|
github.com/syndtr/goleveldb/leveldb/cache
|
||||||
github.com/syndtr/goleveldb/leveldb/comparer
|
github.com/syndtr/goleveldb/leveldb/comparer
|
||||||
|
|
Loading…
Reference in New Issue