惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

IT之家
IT之家
NISL@THU
NISL@THU
The Hacker News
The Hacker News
C
CXSECURITY Database RSS Feed - CXSecurity.com
T
Tenable Blog
Forbes - Security
Forbes - Security
V2EX - 技术
V2EX - 技术
Webroot Blog
Webroot Blog
Schneier on Security
Schneier on Security
T
The Exploit Database - CXSecurity.com
T
Tor Project blog
C
Cisco Blogs
TaoSecurity Blog
TaoSecurity Blog
The Last Watchdog
The Last Watchdog
PCI Perspectives
PCI Perspectives
O
OpenAI News
C
Cyber Attacks, Cyber Crime and Cyber Security
K
KPMG report finds enterprise disconnect between AI and its ROI | CIO
cs.CL updates on arXiv.org
cs.CL updates on arXiv.org
Google Online Security Blog
Google Online Security Blog
宝玉的分享
宝玉的分享
freeCodeCamp Programming Tutorials: Python, JavaScript, Git & More
量子位
D
Docker
AI
AI
Blog — PlanetScale
Blog — PlanetScale
S
Security @ Cisco Blogs
S
Schneier on Security
The GitHub Blog
The GitHub Blog
W
WeLiveSecurity
云风的 BLOG
云风的 BLOG
M
MIT News - Artificial intelligence
P
Privacy International News Feed
Cyber Security Advisories - MS-ISAC
Cyber Security Advisories - MS-ISAC
H
Hackread – Cybersecurity News, Data Breaches, AI and More
B
Blog
C
Check Point Blog
A
About on SuperTechFans
D
Darknet – Hacking Tools, Hacker News & Cyber Security
cs.AI updates on arXiv.org
cs.AI updates on arXiv.org
Application and Cybersecurity Blog
Application and Cybersecurity Blog
Engineering at Meta
Engineering at Meta
I
InfoQ
T
Threat Research - Cisco Blogs
Project Zero
Project Zero
Cloudbric
Cloudbric
MongoDB | Blog
MongoDB | Blog
Cisco Talos Blog
Cisco Talos Blog
L
Lohrmann on Cybersecurity
S
Securelist

祈雨的笔记

安全多方计算MPC spark原理解析 kueue执行源码分析 spark on k8s执行源码分析 spark-operator源码解析 系统压测遇到的缓存击穿问题 我的世界PC与安卓联机 蚂蚁金服流量投放平台的AIG改造 G1大对象致Old区占用率高 日志打印导致接口响应率下跌分析 Groovy加载类导致OOM分析 ERROR日志打印导致CPU满载 记OceanBase死锁超时 应用发版期间服务响应超时 Ark Serverless初探 系统优化复盘一二三 The user specified as a definer does not exist Kong网关初探 API网关选型调研 CPU火焰图常用工具 配置中心选型调研 root操作Nginx导致用户组错误 基于Proxifier使用代理 FastJSON字段智能匹配踩坑 Nacos初探 记一次Nginx服务器CPU满荷载故障 基于券系统分库分表的思考 limit不参与SQL成本计算致索引失效 Linux常用性能监控命令 golang低版本http2偶现400 hostname in certificate didn't match 常见对称加密原理以及应用 tcp_tw_recycle引起的TCP握手失败 记一次mysql执行DDL导致锁表 mysql磁盘占用查看 mysql对text字段update致磁盘增长 elasticsearch报错index read-only TIME_WAIT与Http的Keep-Alive 记一次TIME_WAIT导致连接数报警 记一次生产事故OOM问题排查 webservice复杂加密签名(2)java调用 webservice复杂加密签名(1)SoapUI mysql延时关联 利用中间人拦截实现APP内H5窜改 MySQL表字符集不同导致关联查询索引失效 通过SSH隧道远程办公 数据落盘方案 BeanDefinitionRegistryPostProcessor扩展 mysql空间索引 HTTPS攻击 spring循环依赖过程解析 elasticsearch性能优化 mysql IS NULL 使用索引 mysql字符集utf8mb4失效踩坑 常用加密算法 xml与javaBean转换 初探InnoDB MVCC源码实现 mysql索引原理 redis之list源码分析 redis之key过期源码分析 redis之string源码分析 redis之hash源码分析 线程池之ThreadPoolExecutor mysql数据页结构 Using temporary与Using filesort mysql回表致索引失效 springboot(28)HTTP连接池 定时任务之ScheduledThreadPoolExecutor elasticsearch常用script聚合 elasticsearch实现like查询 elasticsearch实现乐观锁 elasticsearch准实时原理 springboot(27)自定义缓存读写机制CachingConfigurerSupport optimizer tracing arthas常用命令 HTTP和HTTPS详解 redis集群选举机制 kafka消息重试 一点压力测试的经验 kafka架构概念 explain分析sql语句字段的解释 JVM问题分析处理手册 logstash过滤器(2)date logstash过滤器(3)dissect logstash编码器(1)json logstash编码器(2)multiline logstash表达式 logstash输入(1)通用选项 logstash输入(3)file logstash过滤器(1)通用选项 logstash输入(2)stdin logstash安装 记一次前端vConsole导致JSON序列化错误排查 解决多个异步操作嵌套问题 fastjson反序列化失败autoType is not support RTMP串流服务 POI自动调整列宽错误 Nginx+Lua实现动态黑名单 使用curl命令模拟POST和GET请求 ResponseEntity下载文件
redis分布式锁RedissonLock的实现细节
祈雨的笔记 · 2020-04-21 · via 祈雨的笔记

简单使用

1
2
3
4
5
6
7
8
9
10
String key = "key-lock";
RLock lock = redisson.getLock(key);
lock.lock();
try {

} catch (Exception e){
log.error(e.getMessage(), e);
} finally {
lock.unlock();
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
String key = "key-tryLock";
long maxWaitTime = 3_000;
RLock lock = redisson.getLock(key);
if (lock.tryLock(maxWaitTime, TimeUnit.MILLISECONDS)){
try {

} catch (Exception e){
log.error(e.getMessage(), e);
} finally {
lock.unlock();
}
} else {
log.debug("redis锁竞争失败");
}

流程图

多个线程节点锁竞争的正常流程如下图:

image

多个线程节点锁竞争,并出现节点下线的异常流程如下图:

image

源码解析

RedissonLock是可重入锁,使用redis的hash结构作为锁的标识存储,锁的名称作为hash的key,UUID + 线程ID作为hash的field,锁被重入的次数作为hash的value。如图所示:

image

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
private void lock(long leaseTime, TimeUnit unit, boolean interruptibly) throws InterruptedException {
long threadId = Thread.currentThread().getId();

Long ttl = tryAcquire(leaseTime, unit, threadId);
if (ttl == null) {
return;
}



RFuture<RedissonLockEntry> future = subscribe(threadId);
commandExecutor.syncSubscription(future);

try {
while (true) {
ttl = tryAcquire(leaseTime, unit, threadId);

if (ttl == null) {
break;
}

if (ttl >= 0) {

try {
getEntry(threadId).getLatch().tryAcquire(ttl, TimeUnit.MILLISECONDS);
} catch (InterruptedException e) {
if (interruptibly) {
throw e;
}
getEntry(threadId).getLatch().tryAcquire(ttl, TimeUnit.MILLISECONDS);
}
} else {
if (interruptibly) {
getEntry(threadId).getLatch().acquire();
} else {
getEntry(threadId).getLatch().acquireUninterruptibly();
}
}
}
} finally {

unsubscribe(future, threadId);
}
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
public boolean tryLock(long waitTime, long leaseTime, TimeUnit unit) throws InterruptedException {
long time = unit.toMillis(waitTime);
long current = System.currentTimeMillis();
long threadId = Thread.currentThread().getId();
Long ttl = tryAcquire(leaseTime, unit, threadId);
if (ttl == null) {
return true;
}

time -= System.currentTimeMillis() - current;
if (time <= 0) {
acquireFailed(threadId);
return false;
}

current = System.currentTimeMillis();
RFuture<RedissonLockEntry> subscribeFuture = subscribe(threadId);
if (!await(subscribeFuture, time, TimeUnit.MILLISECONDS)) {
if (!subscribeFuture.cancel(false)) {
subscribeFuture.onComplete((res, e) -> {
if (e == null) {
unsubscribe(subscribeFuture, threadId);
}
});
}
acquireFailed(threadId);
return false;
}

try {
time -= System.currentTimeMillis() - current;
if (time <= 0) {
acquireFailed(threadId);
return false;
}

while (true) {
long currentTime = System.currentTimeMillis();
ttl = tryAcquire(leaseTime, unit, threadId);

if (ttl == null) {
return true;
}

time -= System.currentTimeMillis() - currentTime;
if (time <= 0) {
acquireFailed(threadId);
return false;
}

currentTime = System.currentTimeMillis();
if (ttl >= 0 && ttl < time) {
getEntry(threadId).getLatch().tryAcquire(ttl, TimeUnit.MILLISECONDS);
} else {
getEntry(threadId).getLatch().tryAcquire(time, TimeUnit.MILLISECONDS);
}

time -= System.currentTimeMillis() - currentTime;
if (time <= 0) {
acquireFailed(threadId);
return false;
}
}
} finally {
unsubscribe(subscribeFuture, threadId);
}
}

RedissonLock实现的是可重入锁,通过redis的hash结构实现,而非加单的set nx ex。为了实现原子性的复杂的加锁逻辑,而通过lua脚本实现。获取锁会有如下三种状态:

  1. 锁未被任何线程占用,则锁获取成功,返回null
  2. 锁被当前线程占用,则锁获取成功并进行锁的重入,对锁的重入计数+1,返回null
  3. 锁被其他线程占用,则锁获取失败,返回该锁的自动过期时间ttl
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
<T> RFuture<T> tryLockInnerAsync(long leaseTime, TimeUnit unit, long threadId, RedisStrictCommand<T> command) {
internalLockLeaseTime = unit.toMillis(leaseTime);

return commandExecutor.evalWriteAsync(getName(), LongCodec.INSTANCE, command,
"if (redis.call('exists', KEYS[1]) == 0) then " +
"redis.call('hset', KEYS[1], ARGV[2], 1); " +
"redis.call('pexpire', KEYS[1], ARGV[1]); " +
"return nil; " +
"end; " +
"if (redis.call('hexists', KEYS[1], ARGV[2]) == 1) then " +
"redis.call('hincrby', KEYS[1], ARGV[2], 1); " +
"redis.call('pexpire', KEYS[1], ARGV[1]); " +
"return nil; " +
"end; " +
"return redis.call('pttl', KEYS[1]);",
Collections.<Object>singletonList(getName()), internalLockLeaseTime, getLockName(threadId));
}

当锁因为被其他线程占用而 使用redis的发布订阅pub/sub功能,通过监听锁的释放通知(在其他线程通过RedissonLock释放锁时,会通过发布订阅pub/sub功能发起通知),等待锁被其他线程释放。通过如此的线程唤醒而非自旋的操作,提高了锁的效率。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
public RFuture<E> subscribe(String entryName, String channelName) {
AtomicReference<Runnable> listenerHolder = new AtomicReference<Runnable>();
AsyncSemaphore semaphore = service.getSemaphore(new ChannelName(channelName));
RPromise<E> newPromise = new RedissonPromise<E>() {
@Override
public boolean cancel(boolean mayInterruptIfRunning) {
return semaphore.remove(listenerHolder.get());
}
};

Runnable listener = new Runnable() {

@Override
public void run() {
E entry = entries.get(entryName);
if (entry != null) {
entry.aquire();
semaphore.release();
entry.getPromise().onComplete(new TransferListener<E>(newPromise));
return;
}

E value = createEntry(newPromise);
value.aquire();

E oldValue = entries.putIfAbsent(entryName, value);
if (oldValue != null) {
oldValue.aquire();
semaphore.release();
oldValue.getPromise().onComplete(new TransferListener<E>(newPromise));
return;
}

RedisPubSubListener<Object> listener = createListener(channelName, value);
service.subscribe(LongCodec.INSTANCE, channelName, semaphore, listener);
}
};
semaphore.acquire(listener);
listenerHolder.set(listener);

return newPromise;
}

由于是可重入锁则需要在释放锁的时候做订阅通知,因此释放锁的操作同样是lua脚本实现。锁的释放会有如下三个状态:

  1. 等待释放的锁不存在或者不是当前线程持有,返回null
  2. 等待释放的锁被当前线程持有,且该锁当前被重入多次,则锁的重入计数-1,返回0
  3. 等待释放的锁被当前线程持有,且该锁当前未被重入,则锁的删除并发布该锁释放的订阅通知,返回1
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
protected RFuture<Boolean> unlockInnerAsync(long threadId) {
return commandExecutor.evalWriteAsync(getName(), LongCodec.INSTANCE, RedisCommands.EVAL_BOOLEAN,
"if (redis.call('hexists', KEYS[1], ARGV[3]) == 0) then " +
"return nil;" +
"end; " +
"local counter = redis.call('hincrby', KEYS[1], ARGV[3], -1); " +
"if (counter > 0) then " +
"redis.call('pexpire', KEYS[1], ARGV[2]); " +
"return 0; " +
"else " +
"redis.call('del', KEYS[1]); " +
"redis.call('publish', KEYS[2], ARGV[1]); " +
"return 1; "+
"end; " +
"return nil;",
Arrays.<Object>asList(getName(), getChannelName()), LockPubSub.UNLOCK_MESSAGE, internalLockLeaseTime, getLockName(threadId));

}

Watchdog

RedissonLock为了避免应用获取锁后宕机,因为没人来释放锁而导致死锁情况的出现,默认每次锁的占用只有30秒的时间(org.redisson.config.Config#lockWatchdogTimeout = 30 * 1000)。于是便有了Watchdog设计,由独立的线程定时给未释放的锁续期,默认锁有效期的三分之一的时长即每10秒给锁自动续期。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
private void renewExpiration() {
ExpirationEntry ee = EXPIRATION_RENEWAL_MAP.get(getEntryName());
if (ee == null) {
return;
}


Timeout task = commandExecutor.getConnectionManager().newTimeout(new TimerTask() {
@Override
public void run(Timeout timeout) throws Exception {
ExpirationEntry ent = EXPIRATION_RENEWAL_MAP.get(getEntryName());
if (ent == null) {
return;
}
Long threadId = ent.getFirstThreadId();
if (threadId == null) {
return;
}

RFuture<Boolean> future = renewExpirationAsync(threadId);
future.onComplete((res, e) -> {
if (e != null) {
log.error("Can't update lock " + getName() + " expiration", e);
return;
}

if (res) {
renewExpiration();
}
});
}
}, internalLockLeaseTime / 3, TimeUnit.MILLISECONDS);

ee.setTimeout(task);
}

protected RFuture<Boolean> renewExpirationAsync(long threadId) {
return commandExecutor.evalWriteAsync(getName(), LongCodec.INSTANCE, RedisCommands.EVAL_BOOLEAN,
"if (redis.call('hexists', KEYS[1], ARGV[2]) == 1) then " +
"redis.call('pexpire', KEYS[1], ARGV[1]); " +
"return 1; " +
"end; " +
"return 0;",
Collections.<Object>singletonList(getName()),
internalLockLeaseTime, getLockName(threadId));
}