惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

V
Vulnerabilities – Threatpost
U
Unit 42
F
Fortinet All Blogs
aimingoo的专栏
aimingoo的专栏
P
Proofpoint News Feed
F
Full Disclosure
月光博客
月光博客
Engineering at Meta
Engineering at Meta
博客园_首页
The Register - Security
The Register - Security
G
Google Developers Blog
The Cloudflare Blog
博客园 - Franky
K
Kaspersky official blog
A
Arctic Wolf
Scott Helme
Scott Helme
C
Cisco Blogs
Hugging Face - Blog
Hugging Face - Blog
C
Check Point Blog
NISL@THU
NISL@THU
AI
AI
D
DataBreaches.Net
Cyber Security Advisories - MS-ISAC
Cyber Security Advisories - MS-ISAC
Stack Overflow Blog
Stack Overflow Blog
Project Zero
Project Zero
The GitHub Blog
The GitHub Blog
H
Hackread – Cybersecurity News, Data Breaches, AI and More
量子位
Vercel News
Vercel News
T
Tor Project blog
P
Privacy International News Feed
D
Docker
I
Intezer
L
LangChain Blog
P
Proofpoint News Feed
Security Latest
Security Latest
C
CXSECURITY Database RSS Feed - CXSecurity.com
T
Threatpost
博客园 - 聂微东
AWS News Blog
AWS News Blog
Martin Fowler
Martin Fowler
P
Privacy & Cybersecurity Law Blog
V
V2EX
Last Week in AI
Last Week in AI
C
Cybersecurity and Infrastructure Security Agency CISA
The Hacker News
The Hacker News
T
Tenable Blog
Blog — PlanetScale
Blog — PlanetScale
钛媒体:引领未来商业与生活新知
钛媒体:引领未来商业与生活新知
T
Tailwind CSS Blog

博客园 - Johnson_wang

对云服务器挂载cos桶 记录一次xxlJob升级版本导致的 xxl-job remoting error(Connection reset) 记一次NoClassDeffoundEror问题解决过程 nginx配置支持ws,并解决跨域 关于消费端接入dubbo,连接失败问题 RedisTemplate关于key出现前缀\xac\xed\x00\x05t\x00\x0f - Johnson_wang - 博客园 ES 单索引大表拆分 关于ES索引被聚合查询导致filedata堵塞 (pressure too high, (smooth) bulk request circuit break) 发送HTML格式邮件 elasticsearch报错FORBIDDEN/12/index read-only / allow delete spring Boot 相关问题以及修复(后续待补充) mybatis数据加解密处理方案 synchronized的实现原理 Scheduler踩坑记录 关于RedisTemplate的map存储踩坑记录 关于HashMap的加载因子相关理解 Mybatis 分页插件PageHelper 遇坑 Linux 下 Mysql忘记密码重置 Eclipse MAT和jvisualvm分析内存溢出
队列缓存区-db写入
Johnson_wang · 2021-01-04 · via 博客园 - Johnson_wang

近期需要写入大量操作记录到db中,在不考虑高并发的情况下做一个简单缓存区,使用阻塞队列,定时将队列数据刷进db中。

代码如下:

1.建立监听器

 1 /**
 2  * 监听器
 3  *
 4  * @author johnson
 5  * @date 2020/12/31 11:03
 6  */
 7 @Component
 8 public class MyListener implements ApplicationRunner {
 9 
10     @Autowired
11     private AutoThreadConfig autoThreadConfig;
12 
13     ScheduledExecutorService schedule = Executors.newScheduledThreadPool(1);
14 
15     @Override
16     public void run(ApplicationArguments args) throws Exception {
17         System.out.println("启动监听器....");
18         //设置延迟1s 每个10s执行依次数据写入
19         schedule.scheduleAtFixedRate(new MyThread(),1L, 10L,TimeUnit.SECONDS);
20     }
21 
22    class MyThread implements Runnable{
23        @Override
24        public void run() {
25            SimpleDateFormat sdf = new SimpleDateFormat("HH:mm:ss:sss");
26             String time = sdf.format(new Date());
27            System.out.println("执行写入,时间:"+time);
28            autoThreadConfig.push();
29        }
30    }
31 }

2.建立单例队列

 1 public class LogQueue {
 2 
 3     private static volatile BlockingQueue<String> LogDBEntityQueue;
 4 
 5     public LogQueue(){
 6     }
 7 
 8     public static BlockingQueue<String> getOrderLogQueue(){
 9         if(LogDBEntityQueue == null){
10             synchronized (LogQueue.class){
11                 if(LogDBEntityQueue == null){
12                     LogDBEntityQueue = new LinkedBlockingDeque<>();
13                 }
14             }
15         }
16         return LogDBEntityQueue;
17     }
18 }

3.完成数据写入方法

 1 /**
 2  * TODO
 3  *
 4  * @author johnson
 5  * @date 2020/12/31 11:01
 6  */
 7 @Component
 8 public class AutoThreadConfig {
 9 
10 
11     private BlockingQueue<String> blockingQueue = LogQueue.getOrderLogQueue();
12 
13 
14     public void push(){
15         if(CollectionUtils.isEmpty(blockingQueue)){
16             return;
17         }
18         List<String> list = new ArrayList<>();
19         try {
20             Queues.drain(blockingQueue,list,blockingQueue.size(),60, TimeUnit.SECONDS);
21             System.out.println("本次写入数据量:"+ list.size());
22             list.stream().forEach(str ->{
23                 System.out.println(str);
24             });
25 
26         }catch (Exception e){
27             e.printStackTrace();
28         }
29 
30         return;
31     }
32 
33 
34     public void save(String string){
35         blockingQueue.offer(string);
36     }
37 
38 }

4.测试类

 1 /**
 2  * 传入数据
 3  *
 4  * @author johnson
 5  * @date 2020/12/31 13:38
 6  */
 7 @RestController
 8 public class SaveDataController {
 9 
10 
11 
12     @Autowired
13     private AutoThreadConfig autoThreadConfig;
14 
15 
16     @GetMapping("/save")
17     public Boolean save(@RequestParam("str") String str){
18         autoThreadConfig.save(str);
19         return true;
20     }
21 }

在调用save方法后,数据写入到队列中,队列在服务启动后,每隔10s自动将队列数据打印出来