惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

美团技术团队
W
WeLiveSecurity
Stack Overflow Blog
Stack Overflow Blog
L
LangChain Blog
S
SegmentFault 最新的问题
Apple Machine Learning Research
Apple Machine Learning Research
Google DeepMind News
Google DeepMind News
F
Full Disclosure
奇客Solidot–传递最新科技情报
奇客Solidot–传递最新科技情报
The Register - Security
The Register - Security
G
Google Developers Blog
C
Check Point Blog
GbyAI
GbyAI
A
About on SuperTechFans
V
Vulnerabilities – Threatpost
T
The Blog of Author Tim Ferriss
T
Tor Project blog
AWS News Blog
AWS News Blog
Cyberwarzone
Cyberwarzone
C
CERT Recently Published Vulnerability Notes
MongoDB | Blog
MongoDB | Blog
Latest news
Latest news
aimingoo的专栏
aimingoo的专栏
U
Unit 42
Y
Y Combinator Blog
P
Privacy International News Feed
Cisco Talos Blog
Cisco Talos Blog
S
Securelist
S
Schneier on Security
雷峰网
雷峰网
cs.CV updates on arXiv.org
cs.CV updates on arXiv.org
Attack and Defense Labs
Attack and Defense Labs
P
Proofpoint News Feed
C
Cisco Blogs
Webroot Blog
Webroot Blog
T
Troy Hunt's Blog
Google Online Security Blog
Google Online Security Blog
月光博客
月光博客
P
Privacy & Cybersecurity Law Blog
Security Archives - TechRepublic
Security Archives - TechRepublic
罗磊的独立博客
Cloudbric
Cloudbric
钛媒体:引领未来商业与生活新知
钛媒体:引领未来商业与生活新知
Recent Commits to openclaw:main
Recent Commits to openclaw:main
Application and Cybersecurity Blog
Application and Cybersecurity Blog
Hacker News: Ask HN
Hacker News: Ask HN
H
Hackread – Cybersecurity News, Data Breaches, AI and More
博客园 - 司徒正美
freeCodeCamp Programming Tutorials: Python, JavaScript, Git & More
Microsoft Security Blog
Microsoft Security Blog

博客园 - 正怒月神

windows 安装 openclaw docker-compose 启动 elk linux mysql 备份 maven 打包时优先选择本地仓库 nginx 部署2个相同的vue minio 设置IP MapStruct-plus cannot find converter from jenkins pipeline 发布 jar并运行 MapperStruct 嵌套模型中 List<> 转 List<String> Easy es问题总结 Docker 无法拉取 springmvc 多事务提交和回滚 springMvc 配置 UReport2 java 通过 microsoft graph 调用outlook(三) JPA Example 默认 join jackson.dataformat.xml 反序列化 对象中包含泛型 OR-TOOL 背包算法 解决JIRA、Confluence自动注销登录的问题 java 通过 microsoft graph 调用outlook(二)
Logstash docker发布
正怒月神 · 2024-07-18 · via 博客园 - 正怒月神

一 下载Logstash

不废话了,我下载的7.17.6

二 新增配置文件

在logstash/pipeline中,添加logstash.conf

input {
  jdbc {
# 连接
    jdbc_connection_string => "jdbc:mysql://192.168.1.1:3306/kintech-cloud-bo?characterEncoding=UTF-8&useSSL=false"
# 账号
    jdbc_user => "root"
# 密码
    jdbc_password => "xxxx"
# docker中的mysql驱动位置
    jdbc_driver_library => "/app/mysql.jar"
# 驱动
    jdbc_driver_class => "com.mysql.cj.jdbc.Driver"
# 查询语句,:sql_last_value为固定写法
    statement => "SELECT * FROM student where update_time>:sql_last_value"
# 同步间隔,每分钟
    schedule => "* * * * *"
# 启用字段(增量更新)
    use_column_value => true
# 字段类型(update_time)(增量更新)
    tracking_column_type => "timestamp"
# 字段名称(增量更新)
    tracking_column => "update_time"
  }
}
 
output {
  elasticsearch {
#es 地址,不要用local和127.0.0.1
    hosts => "192.168.1.2:9200"
#索引名称
    index => "bo_sop_content"
  }
}

三 启动docker

将logstash.conf和logstash.yml(这个不用改),以及mysql.jar 添加到数据卷。

docker run -d 
-v /root/docker/elk/logstash/pipeline/logstash.conf:/usr/share/logstash/pipeline/logstash.conf 
-v /root/docker/elk/logstash/config/logstash.yml:/usr/share/logstash/config/logstash.yml 
-v /root/lib/mysql.jar:/app/mysql.jar --name=logstash logstash:7.17.6

四 设置es分词器

# 设置分词器
PUT  192.168.1.247:9200/default
{
  "settings": {
    "analysis": {
      "analyzer": {
        "my_english_analyzer": {
          "type": "ik_smart",       
          "max_token_length": 5,    
          "stopwords": "_english_"  
        }
      }
    }
  }
}

五 kibana查询

GET /bo_sop_content/_search
{
  "query": {
    "multi_match": {
      "query":    "归档,hello,sop content",
      "fields": [ "sop_title", "sop_content" ] 
    }
  }
}

两条符合查询