惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

博客园 - 聂微东
C
CXSECURITY Database RSS Feed - CXSecurity.com
H
Help Net Security
P
Proofpoint News Feed
V
Visual Studio Blog
WordPress大学
WordPress大学
Recent Announcements
Recent Announcements
钛媒体:引领未来商业与生活新知
钛媒体:引领未来商业与生活新知
freeCodeCamp Programming Tutorials: Python, JavaScript, Git & More
T
Tailwind CSS Blog
Hugging Face - Blog
Hugging Face - Blog
GbyAI
GbyAI
Cyber Security Advisories - MS-ISAC
Cyber Security Advisories - MS-ISAC
MyScale Blog
MyScale Blog
美团技术团队
Stack Overflow Blog
Stack Overflow Blog
Recorded Future
Recorded Future
Blog — PlanetScale
Blog — PlanetScale
宝玉的分享
宝玉的分享
H
Hackread – Cybersecurity News, Data Breaches, AI and More
I
InfoQ
Microsoft Security Blog
Microsoft Security Blog
The Register - Security
The Register - Security
Y
Y Combinator Blog
Last Week in AI
Last Week in AI
F
Full Disclosure
Jina AI
Jina AI
博客园 - 司徒正美
小众软件
小众软件
让小产品的独立变现更简单 - ezindie.com
让小产品的独立变现更简单 - ezindie.com
S
SegmentFault 最新的问题
量子位
aimingoo的专栏
aimingoo的专栏
Apple Machine Learning Research
Apple Machine Learning Research
U
Unit 42
Vercel News
Vercel News
L
LangChain Blog
云风的 BLOG
云风的 BLOG
The GitHub Blog
The GitHub Blog
B
Blog RSS Feed
有赞技术团队
有赞技术团队
腾讯CDC
Martin Fowler
Martin Fowler
博客园 - 叶小钗
博客园 - 三生石上(FineUI控件)
CTFtime.org: upcoming CTF events
CTFtime.org: upcoming CTF events
D
DataBreaches.Net
Engineering at Meta
Engineering at Meta
F
Fortinet All Blogs
M
MIT News - Artificial intelligence

博客园 - 小潘

GreenPlum tidb 性能比较 hbase master 无法启动 hbase 集群写入能力优化-预分区、TTL的应用 TiSpark 初级应用 ETL-kettle 核心执行逻辑 HBase 写入优化 HBase ttl 验证 大数据多维分析平台的实践 TiDB在特来电的探索 Druid 基础使用-操作篇(Pivot、plyql) Druid安装-单机 phoenix 索引修复-基本流程 Phoenix -修复表索引 oracle sql跟踪及‘相关问题 数据交换平台 reportview 实践学习 linq 排序 学习一 开始学习LINQ
Druid 基础使用-操作篇(Imply )
小潘 · 2016-11-25 · via 博客园 - 小潘

一、Imply

    Druid 原生的配置较麻烦,在上一篇单机版安装中有所涉及
   Imply 基于Druid 进行了一些组件的开发,提供开源社区版本和商业版,简化了部署,开发了一些应用.https://imply.io/product

二、

安装

  1. 下载nodejs 安装(http://jingyan.baidu.com/article/dca1fa6f48f478f1a5405272.html)
  2. .安装imply
    1. 下载最新版本 imply-1.3.1.tar  https://imply.io/download
    2. tar -xzf imply-1.3.1.tar
  3. 启动
     1[root@Druid imply-1.3.1]# bin/supervise -c conf/supervise/quickstart.conf  -可以nohup 后台执行

三、Imply 数据的发送
        1.修改 tranquility 组件下server.josn 中的数据相关信息 --表名称、维度列、指标列
         修改后的数据结构如下
         

{
    "dataSources": {
        "pageviews": {
            "spec": {
                "dataSchema": {
                    "dataSource": "pageviews",
                    "parser": {
                        "type": "string",
                        "parseSpec": {
                            "timestampSpec": {
                                "format": "auto",
                                "column": "time"
                            },
                            "dimensionsSpec": {
                                "dimensions": [
                                    "url",
                                    "user"
                                ]
                            },
                            "format": "json"
                        }
                    },
                    "granularitySpec": {
                        "type": "uniform",
                        "segmentGranularity": "hour",
                        "queryGranularity": "none"
                    },
                    "metricsSpec": [
                        {
                            "name": "views",
                            "type": "count"
                        },
                        {
                            "name": "latencyMs",
                            "type": "doubleSum",
                            "fieldName": "latencyMs"
                        }
                    ]
                },
                "ioConfig": {
                    "type": "realtime"
                },
                "tuningConfig": {
                    "type": "realtime",
                    "maxRowsInMemory": "100000",
                    "intermediatePersistPeriod": "PT10M",
                    "windowPeriod": "PT10M"
                }
            },
            "properties": {
                "task.partitions": "1",
                "task.replicants": "1"
            }
        }
    },
    "properties": {
        "zookeeper.connect": "localhost",
        "druid.discovery.curator.path": "/druid/discovery",
        "druid.selectors.indexing.serviceName": "druid/overlord",
        "http.port": "8200",
        "http.threads": "8"
    }

View Code

        2.重新启动Imply --这个地方有个疑问  如何动态的设置表的名称呢?tranquility 重启 原因在于重启的时候指定了server.json 这个配置文件?

      3.在linnux系统中进行数据的发送
                curl -XPOST -H'Content-Type: application/json' --data-binary @../003.jsonhttp://*。*。*。*:8200/v1/post/pageviews  --pageviews 必须提前声明,否则回提示数据源未定义 ,如何动态增加呢

           003.josn 数据源的数据,一定要注意time 数据,一是时间最好是当前时间,否则tranquility 仅能收到数据,不会想Druid进行数据的发送,比如  receive 3 send 0

          

        如果一切正常,将会受到 received 3 send 3

   4.c# 代码进行json数据的发送 --post json 
       

 1 /// <summary>
 2         /// 插入数据,基于服务端已经有的一个表pageviews
 3         /// </summary>
 4         [TestMethod]
 5         public void InsertData()
 6         {
 7             string url = "http://*.*.*.*:8200/v1/post/pageviews"; //发送数据
 8             string data = PostHttp(url, GetData());
 9             DruiExecuteResult result = JsonConvert.DeserializeObject<DruiExecuteResult>(data);
10 
11             Assert.IsTrue(result.result.received == "100");
12             Assert.IsTrue(result.result.received == "100");
13         }
14          public string GetData()
15         {
16             StringBuilder sb = new StringBuilder();
17             string temp = string.Empty;
18             string ISO8601time = string.Empty;
19             for (int i = 0; i < 100; i++)
20             {
21                 ISO8601time = DateTime.Now.ToString("yyyy-MM-ddTHH:mm:sszzzz", DateTimeFormatInfo.InvariantInfo);
22                 temp = "{\"time\":\"" + ISO8601time + "\",\"url\":\"test" + i.ToString() + "\",\"user\":\"hello" + i.ToString() + "\",\"latencyMs\":" + i.ToString() + "}";
23                 sb.AppendLine(temp);
24             }
25 
26             string result = sb.ToString();
27 
28             return result;
29         }
30           private static string PostHttp(string url, string body)
31         {
32             HttpWebRequest httpWebRequest = (HttpWebRequest)WebRequest.Create(url);
33             httpWebRequest.ContentType = "application/x-www-form-urlencoded;charset=UTF-8";
34             httpWebRequest.ContentType = "application/json";
35             //httpWebRequest.ContentType = "text/plain";
36 
37             httpWebRequest.Method = "POST";
38             httpWebRequest.Timeout = 30000;
39             httpWebRequest.KeepAlive = false;
40             byte[] btBodys = Encoding.UTF8.GetBytes(body);
41             httpWebRequest.ContentLength = btBodys.Length;
42             string responseContent = string.Empty;
43             HttpWebResponse httpWebResponse = null;
44             StreamReader streamReader = null;
45             try
46             {
47                 httpWebRequest.GetRequestStream().Write(btBodys, 0, btBodys.Length);
48                 httpWebResponse = (HttpWebResponse)httpWebRequest.GetResponse();
49                 streamReader = new StreamReader(httpWebResponse.GetResponseStream());
50                 responseContent = streamReader.ReadToEnd();
51             }
52             catch (Exception er)
53             {
54                 throw new Exception("执行出现异常:" + "数据:" + body, er);
55             }
56             finally
57             {
58                 if (httpWebResponse != null)
59                 {
60                     httpWebResponse.Close();
61                 }
62                 if (streamReader != null)
63                 {
64                     streamReader.Close();
65                 }
66                 httpWebRequest.Abort();
67             }
68 
69             return responseContent;
70         }

View Code