惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

AWS News Blog
AWS News Blog
T
Tenable Blog
Project Zero
Project Zero
T
The Exploit Database - CXSecurity.com
L
LINUX DO - 热门话题
T
Threat Research - Cisco Blogs
T
Threatpost
Security Latest
Security Latest
C
Cisco Blogs
L
Lohrmann on Cybersecurity
S
Security @ Cisco Blogs
Google Online Security Blog
Google Online Security Blog
NISL@THU
NISL@THU
AI
AI
V
Vulnerabilities – Threatpost
Google DeepMind News
Google DeepMind News
C
Cyber Attacks, Cyber Crime and Cyber Security
C
CXSECURITY Database RSS Feed - CXSecurity.com
The Last Watchdog
The Last Watchdog
G
GRAHAM CLULEY
Cloudbric
Cloudbric
H
Hackread – Cybersecurity News, Data Breaches, AI and More
H
Hacker News: Front Page
U
Unit 42
A
Arctic Wolf
CTFtime.org: upcoming CTF events
CTFtime.org: upcoming CTF events
MyScale Blog
MyScale Blog
O
OpenAI News
Scott Helme
Scott Helme
V2EX - 技术
V2EX - 技术
P
Proofpoint News Feed
博客园 - 叶小钗
Hugging Face - Blog
Hugging Face - Blog
云风的 BLOG
云风的 BLOG
V
Visual Studio Blog
Application and Cybersecurity Blog
Application and Cybersecurity Blog
Cyberwarzone
Cyberwarzone
博客园 - 【当耐特】
H
Heimdal Security Blog
S
Schneier on Security
阮一峰的网络日志
阮一峰的网络日志
Help Net Security
Help Net Security
D
DataBreaches.Net
Y
Y Combinator Blog
Hacker News - Newest:
Hacker News - Newest: "LLM"
TaoSecurity Blog
TaoSecurity Blog
K
Kaspersky official blog
N
News and Events Feed by Topic
WordPress大学
WordPress大学
P
Palo Alto Networks Blog

博客园 - cindy_zl

SAP SD销售与分销模块 Agent认知框架ReAct&Plan RAG系统优化 RAG检索优化 RAG架构 SAA-C02-架构师助理级-英文版-Q1 scala-数组 navicat mysql创建存储过程报错 安装完kali出现中文乱码 SPARK SQL ERROR: Detected cartesian product for INNER join between logical plans报错解决方法 scala的lazy关键字 scala集合三大类(seq序列,set集,map映射)——map映射 scala集合三大类(seq序列,set集,map映射)——set集合 scala集合三大类(seq序列,set集,map映射)——list序列 scala元组及拉链操作 scala的map映射 scala数组 scala方法和涵数的声明以及方法转换成涵数 scala的基础数据类型&if条件表达式&for循环
Spark SQL是处理结构化的数据
cindy_zl · 2019-10-04 · via 博客园 - cindy_zl

Spark SQL是处理结构化的数据,可以存储在二维表中,类似数据库中的表一样存储数据

Spark1.x

      val sqlContext = new SparkContext(conf)

      val sqlContext = new SQLContext(sc)

     //将RDD和Schema信息关联到一起,1,RDD和case class 2,RDD和StructType

    //case class Person将RDD中的数据转换成case class 属性相对应的类型,然后设置到case class中

    val rdd:RDD[Person] = ....

    //将RDD转换成DataFrame

    val df = rdd.toDF

    //对df进行操作(1,直接使用DataFrame上的算子DSL。2,写SQL)

   //将df注册成临时表

   df.registerTempTable("t_person")

  //执行SQL

  val result :DataFrame = sqlContext.sql("select * from t_person");

   result.show()

Spark2.x

val spark = SparkSession.builder().appName("a").master("local[*]").getOrCreate()

//创建DF

val df = spark.createDataFrame(RDD[Row], schema)

//DSL 和 SQL

df.createTempView("v_user")

//执行SQL

val result:DataFrame = spark.sql("select * from t_user")

//执行action

result.show()

//