惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

博客园 - 三生石上(FineUI控件)
G
Google Developers Blog
Vercel News
Vercel News
Cyber Security Advisories - MS-ISAC
Cyber Security Advisories - MS-ISAC
N
Netflix TechBlog - Medium
奇客Solidot–传递最新科技情报
奇客Solidot–传递最新科技情报
钛媒体:引领未来商业与生活新知
钛媒体:引领未来商业与生活新知
Engineering at Meta
Engineering at Meta
B
Blog
博客园_首页
量子位
博客园 - 叶小钗
L
LangChain Blog
T
The Blog of Author Tim Ferriss
云风的 BLOG
云风的 BLOG
Blog — PlanetScale
Blog — PlanetScale
F
Fortinet All Blogs
S
SegmentFault 最新的问题
宝玉的分享
宝玉的分享
D
DataBreaches.Net
雷峰网
雷峰网
The Cloudflare Blog
freeCodeCamp Programming Tutorials: Python, JavaScript, Git & More
Threat Intelligence Blog | Flashpoint
Threat Intelligence Blog | Flashpoint
Last Week in AI
Last Week in AI
P
Proofpoint News Feed
TaoSecurity Blog
TaoSecurity Blog
罗磊的独立博客
MongoDB | Blog
MongoDB | Blog
The GitHub Blog
The GitHub Blog
I
Intezer
H
Help Net Security
The Hacker News
The Hacker News
The Register - Security
The Register - Security
cs.CL updates on arXiv.org
cs.CL updates on arXiv.org
AWS News Blog
AWS News Blog
V
V2EX
Microsoft Security Blog
Microsoft Security Blog
T
Tenable Blog
Spread Privacy
Spread Privacy
A
Arctic Wolf
P
Proofpoint News Feed
T
Threat Research - Cisco Blogs
Schneier on Security
Schneier on Security
C
CERT Recently Published Vulnerability Notes
让小产品的独立变现更简单 - ezindie.com
让小产品的独立变现更简单 - ezindie.com
The Last Watchdog
The Last Watchdog
Latest news
Latest news
T
Troy Hunt's Blog
L
LINUX DO - 热门话题

博客园 - 我想我是青蛙

将golang程序注册为windows服务 关于听歌这回事 c#读取apk 信息 golang 读取mongob数据写入sqlserver golang 通用Contains方法 golang读取文件信息插入mongodb PetaPoco介绍 白话MongoDB(三)(转载) 白话MongoDB(二)(转载) 白话MongoDB(一) (转载) 给文章加入关键字链接 针对firefox ie6 ie7 ie8的css样式hack (转载) 好久不写日志了,现在开始,好好写了。。 sharepoint 查询calendar recurrence sharepoint之lookup字段 sharepoint获取Audiences 获取exchangeserve的calendar的item sharepoint错误处理 c#上传下载ftp(支持断点续传)
CentOS上安装spark standalone mode(转载)
我想我是青蛙 · 2013-11-07 · via 博客园 - 我想我是青蛙

原文链接 http://blog.csdn.net/chenxingzhen001/article/details/11072765

参考:

http://spark.incubator.apache.org/docs/latest/

http://spark.incubator.apache.org/docs/latest/spark-standalone.html

 http://www.yanjiuyanjiu.com/blog/20130617/

1.安装JDK

2.安装scala 2.9.3

Spark 0.7.2 依赖 Scala 2.9.3, 我们必须要安装Scala 2.9.3.

下载 scala-2.9.3.tgz 并 保存到home目录(已经在sg206上).
$ tar -zxf scala-2.9.3.tgz
$ sudo mv scala-2.9.3 /usr/lib
$ sudo vim /etc/profile
# add the following lines at the end
export SCALA_HOME=/usr/lib/scala-2.9.3
export PATH=$PATH:$SCALA_HOME/bin
# save and exit vim
#make the bash profile take effect immediately
source /etc/profile
# test
$ scala -version

3.building spark

cd /home

tar -zxf spark-0.7.3-sources.gz

cd spark-0.7.3

sbt/sbt package  (需要git环境  yum install git)

4.配置文件

 spark-env.sh

############

export SCALA_HOME=/usr/lib/scala-2.9.3
export SPARK_MASTER_IP=172.16.48.202
export SPARK_WORKER_MEMORY=10G

#############

slaves

将从节点IP添加至slaves配置文件

5.启动和停止

bin/start-master.sh - Starts a master instance on the machine the script is executed on.
bin/start-slaves.sh - Starts a slave instance on each machine specified in the conf/slaves file.
bin/start-all.sh - Starts both a master and a number of slaves as described above.
bin/stop-master.sh - Stops the master that was started via the bin/start-master.sh script.
bin/stop-slaves.sh - Stops the slave instances that were started via bin/start-slaves.sh.
bin/stop-all.sh - Stops both the master and the slaves as described above.