惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

CTFtime.org: upcoming CTF events
CTFtime.org: upcoming CTF events
D
Darknet – Hacking Tools, Hacker News & Cyber Security
F
Fortinet All Blogs
小众软件
小众软件
博客园_首页
奇客Solidot–传递最新科技情报
奇客Solidot–传递最新科技情报
Microsoft Azure Blog
Microsoft Azure Blog
MongoDB | Blog
MongoDB | Blog
罗磊的独立博客
大猫的无限游戏
大猫的无限游戏
量子位
N
Netflix TechBlog - Medium
B
Blog
P
Proofpoint News Feed
月光博客
月光博客
Apple Machine Learning Research
Apple Machine Learning Research
人人都是产品经理
人人都是产品经理
云风的 BLOG
云风的 BLOG
Cyber Security Advisories - MS-ISAC
Cyber Security Advisories - MS-ISAC
腾讯CDC
Engineering at Meta
Engineering at Meta
Y
Y Combinator Blog
AI
AI
Stack Overflow Blog
Stack Overflow Blog
U
Unit 42
M
MIT News - Artificial intelligence
Vercel News
Vercel News
D
DataBreaches.Net
P
Palo Alto Networks Blog
宝玉的分享
宝玉的分享
让小产品的独立变现更简单 - ezindie.com
让小产品的独立变现更简单 - ezindie.com
Cisco Talos Blog
Cisco Talos Blog
T
Threatpost
The Hacker News
The Hacker News
K
KPMG report finds enterprise disconnect between AI and its ROI | CIO
Security Latest
Security Latest
钛媒体:引领未来商业与生活新知
钛媒体:引领未来商业与生活新知
有赞技术团队
有赞技术团队
Attack and Defense Labs
Attack and Defense Labs
Recent Announcements
Recent Announcements
Hugging Face - Blog
Hugging Face - Blog
Webroot Blog
Webroot Blog
Cyberwarzone
Cyberwarzone
美团技术团队
博客园 - 司徒正美
Cloudbric
Cloudbric
J
Java Code Geeks
T
Tailwind CSS Blog
The Last Watchdog
The Last Watchdog
A
About on SuperTechFans

博客园 - Nucky_yang

windows系统下安装openclaw,无权限打开软件,无权限打开浏览器,无权限读写文件的问题。 presto集成 hive(转载) java可重入锁与不可重入锁 的设计 动态规划 背包问题 java ThreadPoolExecutor线程池在美团的最佳实践 回溯 八皇后问题 与 0-1背包 技术学习 线程间通信 计算机网络基础知识总结(各种协议) 大数据Phoenix专题 ClickHouse深度揭秘 hbase 查询组件phoenix redis的设计与实现 第二版 位图 Java并发编程:volatile关键字解析 socket 多路复用原理和代码 select poll epoll mongDB需要注意的事项和 监控命令mongostat mongotop mongoDB的事务 mongoDB学习 mongo的聚合框架、join
just a demo
Nucky_yang · 2021-01-25 · via 博客园 - Nucky_yang
<!-- demodemo.xml -->
<
dependency> <groupId>org.apache.hadoop</groupId> <artifactId>hadoop-mapreduce-client-core</artifactId> <version>2.7.0</version> </dependency> <dependency> <groupId>org.apache.hadoop</groupId> <artifactId>hadoop-hdfs</artifactId> <version>2.7.0</version> </dependency> <dependency> <groupId>org.apache.hadoop</groupId> <artifactId>hadoop-common</artifactId> <version>2.7.0</version> </dependency> <profiles> <profile> <id>id11111</id> <properties> <jar.name>xxxxx</jar.name> <main.class>mainxxxxx</main.class> <java.version>1.8</java.version> </properties> <build> <plugins> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-jar-plugin</artifactId> <configuration> <archive> <manifest> <addClasspath>true</addClasspath> <classpathPrefix>lib</classpathPrefix> <!-- main函数 --> <mainClass>${main.class}</mainClass> </manifest> </archive> <includes> <include>com/rs/java/core/*.class</include> <include>**/*.properties</include> </includes> </configuration> </plugin> </plugins> </build> </profile> </profiles>

java  demo

Configuration conf = new Configuration();
String[] otherArgs = new GenericOptionsParser(conf, args).getRemainingArgs();

Job job = Job.getInstance(conf, "jboname");
job.setJarByClass(AAA.class);
job.setOutputValueClass(Text.class);
job.setOutputKeyClass(Text.class);
job.setMapperClass(FirstMapper.class); //模式1,数据一天跑一次
job.setCombinerClass(FirstComb.class);
job.setReducerClass(FirstReducer.class);
job.setInputFormatClass(CombineSmallfileInputFormat.class);
FileInputFormat.addInputPath(job, new Path(otherArgs[0]));
FileOutputFormat.setOutputPath(job, new Path(otherArgs[1]));
if(!job.waitForCompletion(true)) return ;

posted @ 2021-01-25 00:29  Nucky_yang  阅读(86)  评论()    收藏  举报