惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

酷 壳 – CoolShell
酷 壳 – CoolShell
aimingoo的专栏
aimingoo的专栏
Microsoft Security Blog
Microsoft Security Blog
NISL@THU
NISL@THU
T
Threatpost
T
The Exploit Database - CXSecurity.com
T
Threat Research - Cisco Blogs
S
Securelist
Cyber Security Advisories - MS-ISAC
Cyber Security Advisories - MS-ISAC
人人都是产品经理
人人都是产品经理
B
Blog RSS Feed
S
Secure Thoughts
MyScale Blog
MyScale Blog
O
OpenAI News
P
Palo Alto Networks Blog
美团技术团队
C
Cyber Attacks, Cyber Crime and Cyber Security
TaoSecurity Blog
TaoSecurity Blog
量子位
L
Lohrmann on Cybersecurity
G
GRAHAM CLULEY
让小产品的独立变现更简单 - ezindie.com
让小产品的独立变现更简单 - ezindie.com
T
Tailwind CSS Blog
Know Your Adversary
Know Your Adversary
Recent Commits to openclaw:main
Recent Commits to openclaw:main
Simon Willison's Weblog
Simon Willison's Weblog
宝玉的分享
宝玉的分享
PCI Perspectives
PCI Perspectives
Threat Intelligence Blog | Flashpoint
Threat Intelligence Blog | Flashpoint
C
Cybersecurity and Infrastructure Security Agency CISA
T
Tenable Blog
I
InfoQ
D
Darknet – Hacking Tools, Hacker News & Cyber Security
Microsoft Azure Blog
Microsoft Azure Blog
Recent Announcements
Recent Announcements
S
Security @ Cisco Blogs
S
Schneier on Security
B
Blog
cs.CL updates on arXiv.org
cs.CL updates on arXiv.org
The Cloudflare Blog
AWS News Blog
AWS News Blog
IT之家
IT之家
V
Vulnerabilities – Threatpost
The Hacker News
The Hacker News
H
Heimdal Security Blog
I
Intezer
A
Arctic Wolf
K
KPMG report finds enterprise disconnect between AI and its ROI | CIO
H
Help Net Security
W
WeLiveSecurity

博客园 - DHclly

Aspose.Words 合并单元格的原理、配置方式、代码示例以及常见误区 Gpustack 运行一段时间后出现 Failed to initialize NVML: Unknown Error 解决办法 wps dispimg python 解析实现参考 NVIDIA GPU 计算能力( compute capability,SM version)兼容性查询 wsl 和win主机互相访问 nginx 根路径同时代理 http ws sse 三种请求 在 X86_64(amd64) 平台上的docker支持打包跨平台的镜像(如arm64) 以图搜图功能介绍 docker 容器调试技巧 open ai sdk 的额外请求头说明 x-stainless 大模型常见的概念 创建软连接的几种方式 基于node.js 的 web server 实现 对个人的警醒 jQuery对象与DOM对象之间的转换方法 【BUG】浏览器控制台提示:net::ERR_INVALID_CHUNKED_ENCODING 200 (OK) 的解决思路 实用浏览器脚本 关于Lambda表达式(箭头函数)的get属性访问器和常规的get属性访问器的差异 转换字符串为二进制编码字符串
Amazon S3 Tools:S3cmd 介绍
DHclly · 2024-11-22 · via 博客园 - DHclly

Amazon S3 Tools: Command Line S3 Client and S3 Backup for Windows, Linux: s3cmd, s3express(https://s3tools.org/s3cmd)

S3cmd : Command Line S3 Client and Backup for Linux and Mac S3cmd:适用于 Linux 和 Mac 的命令行 S3 客户端和备份

Amazon S3 is a reasonably priced data storage service. Ideal for off-site file backups, file archiving, web hosting and other data storage needs. It is generally more reliable than your regular web hosting for storing your files and images. Check out about Amazon S3 to find out more.
Amazon S3 是一种价格合理的数据存储服务。非常适合异地文件备份、文件归档、网络托管和其他数据存储需求。它通常比存储文件和图像的常规网络托管更可靠。查看Amazon S3以了解更多信息。

S3cmd is a free command line tool and client for uploading, retrieving and managing data in Amazon S3 and other cloud storage service providers that use the S3 protocol, such as Google Cloud Storage or DreamHost DreamObjects. It is best suited for power users who are familiar with command line programs. It is also ideal for batch scripts and automated backup to S3, triggered from cron, etc.
S3cmd 是一个免费的命令行工具和客户端,用于上传、检索和管理 Amazon S3 和其他使用 S3 协议的云存储服务提供商(例如 Google Cloud Storage 或 DreamHost DreamObjects)中的数据。它最适合熟悉命令行程序的高级用户。它也是批处理脚本和自动备份到 S3(从 cron 触发)的理想选择。

S3cmd is written in Python. It's an open source project available under GNU Public License v2 (GPLv2) and is free for both commercial and private use. You will only have to pay Amazon for using their storage.
S3cmd是用Python编写的。它是一个根据 GNU 公共许可证 v2 (GPLv2) 提供的开源项目,并且免费供商业和私人使用。您只需向亚马逊支付使用其存储空间的费用。

Lots of features and options have been added to S3cmd, since its very first release in 2008.... we recently counted more than 60 command line options, including multipart uploads, encryption, incremental backup, s3 sync, ACL and Metadata management, S3 bucket size, bucket policies, and more!
自 2008 年首次发布以来,S3cmd 中添加了许多功能和选项......我们最近统计了超过 60 个命令行选项,包括分段上传、加密、增量备份、 s3 同步、ACL 和元数据管理、S3存储桶大小、存储桶策略等等!

S3cmd System Requirements: S3cmd requires Python 2.6 or newer. S3cmd version 2 is also compatible with Python 3.x. S3cmd runs on Linux (Fedora, RedHat, SuSe, Ubuntu, Debian, etc.) and on Apple MAC.
S3cmd 系统要求: S3cmd 需要Python 2.6 或更高版本。 S3cmd 版本 2 还与 Python 3.x 兼容。 S3cmd 在Linux (Fedora、RedHat、SuSe、Ubuntu、Debian 等)和Apple MAC上运行。

S3Express : Command Line S3 Client and S3 Backup for Windows S3Express:适用于 Windows 的命令行 S3 客户端和 S3 备份

S3Express is a commercial S3 command line tool for Windows. Differently from S3cmd, S3Express is designed to run specifically on Windows, it is self-contained in one executable (s3express.exe) and does not require any additional libraries or software to be installed to run. It's very compact and has very small footprint: the entire program is less than 5MB. It's very easy to install on Windows servers, clients, desktops and laptop computers alike.
S3Express 是一个用于 Windows 的商业 S3 命令行工具。与 S3cmd 不同,S3Express 专为在 Windows 上运行而设计,它独立于一个可执行文件 (s3express.exe) 中,不需要安装任何额外的库或软件即可运行。它非常紧凑,占用空间非常小:整个程序不到 5MB。它非常容易安装在Windows 服务器、客户端、台式机和笔记本电脑上。

S3Express is developed and maintained by TGRMN Software, the company behind ViceVersa PRO, professional software for file backup, file synchronization and file copy on Windows.
S3Express 由 TGRMN Software 开发和维护,TGRMN Software 是ViceVersa PRO背后的公司,是 Windows 上用于文件备份、文件同步和文件复制的专业软件。

With S3Express you can list and query S3 objects using conditional filters, manage S3 objects' metadata and ACLs, upload files to S3 using multipart uploads and multiple concurrent threads, upload only new or changed files to S3 for automated backup (= incremental backup), delete multiple S3 objects, copy S3 objects, etc. using the Windows command line.
使用 S3Express,您可以使用条件过滤器列出和查询 S3 对象,管理 S3 对象的元数据和 ACL,使用分段上传和多个并发线程将文件上传到 S3,仅将新的或更改的文件上传到 S3 以进行自动备份(=增量备份),使用 Windows 命令行删除多个 S3 对象、复制 S3 对象等。

All operations in S3Express are multithreaded (fast), automatically retryable (network-failure resistant), and interruptible (= all commands can be stopped and restarted at any time). Connections to Amazon S3 are made using secure http (https), which is an encrypted version of the HTTP protocol, to protect your files while they're in transit to and from Amazon S3 storage servers.
S3Express 中的所有操作都是多线程的(快速)、可自动重试(抗网络故障)和可中断(= 所有命令都可以随时停止和重新启动)。与 Amazon S3 的连接是使用安全 http (https)(HTTP 协议的加密版本)建立的,以在文件与 Amazon S3 存储服务器之间传输时保护您的文件。

S3Express System Requirements: Windows 11, Windows 10, Windows 8 / 8.1, Windows 7, Windows Vista, XP, Windows Server 2022, 2019, 2016, 2012, 2008 , 2003, incl. R2.
S3Express 系统要求:Windows 11、Windows 10、Windows 8 / 8.1、Windows 7、Windows Vista、XP、Windows Server 2022、2019、2016、2012、2008、2003,包括。 R2。
Self-contained: S3Express does not require any additional libraries or software to run. The entire S3Express installation is less than 8MB in size. No changes whatsoever applied to the operative system, S3Express can run without admin rights.
自包含:S3Express 不需要任何额外的库或软件即可运行。整个 S3Express 安装大小不到 8MB。操作系统无需进行任何更改,S3Express 无需管理员权限即可运行。

安装

Ubuntu/Debian:

sudo apt-get update
sudo apt-get install s3cmd

CentOS/RHEL:

sudo yum install epel-release
sudo yum install s3cmd

MacOS (Homebrew):

brew install s3cmd

python

pip3 install s3cmd

配置

s3cmd --configure

常用指令

# 获取桶列表
s3cmd ls

# 创建一个名为new-bucket的桶
s3cmd mb s3://new-bucket

# 删除一个名为empty-bucket的桶(前提是该桶为空)
s3cmd rb s3://empty-bucket

# 将本地文件/home/user/file.txt上传到桶my-bucket中,存储名为uploaded_file.txt
s3cmd put /home/user/file.txt s3://my-bucket/uploaded_file.txt

# 从桶my-bucket中下载文件downloaded_file.txt到本地路径/home/user/
s3cmd get s3://my-bbucket/downloaded_file.txt /home/user/downloaded_file.txt

# 删除桶my-bucket中的文件delete_file.txt
s3cmd del s3://my-bucket/delete_file.txt

# 在桶my-bucket中创建一个模拟目录(S3无真正目录概念,以创建以/结尾的对象模拟)名为test_dir
s3cmd put -P /dev/null s3://my-bucket/test_dir/

# 查看桶my-bucket中文件metadata_file.txt的元数据信息
s3cmd info s3://my-bucket/metadata_file.txt

# 查看桶my-bucket的元数据信息
s3cmd info s3://my-bucket

# 同步本地目录/home/user/local_dir到桶my-bucket中
s3cmd sync /home/user/local_dir s3://my-bucket

# 同步桶my-bucket中的内容到本地目录/home/user/target_dir
s3cmd sync s3://my-bucket /home/user/target_dir

# 设置桶my-bucket的访问权限为公共可读(示例,实际权限设置可根据需求调整)
s3cmd setacl s3://my-bucket --acl-public

# 获取桶my-bucket的访问权限信息,这个指令无效了
s3cmd getacl s3://my-bucket
# 改用
s3cmd info s3://my-bucket

# 复制桶my-bucket中的文件original_file.txt为copied_file.txt
s3cmd cp s3://my-bucket/original_file.txt s3://my-bucket/copied_file.txt

# 列出桶my-bucket中以prefix开头的所有对象(可用于类似按前缀筛选文件)
s3cmd ls s3://my-bucket/prefix*

# 重命名桶my-bucket中的文件old_name.txt为new_name.txt
s3cmd mv s3://my-bucket/old_name.txt s3://my-bucket/new_name.txt

# 授权用户user1对桶my-bucket具有只读权限(假设已配置好相关用户及权限体系,示例具体权限设置可根据实际情况调整)
s3cmd setacl s3://my-bucket --acl-user=user1:READ

# 撤销用户user1对桶my-bucket的所有权限(假设已配置好相关用户及权限体系)
s3cmd setacl s3://my-bucket --acl-user=user1:

# 授权用户user1对桶my-bucket中的文件specific_file.txt具有读写权限(假设已配置好相关用户及权限体系,示例具体权限设置可根据实际情况调整)
s3cmd setacl s3://my-bucket/specific_file.txt --acl-user=user1:RW

# 查看桶my-bucket中所有用户的权限信息(假设已配置好相关用户及权限体系)
s3cmd getacl s3://my-bucket --acl-users

# 给桶my-bucket添加默认访问权限为私有(示例,实际权限设置可根据需求调整)
s3cmd setacl s3://my-bucket --acl-default=PRIVATE

# 检查桶my-bucket是否存在(返回0表示存在,非0表示不存在)
s3cmd exists s3://my-bucket

# 计算桶my-bucket中文件total_file.txt的MD5值
s3cmd md5 s3://my-bucket/total_file.txt

# 查看桶my-bucket中文件total_file.txt的存储类型
s3cmd storageclass s3://my-bucket/total_file.txt

# 移动桶my-bucket中的文件from_file.txt到桶other-bucket中,存储名为to_file.txt
s3cmd mv s3://my-bucket/from_file.txt s3://other-bucket/to_file.txt