惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

Cyberwarzone
Cyberwarzone
F
Full Disclosure
V
Visual Studio Blog
freeCodeCamp Programming Tutorials: Python, JavaScript, Git & More
有赞技术团队
有赞技术团队
J
Java Code Geeks
博客园 - 【当耐特】
OSCHINA 社区最新新闻
OSCHINA 社区最新新闻
博客园 - 叶小钗
L
LINUX DO - 最新话题
T
Threatpost
S
SegmentFault 最新的问题
Vercel News
Vercel News
云风的 BLOG
云风的 BLOG
C
Cyber Attacks, Cyber Crime and Cyber Security
Google DeepMind News
Google DeepMind News
Know Your Adversary
Know Your Adversary
S
Schneier on Security
V
Vulnerabilities – Threatpost
D
DataBreaches.Net
G
GRAHAM CLULEY
Latest news
Latest news
P
Privacy International News Feed
D
Darknet – Hacking Tools, Hacker News & Cyber Security
C
CXSECURITY Database RSS Feed - CXSecurity.com
Scott Helme
Scott Helme
L
Lohrmann on Cybersecurity
T
The Exploit Database - CXSecurity.com
Security Latest
Security Latest
G
Google Developers Blog
L
LangChain Blog
MyScale Blog
MyScale Blog
Project Zero
Project Zero
N
News and Events Feed by Topic
Hacker News - Newest:
Hacker News - Newest: "LLM"
大猫的无限游戏
大猫的无限游戏
P
Proofpoint News Feed
Blog — PlanetScale
Blog — PlanetScale
阮一峰的网络日志
阮一峰的网络日志
N
News | PayPal Newsroom
www.infosecurity-magazine.com
www.infosecurity-magazine.com
cs.AI updates on arXiv.org
cs.AI updates on arXiv.org
SecWiki News
SecWiki News
T
Tor Project blog
C
Check Point Blog
Google Online Security Blog
Google Online Security Blog
GbyAI
GbyAI
The Last Watchdog
The Last Watchdog
cs.CL updates on arXiv.org
cs.CL updates on arXiv.org
WordPress大学
WordPress大学

DigitalOcean Community Tutorials

Mastering grep with Regular Expressions for Efficient Text Search It's Time to Break Up with Your Cloud: Why AI Teams are Switching We Built a Private-Document AI App to Test Platform Security. Here Is What We Could Actually Verify. PostgreSQL Explained: A Complete Beginner-to-Advanced Guide How To Install and Configure Postfix on Ubuntu How To Build a Web Application Using Flask in Python 3 Build AI Reading List with DigitalOcean Functions and Mistral How To Concatenate Strings in Python How to Allow MySQL Remote Access Securely How To Install and Use Docker on Rocky Linux How To Build a Multi-Agent AI System with Docker Agent DSPy Use Cases: Build Optimized LLM Pipelines How To Submit AJAX Forms with jQuery Build an AI-Powered GPU Fleet Optimizer with the DigitalOcean AI Platform ADK Monitor GPU Utilization in Real Time: A Complete Guide Reduce File Size of Images in Linux - CLI and GUI methods Reduce PDF File Size in Linux: Tools and Methods How To Set Up a Private Docker Registry on Ubuntu How To Troubleshoot Terraform: Errors and Fixes How to Use Go Modules Python Multiprocessing Example: Process, Pool & Queue Convert Class Components to Functional Components with React Hooks How To Install and Configure Ansible on Ubuntu LLM Tokenizers Simplified: BPE, SentencePiece, and More How To Monitor System Authentication Logs on Ubuntu How to Use Traceroute and MTR to Diagnose Network Issues How to Deploy Postgres to Kubernetes Cluster Importing Packages in Go: A Complete Guide Create RAID Arrays with mdadm on Ubuntu How To Make an HTTP Server in Go How To Set Up Time Synchronization on Ubuntu How To Use Struct Tags in Go apt-key Deprecation: Add Repositories with GPG on Ubuntu Linux ps Command: 20 Real-World Examples Python struct.pack and struct.unpack for Binary Data Deadlock in Java: Examples, Detection, and Prevention How To Use Find and Locate to Search for Files on Linux Structured Resume Skill Extraction Using Mistral-7B Inference How to Use the Python Main Function How to Set Up NemoClaw on a DigitalOcean Droplet with 1-Click Build an End-to-End RAG Pipeline for LLM Applications From Single to Multi-Agent Systems: Key Infrastructure Needs Back Up Data to Object Storage Using Restic How to Generate Videos with LTX-2.3 on DigitalOcean GPU Droplets How To Install LAMP Stack (Apache, MySQL, PHP) on Ubuntu How To Use Variadic Functions in Go Generate UUIDs with uuidgen on Linux How To Use EJS to Template Your Node Application How to Install Node.js on Ubuntu (Step-by-Step Guide) MongoDB Indexes: Improve Query Performance with Node.js LLM Tool Calling with DigitalOcean AI Platform and Databases What are Text Diffusion Models? - An Overview Crafting a Game from Scratch with GPT-5.4 Building Long-Term Memory in AI Agents with LangGraph and Mem0 How To Install PHP 7.4 and Set Up a Local Development Environment on Ubuntu 20.04 Build a GraphQL API in Go to Upload Files to Spaces How To Lint and Format Code with ESLint in Visual Studio Code Train YOLO26 for Retail Object Detection on DigitalOcean GPUs How To Work with JSON in MySQL How to Use the JavaScript .map() Method Building a Scalable App with MongoDB Using DigitalOcean's MCP Server How to Create an SSH Key in Linux: Easy Step-by-Step Guide Measure MySQL Query Performance with mysqlslap How To Use *args and **kwargs in Python 3 Nemotron 3 helped me find the perfect dish rack? A2A vs MCP - How These AI Agent Protocols Actually Differ How To Install and Manage Supervisor Docker Container Images with Watchtower on Ubuntu Getting Started with Qwen3.5 Vision-Language Models How To Create a New Sudo-Enabled User on Ubuntu How to Use Ansible to Install and Set Up Docker on Ubuntu How To Enable Remote Desktop Protocol Using xrdp on Ubuntu 22.04 How To Convert a String to a List in Python How To Check If a String Contains Another String in Python How to Read a Properties File in Python Python Command Line Arguments: sys.argv, argparse, getopt Mastering Grep command in Linux/Unix: A Beginner's Tutorial Understanding Python Data Types How to Implement a Stack in C With Code Examples Python os.system() vs subprocess: Run System Commands How To Install and Use Docker Compose on Ubuntu How to Add and Delete Users on Ubuntu How To Order Query Results in Laravel Eloquent How To Define and Use Handlers in Ansible Playbooks How To Install and Use SQLite on Ubuntu How To Install and Use Homebrew on macOS How To Manage DateTime with Carbon in Laravel and PHP How To Install Git on Ubuntu How To Install and Secure Redis on Ubuntu How To Build and Install Go Programs on Linux Using ldflags to Set Version Information for Go Applications How To Build a Node.js Application with Docker How To Add JavaScript to HTML How To Reset Your MySQL or MariaDB Root Password How To Add Images in Markdown How To Set Up a Production Elasticsearch Cluster with Ansible How To Set Up a Firewall Using firewalld on CentOS Understanding Systemd Units and Unit Files How To Set Up Replication in MySQL How To Use the .htaccess File
How to Download Files with cURL
bhogan · 2026-03-18 · via DigitalOcean Community Tutorials

Client URL, or cURL, is a library and command-line utility for transferring data between systems. It supports many protocols and tends to be installed by default on many Unix-like operating systems. Because of its general availability, it is a great choice for downloading a file to your local system, especially in a server environment.

In this tutorial, you’ll use the curl command to download a text file from a web server. You’ll view its contents, save it locally, and tell curl to follow redirects if files have moved. This knowledge is particularly useful when working with REST APIs or setting up Node.js applications.

Downloading files from the Internet can be dangerous, so be sure you are downloading from reputable sources. In this tutorial, you’ll download files from DigitalOcean, and you won’t be executing any files you download.

Key Takeaways:

  • cURL is a powerful and flexible command-line tool used for transferring data across a wide range of protocols, making it a staple for developers and system administrators.
  • By default, cURL retrieves remote content and displays it in the terminal, which is useful for quickly inspecting files or API responses.
  • You can save downloaded files locally using the -O option to preserve the original filename or the -o option to define a custom name.
  • cURL does not automatically follow HTTP redirects, so you must use the -L or --location flag to ensure you reach the final destination URL.
  • It supports multiple authentication methods, including basic authentication with credentials and token-based authentication using HTTP headers.
  • cURL provides built-in options for handling unreliable networks, such as resuming interrupted downloads, setting timeouts, and retrying failed requests.
  • It integrates seamlessly into shell scripts, making it ideal for automating downloads, interacting with APIs, and supporting CI/CD workflows.
  • When issues arise, cURL offers debugging capabilities like verbose output and header inspection, and tools like wget can serve as alternatives for specific download use cases.

Step 1 — Fetching remote files

Out of the box, without any command-line arguments, the curl command will fetch a file and display its contents to the standard output.

Let’s give it a try by downloading the robots.txt file from Digitalocean.com:

  1. curl https://www.digitalocean.com/robots.txt

You’ll see the file’s contents displayed on the screen:

Output

User-agent: * Disallow: sitemap: https://www.digitalocean.com/sitemap.xml sitemap: https://www.digitalocean.com/community/main_sitemap.xml.gz sitemap: https://www.digitalocean.com/community/questions_sitemap.xml.gz sitemap: https://www.digitalocean.com/community/users_sitemap.xml.gz

Give curl a URL and it will fetch the resource and display its contents.

Saving Remote Files

Fetching a file and displaying its contents is all well and good, but what if you want to actually save the file to your system?

To save the remote file to your local system, with the same filename as the server you’re downloading from, add the --remote-name argument, or use the -O option:

  1. curl -O https://www.digitalocean.com/robots.txt

Your file will download:

Output

% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 286 0 286 0 0 5296 0 --:--:-- --:--:-- --:--:-- 5296

Instead of displaying the contents of the file, curl displays a text-based progress meter and saves the file to the same name as the remote file’s name. You can check on things with the cat command:

  1. cat robots.txt

The file contains the same contents you saw previously:

Output

User-agent: * Disallow: sitemap: https://www.digitalocean.com/sitemap.xml sitemap: https://www.digitalocean.com/community/main_sitemap.xml.gz sitemap: https://www.digitalocean.com/community/questions_sitemap.xml.gz sitemap: https://www.digitalocean.com/community/users_sitemap.xml.gz

Now let’s look at specifying a filename for the downloaded file.

Step 2 — Saving Remote Files with a Specific File Name

You may already have a local file with the same name as the file on the remote server.

To avoid overwriting your local file of the same name, use the -o or --output argument, followed by the name of the local file you’d like to save the contents to.

Execute the following command to download the remote robots.txt file to the locally named do-bots.txt file:

  1. curl -o do-bots.txt https://www.digitalocean.com/robots.txt

Once again, you’ll see the progress bar:

Output

% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 286 0 286 0 0 6975 0 --:--:-- --:--:-- --:--:-- 7150

Now use the cat command to display the contents of do-bots.txt to verify it’s the file you downloaded:

  1. cat do-bots.txt

The contents are the same:

Output

User-agent: * Disallow: sitemap: https://www.digitalocean.com/sitemap.xml sitemap: https://www.digitalocean.com/community/main_sitemap.xml.gz sitemap: https://www.digitalocean.com/community/questions_sitemap.xml.gz sitemap: https://www.digitalocean.com/community/users_sitemap.xml.gz

By default, curl doesn’t follow redirects, so when files move, you might not get what you expect. Let’s look at how to fix that.

Step 3 — Following Redirects

Thus far, all of the examples have included fully qualified URLs that include the https:// protocol. If you happened to try to fetch the robots.txt file and only specified www.digitalocean.com, you would not see any output, because DigitalOcean redirects requests from http:// to https://:

You can verify this by using the -I flag, which displays the request headers rather than the contents of the file:

  1. curl -I www.digitalocean.com/robots.txt

The output shows that the URL was redirected. The first line of the output tells you that it was moved, and the Location line tells you where:

Output

HTTP/1.1 301 Moved Permanently Date: Wed, 18 Mar 2026 05:27:02 GMT Content-Length: 0 Connection: keep-alive CF-RAY: 9de1d370fd80c537-BLR location: https://www.digitalocean.com/robots.txt CF-Cache-Status: MISS Expires: Wed, 18 Mar 2026 07:27:02 GMT Cache-Control: public, max-age=7200 Server: cloudflare . . .

You could use curl to make another request manually, or you can use the --location or -L argument which tells curl to redo the request to the new location whenever it encounters a redirect. Give it a try:

  1. curl -L www.digitalocean.com/robots.txt

This time, you see the output as curl followed by the redirect:

Output

User-agent: * Disallow: sitemap: https://www.digitalocean.com/sitemap.xml sitemap: https://www.digitalocean.com/community/main_sitemap.xml.gz sitemap: https://www.digitalocean.com/community/questions_sitemap.xml.gz sitemap: https://www.digitalocean.com/community/users_sitemap.xml.gz

You can combine the -L argument with some of the aforementioned arguments to download the file to your local system:

  1. curl -L -o do-bots.txt www.digitalocean.com/robots.txt

Warning: Many resources online will ask you to use curl to download scripts and execute them. Before you run any scripts you have downloaded, it’s good practice to check their contents before making them executable and running them. Use the less command to review the code to ensure it’s something you want to run.

Step 4 — Downloading Files with Authentication

Some web files are protected and require authentication. curl allows you to handle these cases easily. This is particularly useful when working with proxy servers or secure API endpoints.

Basic Authentication (Username & Password)

To access a file that requires login credentials, use the -u flag:

  1. curl -u username:password -O https://example.com/securefile.zip

Token-Based Authentication

You can also use headers to pass API tokens:

  1. curl -H "Authorization: Bearer YOUR_TOKEN" -O https://api.example.com/protected/data.json

For better security, avoid hardcoding sensitive data. Instead, use environment variables or configuration files.

Step 5 — Handling Timeouts, Retries, and Resuming Downloads

Robust scripts need to account for network interruptions and delays.

Resume Interrupted Downloads

Use the -C - option to resume:

  1. curl -C - -O https://example.com/largefile.iso

Set Timeouts

Prevent curl from hanging indefinitely:

  1. curl --max-time 30 -O https://example.com/file.txt

Retry Failed Downloads

Automatically retry failed downloads up to 3 times:

  1. curl --retry 3 -O https://example.com/file.txt

Step 6 — Automating Downloads with Shell Scripts

Automating downloads can be useful in CI/CD pipelines or regular backups. This is especially relevant when working with Node.js applications or REST APIs that require regular data updates.

Sample Script

#!/bin/bash
URL="https://example.com/file.zip"
DEST="/home/user/downloads/file.zip"
curl -L -o "$DEST" "$URL"

Make the script executable with chmod +x script.sh, then schedule it with cron or use it in a deployment pipeline.

Step 7 — Troubleshooting Common Download Issues

Sometimes downloads might fail or behave unexpectedly. Here are some common issues and their solutions:

File Not Downloading

If curl isn’t downloading your file, try these troubleshooting steps:

  1. curl -I https://example.com/file.zip
  • Verify if the server requires a specific user agent:

    1. curl -A "Mozilla/5.0" -O https://example.com/file.zip
  • Check for SSL/TLS issues:

    1. curl -v -O https://example.com/file.zip
  • Try with different protocols if available:

    1. # Try HTTP if HTTPS fails
    2. curl -O http://example.com/file.zip
  • Check if the file exists and you have permissions:

    1. curl -u username:password -O https://example.com/file.zip

If you’re still having issues, the verbose output (-v) will help identify the problem:

  1. curl -v -O https://example.com/file.zip

Step 8 — Using wget as an Alternative

While curl is powerful, sometimes wget might be a better choice for certain download scenarios. wget is specifically designed for downloading files and has some features that make it particularly useful:

Basic wget Usage

To download a file with wget:

  1. wget https://example.com/file.zip

Key wget Features

  • Automatic retry on failure:

    1. wget -t 3 https://example.com/file.zip
  • Download in the background:

    1. wget -b https://example.com/file.zip
  • Limit download speed:

    1. wget --limit-rate=200k https://example.com/file.zip
  • Download entire websites:

    1. wget --mirror --convert-links --adjust-extension --page-requisites --no-parent https://example.com

When to Choose wget over curl

  • When you need recursive downloading
  • For mirroring entire websites
  • When you want automatic retry functionality
  • For simpler download commands (wget has fewer options to remember)

When to Stick with curl

  • When you need to interact with APIs
  • For more complex HTTP requests
  • When you need to send data to servers
  • For better script integration

FAQs

1. What is the difference between -O and -o in cURL?

The -O (uppercase “o”) option in curl saves the downloaded file using the original filename as provided by the server in the URL or HTTP headers. This is particularly useful when you want to preserve the server’s naming convention, or when downloading multiple files in a batch without having to specify each filename manually.

On the other hand, the -o (lowercase “o”) option allows you to specify a custom filename for the downloaded file. This is helpful for organizing your downloads, preventing filename collisions, or when you want to give the file a more meaningful name locally.

Example:

  1. curl -O https://example.com/file.zip # Saves as file.zip
  2. curl -o custom-name.zip https://example.com/file.zip # Saves as custom-name.zip

2. How do I resume an interrupted download with cURL?

If your download was interrupted due to a network error or system reboot, you can resume it using the -C - option. This tells curl to continue the download from where it left off, assuming the server supports HTTP range requests.

Example:

  1. curl -C - -O https://example.com/largefile.iso

This is particularly useful for large ISO files or multi-gigabyte datasets, especially when working with unreliable internet connections or automating downloads in scripts.

3. Can I download files behind authentication with cURL?

Yes, curl supports both basic and token-based authentication methods.

For basic authentication using a username and password:

  1. curl -u username:password -O https://secure.example.com/file.zip

For token-based authentication, use the Authorization header:

  1. curl -H "Authorization: Bearer YOUR_TOKEN" -O https://api.example.com/file.zip

Always avoid hardcoding sensitive credentials. Use environment variables or configuration files when scripting to enhance security.

4. What should I do if the download URL redirects?

Some URLs redirect from http to https, or from an old endpoint to a new one. By default, curl does not follow these redirects.

To handle this, add the -L or --location option:

  1. curl -L -O http://example.com/download

This is especially important when accessing public APIs, shortened URLs, or migrating download links.

5. Is cURL available on Windows?

Yes, cURL is included by default in Windows 10 and later. You can use it directly from Command Prompt or PowerShell.

For older versions, or for enhanced Unix-like behaviour, you can install it via:

Once installed, you can run cURL commands just like on Linux or macOS. This enables cross-platform scripting and consistent development workflows.

6. How can I download multiple files at once with cURL?

You can download multiple files in a single command using either a list of URLs or a pattern. Here are two common approaches:

Using multiple URLs:

  1. curl -O https://example.com/file1.zip -O https://example.com/file2.zip

Using a pattern with brace expansion:

  1. curl -O https://example.com/file{1..5}.zip

For more complex scenarios, you can also use a text file containing URLs and the -K option:

  1. # urls.txt contains one URL per line
  2. curl -K urls.txt

7. How do I handle SSL/TLS certificate issues with cURL?

Sometimes, you might encounter SSL certificate errors when downloading from servers with expired or self-signed certificates (SSL). While not recommended for production use, you can bypass certificate verification using the -k or --insecure option:

  1. curl -k -O https://example.com/file.zip

For a more secure approach, you can specify a custom certificate:

  1. curl --cacert /path/to/certificate.pem -O https://example.com/file.zip

Remember that bypassing certificate verification can expose you to security risks, so use these options cautiously.

8. How can I monitor download progress and speed with cURL?

By default, cURL shows a progress bar, but you can customise the output using various options:

For a simple progress bar:

  1. curl -# -O https://example.com/file.zip

For detailed progress information:

  1. curl -w "\nDownloaded: %{size_download} bytes\nSpeed: %{speed_download} bytes/sec\nTime: %{time_total} seconds\n" -O https://example.com/file.zip

You can also create a custom progress format using the -w option with various variables like %{speed_download}, %{time_total}, and %{size_download}.

Conclusion

curl lets you quickly download files from a remote system. It supports a wide range of protocols like HTTP, HTTPS, FTP, and more—making it a reliable and script-friendly choice for file transfers. But it doesn’t stop there.

From simple downloads to complex API interactions, curl can handle everything from setting custom headers and authentication to managing redirects and resumable downloads. It’s a staple tool for developers, sysadmins, and DevOps engineers who need precise control over network communication without relying on heavyweight tools.

Whether you’re automating tasks in a CI/CD pipeline, integrating data from external sources, or testing endpoints in REST APIs or Node.js applications, curl fits naturally into modern development workflows.

To dive deeper into all its capabilities, view the manual page by running:

  1. man curl

Or explore online examples to sharpen your command-line skills even further.

For more Linux-related tutorials, check out the following articles:

Creative CommonsThis work is licensed under a Creative Commons Attribution-NonCommercial- ShareAlike 4.0 International License.