惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

博客园 - 叶小钗
Cyber Security Advisories - MS-ISAC
Cyber Security Advisories - MS-ISAC
Microsoft Security Blog
Microsoft Security Blog
罗磊的独立博客
大猫的无限游戏
大猫的无限游戏
美团技术团队
让小产品的独立变现更简单 - ezindie.com
让小产品的独立变现更简单 - ezindie.com
钛媒体:引领未来商业与生活新知
钛媒体:引领未来商业与生活新知
aimingoo的专栏
aimingoo的专栏
腾讯CDC
WordPress大学
WordPress大学
Apple Machine Learning Research
Apple Machine Learning Research
F
Fortinet All Blogs
G
Google Developers Blog
MongoDB | Blog
MongoDB | Blog
Microsoft Azure Blog
Microsoft Azure Blog
小众软件
小众软件
Engineering at Meta
Engineering at Meta
博客园_首页
B
Blog RSS Feed
D
Docker
M
MIT News - Artificial intelligence
爱范儿
爱范儿
I
InfoQ

DigitalOcean Community Tutorials

It's Time to Break Up with Your Cloud: Why AI Teams are Switching We Built a Private-Document AI App to Test Platform Security. Here Is What We Could Actually Verify. PostgreSQL Explained: A Complete Beginner-to-Advanced Guide How To Install and Configure Postfix on Ubuntu How To Build a Web Application Using Flask in Python 3 Build AI Reading List with DigitalOcean Functions and Mistral How To Concatenate Strings in Python How to Allow MySQL Remote Access Securely How To Install and Use Docker on Rocky Linux How To Build a Multi-Agent AI System with Docker Agent DSPy Use Cases: Build Optimized LLM Pipelines How To Submit AJAX Forms with jQuery Build an AI-Powered GPU Fleet Optimizer with the DigitalOcean AI Platform ADK Monitor GPU Utilization in Real Time: A Complete Guide Reduce File Size of Images in Linux - CLI and GUI methods Reduce PDF File Size in Linux: Tools and Methods How To Set Up a Private Docker Registry on Ubuntu How To Troubleshoot Terraform: Errors and Fixes How to Use Go Modules Python Multiprocessing Example: Process, Pool & Queue Convert Class Components to Functional Components with React Hooks How To Install and Configure Ansible on Ubuntu LLM Tokenizers Simplified: BPE, SentencePiece, and More How To Monitor System Authentication Logs on Ubuntu How to Use Traceroute and MTR to Diagnose Network Issues How to Deploy Postgres to Kubernetes Cluster Importing Packages in Go: A Complete Guide Create RAID Arrays with mdadm on Ubuntu How To Make an HTTP Server in Go How To Set Up Time Synchronization on Ubuntu
Visualizing DigitalOcean Spaces Access Logs With GoAccess
Akshit Pratiush, Anish Singh Walia · 2026-05-07 · via DigitalOcean Community Tutorials

Introduction

DigitalOcean Spaces is an S3-compatible object storage service that supports detailed access logging. Spaces can automatically generate access logs for buckets and store the logs in a bucket you specify. Access logs include:

These access logs capture reads, writes, and deletions of objects within the bucket, requests made to origin endpoints, and accesses to the bucket’s CDN endpoints if CDN is enabled.

However, there’s a gap.

While Spaces provides the logs, it does not natively offer a built-in analytics or visualization layer. This is where tools like GoAccess come in, allowing teams to transform raw log files into meaningful, interactive dashboards. This tutorial takes raw, S3-style access logs from Spaces and turns them into a human-readable, real-time dashboard using lightweight, open-source tooling.

In this tutorial, you will learn how to:

This guide is especially useful for developers, DevOps engineers, and platform teams who want better visibility into Spaces usage.

Key takeaways

  • You can turn DigitalOcean Spaces access logs into a searchable HTML analytics dashboard with GoAccess in a few repeatable steps.
  • Access logs are delivered asynchronously, so your workflow should include periodic sync and report refresh.
  • A consistent S3 log format and a quick validation pass before report generation prevent most parsing failures.
  • Internal runbooks should include log retention, key scoping, and scheduled report updates for production use.

Prerequisites

Before you begin, make sure you have:

Step 1: Create a Droplet

Create a new Droplet using Ubuntu. A basic Droplet (1 vCPU / 1 GB RAM) is sufficient for moderate log volumes. Once created, SSH into the Droplet.

If you are new to Spaces setup, review How to Create a Spaces Bucket before continuing.

Step 2: Install GoAccess

GoAccess is a real-time web log analyzer and interactive viewer. We’ll use it to visualize our Spaces access logs in a browser-friendly HTML report.

Install GoAccess:

This updates your package list and installs GoAccess on your server:

sudo apt update
sudo apt install -y goaccess

Verify the installation:

Check that GoAccess was installed correctly by printing its version:

goaccess --version

This should output the installed version number, confirming it’s ready to use.

Step 3: Install AWS CLI v2

The AWS CLI v2 is required to interact with DigitalOcean Spaces, which uses S3-compatible APIs. With this tool, you’ll be able to download your access logs programmatically.

Install AWS CLI v2:

curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o /tmp/awscliv2.zip
sudo apt install -y unzip
unzip /tmp/awscliv2.zip -d /tmp
sudo /tmp/aws/install
aws --version
  • The first command downloads the AWS CLI installer as a zip file.
  • The next commands extract and install it.
  • aws --version confirms the CLI is installed and accessible.

Step 4: Configure Spaces access keys

You’ll need to provide credentials so the AWS CLI can access your Spaces bucket.

Set up the credentials folder and open the credentials file:

mkdir -p ~/.aws
nano ~/.aws/credentials
  • mkdir -p ~/.aws ensures the configuration directory exists.
  • nano ~/.aws/credentials creates or opens the credentials file for editing.

Add your Spaces keys:

[default]
aws_access_key_id = YOUR_SPACES_ACCESS_KEY
aws_secret_access_key = YOUR_SPACES_SECRET_KEY

Replace YOUR_SPACES_ACCESS_KEY and YOUR_SPACES_SECRET_KEY with your actual credentials from DigitalOcean.

Configure the region and output format:

nano ~/.aws/config
[default]
region = sgp1
output = json

The region should match your Spaces region (e.g., sgp1 for Singapore); output = json sets the default AWS CLI output format.

Step 5: Download logs from Spaces

Now, you’ll pull the access logs from your Spaces bucket down to your Droplet.

Set variables for your log bucket and prefix, and endpoint:

LOG_BUCKET="your-log-bucket-name"
LOG_PREFIX="spaces-logs-prefix/" # Prefix path where logs are written
ENDPOINT="https://sgp1.digitaloceanspaces.com"

Change LOG_BUCKET and LOG_PREFIX to match your setup.

Create a local directory for logs and sync them:

mkdir -p /var/log/spaces-logs
cd /var/log/spaces-logs

# Copy all logs in the prefix to local storage
aws --endpoint-url "$ENDPOINT" s3 cp "s3://$LOG_BUCKET/$LOG_PREFIX" ./ --recursive
  • The mkdir and cd commands set up a working directory for your logs.
  • The aws s3 cp command recursively copies log files from Spaces to your local directory.

Example variable values:

LOG_BUCKET="akshit-logs"
LOG_PREFIX="logs/"
ENDPOINT="https://sgp1.digitaloceanspaces.com"

Use these as a template for your own values.

Step 6: Decompress .gz files

Access log files from Spaces are typically compressed with gzip to save space.

To decompress all .gz files to plain .log files:

find . -type f -name "*.gz" -exec gunzip -kf {} \;
  • This command searches for all .gz files and decompresses them in place.
  • The -k option keeps the original .gz files, and -f forces overwrite if needed.
  • After running this, you’ll have uncompressed log files ready to process.

Step 7: Combine logs into one file

To simplify analysis, combine all the decompressed log files into a single file:

find . -type f -name "*.log" -print0 | xargs -0 cat -- > combined.log
  • This finds every .log file and concatenates them into combined.log.
  • Using one combined file makes it easier for GoAccess to generate a unified report.

Sanity check:

wc -l combined.log
  • This shows the number of lines in combined.log, so you can confirm that data has been collected.

Step 8: Generate the report

Now, use GoAccess to analyze the combined log file and create your dashboard.

Recommended GoAccess command:

goaccess /var/log/spaces-logs/combined.log \
  --log-format='%^ %^ [%d:%t %^] %h %^ %^ %^ %^ "%r" %s %^ %b %^ %^ %^ "%R" "%u" %^ %^ %^ %^ %^ %^' \
  --date-format=%d/%b/%Y \
  --time-format=%T \
  -a -o /var/www/html/spaces_report.html
  • Specifies the expected log format, which matches the S3 log structure.
  • -a enables analytics for all log entries.
  • -o defines the output path for the HTML dashboard.

If you are working with CDN logs in CloudFront format, use instead:

goaccess /var/log/spaces-logs/combined.log \
  --log-format=CLOUDFRONT \
  -a -o /var/www/html/spaces_report.html
  • The built-in CLOUDFRONT format covers standard CDN logs.

Why this format works

The log format string tells GoAccess how to parse each field in the S3 log entry:

  • %d/%b/%Y and %T match date and time formatting, e.g., [04/May/2016:12:11:43 +0000].
  • %^ skips fields you don’t want to process.
  • %h grabs the IP address.
  • "%r" captures the HTTP request line in quotes.
  • %s, %b, "%R", and "%u" capture response code, bytes sent, referrer, and user agent, respectively.

Quick validation for report generation: Before deploying, you can quickly generate a preview report to ensure parsing works:

goaccess /var/log/spaces-logs/combined.log \
  --log-format='%^ %^ [%d:%t %^] %h %^ %^ %^ %^ "%r" %s %^ %b %^ %^ %^ "%R" "%u" %^ %^ %^ %^ %^ %^' \
  --date-format=%d/%b/%Y \
  --time-format=%T \
  --no-global-config \
  -o /tmp/spaces_report_test.html

This creates a test dashboard in /tmp. Check that data displays as expected before publishing live.

Step 9: Install and configure Nginx

Nginx is a lightweight web server used to serve your GoAccess HTML dashboard.

Install Nginx:

sudo apt update
sudo apt install -y nginx
  • Updates the package list and installs Nginx.

Set permissions so the web server can read your report:

sudo chown -R www-data:www-data /var/www/html
sudo chmod -R 755 /var/www/html
  • Changes ownership and permissions for the web root so Nginx can serve the dashboard.

Start and enable Nginx:

sudo systemctl enable --now nginx
  • Starts Nginx now and configures it to start automatically on reboot.

Now, open http://YOUR_DROPLET_IP/spaces_report.html in your web browser.

You will see an interactive dashboard visualizing your Spaces access logs—enabling quick insights into storage activity, requests, and usage patterns.

Operational tips for production

  • Schedule sync and report generation with cron to keep dashboards current.
  • Keep Spaces keys scoped to least privilege and rotate them on a regular schedule.
  • Store generated reports behind access controls if logs contain sensitive metadata.
  • Track parsing failures by testing each new log format variant before automation.

FAQs

1. How do I check whether Spaces access logging is enabled?

Use the bucket Settings tab in the control panel or run aws s3api get-bucket-logging against the Spaces endpoint. The response includes LoggingEnabled when logging is active.

2. Where are Spaces access logs stored?

Spaces stores logs in the destination bucket and prefix you set in the logging configuration. Keep destination and source buckets separate, and use the same region for best performance.

3. Can GoAccess parse S3-compatible logs in real time?

Yes, GoAccess can process S3-compatible logs when you provide a matching --log-format, --date-format, and --time-format. For continuously updated dashboards, run sync and generation on a schedule.

4. Why is my GoAccess report empty or missing requests?

The most common causes are mismatched log format tokens, an empty combined.log, or delays in asynchronous log delivery. Validate with wc -l combined.log and test parsing to /tmp/spaces_report_test.html.

5. How long does it take for Spaces access logs to appear?

Access logs are delivered asynchronously. In most cases logs appear within about an hour, but delays can be longer during peak periods.

Conclusion

In this tutorial, you configured DigitalOcean Spaces access logging, synced logs locally using the S3-compatible AWS CLI, and transformed raw log data into an interactive HTML dashboard using GoAccess. This workflow provides a lightweight and cost-effective way to gain visibility into your storage usage without relying on heavy observability stacks.

By combining Spaces’ built-in logging with open-source tooling, you now have a practical foundation for monitoring traffic patterns, debugging issues, and understanding how your data is being accessed.

Next steps

To extend this setup, review the Spaces documentation and test DigitalOcean Spaces with your production logging workflow. You can also connect your broader infrastructure with DigitalOcean Kubernetes when you are ready to scale analytics pipelines.

Still looking for an answer?

Creative CommonsThis work is licensed under a Creative Commons Attribution-NonCommercial- ShareAlike 4.0 International License.