Quick answer: A Lemmy scraper calls the public /api/v3/ REST API that every Lemmy instance exposes by design and returns posts (and optionally comments) as flat JSON rows. The Apify Actor below handles cursor pagination, exponential backoff on rate limits, browser fingerprint rotation, and denormalised output — one row per post or comment, community metadata included. Cost: $0.002 per post row, $0.001 per comment row, $0.05 per run start (~$2.05 per 1,000 posts).

When Reddit tightened its API terms in mid-2023, a wave of communities migrated to Lemmy. Three years later, lemmy.world hosts more than 39,000 subscribers in c/asklemmy alone, alongside dozens of active instances — lemmy.ml, beehaw.org, sh.itjust.works, programming.dev — each running the same open-source software and the same versioned REST API. For researchers, journalists, and NLP engineers, Lemmy is Reddit-shaped conversation data without the OAuth application process or the restricted academic access tier.

The complication: "same API" still means cursor-based post pagination, integer-page comment pagination, a 17-token sort enum that changed between minor versions (bare Top was silently removed in v0.19), and a federated post model where the post_ap_id field points back to the originating instance — not the one you queried. We absorb all of that for you.

What is Lemmy? 🌐

Lemmy is a free, open-source (AGPL-3.0) link-aggregator and discussion platform that runs the ActivityPub federation protocol. Think Reddit, but every "subreddit" is a community hosted on an independently operated server (an instance), and any two instances can subscribe to each other's communities. The result is a federated network: c/asklemmy@lemmy.world is reachable from lemmy.ml, and posts federate across every subscribed instance.

Each instance exposes a versioned JSON REST API at /api/v3/. Unauthenticated GET endpoints — community metadata, post lists, comment lists — are open by design, because read access is required for federation to work. That's the surface this Actor targets.

Does Lemmy have an official scraping API or data export?

No bulk export exists. The /api/v3/ REST API is excellent for reading data programmatically — versioned, well-documented, authentication-free for public reads. But it gives you one community at a time, one page at a time, with a cursor token you must thread through each request. There is no "export all posts" endpoint, no webhook, no streaming feed. Getting a dataset means writing a crawler — and keeping it working as Lemmy's minor versions ship API changes.

What the data looks like 📤

Each post row is a flat, typed object with community context, post metadata, author fields, and a scraped_at timestamp — no nested objects, no joins for a CSV export. Here is a real post row, exactly as the Actor writes it to the Apify dataset:

{
  "row_type": "post",
  "instance_url": "https://lemmy.world",
  "community_actor_id": "https://lemmy.world/c/asklemmy",
  "community_name": "asklemmy",
  "community_title": "Ask Lemmy",
  "community_description": "A Lemmy equivalent of Ask Reddit.",
  "community_subscribers": 39567,
  "community_posts_count": 8853,
  "community_local": true,
  "post_id": 46934299,
  "post_ap_id": "https://leminal.space/post/35446889",
  "post_url": "https://leminal.space/post/35446889",
  "post_title": "What's something that you feel genuinely sad about?",
  "post_body": null,
  "post_external_url": null,
  "post_score": 34,
  "post_upvotes": 38,
  "post_downvotes": 4,
  "post_comments_count": 18,
  "post_published": "2026-05-16T09:31:47.723504Z",
  "post_updated": null,
  "post_nsfw": false,
  "post_featured_community": false,
  "post_locked": false,
  "comment_id": null,
  "comment_ap_id": null,
  "comment_path": null,
  "comment_content": null,
  "comment_score": null,
  "comment_published": null,
  "comment_parent_id": null,
  "author_actor_id": "https://leminal.space/u/FosterMolasses",
  "author_name": "FosterMolasses",
  "author_display_name": null,
  "author_bot_account": false,
  "author_published": "2025-01-17T14:35:51.850105Z",
  "scraped_at": "2026-05-16T12:00:00.000Z"
}

Comment rows carry the same community and post fields — so a flat CSV export stays self-contained — plus the comment-specific fields: comment_id, comment_ap_id, comment_content, comment_score, comment_path, comment_published, and comment_parent_id (derived from the path, for tree reconstruction). The row_type discriminator tells you whether the comment fields are populated or null.

The naive approach (and why it falls apart) ⚙️

The Lemmy API is genuinely open, which tempts you into a three-line script. Here is what breaks before you finish writing it:

Cursor pagination is opaque and version-sensitive. GET /api/v3/post/list returns a next_page token in the response body. Miss it, hand it to the wrong endpoint, or pass the bare page integer the older API accepted, and you get the first page again in a silent loop. Comment pagination uses a different mechanism — an integer page parameter (1-based), not a cursor — so a single abstraction won't cover both.

The sort enum changed without a major version bump. Lemmy v0.19 silently removed the bare Top sort token and replaced it with seven compound tokens: TopHour, TopSixHour, TopTwelveHour, TopDay, TopWeek, TopMonth, TopYear. Pass bare Top and the API returns {"error": "unknown"} with a 200 status — no exception, just a payload that looks valid until you notice the posts array is empty. We enforce the 17-token enum in Pydantic validation and reject bare Top before any network call.

Federated posts carry a remote post_ap_id. When c/asklemmy@lemmy.world receives a federated post from leminal.space, the post.ap_id field points back to https://leminal.space/post/35446889 — the authoritative ActivityPub URL — not to lemmy.world. A naive scraper that builds local URLs from post_id gets this wrong for every federated post. We capture both post_ap_id and post_url as separate fields, exactly as the API returns them.

Rate limits need careful handling. Lemmy instances are community-operated servers with limited capacity. We retry 408, 429, and 503 responses with exponential backoff — base 2 seconds, doubling per attempt, capped at 30 seconds, max 5 attempts — and honour Retry-After headers when present. We rotate the browser fingerprint via curl-cffi impersonation (Chrome, Firefox, and Safari TLS profiles). If a run is interrupted, we surface the partial count via Actor.set_status_message rather than returning a green status with a quietly truncated dataset.

You want a flat table, not nested JSON. The Lemmy API returns community metadata, post metadata, and comment metadata in separate layers. We denormalise all three onto every row at write time, so a CSV export is immediately usable without joins.

The Actor 🔥

The scraper is packaged as an Apify Actor: Lemmy Community Scraper.

Set the instance URL and community name in the Apify Console and click Start, or run it from Python:

from apify_client import ApifyClient

client = ApifyClient("YOUR_APIFY_TOKEN")

run = client.actor("DevilScrapes/lemmy-community-scraper").call(
    run_input={
        "instanceUrl": "https://lemmy.world",
        "communityName": "asklemmy",
        "sort": "TopWeek",
        "maxPosts": 500,
        "includeComments": True,
        "maxCommentsPerPost": 25,
        "useProxy": False,
    }
)

for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item["row_type"], item["post_title"])

The sort parameter accepts any of the 17 Lemmy v0.19 tokens: TopWeek returns the highest-scoring posts of the last seven days, Hot (the default) blends recency and engagement, New is strictly chronological.

For a federated community hosted on a different instance, use the @instance form for communityName:

{
  "instanceUrl": "https://lemmy.ml",
  "communityName": "memes@lemmy.world",
  "sort": "Hot",
  "maxPosts": 100,
  "includeComments": false,
  "useProxy": false
}

The Actor queries lemmy.ml's view of the memes community hosted on lemmy.world. Different instances cache different subsets of federated content, so the instanceUrl parameter controls exactly which view you get.

Use cases 💡

Reddit-migration research. Track how communities move from Reddit to Lemmy after platform policy shifts. The community_subscribers, community_posts_count, and post timestamps give you the time-series inputs for a migration study across instances.

Newsroom monitoring. Subscribe to journalism, politics, or breaking-news communities and pipe the latest TopDay posts to a Slack alert or a Google Sheet via Apify integrations. The Actor runs on Apify Schedules — set it to hourly or daily without writing any cron infrastructure.

Brand monitoring on the Fediverse. Lemmy is a growing channel for product complaints, support discussions, and competitor mentions that won't show up in a Reddit or Twitter social-listening tool. Scrape relevant communities on a schedule and search the post_title, post_body, and comment_content fields for brand terms.

NLP and RAG corpus building. Lemmy's public REST API is more accessible than Reddit's gated API for building training corpora. The flat, denormalised output feeds directly into a Pandas dataframe or a vector store. Use includeComments: true and a permissive maxCommentsPerPost to capture full threads.

Comment-tree reconstruction. Each comment row carries comment_id and comment_parent_id (derived from Lemmy's dot-notation comment_path). Group rows by post_id, connect parents and children on comment_parent_id, and you have the full discussion tree — ready for a networkx graph or a moderation pipeline.

Pricing — exact numbers 💰

Pay-per-event. You pay for rows that land; the start fee covers the Actor boot.

Comment rows are priced at half the post-row rate because they are higher-volume and have lower per-row commercial value for most analytics workflows. If you only need post-level data, leave includeComments off — cost and run time drop substantially.

Apify's $5 free trial credit covers your first ~2,400 post rows with no credit card required. There are zero competing Lemmy scrapers on the Apify Store, so this is the only hosted option.

The technically interesting bit

The detail worth calling out: comment_parent_id is derived at parse time from Lemmy's dot-notation path field. Lemmy stores the full ancestry of every comment as a string like "0.12345.67890" in comment.path. The leading 0 is always the root; the final segment is the comment's own ID; every segment in between is an ancestor. To get comment_parent_id for tree reconstruction, the parser takes the penultimate segment — the immediate parent — or returns null for top-level comments whose path is "0.<id>".

This matters because Lemmy's REST API does not return a dedicated parent_id field. The path string is the canonical representation of thread ancestry, so deriving the parent ID from it is the only reliable way to reconstruct the comment tree. We handle that derivation in the parser, verified against live responses on both lemmy.world and lemmy.ml.

Limitations 🚧

• Public communities only. Private or restricted communities require authentication and are out of scope.
• One community per run. For a multi-community sweep, run the Actor multiple times (or fan out runs in parallel via the Apify API).
• No media download. Image, video, and external link URLs are captured as fields, but the Actor does not fetch the media content itself.
• No real-time streaming. The Actor takes a snapshot at run time. For a live feed, schedule recurring runs via Apify Schedules.
• Lemmy v0.19 field paths. Verified on v0.19.18 on lemmy.world and lemmy.ml. Pydantic's extra="ignore" absorbs additive API changes, but breaking field removes in a future version will require an Actor update.
• 7-day default storage retention on the Apify FREE tier. Export the dataset immediately after the run, or upgrade for longer retention.
• Comment tree is flat on output. The dataset is flat rows; callers reconstruct the tree from comment_parent_id.

FAQ ❓

Is scraping public Lemmy data legal?
Lemmy is AGPL-3.0 free and open-source software, and public communities are designed to be publicly readable — the unauthenticated REST API exists to support federation and third-party clients. This Actor reads only what the public API exposes, at a polite rate. Always verify the terms of service of the specific instance you query, and check your jurisdiction's data-protection rules before using scraped data commercially.

Do I need a Lemmy account or API key?
No. Lemmy's public /api/v3/ REST API requires no login, no API key, and no OAuth flow for read access to public communities.

Is there an official Lemmy export or bulk API?
No. The /api/v3/ REST API is excellent for programmatic access, but there is no bulk-export endpoint. You get one page per request and must manage pagination yourself — which is what this Actor handles for you.

Why does sort=Top not work?
Lemmy v0.19 removed the bare Top token and replaced it with compound tokens that embed a time range: TopHour, TopSixHour, TopTwelveHour, TopDay, TopWeek, TopMonth, TopYear, TopAll. Passing bare Top returns {"error": "unknown"}. This Actor validates the sort token via Pydantic before any network call and rejects bare Top at startup — pick the compound token whose time range you need.

Try it

The Actor is live on the Apify Store: apify.com/DevilScrapes/lemmy-community-scraper.

Free $5 trial credit, no credit card required. Point it at c/asklemmy@lemmy.world with sort=TopWeek and maxPosts=100 and you will have a week's worth of community posts — with community metadata and author fields — in a flat, export-ready dataset in under a minute.

If there is a field missing or a use case the output shape does not support, drop a note in the comments — I ship based on what people actually need.

Further reading:

• Lemmy API documentation (v0.19)
• ActivityPub protocol specification
• Apify platform documentation — Actors and datasets

Built by Devil Scrapes — Apify Actors that do the dirty work so your dataset stays clean. Pay-per-event, transparent pricing, no junk fields. 😈