Imagine you’re an architect handed this brief: a prime-time televised guessing game, launching in weeks, expected to hit 8 million total registrations and 1 million concurrent users. It runs on Experience Cloud. It needs a conversational agent. It cannot go down, especially during the televised peak.
This is the scenario we worked through in the latest Think Like an Architect episode. In Think Like an Architect, we solve scenarios live so you can build the mental muscles needed to evaluate requirements, weigh options, and justify a solution direction instead of simply reviewing a finished architectural design. This recap walks you through each step of the scenario and the thinking behind it.
To design scalable solutions, we use a repeatable three-step method to move from raw business requirements to justified architectural decisions:
Step inside the architecture behind a high-stakes, large-scale digital experience designed to handle millions of users in real time. Use the What/How/Why method to diagram Big Game–level scale challenges and learn how the team tackled extreme volume considerations and lessons learned.
In the “What” step, let’s look at the specific challenge from this episode.
We start with a raw business requirement and use highlighting to indicate the key parts, as shown below.
From those highlights, four High-Level Requirements emerge:
Note that HLRs paraphrase rather than use the exact words from the business requirement. This is essential in validating with the business stakeholders that you understood the intention of their requirement.
Two groupings made sense as we moved into the “How” step. HLRs 2 and 3 are closely related as toxicity detection needs to happen in the conversational layer. HLR 4’s load balancing concern belongs with HLR 1, because you can’t talk about supporting 1 million concurrent users without talking about how they’re distributed. So we tackle HLR 1 + load balancing together, then HLRs 2 + 3 together, then the scale testing and zero-downtime strategy.
To continue with evaluating solution options, let’s move into the “How” step by following the trail of questions, assumptions, and decisions captured during our live session.
In the “How” step, we evaluate solution options for the HLRs. To keep your thinking organized, track your Questions, Assumptions, and Decisions alongside drawing the system landscape piece by piece.
The first question that comes to mind is: what’s the org strategy? Closely tied to that is the license strategy, because different Experience Cloud license types have very different scalability implications.
The requirement doesn’t suggest advanced sharing or complex permission models — it’s a public game. That points to a CC (Community Cloud) license. But even with a CC license, the platform has limits on both total users per org and concurrent users per org. At 8 million registrations and 1 million concurrent, a single org won’t work. We’re in multi-org territory.
With multi-org confirmed, the registration question comes next: what are the registration considerations? Three assumptions surface:
These assumptions point to a CDN. Akamai is the decision here, sitting in front of the orgs and handling three things: IP-based geo-filtering for US/Canada eligibility, fast delivery of static content, and a waiting room to manage traffic spikes at peak.
The next question is: how do we route users to the correct org? New users need to land in an org that has capacity for their registration. Returning users need to be routed back to the specific org where their account lives also known as what we call shard registration.
The decision is a pre-login infrastructure layer hosted on AWS. This is a natural fit because AWS shares a trust boundary with Salesforce. For new users, round-robin distribution ensures registrations are spread evenly across orgs. For returning users, a shard registry stored in AWS DynamoDB maps each user to their org so they’re routed back correctly on every subsequent visit.
With the scale and routing architecture in place, we move to the agent. The first question is: what kind of agent?
In an Experience Cloud context with a public-facing site, not an employee portal, the answer is an Agentforce Service Agent. It’s designed for external engagement, and it lives where the users are.
Next question: where does the knowledge live? Two categories of knowledge matter here. The rules and hints need to be consistent across all orgs. The conversation history is per-user, so it lives naturally in whichever org the user is registered in that’s already solved by the multi-org routing layer.
For the rules and hints, the simplest approach is a prompt template deployed to all orgs. The assumption here is that the rules and hints are not enormous documents but bounded content that fits cleanly into a template.
For toxicity detection, the answer is already built in. The Agentforce Trust Layer handles bidirectional content filtering — it applies to both what the agent says and what players type. This is out-of-the-box behavior, not a custom build.
This is a prime-time televised event, so the core question is: how can we have assurance that this will scale? The answer is to test at 3x the expected peak. If the architecture holds at 3x, you have a meaningful safety margin for game day.
When you run a scale test at that volume, you will find things. That leads to the next question: when findings come in, how do you tackle them? This is where Scale Center and Apex Guru come in. Scale Center provides observability meaning that you can see where the system falls apart and holds under pressure. Apex Guru gives you targeted recommendations for Apex code that needs to be rewritten or optimized to perform at scale.
Last question for this HLR: what’s the environment strategy? Given that we’re already in a multi-org setup at this scale, the only meaningful testing environment is a Full Copy Sandbox. The 3x target makes this clear as well, you need a production-grade environment to validate production-scale load.
During the live stream, we emphasized that an architect’s value is not in the solution, but in the rationale. The questions, assumptions, and decisions you capture during the “How” step are the raw material for an Architectural Decision Record that explains your thinking to stakeholders, to other architects, and to yourself six months from now.
A few specific ADR principles worth internalizing from this episode:
For a step-by-step guide to building ADRs, including how to use LLMs to help, see this blog on how to create an Architectural Decision Record with the help of LLMs.
Joining us for this episode were Karishma Lalwani, Salesforce Certified Technical Architect and Senior Director of Product at Salesforce, and Sebasten Raffal, Certified Technical Architect and Director AI Architect at Salesforce. Here’s a selection of the questions from the session.
Should you use one big agent action that handles multiple steps, or multiple smaller actions? Multiple smaller actions is the better design. It gives the agent more flexibility to orchestrate and makes each action reusable across multiple contexts. As Sebasten noted, consumption-based licensing in Agentforce is tied to what you use, not how you structured the actions so there’s no architectural penalty for composability.
Would you consider a standby conversational agent on a different LLM vendor as a failover? Salesforce already handles this within the platform. Every conversation goes through the LLM gateway, and within that gateway, Salesforce has load balancing and failover infrastructure across LLM vendors.
Why can’t a single org handle 8M users? The total number of Experience Cloud users a single org can support is in the hundreds of thousands, not millions. The concurrent user limits compound this further. Multi-org isn’t a preference or a default, but based on the scale of the solution, it is the only viable architecture.
What are the governance implications of a multi-org strategy? The primary complexity is release management. You need to ensure the same deployment packages, configurations, and customizations go out to every org consistently. In this scenario, each user belonged to exactly one org and no data sync between orgs was required beyond the prompt template. For a long-lived multi-org deployment, the administration overhead is more significant. Packaging the Experience Cloud definition and deploying it uniformly across orgs (as mentioned by audience participants during the session) is the right approach.
What about aggregating data across orgs to create a leaderboard? This requires an aggregation layer outside the individual orgs. One approach used successfully in practice: MuleSoft extracts data from each org on a scheduled basis and writes it to a central repository org. Tableau or CRM Analytics on top of that gives you the dashboard, and you can expose it externally if needed.
What are some other real-world uses of org sharding? Karishma shared from direct experience. During the COVID-19 vaccination rollout, multiple states and countries used Salesforce as their booking platform, implementing the same CDN + waiting room + org sharding pattern to manage sudden surges in traffic. She also talked about how this is a common experience that she encountered as a consumer when ordering international soccer tickets. These aren’t edge cases and any scenario with high-burst, time-sensitive public access hits these architectural constraints.
Isn’t a waiting room a frustrating user experience? It’s an industry-standard pattern for high-burst scenarios, and users have come to recognize it. The alternative of showing an error page when the system is overwhelmed is a much worse experience. A waiting room that tells you “your turn in 9 minutes” is manageable. One pre-registration strategy can reduce waiting room friction significantly: register users ahead of the event opening. If they’ve already completed the OTP flow and account setup in advance, the surge at launch is much smaller, because the high-cost part of registration is already done.
How do you ensure applicants are real people? The defense is layered. Akamai handles the first gate with IP-based filtering and traffic pattern detection. AWS has built-in threat detection for request blasting from single IPs. Inside Salesforce, an email-based OTP flow adds another verification step. In previous real deployments, users were also asked to upload verifiable identity documents. Each layer catches something the others don’t and the combined strategy is what makes it robust.
How do you handle the cost reality of 3x scale testing with AI in the mix? Testing at 3x volume on an AI-heavy solution means 3x LLM calls, and that has a real cost. The right framing is that this is an investment in derisking the launch. Estimate it upfront, bring it to stakeholders with context, and let them make an informed decision. The comparison to make: what does one minute of downtime cost during a live televised event? Karishma cited analysis showing that for high-stakes launches, this can be in the thousands of dollars per minute in lost customer acquisition.
Scale testing doesn’t need to be weeks-long. For most projects, the strategy is one x volume for the bulk of testing, then a dedicated three- to four-day scale test budget to stress the full end-to-end stack in this case Akamai, AWS, Salesforce app layer, database, LLM gateway. This is especially important for recurring peak events like Black Friday, tax season, or major sporting events, where a quarterly or annual scale test cadence makes sense.
An important takeaway for any architect is to first solve the problem with the simplest, most standard tool or functionality available, then prove why it won’t work before moving to more custom or made-to-measure solutions. In this architecture, that principle shows up repeatedly, such as leveraging prompt templates before Salesforce Knowledge, and Platform Events before Kafka.
Get monthly curated content, technical resources, and event updates designed to support your Salesforce Architect journey.
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。