Friday, 19 June 2026
As one of the authors of the Well-Known URI specification and current Designated Expert for the registry, I field a lot of questions about how they should be used, and end up coaching a lot of folks on how to best use them. Below, I’ve summarised how I think about them. Note that these aren’t all requirements for registration – just what I consider good practice.
What Well-Known URIs are Good For
Well-known URIs work best when the client – whether it’s a browser, bot, or other software – knows the site1 and needs to discover something about the whole site in an efficient way.
robots.txt is the perfect example – it pre-dated the RFC so it doesn’t use well-known URIs, but was a major part of the reason we reserved a space for them. A crawler needs to know what the access policies for the site are, and putting it in one central place for the site avoids the need to check headers and content on every response (which would defeat many of the purposes of having such a policy).
A well-known location doesn’t have to contain policy, though. Any mechanism where the client already knows the site but needs to learn something about or interact with it as a whole is a candidate for becoming a well-known URI. For example, the change-password well-known location allows clients to change their password for a site.
When They’re the Wrong Tool
While well-known locations solve real problems for some protocols, in other cases it seems like designers are specifying a well-known URI because it seems like the thing to do. Some proposals register one expecting it to confer legitimacy, or boost adoption — as though a slot in the registry were a credential. It isn’t. A well-known URI solves a specific problem (the client knows the site, and needs something site-wide); if your protocol doesn’t have that problem, a registration may only create new ones — and won’t bring the adoption you’re hoping for.
Similarly, some proposals for well-known locations are effectively using them as a URL shortener. Instead of conveying a full URL in a protocol, they only need to convey the relevant site – the well-known location fills in the rest.
The problem is that this pattern locks you into a 1:1 relationship between services and sites. If a deployment ever needs more than one service, they’ll need to create a different site, and find a way to direct users to the appropriate one.
Using a well-known location is reasonable if your protocol can genuinely only carry a hostname. Often, though, it’s done just for convenience — and maybe to make the protocol feel more “official” — causing unnecessary rigidity in deployments. If your protocol can use a real URL, don’t bother with a well-known location.
Common Pitfalls and Tradeoffs
Even when well-known locations are the right tool, the assumptions we make about sites don’t always hold true, and can create significant complications. If you’re defining a well-known URI for your protocol, you should be aware of the issues below.
Discovery Mechanisms
Many protocols try to use a well-known location as a discovery mechanism, with the idea that “the user already knows the site.”
The problem is that the reality is fuzzier than it sounds at first – there may be a mismatch between the scope of the user’s current interaction and where discovery happens. For example, if the client starts with “login.example.com”, should they look up the well-known URI on that site, or on “example.com”? Should they follow redirects from one to the other? What site(s) should the publisher make a well-known URI on available to assure interoperability?
This especially matters in cases where the protocol isn’t really about Web sites, but instead it’s just leveraging HTTP to get something else done. For example, it may be tempting to specify that a well-known location for a registrable domain name be located at the apex, but that can be operationally difficult for some.
If your protocol falls into this category, consider both what’s being discovered and what your users start with, then work out how they reliably find the right hostname without assuming too much about their architecture.
Content Metadata
Some protocols try to use a well-known location as a way to learn about the content on the site. After all, that’s how /robots.txt works.
While that pattern works for some kinds of metadata, many sites represent multiple publishers (for example, the old /~username/ convention). Putting content metadata in a central location either puts that mechanism off-limits to those users, or requires the administrator to develop complex infrastructure to support and oversee their control of it.
These kinds of deployments surface a tradeoff between convenience and granularity, and often necessitate creation of a parallel metadata mechanism – for example, in HTTP response headers or in the content itself – along with the rationalisation of the different metadata attachment mechanisms.
That’s not to say that Well-Known URIs can’t be used for content metadata; just that they’re a lot more work than you might think. So if your protocol is using well-known locations for resource metadata, make sure you consider these tradeoffs carefully. Not all sites are like the ones you’re used to – the Web is a big place.
Other Considerations
A few other things regularly come up.
First, some proposals have already defined a fixed location at the root (like /robots.txt – usually because they only learned about Well-Known URIs later). If so, they should have a transition plan for existing deployments. Generally, proponents over-focus on their current deployed footprint; with a good transition plan over a reasonable amount of time, moving over to a Well-Known URI is manageable.
Many proposals assume http and https URIs implicitly. Well-known URIs apply to other URI schemes too, so the relevant schemes should be enumerated explicitly.
Finally, make sure you register your well-known location. That link contains guidance about when to register and how to choose a name; unlike the advice here, it does affect your chances of successful registration.