惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

cs.CV updates on arXiv.org
cs.CV updates on arXiv.org
小众软件
小众软件
博客园_首页
博客园 - 聂微东
V
V2EX
WordPress大学
WordPress大学
freeCodeCamp Programming Tutorials: Python, JavaScript, Git & More
OSCHINA 社区最新新闻
OSCHINA 社区最新新闻
罗磊的独立博客
酷 壳 – CoolShell
酷 壳 – CoolShell
博客园 - 司徒正美
博客园 - 三生石上(FineUI控件)
奇客Solidot–传递最新科技情报
奇客Solidot–传递最新科技情报
S
SegmentFault 最新的问题
J
Java Code Geeks
Last Week in AI
Last Week in AI
The Cloudflare Blog
月光博客
月光博客
雷峰网
雷峰网
宝玉的分享
宝玉的分享
钛媒体:引领未来商业与生活新知
钛媒体:引领未来商业与生活新知
Hugging Face - Blog
Hugging Face - Blog
让小产品的独立变现更简单 - ezindie.com
让小产品的独立变现更简单 - ezindie.com
有赞技术团队
有赞技术团队
人人都是产品经理
人人都是产品经理
博客园 - Franky
腾讯CDC
Jina AI
Jina AI
博客园 - 叶小钗
大猫的无限游戏
大猫的无限游戏
阮一峰的网络日志
阮一峰的网络日志
量子位
爱范儿
爱范儿
美团技术团队
T
Tailwind CSS Blog
博客园 - 【当耐特】
D
Docker
IT之家
IT之家
V
Visual Studio Blog
P
Proofpoint News Feed
L
LangChain Blog
Engineering at Meta
Engineering at Meta
C
Check Point Blog
G
Google Developers Blog
Google DeepMind News
Google DeepMind News
云风的 BLOG
云风的 BLOG
Cyber Security Advisories - MS-ISAC
Cyber Security Advisories - MS-ISAC
Microsoft Azure Blog
Microsoft Azure Blog
B
Blog RSS Feed
Recorded Future
Recorded Future

Catchpoint Blog

SRE Report: AI optimism and the economics of effort SRE Report: Why fast is what users trust SRE Report 2026: What surprised us, what didn't, and why the gaps matter most The SRE Report 2026: Defensible Ns Why Synthetic Tracing Delivers Better Data, Not Just More Data A New Chapter: LogicMonitor + Catchpoint – A Personal Note from Mehdi Mezmo + Catchpoint deliver observability SREs can rely on The four pillars holding up your digital business, and what happens when they crumble When payments pause: lessons from a global payments outage Observability 2025 Decoded: What the DZone Report Means for SLO-Driven Ops The next evolution of WebPageTest has arrived, and it’s a game-changer The Monitoring Blind Spot That Could Cost You Black Friday Powering Mexico’s Digital Future: Expanded Internet Observability with Catchpoint The Next Chapter of WebPageTest: Your New Experience Starts Soon SRE Report Retrospectives — Have AIOps Predictions Held Up? Session Replay explained: A guide to seeing digital experience through your user’s eyes Making the invisible visible: Are your cloud firewalls and DDoS protection really working? Why it’s time to move beyond APM: Monitoring from the user’s perspective When metrics mislead: Inside the 2025 Retail Web Performance Benchmark The vendor trap: why your next outage won’t be your fault—but will be your problem LLMs don’t stand still: How to monitor and trust the models powering your AI Semantic Caching: What We Measured, Why It Matters The Annual SRE Survey Is Open—We Want to Hear from You Observability isn’t about the tool. It’s about the truth Invisible dependencies, visible impact: Lessons from the Google Cloud outage Real-time detection of BGP blackholing and prefix hijacks Leading analyst firm reveals the real cost of internet disruptions The Power of Over 3000 Intelligent Observability Agents Monitoring in the Age of Complexity: 5 Assumptions CIOs Need to Rethink Why Intelligent Traffic Steering is Critical for Performance and Cost Optimization Retail digital performance event recap: Key insights from IBM & Catchpoint Zendesk outage: A case for proactive monitoring and faster incident response Silence during chaos: Why the X outage is a call to arms for proactive monitoring The $1 Million Lesson: Building a Culture of Quality Through SLAs When AI tools fail: How to map your AI dependencies for proactive visibility Why Super Bowl 2025 was a triumph for Internet Resilience Why Internet Performance Monitoring is the new health check for IT organizations Why use Playwright in Catchpoint for synthetic monitoring Introducing WebPageTest Expert Plan: Real-Time Insights, Synthetic + RUM together in One Platform The shift to digital: How businesses are reshaping their priorities for 2025 The SRE Report 2025's Call to Action Monitoring in the Age of the Internet: DEM, IPM, and APM—What You Need to Know SSL Monitoring, Trust, and McLOVIN Performing for the holidays: Look beyond uptime for season sales success Lessons from Microsoft’s office 365 Outage: The Importance of third-party monitoring Web Performance Experts Look into the Future of Web Performance The hidden challenges of Internet Resilience: Key insights from 2024 report When SSL Issues aren’t just about SSL: A deep dive into the TIBCO Mashery outage The curious case of Marriott and the untold impact of web performance on revenue Preparing for the unexpected: Lessons from the AJIO and Jio Outage It’s time to stop neglecting the elephant in the room: Performance Matters! The Need for Speed: Highlights from IBM and Catchpoint’s Global DNS Performance Study Learnings from ServiceNow’s Proactive Response to a Network Breakdown Webinar Recap: Taking Web Performance to the Next Level Use the Catchpoint Terraform Provider in your CI/CD workflows Is the Internet ready for L4S? Takeaways from the CrowdStrike outage: third-parties can pose risk July 19th global IT outage reminds us of digital complexity 5 Actions you can take to improve digital performance 2024: A banner year for Internet Resilience APM vs Observability: Both-and, not either-or AppAssure: Ensuring the resilience of your Tier-1 applications just became easier APM vs observability: why your definitions are broken APM vs Observability: What comes next? APM vs Observability: Observing beyond APM Achieving stability with agility in your CI/CD pipeline AWS Outage: How do you prepare for the failure of your own safety net? Agentic AI: Powerful But Fragile—What You Need to Know Catch frustration before it costs you: New tools for a better user experience Catchpoint Expands Observability Network to Barcelona: A Growing Internet Hub Catchpoint Peak Performance Summit 2025: Redefining Observability for the Outcome Economy Catchpoint named a leader in the 2024 Gartner® Magic Quadrant™ for Digital Experience Monitoring Consolidation and Modernization in Enterprise Observability Connected Devices: Unlocking the next frontier of Internet Performance Monitoring Cloud Monitoring's Blind Spot: The User Perspective Cloudflare’s Resolver Outage: More Than Just DNS Cloudflare outage: another wake-up call for resilience planning Demystifying API Monitoring and Testing with IPM Creating the IPM Category: Catchpoint’s Journey to Leadership and the LogicMonitor Era Critical Requirements for Modern API Monitoring Customer Survey 2024: Unveiling insights and impact Did Delta's slow web performance signal trouble before CrowdStrike? Diagnosing Wi-Fi failures that traditional tools miss: a case study DNS misconfiguration can happen to anyone - the question is how fast can you detect it? ECN explained: Navigate congestion for faster, smoother data delivery Don’t get caught in the dark: Lessons from a Lumen & AWS micro-outage Escalating risk, shrinking margins: The 2025 Internet Resilience Report From refresh to results: the metrics that shaped Election Day 2024 coverage Fast and furious: The importance of performance in the digital age Getting Started with Traceroute From the source to the edge: the six agent types you can’t ignore From SEO to AEO: Why Web Performance Is the Key to AI Search Success Going for gold: Testing the resilience of Olympic websites Here’s the proof: What the fastest sites on the web have in common Google’s Agent-to-Agent (A2A) Protocol is here—Now Let’s Make it Observable How IPM helped a top tech brand catch an OpenAI outage before it became a crisis How AI Turns Monitoring From “What Now?” Into “What’s Next?” How SAP achieved world-class uptime through modern observability How to Monitor AI Agents in Commerce Systems
When BGP becomes UX: The inside story of a SaaS routing decision gone wrong (or right)
2025-10-02 · via Catchpoint Blog

in this blog post

Most operations teams trust their green dashboards. If the internal monitoring says everything is healthy, the app must be fine, right? But as the Internet keeps proving, what’s green inside the firewall can look red for customers outside of it. Sometimes, a single change in how web traffic moves can suddenly slow logins, disrupt websites, or hurt business results, even if everything looks fine inside.  

This happened recently to a global SaaS platform specializing in IT service management and workflow automation. When they changed their IP routing strategy (essentially, how they send traffic to customers), nothing broke internally. SLAs looked clean. But customers suffered slowed logins, and the kind of unpredictable, frustrating performance that’s often caused by spiky SSL handshakes behind the scenes.

The cause? A quiet BGP routing decision that flipped how traffic moved across the US. This wasn’t just about performance. It was a lesson in how IT decisions shape what real people experience, every day. Let’s unpack what happened, why it matters for every business, and what teams need to learn to avoid hidden pain points.  

When a network decision becomes a customer issue

Let’s look at what actually happened to real users. Browser-based performance tests revealed just how much the routing change affected login times and security handshakes across the country, sometimes multiplying delays tenfold.  

Here’s what the data showed:  

Browser-based tests reveal dramatic regional shifts in login speed after a BGP routing change: East Coast latency surged while Midwest and West users saw significant improvement

What was changed: /24 vs /20  

Until late August, most customers were using addresses in a specific range, the ‘/24 block’ (for the tech crowd: X.X.23.0/24) routed through a security service called Neustar. Neustar has connections across the country, which helped speed up traffic and protected users from online threats. For customers on the East Coast (like Philadelphia), this setup meant web requests reached the provider quickly, with lightning-fast response times (about 9 milliseconds) and almost no delays.

Then came the pivotal change. The SaaS provider moved customers from those addresses to a larger pool called the “/20 block.” On paper, this looked like a routine update. In practice, it was a big deal: it meant Neustar was no longer guiding the traffic. Instead, customers’ data was sent directly over the internet using providers like Lumen.

This was especially rough for East Coast users. Their requests now had to take a much longer journey, getting bounced across the country before arriving at the destination server in Phoenix. Users felt the pain immediately. Latency shot up by 10x, and basic tasks like logging in or establishing a secure connection suddenly took 5–10 times longer. For customers, everything just felt… slow.

How the changes played out in three cities

This BGP flip essentially redistributed those who suffered and who benefited. Here’s how these changes played out in three real cities, backed by the browser-based measurements you saw above:

Philadelphia (East Coast):

  • Before:
    Fast route (via Neustar), response time: ~9 ms
  • After:
    Traffic sent on a long detour through Phoenix, response time: ~61 ms
  • Impact:
    Local users waited over 50 ms longer for everything to load.

Denver (Midwest):

  • Before:
    Multiple cross-country hops, slow and shaky (~93 ms, jitter: 45 ms)
  • After:
    More direct path, smoother and much faster (~51 ms, jitter: 15 ms)
  • Impact:
    Central US users saw a big speed boost and fewer hiccups.

Phoenix (West):

  • Before:
    Odd detours, not local as expected (~69 ms even though they were close)
  • After:
    Data routed the short way, response time: ~17 ms
  • Impact:
    West Coast users finally got the speed they should have had all along.

But it’s not just the numbers that changed. We ran traceroute tests to map out the journey customer traffic actually took before and after the routing shift. These show each handoff along the way—from city to city and network to network—offering a clear, visual proof of what caused the delays (or improvements).

These traceroutes highlight how routing impacted real user performance; fast for some, slow and inconsistent for others, even before the major change

A screenshot of a screenshot of a computerAI-generated content may be incorrect.

Traceroutes showing how the /24 anchored better for the East. The /20 anchored better for the West and Midwest. Neither strategy was consistent nationwide.

Why Neustar mattered, and what changed when it was removed

AS19905 is Vercara (formerly Neustar Security Services), widely used for DDoS mitigation and traffic scrubbing. For the SaaS platform, Neustar wasn’t just speeding up traffic, it acted as a key security and routing partner in the network. By running data through Neustar, the SaaS provider gave many customers a faster, more direct journey, with extra protections along the way. For East Coast users, this meant blazing-fast connections and a nearly flawless experience.

But this story isn’t just about latency, it’s about how the provider decided to split its network. After the routing change, some customers stayed on blocks that still pass through Neustar, getting their traffic scrubbed and distributed efficiently. Others were placed on new blocks that send data directly over major internet providers like Lumen, AT&T, NTT, GTT, or Zayo.

What’s the impact? Depending on which group you belong to, your web experience is completely different. Some users continue to benefit from both high security and fast delivery; others have a longer, less protected route, with more potential for delays.

Bottom line: Neustar’s “hidden role” meant a better, safer, and faster experience for part of the customer base. Removing it—and changing how blocks are routed—redrew the map of user pain and performance, by design.

A screenshot of a computerAI-generated content may be incorrect.

BGP routing before the change

A screenshot of a computerAI-generated content may be incorrect.

BGP routing after the change

Incident timeline

  • July–August 2025: Customers primarily on X.X.23.0/24, routed via Neustar. East Coast enjoyed best performance, West Coast Users connecting to same IP were penalized.
  • Late August 2025: SaaS provider moves tenants to X.X.39.0/24 within the X.X.32.0/20 block, bypassing Neustar. East Coast latency jumps, West improves overall.
  • September 15–18, 2025: BGP monitoring shows X.X.23.0/24 withdrawn globally. Reachability drops to zero, confirming decommission of the old block.
  • October 2025 (present): Only the /20 remains in play, with its uneven routing. This is no longer an anomaly - it is the permanent architecture.

Key takeaways for every business

Here’s what this story really shows about network changes, user experience, and why every organization should care

#1 Every BGP change can affect UX: A single prefix change can add 50+ ms to every transaction. That’s not visible on an internal dashboard but shows up immediately in user experience.

#2 Performance shifts aren’t random — they're routed: There’s rarely a universal win. Optimizing one geography often comes at another’s expense. Without broad external monitoring, you’ll never know whose experience got worse.

# 3 Security layers shape performance. The presence of Neustar as an intermediate ASN created one kind of experience; its absence created another. SaaS vendors juggle performance vs security vs cost. Sadly customers have to live with the consequences. Is there no other choice?  

#4 You can’t rely on internal dashboards alone: The enterprise node in Philadelphia looked fine at layer 4. Only backbone tests and traceroutes revealed the real problem. Without external Internet visibility, ops teams would still be guessing.

How to stay ahead of BGP-driven UX surprises

From an enterprise perspective, this wasn’t about fixing an internal outage. It was about vendor accountability. With Catchpoint Internet Performance Monitoring (IPM) browser tests, traceroutes, and BGP visualizations, the ops team had undeniable evidence: the SaaS provider’s routing choice degraded East Coast performance. They could escalate with proof, not speculation.

This is why Internet-aware observability is non-negotiable:

  • To catch SaaS routing shifts before customers complain.
  • To understand when security layers (like DDoS scrubbing) vanish or change.
  • To hold vendors accountable with hard evidence.
  • To see user experience as it actually happens - outside your firewall.

The Internet is now your user experience

The Internet is now your production stack. A SaaS provider’s decision to announce a /20 instead of a /24 isn’t just a routing artifact - it’s a user experience event. East Coast logins slowed because BGP policy shifted. West Coast users cheered while Philadelphia groaned. That’s the reality of Internet-scale delivery.

Green dashboards inside the enterprise won’t tell you this. Only IPM can. Because when BGP changes, so does your customer’s experience.

Want to see how proactive BGP monitoring works in action? Request your live demo today

Summary

A global SaaS provider changed its BGP routing, which dramatically shifted user experience across the US—East Coast users suffered higher latency, while the Midwest and West saw improvements. Internal dashboards missed these regional impacts, proving only external monitoring reveals the real customer story. The lesson: Every BGP decision matters for UX, and true visibility requires looking outside the firewall.

Most operations teams trust their green dashboards. If the internal monitoring says everything is healthy, the app must be fine, right? But as the Internet keeps proving, what’s green inside the firewall can look red for customers outside of it. Sometimes, a single change in how web traffic moves can suddenly slow logins, disrupt websites, or hurt business results, even if everything looks fine inside.  

This happened recently to a global SaaS platform specializing in IT service management and workflow automation. When they changed their IP routing strategy (essentially, how they send traffic to customers), nothing broke internally. SLAs looked clean. But customers suffered slowed logins, and the kind of unpredictable, frustrating performance that’s often caused by spiky SSL handshakes behind the scenes.

The cause? A quiet BGP routing decision that flipped how traffic moved across the US. This wasn’t just about performance. It was a lesson in how IT decisions shape what real people experience, every day. Let’s unpack what happened, why it matters for every business, and what teams need to learn to avoid hidden pain points.  

When a network decision becomes a customer issue

Let’s look at what actually happened to real users. Browser-based performance tests revealed just how much the routing change affected login times and security handshakes across the country, sometimes multiplying delays tenfold.  

Here’s what the data showed:  

Browser-based tests reveal dramatic regional shifts in login speed after a BGP routing change: East Coast latency surged while Midwest and West users saw significant improvement

What was changed: /24 vs /20  

Until late August, most customers were using addresses in a specific range, the ‘/24 block’ (for the tech crowd: X.X.23.0/24) routed through a security service called Neustar. Neustar has connections across the country, which helped speed up traffic and protected users from online threats. For customers on the East Coast (like Philadelphia), this setup meant web requests reached the provider quickly, with lightning-fast response times (about 9 milliseconds) and almost no delays.

Then came the pivotal change. The SaaS provider moved customers from those addresses to a larger pool called the “/20 block.” On paper, this looked like a routine update. In practice, it was a big deal: it meant Neustar was no longer guiding the traffic. Instead, customers’ data was sent directly over the internet using providers like Lumen.

This was especially rough for East Coast users. Their requests now had to take a much longer journey, getting bounced across the country before arriving at the destination server in Phoenix. Users felt the pain immediately. Latency shot up by 10x, and basic tasks like logging in or establishing a secure connection suddenly took 5–10 times longer. For customers, everything just felt… slow.

How the changes played out in three cities

This BGP flip essentially redistributed those who suffered and who benefited. Here’s how these changes played out in three real cities, backed by the browser-based measurements you saw above:

Philadelphia (East Coast):

  • Before:
    Fast route (via Neustar), response time: ~9 ms
  • After:
    Traffic sent on a long detour through Phoenix, response time: ~61 ms
  • Impact:
    Local users waited over 50 ms longer for everything to load.

Denver (Midwest):

  • Before:
    Multiple cross-country hops, slow and shaky (~93 ms, jitter: 45 ms)
  • After:
    More direct path, smoother and much faster (~51 ms, jitter: 15 ms)
  • Impact:
    Central US users saw a big speed boost and fewer hiccups.

Phoenix (West):

  • Before:
    Odd detours, not local as expected (~69 ms even though they were close)
  • After:
    Data routed the short way, response time: ~17 ms
  • Impact:
    West Coast users finally got the speed they should have had all along.

But it’s not just the numbers that changed. We ran traceroute tests to map out the journey customer traffic actually took before and after the routing shift. These show each handoff along the way—from city to city and network to network—offering a clear, visual proof of what caused the delays (or improvements).

These traceroutes highlight how routing impacted real user performance; fast for some, slow and inconsistent for others, even before the major change

A screenshot of a screenshot of a computerAI-generated content may be incorrect.

Traceroutes showing how the /24 anchored better for the East. The /20 anchored better for the West and Midwest. Neither strategy was consistent nationwide.

Why Neustar mattered, and what changed when it was removed

AS19905 is Vercara (formerly Neustar Security Services), widely used for DDoS mitigation and traffic scrubbing. For the SaaS platform, Neustar wasn’t just speeding up traffic, it acted as a key security and routing partner in the network. By running data through Neustar, the SaaS provider gave many customers a faster, more direct journey, with extra protections along the way. For East Coast users, this meant blazing-fast connections and a nearly flawless experience.

But this story isn’t just about latency, it’s about how the provider decided to split its network. After the routing change, some customers stayed on blocks that still pass through Neustar, getting their traffic scrubbed and distributed efficiently. Others were placed on new blocks that send data directly over major internet providers like Lumen, AT&T, NTT, GTT, or Zayo.

What’s the impact? Depending on which group you belong to, your web experience is completely different. Some users continue to benefit from both high security and fast delivery; others have a longer, less protected route, with more potential for delays.

Bottom line: Neustar’s “hidden role” meant a better, safer, and faster experience for part of the customer base. Removing it—and changing how blocks are routed—redrew the map of user pain and performance, by design.

A screenshot of a computerAI-generated content may be incorrect.

BGP routing before the change

A screenshot of a computerAI-generated content may be incorrect.

BGP routing after the change

Incident timeline

  • July–August 2025: Customers primarily on X.X.23.0/24, routed via Neustar. East Coast enjoyed best performance, West Coast Users connecting to same IP were penalized.
  • Late August 2025: SaaS provider moves tenants to X.X.39.0/24 within the X.X.32.0/20 block, bypassing Neustar. East Coast latency jumps, West improves overall.
  • September 15–18, 2025: BGP monitoring shows X.X.23.0/24 withdrawn globally. Reachability drops to zero, confirming decommission of the old block.
  • October 2025 (present): Only the /20 remains in play, with its uneven routing. This is no longer an anomaly - it is the permanent architecture.

Key takeaways for every business

Here’s what this story really shows about network changes, user experience, and why every organization should care

#1 Every BGP change can affect UX: A single prefix change can add 50+ ms to every transaction. That’s not visible on an internal dashboard but shows up immediately in user experience.

#2 Performance shifts aren’t random — they're routed: There’s rarely a universal win. Optimizing one geography often comes at another’s expense. Without broad external monitoring, you’ll never know whose experience got worse.

# 3 Security layers shape performance. The presence of Neustar as an intermediate ASN created one kind of experience; its absence created another. SaaS vendors juggle performance vs security vs cost. Sadly customers have to live with the consequences. Is there no other choice?  

#4 You can’t rely on internal dashboards alone: The enterprise node in Philadelphia looked fine at layer 4. Only backbone tests and traceroutes revealed the real problem. Without external Internet visibility, ops teams would still be guessing.

How to stay ahead of BGP-driven UX surprises

From an enterprise perspective, this wasn’t about fixing an internal outage. It was about vendor accountability. With Catchpoint Internet Performance Monitoring (IPM) browser tests, traceroutes, and BGP visualizations, the ops team had undeniable evidence: the SaaS provider’s routing choice degraded East Coast performance. They could escalate with proof, not speculation.

This is why Internet-aware observability is non-negotiable:

  • To catch SaaS routing shifts before customers complain.
  • To understand when security layers (like DDoS scrubbing) vanish or change.
  • To hold vendors accountable with hard evidence.
  • To see user experience as it actually happens - outside your firewall.

The Internet is now your user experience

The Internet is now your production stack. A SaaS provider’s decision to announce a /20 instead of a /24 isn’t just a routing artifact - it’s a user experience event. East Coast logins slowed because BGP policy shifted. West Coast users cheered while Philadelphia groaned. That’s the reality of Internet-scale delivery.

Green dashboards inside the enterprise won’t tell you this. Only IPM can. Because when BGP changes, so does your customer’s experience.

Want to see how proactive BGP monitoring works in action? Request your live demo today

This is some text inside of a div block.