Swish: No substitute for platform foundations

Platform engineers from Getswish AB, makers of the Swish Swedish payment processing app, told a similar story of incident management improvement in another livestreamed session. These engineers spearheaded the creation of a new internal incident management platform for the company after a 2021 outage was reported in the press before the company's IT management outsourcing provider was aware of it.

By late 2024, another high-profile incident made national news but was detected and resolved by an SRE team within five hours. Getswish is now working on extending observability and incident management practices to its financial transaction partners.

Between those two points came a comprehensive effort to rearchitect the company's code development process using GitOps and incident management practices using Google's seminal manual on SRE, said Jonas Cronholm-Lundin, head of platform at Getswish AB, during the June 10 DASH session.

"The SRE book that Google published [10] years ago, has a prioritization triangle," Cronholm-Lundin said, referring to the book's Service Reliability Hierarchy, represented by a pyramid, with monitoring at the base "If you have problems with reliability in your system, where do you start? … Adding on release procedures or performance testing and whatnot, will not do any good if you don't build your base in the triangle first. You need to actually understand how production runs and how it works. That is where everything like this needs to start."

The speakers also emphasized the importance of laying platform groundwork before AI agents can be effective. The company's revamped incident management process also now includes runbooks structured according to the OODA loop approach to decision making, Cronholm-Lundin said.

"When agents are coming in, like Bits [AI] SRE, and so on, if you have gathered your post-mortem reports [and] curated good runbooks, that is excellent for your agent to consume in the future when you have future incidents, and also for onboarding purposes," he said. "So it's super important to do the groundwork of actually bootstrapping your internal documentation in this area."

Data classification agents raise fears, opportunities

Datadog made more than 100 updates to its observability platform during DASH, several of them offering an AI-assisted approach to data gathering. These include auto-tagging features in its new Runtime Prioritization Engine for security vulnerabilities and a new Auto-Processing feature for log management with its Observability Pipelines.

Reached by email following the presentation, Cronholm-Lundin told TechTarget that his team is following Datadog's agentic updates with interest and plans to evaluate some, but it's too early to say which will be relevant to the company.

Another DASH attendee who saw a demo of Auto-Processing said he was concerned about a potential loss of skills among engineers as AI-driven automation takes hold.

"My takeaway was that developers no longer need to have discipline in what they generate for log output or how they format it; AI will take care of all the details so humans can focus on iterating faster and shipping more code," said Eric Swanson, senior site reliability engineer at MagicSchool AI, makers of a generative AI platform for education in Denver, Colo., in an online interview during DASH.

"My concern with these types of solutions is that we are choosing to hand our agency over to Agents, and blunting the sharpness of our skills honed through critical thinking," Swanson said, emphasizing that he was speaking on behalf of himself, and not his employer. "The skills themselves may become obsolete, but we run the risk of dulling our thinking next."

 However, one breakout session presenter who has worked extensively with Observability Pipelines noted that AI can offer new opportunities for engineers to improve incident management efficiency.

Not every log generated by applications at US Bank gets a "first-class seat" in Datadog's back end, but Observability Pipelines enables the platform to route the data to other repositories to manage retention costs, such as Amazon's S3, said Bryan Pierson, vice president of enterprise observability engineering at US Bank, during his DASH presentation.

But Pierson's team also came up with a way to route logs to Datadog in the event developers need its AI features, he said.

"Right before DASH, we ended up putting out a proof-of-concept application called Kickflip that puts the power back in the application team's hands to enable their debug logs if they do need them on the platform for use with things like Bits AI," Pierson said.

Beth Pariseau, senior news writer for Informa TechTarget, is an award-winning veteran of IT journalism. Have a tip? Email her or connect on LinkedIn.