Andrew Krug

Toward the end of each year, tens of thousands make the journey to Las Vegas to participate in AWS re:Invent. AWS re:Invent has been the seminal conference for cloud-focused engineers since 2012, offering a space where the global cloud computing community can share and learn the latest insights and solutions.

After going fully virtual last November, AWS implemented a hybrid model for this year’s conference with both in-person and virtual events. The focus on health was front and center for re:Invent 2021, with talks and workshops on health and technology joined by an added Health and Life Sciences Lounge for industry attendees to dive deeper into health-related topics. There was also a focus on diversity, equity, and inclusion, and the importance of mitigating biases when implementing machine learning techniques in observability and predictive models for large organizations. These two themes were blended with the topics of security, accessibility, and refinement in cloud technologies.

Datadog had three booths this year, with our main booth giving demos and our smaller booths focused on deep dives into Datadog’s latest innovations and machine learning capabilities. We also had a few speaker series on the main stage, which featured talks from Datadog and our partners. Nathan Case, the Senior Director of Security Operations at Resilience, and I discussed businesses scaling successfully through enhanced observability and security techniques, demoing the significance of automating instrumentation and adding context to alerts. Datadog’s Kirk Kaiser and Senior Principal Engineer at Paypal Jeff Nickoloff also talked about bringing intentional and empathetic observability to dashboards, showcasing the importance of using dashboards to solve the right problems relative to your company and product.

There were many other amazing speakers, companies, and technologies at this year’s conference. In this post, we’ll take a look at some of the show highlights and review what it was like to be back in person at re:Invent 2021.

Refining the cloud

Many vendors showcased refinements and updates to existing products and platforms this year instead of introducing new ones, building on the success of cloud technology over the last decade. AWS CEO Adam Selipsky noted in his keynote how cloud computing is now the default option for many companies, calling it “an enabler of a fundamental shift in the way that business functions.” The emphasis on new features and updates at re:Invent 2021 was a big indicator for how successful the cloud industry has become.

One of Selipsky’s big-splash announcements was AWS Lake Formation getting cell-level security. If you’re not familiar with Lake Formation, you can think of it like a big spreadsheet that manages your different data lakes. Instead of everyone seeing all the individual rows and columns, you can now use the power of AWS IAM and KMS to grant access or restrict on a cell-by-cell basis—making the hardest problem simply classifying data. For more information, check out their page detailing AWS Lake Formation.

Accessibility and ease

It’s no shock that everyone out there is having a challenging time hiring builders. Every kind of engineering resource is in short supply, and this was amplified by AWS and customer breakouts throughout re:Invent 2021. From the AWS Community Builders program to the extensive list of certification paths for DevOps, machine learning, and many more, it’s clear that AWS is committed to growing the pool of cloud builders. This year, we also saw a significant product pivot to creating low-code and no-code solutions to empower users who are less inclined to software engineering.

Breaking down silos with SageMaker Canvas

New low-code solutions surfaced on the first day with the announcement of Amazon SageMaker Canvas, a visual, no-code machine learning capability for business analysts. SageMaker Canvas is unique because it’s designed to break down silos across the machine learning pipeline by simply keeping data in a single platform and allowing stakeholders in the business to see and understand all the major stages of the machine learning modeling process. This helps save time and reduce errors, and results in fewer copies of the data streaming through the network. You can learn more about AWS SageMaker Canvas here.

SageMaker Studio Lab and accessible machine learning

If SageMaker Canvas wasn’t enough to entice companies on machine learning, AWS released another project called SageMaker Studio Lab. SageMaker Studio Lab seeks to make machine learning accessible to everyone and reduce the barriers to getting familiar with machine learning concepts, all within a no-setup, free development environment. The possible applications for corporate training, academia, and professional development are certainly exciting. To experience what’s possible with SageMaker Studio Lab, check out the AWS Disaster Response Hackathon, created to foster machine learning solutions for pressing challenges related to natural disaster preparedness.

Security in everything

It was just a few re:Invent conferences ago we heard AWS VP and CTO Werner Vogels proclaim, “Security is everyone’s job.” If you’ve followed the number of security talks across re:Invent since then, it might appear that security-focused content has diminished. However, security content was in nearly every presentation at re:Invent 2021.

Access Analyzer

Security was top-of-mind in nearly every product, demo, and pipeline at this year’s re:Invent. There were some amazing sessions on the pillars of AWS security services, including one on IAM policies and another on KMS and confidentiality. Both of these deep dives had features on Access Analyzer, a free tool users can enable to analyze policies in AWS Organizations. Similar to AWS CloudTrail, it’s one of those no-brainer, high-value tools everyone should use. Brigid Johnson walked users through the IAM primer and how Access Analyzer fits in by using logic-based reasoning to determine safe access to resources. If you’re familiar with Brigid’s prior work, her demos included plenty of unicorns and horses.

Colm MacCarthaigh also described how engineering teams and architecture processes can help you treat customer data as radioactive. Using cryptographic controls to protect both the customer and the business at the same time isn’t new, but much of this technology is more accessible and easier to implement than ever before. Datadog has a great integration with IAM Access Analyzer that you can find out more about here.

Observability front and center

As environments grow increasingly more complex, it’s clear observability isn’t just a theme but a core principle in design and operations. Products like Datadog, in combination with excellent open source standards like OpenTelemetry, are paving the road to build observability into the very DNA of the cloud services we create. Be sure to check out other Datadog talks at re:Invent 2021, including Matthew Williams and Senior Partner Solutions Architect at AWS Shivansh Singh on telemetry and business metrics and Rachel White on monitoring Amazon Elastic MapReduce workloads at scale.

The theme of increasingly more mature products is here to stay, and regardless of the form re:Invent 2022 takes, we’re sure to see observability continue to evolve and grow. It’s our prediction that this won’t just be valuable for operations; observability is likely to become the driving force in detecting misconfigurations, attacks, and vulnerabilities in all your environments. If you’re already using Datadog, you can start monitoring the security of your infrastructure here. Otherwise, sign up for a 14-day free trial.