Channel Title: USENIX Channel Description: As part of USENIX's commitment to the Open Access to research movement, slides, proceedings, and audio and video recordings from many of our events are now freely available to the general public shortly after they take place. Selected videos are added to this channel; additional recordings can be found on the USENIX Web site. <video> SREcon24 Europe/Middle East/Africa - SRE Stakeholders: A Spotter’s Guide Description: SRE Stakeholders: A Spotter’s Guide Dave O'Connor For Every SRE or SRE-adjacent team in any organisation, there are many kinds of stakeholders; people who care (or don't care!) about how your team operates, and the outcomes of that. They differ massively in how they view your team, and in how they, in turn, should be viewed, and managed. In a timeline that doesn't contain a canonical book setting out what SRE is here for and how it achieves that, the sad and annoying answer is that "it depends". Because of this, we need to get good (or remain good) at stakeholder management and communications about why we're here, and what we do. While primarily useful to SRE leadership, the kinds of stakeholders you run into can be useful to know for any SRE. Learn to spot the different stakeholders in your life, what they (generally) care about, and how you can help reduce misunderstandings and tension, no matter where you're sitting. View the full SREcon24 Europe/Middle East/Africa program at https://www.usenix.org/conference/srecon24emea/program </video> <video> SREcon24 Europe/Middle East/Africa - Rock around the Clock (Synchronization): Improve Performance... Description: Rock around the Clock (Synchronization): Improve Performance with High Precision Time! Lerna Ekmekcioglu, Clockwork Systems Is the app slow or the network lagging? When it comes to latency in distributed systems, it can be hard to identify where exactly the issue is. As businesses increasingly adopt diverse deployment environments —on-premises, cloud, or hybrid— the complexity grows, obscuring visibility into system health. Join me to hear why clock synchronization is key for identifying the true culprit when latency is due to contention in the network. I’ll demo how network contention impacts tail latencies followed by an overview of clock synchronization protocols to date, their pros and cons, and best practices in disciplining clocks, as well as recent algorithms from Stanford Research. With high precision clock synchronization at scale, we gain back visibility into useful one way delay metrics, which act as an early signal for network congestion that help us prevent impact to response times for our end users! View the full SREcon24 Europe/Middle East/Africa program at https://www.usenix.org/conference/srecon24emea/program </video> <video> SREcon24 Europe/Middle East/Africa - OMG WTF SSO: A Beginner’s Guide to Single Sign-On... Description: OMG WTF SSO: A Beginner’s Guide to Single Sign-On (Mis)configuration Adina Bogert-O'Brien SSO protocols are just ways for an identity provider to share information about an authenticated identity with another service. Me having a way to tell my vendor “yeah, that’s Bob” doesn’t tell me what the vendor does with this information, or if the vendor always asks me who’s coming in the door. A bad SSO implementation can make you think you’re safer, while hiding all the new and fun things that have gone wrong. To get the most out of implementing SSO, I need to know what I’m trying to accomplish and what steps I need to follow to get there. To illustrate why SSO needs to be set up carefully, for each of the things you need to do right, I’ll give you some fun examples of creative ways you and your vendor can do this wrong. We all learn from failure, right??? View the full SREcon24 Europe/Middle East/Africa program at https://www.usenix.org/conference/srecon24emea/program </video> <video> SREcon24 Europe/Middle East/Africa - Enhancing Elasticsearch Performance: Innovative Reindexing... Description: Enhancing Elasticsearch Performance: Innovative Reindexing Strategies Using Dedicated Nodes and KEDA Autoscalers Leila Vayghan, Shopify This talk is about enhancing the search infrastructure of Shopify, a large-scale ecommerce platform that supports over 3 million merchants and handles more than two petabytes of data. This talk explains how we leverage Kubernetes on Google Cloud Platform to ensure high availability and performance, crucial for maintaining our platform's robust search functionality. It will also elaborate on our innovative approach using dedicated reindexing nodes within existing clusters, which significantly improves indexing and reindex performance while cutting infrastructure costs. We will explore the application of Kubernetes Event-Driven Autoscaling (KEDA) to dynamically manage resource allocation, enhancing operational efficiency and reducing on-call fatigue. This strategy not only supports seamless user experiences but also boosts Gross Merchandise Value (GMV) and revenue through improved system responsiveness. This presentation is ideal for those involved in managing large-scale data systems or interested in advanced Elasticsearch optimizations. View the full SREcon24 Europe/Middle East/Africa program at https://www.usenix.org/conference/srecon24emea/program </video> <video> SREcon24 Europe/Middle East/Africa - Selective Reliability Engineering: There Is No Single Source... Description: Selective Reliability Engineering: There Is No Single Source of Truth Elise Burke, Datadog As engineers we design distributed architectures, define project scopes, and ensure that we have a single "source of truth". But what, exactly, do we mean by the phrase? Do we really have only one source of truth - and for that matter, how do we decide what it is? We'll look at some well-known ambiguities in system design and data modeling and then consider more philosophical questions about truth, the sources of truth we accept, and why this ambiguity matters. View the full SREcon24 Europe/Middle East/Africa program at https://www.usenix.org/conference/srecon24emea/program </video> <video> SREcon24 Europe/Middle East/Africa - Panel Discussion: Is Reliability a Luxury Good? Description: Panel Discussion: Is Reliability a Luxury Good? Moderator: Emil Stolarsky, Increase Panelists: Andrew Ellam; Niall Murphy, Stanza; Joan O'Callaghan, Udemy; Avleen Vig View the full SREcon24 Europe/Middle East/Africa program at https://www.usenix.org/conference/srecon24emea/program </video> <video> SREcon24 Europe/Middle East/Africa - Sailing the Database Seas: Applying SRE Principles at Scale Description: Sailing the Database Seas: Applying SRE Principles at Scale Ioannis Androulidakis and Martin Alderete, Booking.com In this talk we will demonstrate how we apply core SRE principles in the field of Database Engineering. More specifically, we will talk about the challenges of operating large-scale database systems in multiple cloud environments and how adopting best SRE practices dramatically improved our daily workflows and operations. We will share insights and concrete use cases around the following topics: Monitoring Distributed Systems, Eliminating Toil and Postmortem Culture. This talk will equip attendees with ideas and guidelines to better understand and efficiently operate their database systems such as choosing the right SLIs and SLOs, automating capacity planning and embracing a postmortem culture after outages. View the full SREcon24 Europe/Middle East/Africa program at https://www.usenix.org/conference/srecon24emea/program </video> <video> SREcon24 Europe/Middle East/Africa - I Can OIDC You Clearly Now: How We Made Static Credentials a... Description: I Can OIDC You Clearly Now: How We Made Static Credentials a Thing of the Past Iain Lane and Dimitris Sotirakis, Grafana Labs At Grafana Labs, we tackled a thorny problem: managing secrets in an open-source CI/CD pipeline. Our journey from static secrets to OIDC-based access wasn't just about better security—it was about empowering our engineers. We'll walk you through how we leveraged OIDC and GitHub Actions to create a "secretless" system for accessing cloud resources, complete with shared jobs and abstractions that make secure access simple. But it wasn't all smooth sailing. We'll share war stories, including a security hiccup that taught us valuable lessons. If you're drowning in a sea of secrets or just want to sleep better at night, come and learn how we boosted security while cutting operational headaches. You'll walk away with practical strategies for implementing OIDC-based access that'll make your engineers happy and your security team even happier. View the full SREcon24 Europe/Middle East/Africa program at https://www.usenix.org/conference/srecon24emea/program </video> <video> SREcon24 Europe/Middle East/Africa - Achieving Excellence: SLO Thresholds That Transform Service... Description: Achieving Excellence: SLO Thresholds That Transform Service Quality Thiara Ortiz, Netflix At Netflix, ensuring exceptional quality for our streaming platform is crucial. Every time a Netflix member sits down, reclines in their chair, and turns on their TV, it's a moment of truth. It's our opportunity to deliver a spectacular service with amazing quality of experience. Misses, errors, or high latency—whether due to ISP configuration changes, code deployment, or catastrophic fallback—impact how our service is perceived. In this talk, I'll share methods for defining thresholds for SLOs, ranging from intuition and industry best practices to advanced techniques like A/B experimentation. At Netflix, properly defining SLOs allows us to ensure industry-leading quality of experience for our members. View the full SREcon24 Europe/Middle East/Africa program at https://www.usenix.org/conference/srecon24emea/program </video> <video> SREcon24 Europe/Middle East/Africa - SRE Saga: The Song of Heroes and Villains Description: SRE Saga: The Song of Heroes and Villains Daria Barteneva, Microsoft Azure SRE team require a balance of technical and soft skills, creativity and teamwork to be successful. Drawing parallels between the roles, challenges and dynamics of Dungeons and Dragons party and an SRE team will help us to explore SRE journey from the team inception to developing ideal makeup in terms of tenure/seniority, skillset and align it with the context SRE team could be part of. We will share practical examples that helps SRE teams building resiliency and effective collaboration while dealing with challenges. We will also explore different mechanisms that can channel "super hero" energy to make team stronger and nurture the talent, helping team to keep the balance of distributed knowledge and accountability. In this talk we will discuss: • Examples of functional SRE team setups • Common challenges SRE team may encounter • Developing early in career SRE • Dealing with the change and building resilience • Identifying red flags and avoiding long term problems View the full SREcon24 Europe/Middle East/Africa program at https://www.usenix.org/conference/srecon24emea/program </video> <video> SREcon24 Europe/Middle East/Africa - Survivor: MySQL Island – Outwit, Outplay, Outlast Metadata... Description: Survivor: MySQL Island – Outwit, Outplay, Outlast Metadata Locking Challenges Julia Jablonska, Capsule CRM Think you understand MySQL metadata locks? Join this interactive session to test your knowledge and take a deep dive into the intricacies of MySQL's locking mechanisms. We'll explore real-world scenarios, such as creating tables with foreign key constraints and adding indexes, to see how metadata locks can impact performance and stability. Through live voting you'll gain insights into what's happening behind the scenes and learn practical tips for managing database migrations. View the full SREcon24 Europe/Middle East/Africa program at https://www.usenix.org/conference/srecon24emea/program </video> <video> SREcon24 Europe/Middle East/Africa - Mnemonic Rules for Eponymous Laws or: There’s a Law for That! Description: Mnemonic Rules for Eponymous Laws or: There’s a Law for That! Peter Burkholder, U.S. Government As SREs, referencing named laws like Brook’s Law, Galls Law, or Jevons Paradox can help strengthen our arguments. But remembering which law applies when is challenging. In this talk, I'll highlight the most useful tech and behavioral science laws for SRE work, offer mnemonic tips for recalling them, and share real-world examples. We'll finish with a quick quiz to ensure you're ready to apply these concepts in your role. View the full SREcon24 Europe/Middle East/Africa program at https://www.usenix.org/conference/srecon24emea/program </video> <video> SREcon24 Europe/Middle East/Africa - The Frontiers of Reliability Engineering Description: The Frontiers of Reliability Engineering Heinrich Hartmann, Zalando SE We take the 10s anniversary of SRECon as an occasion to reflect over the past decade of advancements in Reliability Engineering and provide an overview about the Frontiers we are facing today. Within Zalando we followed major trends of the industry in outsourcing hardware provisioning to AWS, package applications into Docker images, fully automated deployments (CI/CD), and implemented Distributed Tracing for Microservice Observability. Despite these advances, many challenges remain in building reliable, observable software systems and new areas arose which require new methods and tools. In the talk we are proving a number of conceptual view that help to map out the larger Reliability Engineering landscape and zone-in on 3 specific frontiers that we are actively investing in at Zalando: (1) Data Operations and Monitoring Event Based Systems (2) Mobile Observability (3) Effective Management Practices for Reliability. View the full SREcon24 Europe/Middle East/Africa program at https://www.usenix.org/conference/srecon24emea/program </video> <video> SREcon24 Europe/Middle East/Africa - You Depend on Time, This Is How It Works and You Won’t... Description: You Depend on Time, This Is How It Works and You Won’t Believe It Philip Rowlands, Jane Street This is a talk about calendars, clocks, and computers. We’ll look at the metrology of the second, from candles to atoms, and consider how your phone always seems to know the right time. If you’ve ever wondered why is today Thursday? or how was the Gregorian calendar adopted? then come and learn the mistakes to avoid the next time you are the Pope. If you’ve ever wondered why do these two clocks disagree? then come and learn about the challenges of finding the elusive perfect tick, and why it’s not at the top of Mount Everest. And if you’ve ever wondered how calendars and clocks work together in modern computer systems, then come and learn about protocols and APIs for keeping clocks reliable and accurate. View the full SREcon24 Europe/Middle East/Africa program at https://www.usenix.org/conference/srecon24emea/program </video> <video> SREcon24 Europe/Middle East/Africa - Dude, You Forgot the Feedback: How Your Open Loop Control... Description: Dude, You Forgot the Feedback: How Your Open Loop Control Planes Are Causing Outages Laura de Vesine, Datadog, Inc. It's a strong principle of good UX design that users should get feedback about the results of their actions, to help prevent errors. Experienced SREs know to build in additional observability to systems to watch our systems change as we mutate them, but these are typically out-of-band and require a conscious, deliberate action to observe -- so getting good feedback into our actions requires constant vigilance and training of new users. What if we instead built control planes that tell us exactly what we've done, and what effect that is having? This talk explores various patterns of "fire and forget" control planes in production systems, how each one contributes to outages, and some simple solutions to build better tools for operations. View the full SREcon24 Europe/Middle East/Africa program at https://www.usenix.org/conference/srecon24emea/program </video>

Contacts and Links

Description

As part of USENIX's commitment to the Open Access to research movement, slides, proceedings, and audio and video recordings from many of our events are now freely available to the general public shortly after they take place. Selected videos are added to this channel; additional recordings can be found on the USENIX Web site.

Videos from channel