August 16, 2022
Evelina Vrabie joins us to share her insights into measuring the success of engineering teams. She tells us about the role of an engineering manager as well as the four types of success. Then, she walks us through how to measure productivity and high performance through research-based frameworks.
Evelina Vrabie is an Engineering Manager at Hopin. She is an engineering leader with over 15 years of experience, with a strong entrepreneurial, management and technical background. In her, Master's degree she explored the use of artificial intelligence in work therapy for high performance environments. Her work culminated in co-founding Touco Labs, a FinTech with the purpose of helping personal finance. Evelina's blog is called Jumpstart and she often publishes for CTO Craft as well. Follow these resources to connect with her.
Evelina begins by sharing what an engineering manager or leader means to her. Her view is that technical leaders bring their unique perspectives on technology to help the strategic execution of their companies. In practice, that means:
In order to achieve this, leaders have to develop the ability to understand and shape a particular function in a company. In a small startup, leadership roles often span across many areas from product, to technology to finance. In larger companies, leadership roles often become more specialised, but still require an understanding on other functionalities. This can be very difficult to do, as leaders often need to take into account problems that aren't even in the technology domain.
Evelina believes that the success of an individual or a team needs to start with where they are, their context and their organisation. Understanding the company goals and strategies, as well as the fact that they will be different at different times. There are always multiple successful paths or strategies, each coming at a different cost. Just like in a complex, distributed system, which is never fully healthy, nor is an organisation fully successful. Success is variable, especially in technology startups which are often compared to rollercoasters.
There are at least four types of success that Evelina has dealt with in her experience as an engineering leader:
All of these types of successes and outcomes require engineering leaders to employ different types of measurement metrics.
As we begin to look at measuring software delivery success, Evelina recommends that leaders start by identifying first the problems they need to solve. Then, they can proceed to identifying the outcomes they want to measure. Once these outcomes are identified, they can be placed into a framework like OKRs:
As technology leaders, we need to understand things that are outside of technology. One example Evelina gives is understanding the basics of a marketing funnel, when trying to improve the number of users for a service.
Teams should be heavily involved when setting the success strategy, especially in the metric setting part of the process. The engineers on your team should understand their success metrics very well. Leaders need to understand that their teams never exist in a vacuum and are always affected by factors outside of only technology.
High performing organisations invest in software delivery success and organisational success. When it comes to organisational success, the folks behind the book "Accelerate: The Science of Lean Software and DevOps" have been pushing new research on scaling high performing technology organisations. The whole idea behind this research is to not even start with measuring as a goal, instead we need to ensure that the company has in place at least a dozen key capabilities that are proven to increase both delivery and organisational performance. Evelina lists some common examples of these capabilities:
Once these capabilities are in place, we can begin to choose the metrics that we care about. We need to be careful about how we use these metrics and break them down accordingly as well. No matter how many metrics we know and we employ, Evelina thinks leaders should understand that blockages are not caused by technology and code, they are caused by people.
Evelina recommends the article "When Incentives Fail. A Story about Rats, Cobras, Nails, and Atrocities". This article provides many examples from human history of Goodhart's law, which states:
"When a measure becomes a target, it ceases to be a good measure"
Evelina summarises at least 3 types of anti-patterns:
Less than half of startups achieve both organisational and delivery success. Evelina points us to The SPACE of Developer Productivity framework, which has the headline:
There's more to it than you think.
Most technology leaders, which have limited exposure to research, often only emphasise activity metrics, completely disregarding people metrics. The lack of organisational performance leads to loss of productivity, burnout and attrition. Both of these need to be considered if we want to see an impact on performance.
How do you phrase high performance? Evelina thinks of it as:
High performance = Workplace satisfaction * Intrinsic motivation
As technology leaders, we can focus on providing the two factors to this equation:
Evelina thinks we lack good metrics for measuring the motivation and satisfaction in the technology industry. We often think of happiness as a soft metrics, when in fact happiness at work is directly translatable to revenue because software is written by humans - happy people are motivated to deliver good products that others love!
The SPACE framework recommends measuring invisible work as well. Aside from activity metrics, we need to look at communication and collaboration, as well as satisfaction and well-being. We need to learn to communicate timely, accurately and succinctly. This is not an art, it's a science. Very few technology companies invest in communication training, which is especially important for fully remote companies.
Evelina thinks referring to technical debt by this term diminishes our understanding of it. Technical debt is in fact scale work and risk work. Prioritising technical debt is a Goldilocks problem: doing it too early limits our opportunity to do work and gain the market, while doing it too late will degrade performance, again leading to a loss of market share. We need to find the just right moment. It has at least three steps:
Evelina has written an interesting 2-part series on "Data-driven negotiation with SLIs, SLOs and Error Budgets", which you can read on her blog.
Evelina ends with five highlights for engineering leaders:
blogs · 7 min
Daniel Teixeira, Lead of Offensive Security at Form3 discusses exploiting Distroless images, covering the topics of:
September 22, 2022
blogs · 5 min
Natan Yellin joins us to his insights on the challenges of running software at scale, which now involves maintaining more complex system architecture than ever. Then, he walks us through the open-source tool Robusta Dev and how it can make running systems on Kubernetes easier!
September 15, 2022
blogs · 6 min
I have a rough understanding of PKI certificates, how they work, and what TLS is in general. However, I've always struggled to understand the details, particularly from the point of view of an operator. How do I check if a certificate is valid? How do I check who issued it? What does it even mean to "issue" a certificate? To make matters worse, I'm frequently confounded by the variety of different file types used for certificates. Is it a pem, or a crt, or a pub? Speaking of pub, what's the difference between the TLS certificate my server uses to encrypt traffic, and the certificates I use for SSH authentication? In this post, I will answer these questions and then walk though a practical example of using certificates for TLS via a local nginx proxy, modeling the client/server TLS you often see on the web.
August 5, 2022