A short recap:

It turns out that metrics are not as important as we had originally thought 😥

Our holistic developer success platform was slowly taking shape because of direct feedback from participants during our research calls.

In the direction we were heading, how are you would aim to facilitate conversations, teach managerial best practices, and help developers succeed at their companies, in their teams, and on their project tasks.

We started bouncing these ideas off a couple of new participants in a second round of calls. This time we selected engineering managers because they are the “first responders” to problems related to their software engineers’ mental health, wellbeing, and overall happiness. It was great to learn that the idea really resonated with them. The area where all these value items could be addressed was during 1:1 meetings.

Participants started mentioning that 1:1 meetings were a real pain point. The problem was not only how to run 1:1 meetings that would be effective and interesting for their developers, but the sheer amount of time it took to prepare for and conduct them. To our shock and surprise, we learned that engineering managers only had ~5 minutes to prepare for each 1:1 meeting.

When we started discussing ways to run a good 1:1 meeting, more and more started to mention employee wellbeing and company culture were important.

Out of everyone we spoke to, 62% mentioned employee wellbeing as the single most important priority for them.

So, we set our minds on redefining how are you in a way that would help tech leads and engineering managers have meaningful and effective 1:1 meetings.

These meetings would be built around 4 core areas. They are:

• Wellbeing, to see how happy developers are working on projects, in teams, and at the company;
• Team Feedback, gathered from teammates, and investigates potential frictions and encourages synergies;
• Career Growth, where managers help engineers explore ways to upskill, set goals, and forge career paths;
• Work Output, which is a context-based conversation starter about commits, pull requests, code review comments, and documentation updates.

We really started to believe that we were on the right track, but as the solution was evolving, we knew we needed to grow our team.

Krzysztof joined us as a Product Owner, and Ada would continue with research. Both of them would work together to finish defining the final scope of the product. Mariusz and I would take care of the storytelling.

We still had a huge task ahead of us, which was to finalize the direction of the product and find out which metrics would actually matter in a developer-focused tool. This was no small feat as by this point we knew that our potential users weren’t very enthusiastic about individual-level metrics. They deemed them too reductive, even downright dangerous at times. A second important task was creating the right context for incoming 1:1 meetings. Because of this, we wanted to know everything about how engineering managers prepared for and conducted their 1:1s.

Another important thing we had to consider: leaderboard. We decided not to have a ‘leaderboard’ view that would allow developers to compare their performance to other developers. Everybody was a rock star coder in their own way. We actually preferred it if each developer compared themselves in the current sprint to how they did in past sprints. We didn’t want how are you to become a tool of oppression or a trigger for in-fighting, but a solution for developer growth and a way to elevate conversations using data.

More Research Calls

We did our due diligence and looked into improving our research calls. We followed Discovery best practices from people like Teresa Torres, Hiten Shah, and this video on Customer Discovery from Techstars. We made sure to ask our participants about the size of teams and the companies they worked for; the exact number of direct reports, the duration of their 1:1 meetings, and what they comprised.

We’d share a clickable prototype and ask the subjects to pretend they were preparing for a 1:1 meeting with a direct report. Ada, our product researcher, would take meticulous notes, and sometimes follow them in Figma to see where they clicked, what needed explaining, and what was unclear.

After each call, which would usually last 45 minutes to an hour, we’d take a couple of minutes to sum up what we’ve learned to see if we got anything interesting from the calls.

Into Miro We Go

As before, we were still updating our Miro board. Early on we had several hunches about which direction we should take, but that wasn’t enough. Now we needed to quantify our observations and ensure they aligned with our Jobs, Pains, and Gains. Here’s a snapshot of how we tallied how are you's values and benefits.

Based on these findings, we adjusted our value proposition and iterated the scope of the product.

Some more important learnings:

• Engineering managers spend a ton of time on 1on1s and they feel they're too time-consuming
• 1on1s lack structure and because of that, it's really hard to implement effective processes around them
• Engineering managers on average have only about 5 minutes to prepare for their next 1on1
• All the context needed for 1on1s are dispersed between a bunch of different tools (git, JIRA, Slack, email, calendar, bug trackers, etc). It's a nightmare to come up with any clear conclusion about what's going on at the dev's end.
  

Here’s what the Output section looked like in the first version of the app:

The feedback we got was pretty clear - people didn’t care nearly as much about Output, as they cared about Wellbeing and Feedback. In fact, the hierarchy went something like this: Wellbeing, Feedback, Growth, and then Output. While the rest of the screens were less controversial, the users were pretty harsh on Output, so we knew we had to still come up with a better alternative.

And so, we iterated again. And again, until the feedback we got from our users changed. An Overview dashboard was added because we learned that engineering managers only have 5 minutes to prepare for every 1on1. Tabs were rearranged based on the hierarchy participants indicated. We spent many hours brainstorming and iterating each screen to make sure we addressed every possible issue. Our subjects needed to see total value in our product, and we wouldn’t stop until we got it right!

And as it often happens in Discovery, the time we were taking to define all the features started to frustrate other team members. It was time to set a stricter timeline, so we defined the last version of mockups and went on to estimate how much time and money it would take to develop how are you. This process resulted in an MVP of the MVP, but I’m getting ahead of myself - that’s what I’m going to take you through in the next product update. See ya soon!