Reforge Reflection

After taking the Reforge program, I was confident and happy to walk away with a plan for the next stage of dev.lab. dev.lab is the internal developer platform product built to make the lives of software engineers easier and simpler in writing their code and applications. Based on the Reforge material, I created a plan, a feature assessment, and scoring. I’ll tell you what I planned for dev.lab’s expansion and what ended up happening.

What I Thought Was Going to Happen

My plan was after writing the Reforge action plan report, I would share it with everyone. In increasing circles of influence. Making sure everyone was in alignment on the upcoming features and what to work on in what order. Then send it off to executives to review. If they had questions, they could let me know so that I could meet with them one-on-one.

What happened

Early buzz about the platform excited our quantitative strategy department. They were ready to reduce the maintenance work on their older systems and start to use our platform. I asked the advice of our CPO and VP of Engineering. Their advice included an example of an earlier platform product avoiding helping an internal department when it would have been better to assist in the long run. They made it clear it was the dev.lab team’s decision. The majority of my planning fell apart at that point.

Acquisition

Due to technical hurdles, there is still friction in the onboarding and acquisition process. That primary friction is the headache of giving new users access and permissions. Originally I envisioned habit formation occurring through the usage of the product and creation of an application. Habit formation now occurs during new user training. Training courses developed by our in-house trainer have simplified onboarding and significantly reduced time spent on fielding user support questions. I created tutorials and videos that helped with completing some work. The training goes above and beyond in explaining the why and what 12-Factor means. Innersourcing and collaboration on the platform are limited so far. There is a lot of opportunity around cultural change to promote the inner sourcing concept.

Measuring Engagement

Self-Service Portal

FullStory has been a lifesaver here. dev.lab is many products glued together via a self-service portal we built ourselves. FullStory enables collecting metrics of interaction and how users navigate your website. It feels nice to see the success and sometimes struggles. By observing FullStory, users tediously typed in the same project names over and over. A user eventually submitted a feature request. Viewing the recordings helps see parts of the user experience that is not working for users.

Deploys to Production and teams using the product

I use GitLab WebHooks to track applications deployed. The data services department records these events via a database that I query from time to time. This database saves me a lot of time validating this key metric of adoption. I manually track team adoption as reliable automation of this social construct appears to be a wicked problem.

Activation

In the Reforge course, one suggestion to improve activation is the concept of a warm start. In dev.lab there are quite a few new paradigms we are asking users to take on. It is hard to determine where to put this information. Documentation can be a blessing and a curse over time. We might add more user tutorials in the self-service portal in the future. For now, the training significantly improved activation and reduced the time to establish new habits. Our trainer created three courses. Getting started with dev.lab, dev.lab flow, and a course on 12-factor includes peer demos for engineers to illustrate their understanding. I loved these sessions because it lets the team provide direct feedback on the product in a safe group setting.

Communication

I record a weekly Monday morning video to share high-level metrics of adoption. I also use this video to tell why we made dev.lab and share any new messaging. For example, a big push is to remind people that we support engineers, contractors, and interns. People appreciate knowing you are open to feedback. I learned its better to overcommunicate this point. After that, I started to receive better feedback. Slack huddle functionality has made it easier to conduct office hours and help people on demand. I find it difficult to capture and harness attention despite email, Slack, StackOverflow, and constant video updates. If I learned anything during this long journey it is that despite you thinking you are overcommunicating you will find pockets of people that need further clarification.

Testimonials and marketing

Nine testimonials so far. Each week I continue to follow up and ask for more.

“It’s incredibly nice to have a lot of the dev ops heavy lifting done for you. With dev.lab we can dedicate more time and energy towards writing, testing, and configuring our code instead of figuring out why an argument didn’t get passed to a powershell script. The ease of use is not a trade off - we can still get everything we want from a deploy. dev.lab has all the tools you need.” - Satisfied dev.lab user.

I highly recommend making merchandise to advertise your product. Make sure you clarify that you built it because people may have no idea.

Conclusion

It has been quite the journey. Thankfully I have a fantastic team that has labored to make this a successful product journey. 25 Applications in Production with nine teams onboarded since announcing general availability at the end of May.

The next cohort of Reforge starts October 4th, 2021. I plan to take the Mastering Product Strategy Course and share with you what I have learned.