It’s been about 8 weeks since my last newsletter. It doesn’t feel that long, but time operates differently in a quarantine (and when procrastinating). In my defense, I spent 6 of those weeks enrolled in this Reforge course, a training program for product folks, focusing on retention and engagement. Some of you have asked me if I’ve found it helpful, so I jot down some thoughts to share!
When the lockdown in SF first started, I was initially disappointed that I wouldn’t be attending this course in person. But, as the world quickly changed around us, Reforge has been a nice constant in my life. Every Tuesday, I attended Zoom lectures with about 100 other PMs, and spent weekends reviewing the course materials. I dedicated about 5-10 hours each week on the course, which doesn’t seem like a lot, but with more hours at work, I felt like I had little time to do anything else. Except, of course, for the most important things in life like creating a sourdough starter and growing scallions.
I chose this particular program because retention is typically the hardest problem a PM has to solve, but going into it, I was fairly skeptical. I mean, can’t I just keep reading Andrew Chen’s blog? Also, so much of being a PM are about the intangibles. How can you even teach that?
I still think that real life experience trumps all others. If you are lucky enough to be at a startup in growth mode, you will likely learn more about growth there than anywhere else. But for the rest of us, real-life product work at startups is often too messy for structured learning. Though I try to introduce frameworks to my work, a lot of my time is spent firefighting, in which I rely on my instincts way more than I should. It’s literally ready, fire, aim on a regular basis. It helps to take a step back and think about concepts at a more abstract level. Similarly, a lot of blog posts are about “my experience doing X at company Y”, which is often too specific to apply to my current work. Reforge is great at using examples of real-life products like Pinterest so that it is easy to grok the product, but provides fake data to help illustrate the frameworks more clearly.
This idea of putting product stuff into frameworks reminds me a lot of my time spent as a tutor. For a couple years after college, I worked full-time as a private SAT coach in San Diego.
The challenge was the same. How do you consistently improve a student’s score on the Reading section if you can’t turn back time and get them to watch less TV and read more classics growing up? We did it by coaching them on the tactics of the test (better to skip than to guess), shoring up their fundamentals (most of geometry they’ve learned in school is useless), and giving them more confidence. Very little time was spent on the subject matter itself, but it worked. On average, students improved by 240 points, the difference between UC Irvine and Notre Dame.
To teach, you must systemize. And, in systemizing, some nuances are necessarily lost. But, some framework is better than none at all. Often, teams don’t even know how to discuss retention in a productive way. Here’s a quote from by the outgoing CTO at NYTimes that discusses this point:
“Perhaps the single most impactful thing we did was to bring growth product thinking and practice to the Times. Growth product is the concept of leveraging product mechanics to drive growth, through a laser-like focus on the key conversion moments in the customer journey. It has become commonplace in the tech world, but it’s a big adjustment and a major lever for companies that have traditionally relied on direct-marketing driven conversion.
The key for us was finding, and then listening to, some super smart people. I want to thank Andrew Chen, Brian Balfour of Reforge, and Aaron Shildkrout for teaching us how to do it. Reforge in particular really helped us educate ourselves en masse, injecting great thinking and practice into the org. I think about 150 Timesians went through their Growth Series.”
Internal alignment sounds simple but at Entelo, I struggled immensely with this. We knew that retention was bad, but did not know how to break down this problem into solvable parts. Our customers bought annual licenses and when 50% of them didn’t renew after a year, we couldn’t really pin down the reasons why. When we launched new features or fixes to improve retention, we couldn’t determine their results until a year later so it didn’t feel impactful.
In the very first week of Reforge, I already had a few “ah shit” moments, reflecting on all the things we should have done at Entelo. If I’d taken this course 3 years earlier, things might have gone very differently.
At Entelo, we focused intensely on retention on a quarterly basis without realizing that retention is the output of other more granular (and measurable) factors like engagement. Focusing solely on retention is like watching the end of the assembly line wondering why you’ve made 1 car a day instead of 100 cars a day. We never aligned on what it meant for users to be activated or dived deeply into understanding the different levels of engagement. Instead, we relied heavily on our Customer Success team to guide users through product hurdles. Everyone at Entelo was capable and open to trying new things to help improve the product, but we just didn’t know what to do.
Overall, I found Reforge to be incredibly valuable at this point of my career. The instructors, all experienced practitioners, were approachable and engaging. The materials are packed with insights so there’s not a lot of fluff or bullshit. Anyone with 1-4 years of general product experience could benefit from this, but there’s likely limited utility if you’ve already spent a lot of time working on growth. It’s possible to get 80% of the concepts through blogs, books, experience, etc but I think it’s worth it to compress those learnings into a short, focused period.
If anyone here has taken another Reforge course in the past, I would love to hear your experience as I’m thinking about another course in the Fall.
Stay safe out there. Send over any other questions that you might have about Reforge, sourdough, or anything else!