When to scrap the codebase and start from scratch

This is one of those decisions that is so context specific that almost all advice on it is worthless. Nevertheless, I thought I’d share the story of when I had to make this decision as a beginner programmer. There may be a lesson or two.

In the summer of 2018, SharpestMinds was a platform for aspiring data scientists. It was half community, half job board; built on a forked version of an open source chat platform called RocketChat. I owe my career to RocketChat because I learned full-stack web development by modifying and mimicking the patterns that were already there. But, as my skills developed, I got increasingly frustrated with the amount of legacy spaghetti code and an excess of features we didn’t need. But it (mostly) worked and we had (some) users.

Then came the pivot (a story for another day) to income share agreement (ISA) mentorships. The core goal was the same: help aspiring data scientists get jobs, but the business model was much more exciting (incentive alignment!). Before building anything, we validated the concept with a landing page and sign-up form. There was interest. Enough to justify building software. And that’s when the question came up: do we adapt our current platform or build a new one from scratch?

This was the first major decision I had to make since becoming CTO months earlier. I was excited by the idea of ditching our Frankenstein version of RocketChat and it’s sprawling code-base of legacy features (much of it written by me while I was learning). But the general advice in the developer world suggested that “building from scratch” was rarely the right answer – incremental change is usually the right path.

After building an MVP mentor dashboard on the RocketChat app (that way, we have it), we decided to build a new platform from scratch. Iteration time on the old code-base was consistently being bogged down by technical debt. RocketChat’s front end was originally built with Blaze.js and jQuery, which we had been slowly converting to React. The result was a confusing mix bag of frameworks and every new feature inevitably involved a lot of refactoring. The app was bloated, slow, and buggy due to choices made long ago that none of us understood. Newly onboarded mentors were getting confused by the excess of unrelated features. By starting from scratch, we could start with the simplest version of the product and iterate quickly while we searched for product/market fit.

The trade-off was developer time. I was still rather new to web development and building a web app from scratch would involve lots of “discovered” work – tasks and problems I could not predict at the outset. I treated it as a learning opportunity. An added advantage of building from scratch would be a better understanding of the technology and work involved (something I lacked at the time). We relied on a lot of manual work along with the MVP dashboard for mentors awkwardly embedded in our old platform while I buckled down and started from scratch.

A big part of me wanted to switch to a new and hot JavaScript framework. The physicist in me wanted to drop all frameworks and build from the ground-up. But, in the end, I stuck with the what I knew best. RocketChat was built with Meteor.js, a full stack JavaScript framework. This is the environment I learned to program in, so that’s what I was most comfortable developing with.

Sticking with what I knew was the right decision. It’s easy to get tempted by the newest, hottest thing, or to be influenced by the strong opinions of HackerNews commenters (who seem to regularly trash Meteor.js). It was the right decision because it greatly reduced developer time and the faster we moved, the better (the founders were frantically connecting and managing mentorships over email). I did not have to learn a new framework from scratch and I could re-use a lot of the patterns from the old app. It also helped that Meteor.js abstracts away a lot of the challenges – like account management, user authentication, and reactivity – right out of the box.

In a few weeks, we had an MVP web-app. It wasn’t pretty – it barely did anything – but that’s exactly where we wanted to be. Now we could start adding only what was necessary by staying close with our users and understanding what they needed. I may not have made the most optimal decisions while building it, but at least I understood them (and could communicate that to the rest of the team).

It I were to do it again, I would have spent more time planning. Particularly with design and layout. I made some arbitrary choices that I had assumed we would change, but these early decisions have a way of sticking around.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s