How to Take Three Years to Finish Your Side Project
Saturday, April 20, 2019
It's been almost 2 years since I've posted any content on this blog. One of the reasons for my hiatus is that I've been spending a lot of my extra energy on finishing my side project, Flipt, which I can best describe as a ‘self contained feature flag solution’.
This post however is not really about Flipt, at least not directly. This post is about how it took me almost 3 years from inception to feel like I was ready to launch Flipt out into the world. It's mostly a tongue in cheek recounting of mistakes I've made and things that I wish I would have done differently, but also includes some things I've learned in the process. Lets get to it.
The idea behind Flipt came when I realized that every software company that I've worked at that which practiced some form of continuous delivery had a home grown feature flag solution. These solutions were normally OK.. but always seemed to have some major limitation. Usually, they only worked for some parts of the application and were not well suited for when the app was eventually broken up into several systems.
This got me thinking that there should be an easy to use, powerful, open source solution that companies could use instead of trying to roll their own. I was also aware that there were already several competitors in the space, both free and paid, but after looking at them they never seemed to scratch my itch, or were prohibitively expensive.
So, I did what any software engineer with and idea would do, I started building.
Soon, I quickly entered the part of the journey that I will call ‘The Great Wandering’. This period consisted of me mostly spinning my wheels but not really getting anywhere meaningful and lasted for almost a year and a half. Here are some hot tips you can use if you too want to waste countless hours of time and energy with your own projects:
- Re-implement the basic functionality in at least two programming languages. I first implemented the straw man version of Flipt in Ruby, because three years ago I was still learning Go and didn't feel confident enough in my abilities at the time. Eventually, once I realized that I wanted Flipt to be mostly self contained with limited dependencies and a painless install, I realized that Go would be a better choice. So, of course I had to re-write everything.
- Don't write anything down. Just jump straight to code. There's no need to draw anything out on paper such as user flow diagrams, entity relationships, or even just the basic ideas you are trying to represent in code. You can totally hold all of that in your brain and it will translate effortlessly into code. Obviously, I am being sarcastic because I am a little bitter here since this is where I probably wasted the most amount of time. I did exactly what everyone tells you not to do and that is jump straight to code. Instead, I should have closed my laptop and opened a notebook to draw out the basic functionality and some simple wireframes for the UI.
- Change your storage backend several times. Yes. I used 3 different databases over the course of developing Flipt. Initially I went with Postgres because that's the DB I know and love, and obviously I needed this thing to be web scale (TM). However, after I realized that I wanted Flipt to be as simple as possible to get up and running (see above), I knew that Postgres just wasn't gonna cut it. So, I switched to a completely different technology and paradigm and decided to use BoltDB which is an embedded key/value store written in Go. This was great in that it made Flipt extremely fast, as it removed the need for communication over the network to access data. What was not so great was that I had to port (completely re-write) my repository layer to a NoSQL version, which of course meant I had to write code to maintain my own indexes, serialization/deserialization, error checking, etc. Soon after this re-write was complete, I had the sad realization that although this key/value business was the new hotness, what I gained in speed I lost in ease of extensibility. For example if I wanted to add a new feature such as user management, I'd have to completely redo mostly everything again as all the data was stored in binary form and there was no way to easily migrate to a new data model. This resulted in me switching databases AGAIN, back to a SQL based but embeddable solution (SQLite), which meant yes… another re-write of the storage package.
- Try to make everything perfect. Perfect is the enemy of done.. or something like that. Yes you need good documentation, but it doesn't have to be perfect from the start. You probably don't need to spend weeks optimizing your build system. You definitely don't need to teach yourself Sketch so that you can make a logo for your open source project that no one really cares about yet. It's more important to garner feedback in the beginning to make sure you are actually building something of value instead of waisting your energy on things that don't really matter.
Putting it Out There
After many nights and weekends of slogging through the above trials and tribulations, I eventually got to a point where I felt good enough about releasing. Actually, I don't know if it was so much that I felt good enough, I was just tired. Tired of putting in hours after my day job, working on something that only I had seen. So I pushed a tag, created a GitHub release, sent a tweet and prayed someone would notice.
Today I'm open sourcing, Flipt, my self-hosted feature flag application I've been working on for what seems like forever! It's been a labor of ❤️ but I'm finally ready to let it go free:https://t.co/iRzIk7YpeL— git checkout this-guy (@mark_a_phelps) February 16, 2019
Check it out!
I also posted on /r/golang and hoped people wouldn't be too mean. Luckily, I got some good feedback/questions and even ended up being featured in Golang Weekly! I'm still too scared to post on HackerNews though..
In retrospect, I think I wasted so much time described above on purpose (subconsciously) because I was scared to put something out there in the world. When it's only you working on a thing it can be imperfect, it's safe. Once you share what you've been working on with other people it can be judged, you can be judged. The safety net is gone.
What I Would Do Differently
Apart from not making many of the mistakes mentioned above, I think the main thing I would do differently next time is launch much sooner. This seems obvious in retrospect, but for me it is insanely difficult. Call it imposter syndrome or whatever, but I am not good at launching before I feel like something is mostly done and polished. Basically, stop working in a vacuum.
I think it also would have helped if I talked to other engineers to validate my assumptions on why this project should exist before I started building. I know at the very least that Flipt is something that I would actually use and do hope to use in the future at work and on other projects, but it would have helped a lot to know others felt similarly. Since launch, I've met some great people over the Internet and gotten some good feedback from other members of the community as well but I still wish I had started talking to people much sooner, because for the most part, people are awesome!
Well this post is already almost 1500 words so I'm going to call it a day. Let me know on Twitter if you have had a similar experience on one of your side projects!