It’s that time of year again.
Like clockwork, when the leaves start falling and the hoodies begin making their regular appearance, I find myself falling back into a weird pattern.
I begin believing that a decision holds no merit if it isn’t backed by bullet-proof, statistically significant data.
After all, a huge number of successful startup stories (and job postings) mention how they are a data-driven team.
So I’m writing this as a note to myself, and anyone else that stumbles onto this:
Not everything has to start as an A/B test.
And not everything has to be data-backed with statistical significance.
I’m not suggesting that you ignore the data, but there’s a spectrum between “trust your gut 100%” and “let the data decide”.
Especially when the decision is easy to revert.
Copy for the first version of a new landing page? It’s OK to trust your gut feeling if it’s easy to revert or update. You will analyze the results and can introduce new versions very easily if necessary.
User interface for a new feature? Put some early wireframes in front of a couple customers and gauge their gut reactions. You will analyze the results and can introduce a new version, with some work, if necessary.
Two drastically different business models for a new product? Lean more heavily on your research since this is probably a drastic shift in your positioning, burn rate and data model. These are much harder to iterate on since your customers, employees and investors may all be impacted.
It’s easy to fall into analysis paralysis, and take way too long analyzing all possible angles before making a decision.
But sometimes, it’s just a headline change or a really common UI pattern.
So it’s OK to trust the 20 years of experience and countless mess-ups in the past.