Thursday
Oct202011

It's about the team, damnit!

Investors are known for investing not in products or companies, but people. At least the good ones. Because markets change. Products change. The only consistency you have in a startup are the founders. You invest into them assuming that they can adopt to change.

A job is the same. Yes, you ought to take an interest in what you build, be passionate about it, but you and I both know that it might change on a dime. In the end what counts are the people you work with. Are they smart? Are you learning from them? Do you feel your contributions are appreciated? Is your manager giving you enough new challenges that stretch you and enough guidance to make sure you succeed? Are you buying into the company culture?

As obvious as this is, I had to learn this lesson several times. Of course, I do want to work on something that matters as much as the next guy, but I have come to realize that if you find the right people to work with, whatever you build together will be something that matters. You will make it matter. Not only because you are motivated and dedicated, but so is everyone else.

So, for all those job seekers in the tech industry (and outside), focus on learning as much about the team as possible. Great people build great products. Set out to find a team that matches your interest and has an engineering culture you like. Obviously, do keep an eye on whether the projects you'd be working on pique your interest. And for those of you hiring, you need to remember that you cannot compromise on quality in your team, no matter how desperate you are. Because every hire will affect all future hires.

Again, these are obvious lessons, but they are also too easily forgotten.

Sunday
Oct162011

SA500: The Career Fair That Didn't Suck

Career fairs suck. I wasn't looking forward to recruiting for Sulia at last Saturday's SA500 "Recruiting Event" at the New York Stock Exchange. This career fair, though, was different. I talked with tons of energetic and motivated applicants, who were college seniors and recent grads from top schools. Many attendees stayed to chat at length with us about their interests and about our company. I hope that a few of them will join us at Sulia or, at least, that they will join the New York startup scene.

The SA500 Career Fair (photo from Silveira Neto's flickr stream)

Click to read more ...

Friday
Oct142011

Steve Jobs' legacy is not design

Before diving into the deep end of software and hardware development in college, before writing my own micro-kernel OS and C-style compilers, before designing and soldering together circuit boards for my Master's thesis, I spent much of my childhood in my dad's painting studio in Germany. After my geeky years in college I promptly devoted another three years to a small art school in San Francisco. So I've always had one foot in technology and the other one in the visual arts. It was tricky because until recently I was never able to combine the two; it was always one or the other leading me to strange, one-footed balancing acts.

[...]

Much has been written about Steve Jobs and his impact on design and the design community. He elevated design to be a core aspect of any product, not just the icing on the cake afterthought. But to me his legacy is bigger: ...

Click to read more ...

Saturday
Oct082011

Queues: Your Company's Invisible Money Drain

If running an efficient software development operation were as easy as eliminating the $600 toilet seats, everyone would do it. In practice, though, anyone who has been to a 2-hour bug triage can tell you that it's hard to fix inefficiency. But the problem goes deeper than fixing inefficiencies. Lean methods are teaching software companies an even scarier truth: our most glaring inefficiencies are invisible.

Click to read more ...

Friday
Oct072011

Be RaDDD!!!

There are five cornerstones to the RaDDD process, which all lock nicely together, like a clockwork.

  • Project oriented, limited WIP planning
  • Iterationless development
  • Developer owned features
  • Dedicated OCE
  • Staged feature roll-out

Click to read more ...

Sunday
Oct022011

Don't Measure Code Coverage

To better understand the "lean" movement that's sweeping the software industry, I recently worked my way through Donald Reinertsen's The Principles of Product Development Flow. It's not your typical business book filled with fluffy anecdotes, catchy metaphors, and silver bullets. Principles of Product Development Flow is a tough read that's packed with equations and graphs.

Reinertsen advocates making decisions on an economic basis. After all, we develop products to make money. No, it's not the only reason that we do it, but it is a primary reason. Reinertsen advocates that we make decisions on the basis of an economic framework.

Unfortunately, it's hard to map every decision to money made or to money lost. So we often choose to measure and manage proxy variables, easy-to-measure standins for economic return. Some proxy variables are better than others. One particularly bad example of a proxy variable is adherence to schedule. If we measure adherence to schedule, we encourage padding of schedules, an activity that has little economic value.

Test Coverage is a Proxy Variable

One proxy variable that bothers me is test coverage. I'm not concerned only about how coverage is measured (statement coverage, branch coverage, test count). I don't think we should measure it at all. Don't get me wrong. I'm a strong believer in TDD. I just wrote three posts on Javascript unit testing, for crying out loud! I've encouraged my company to invest heavily in test automation, and the investment is paying off. We are able to deploy safely 5+ times a day, and that number will increase as our team grows.

But test automation is not something that makes money. Automated testing is valuable because it drives down the cost of software development. And automated testing is valuable only when it is the best way to drive down the cost of software development. Sometimes, test coverage is not the best way to fight costs.

Optimal Coverage

I propose that there is an optimal amount of test coverage that minimizes the costs of product development. This optimal coverage point will vary among products and organizations. It will be hard to quantify exactly and it will never be 100%. Intuitively, this should make sense. "Perfect is the enemy of good," right?

Let's take a deeper look, inspired by Principles of Product Development Flow. Here are some of the costs related to test automation:

  • Cost of developing tests
  • Opportunity costs
  • Cost (in lost business and in engineering effort) of bugs that tests do not detect

The optimal coverage point is the amount of test coverage that minimizes the sum of those costs. For an off-the-shelf enterprise product, test development cost will likely be low relative to the cost of fixing bugs in systems that are installed at customer sites. But for a web site, in-production bugs are substantially cheaper to fix. We can increase the optimal coverage point (and make it economically valuable to write more tests) by finding ways to make test development cheaper (e.g., by refactoring and by adopting test-friendly frameworks). Likewise, we can make tests less valuable economically by reducing the economic damage that bugs cause.

The latter approach, reducing the economic damage of bugs, is increasingly common, especially for cloud-deployed applications. Nowadays, there are several techniques for doing so.

  • Reduced Batch Size (e.g., continuous deployment). One cost-savings approach is working in small batches (e.g., continuous deployment). Checking in and deploying small code batches lowers variability and reduces the chance of exposing a catastrophic, costly bug.
  • Monitoring.  Bugs do less economic damage when they are found quickly. Etsy, among others, has argued that reducing response time is economically more effective than preventing failure altogether. When I worked at Amazon.com, they improved responsiveness by providing every engineer with a pager that was linked to their ticketing system. (Okay...that was pretty painful. There are better ways!)
  • Gatekeeper tools, such as those employed by Google Chrome, Facebook, and Amazon.com. These tools allow incremental rollout of new features. And they make it easy to disable rogue features without doing new deployments.

Fight the Real Enemy

Measuring code coverage draws attention where it isn't needed. It shifts focus away from costs and towards a proxy of dubious value. If a universal optimal coverage point were easy to determine, then measurement of test coverage might make sense. But that optimal point is hard to derive, it isn't universal, and it will change as innovations lower the costs of bug prevention and of bug fighting.

The real enemy isn't lack of code coverage. It's cost. Challenge yourself and your team to articulate your costs and to find ways of lowering them. You'll certainly put considerable effort into writing testable code and into automating your tests. But remember that test automation is just one approach to reducing costs. Consider the full range of cost fighting tools!

Thursday
Sep292011

A primer on startup processes

So you read my last post on why you should have process and you ask yourself "How can our team be more awesome?" Working for both big companies and startups I have summarized my key experiences as relevant to a startup in this blog post. As our process evolves some of these are bound to change, but it should be good starting point for you and your startup. Next week I'll describe our current process, cheekishly named RaDDD, in more detail. Before I dive into the big ideas a few more quick thoughts of wisdom and caveats:...

Click to read more ...

Sunday
Sep252011

Making Javascript Unit Testing Work (Part 3: My Choices)

In the previous installment of this series, I reviewed Jasmine and JsTestDriver, two exciting tools that have simplified Javascript unit testing. In this post, I'll explain my setup. (Hey, shameless plug, but if you're interested in my evaluation criteria, see my first post on the subject.)

Click to read more ...