The Merit of Software Experiments

To contrast the ideas presented in my last post [On Doing Things the Right Way] I felt it necessary to say something about the other type of software development project - known as "experiments". About half all the projects I worked on were experimental in nature.

I personally have worked on several experiments: Emulators, Data spike detectors, Performance and load testing utilities, Object-oriented version control, Parallelized software build systems, etc. All of these are not in current use, but some have been re-written and deployed as "real" software projects by others.

Experimental software projects are kind of an enigma. They are less planned and more ad-hoc. They are essentially license for the programmer to operate with impunity in designing and steering the behavior of the program. From stakeholders, an experiment rarely defines more than a few requirements. The lead developer determines most aspects of the design.

Conceptually, experiments have business value, but the need they fill is often only secondary and non-critical in nature. The experiment is almost alsways disposable, and lacks commitment from developers and management to succeed beyond its first incarnation.

The failure rate of experiments is extremely high. On the first iteration, with a single developer, they are almost always guaranteed to fail as deployable entities. Depending on their degree of criticality to the business, they may or may not be re-written and permanently deployed, but rarely by the same people who wrote them initially.

In that way, experiments are not real software projects, but more loosely-defined forays into the unknown. Their hidden purpose is to discover facts and measure degree of desirability of a service or program. Unlike prototypes, they are not marketing props but tools of investigation. There is always something to be learned from their construction about business process, technical needs, and obstacles that might be encountered in the dark.

An unsuccessful experiment is one where the business did not gain much for the time and effort placed into it. Little or no advantage or knowledge was gained compared to the cost.

It can be hard to distingush between an unsuccesful experiment and an unsuccessful 'real' software project. The main disctinction is that experiments are acknowledged as experiments from the beginning, even if they are deployed for a time. Real software projects have enough organizational support and commitment to drive things to production. Experiments generally lack that organizational backing.

A successful experiment is one in which time and effort in development gave insight into business needs, helped to define requirements or new architectural approaches for future projects, a kind of initial marker-stone to navigate future development efforts.

Successful experiments can become successful software projects if the organization commits to re-developing and maintaining the software. Re-usability of a successful experiment is not the code, but the ideas behind the code which are adopted by the company and built-upon to gain a product advantage.