When I began research in computational physics in my junior year, the academic job market was bleak. It was 1992, and many physicists were leaving eastern Europe and Russia for jobs in the United States. A year later the supercollider was canceled, further dimming job prospects for my friends in high energy physics. However, I had few worries. My physics courses were fascinating, and I was thrilled with the opportunity to combine my two passions—computer programming and theoretical physics—in practical scientific research. Besides, people told me that with computational physics skills students could easily get computer jobs. Universities had been at the forefront of computer development, far ahead of the computer industry.

In the late 1990’s all that changed. The development of the personal computer had launched the consumer software market a decade earlier. Bill Gates pioneered the idea of selling software, and Microsoft dominated consumer software. Throughout the 1990’s a second consumer software revolution accompanied the explosive growth of the world wide web. Startup companies competed aggressively to profit from and drive the dot-com-revolution. While academics had produced the first web browser and incubated many companies, I believe that academic high-performance computing and the commercial software business bifurcated at that time.

Out of the fierce competition and rapid change of the dot-com-era, Agile software development emerged. While the Agile manifesto is written in the language of commercial or consumer software, its principles resonate with the needs of scientific and engineering programming. Three principles—“we value interactions of individuals over processes and tools,” “working software over comprehensive documentation,” and “responding to change over following a plan”—perfectly describe the decades-long culture of scientific computing. One principle—“Customer collaboration over contract negotiation”—is a little less applicable, since most scientific software is only used by the scientist who wrote the code.

Agile software development never took a hold in academic high-performance computing. This could be viewed as an example of academic arrogance, but I believe there were more pragmatic issues. Scientists often tackle very difficult problems at the forefront of computational abilities. Compilers and other tools in use in the late 1990’s were not good enough to support Agile methodologies in serious scientific computing. C++ compilers and computers themselves were simply too slow and unable to optimize the programming style embraced by the Agile community.

In 2013, we face two challenges. First, the bifurcation of academic and commercial computing fifteen years ago has created a situation where students in computational physics are no longer well-trained for most industry jobs. Teaching students Unix and how to code Fortran or kludgy C++ is not great for their resumes. Second, fierce academic competition and the availability of powerful software packages—especially tools for molecular dynamics, quantum chemistry, and symbolic mathematics—hinders development of new scientific software.

Now is the time for academics to embrace Agile methodologies. The tools for C++ have matured, workstations are supercomputers, and development of the new clang compiler suggests that things will just get better. Science needs reliable software that responds quickly to change. In my experience students embrace Agile methodologies and test driven development once they see how it works. No manifesto or fiat will change academic culture, but pragmatic development of new practices and shared examples help everyone.