SoftwareEngineering

The road to follow if you don't want spaghetti code!

Introduction

The name Software Engineering can mean different things to different people. Rather than enumerate all those nuances and perspectives, it's probably more useful to emphasise areas of consensus.

Probably most would agree that the enterprise of software engineering advocates the design and crafting of robust and re-usable code that is extendible and maintainable.

This is to be contrasted with software that evolves without foresight invariably resulting in ad-hoc and fragile solutions that are difficult to extend or maintain.

What's in it for Me?

Why should you care about extensible and maintainable software? why should invest time in creating robust and reliable code? Well, for better or for worse, software tends to hang around. Once a group of people have some software for doing a job which surpasses some modest level of usability for doing a job, it will tend to get re-used, regardless of whether there is a better or more efficient way of getting the same job done. An even worse fate awaits software which fails the usability test. It will be discarded, only to be re-invented in a myriad of time-wasting ways.

• If you are a PhD student or postdoc, sooner or later you will be tempted to write a quick-and-dirty hack to get a job done right now. However, unless that job is a one-off (and hindsight gleefully points out that they rarely are), the ad-hoc nature of that quick-and-dirty hack will soon become apparent. However, since you invested the time already, you'll persist, trying to retrofit additional functionality. As time goes on, however, the pain inexorably increases and you will end up wasting more and more time. The story goes that a frog will jump out of boiling water, but will die if the heat is increased gradually. All of this is an obstacle to progress. What counts is the amount of research done in a three year period. The argument is that if you adopt a software engineering approach from the outset, you will have more to show for your time come the end of your project.

• If you are a PI, you will be rewarded for adopting a longer-term perspective. You will get more bang for your buck if the tools developed for your research enterprise can be used by more than one student or postdoc and you don't end up re-inventing the wheel during each successive funding round.

OK, but what's Involved?

The traditional waterfall model of software development will espouse the steps of:

Feedback loops in Extreme Programming (wikipedia)

1. Requirements capture & specification
2. Design
3. Implementation
4. Integration
5. Testing & debugging
6. Installation
7. Maintenance

While many would agree with the actions involved, many would take issue with the static and prescriptive nature of the above approach. I would argue that these objections accrue additional weight in the academic setting.

As a counterpoint, more agile and dyanamic approaches have been advocated. These emphasise the virtues of user-centric design, co-design and rapid prototyping. Extreme programming is one such approach.

The subjects covered by the tutorials in this course all have a place in a software engineer's toolbox. They will draw your attention to:

• Software re-use. A wise developer will look to what others have done before and will strive to create code which can be re-used when, all too commonly, a related task raises it's head.
• Code design. Software developed with some forethought will often provide elegant solutions for more than just the immediate task at hand. It is also likely to be more robust and efficient than code which was quickly thrown together with just one job in mind.
• Productivity boosting tools. These will help speed you to your goal. To err is human and we will all need to find and correct software bugs from time to time. A modicum of knowledge of compilers, debuggers, profilers and documentation tools will repay the time invested in learning them many times over.
• Reproducibility. A key tenet of science is that experiments are reproducible. If your experiments involve computers, the task of reproduction is very hard indeed without the aid of a version control system.
• Testing. You wouldn't just switch on a piece of lab equipment and assume that it is working correctly--you'd need to calibrate it first. Likewise, just because a program runs does not mean that it will produce correct results. Tests are an essential part of the code development process, particularly so for science. The more agile development approaches encourage very regular tests. Indeed some methodologies advocate writing the tests before the code itself--the tests form a specification for what the code should do.