Linux Engineering

Software Engineering vs. Development

Engineering Steps

Every engineering process consists of several steps:

  1. Define a general problem or issue to be solved
  2. Analyze the problem in detail, identify the root cause
  3. Investigate and Evaluate all available solutions
  4. Select the best solution or propose a new one if none of the existing solutions can be applied
  5. Implement selected solution

This simple and quite boring process of problem solving is the core principle of engineering. In many cases, this process is not intuitive and very hard to follow. If one solution has been used everywhere for too long, nobody has the incentive to start looking for alternatives. For new, important, long-term projects, this should be different. Current solutions should always be questioned, challenged, evaluated and improved upon.

Simple example of engineering:

  1. Define the problem in general:
    • It’s dark at night, people can’t work
  2. Analyze and identify the root of the problem:
    • Artificial light source is needed to emulate the function of sun at night
  3. Investigate and Evaluate:
    • As available solutions, people use candles and petrol lamps. They have some disadvantages:
      1. Limited brightness of emitted light
      2. A lot of regular maintenance needed
      3. Bad smell
      4. Every now and then a house burns down and people die
    • Is it possible to create efficient workarounds for these disadvantages?
      1. The light intensity of a candle can not be pumped up very much
      2. Burned candles need to be replaced, petrol lamps need petrol. No workaround.
      3. Petrol stinks. Filling a lamp with whisky is unacceptable.
      4. You might make fire escapes mandatory, but that’s about it
  4. Select or propose a better solution:
    • Create new artificial light source without all these disadvantages: a device that burns bright while preventing the actual fire, e.g. by the lack of oxygen
  5. Implement the selected solution

Problem Definition

Simple description of the problem in general. This might not be specific enough to be directly solved, or any possible solution might not be obvious at first. This definition needs to be further analyzed in detail and elaborated on, see Analyze below. Into this category fall problems like: “Earth gets warmer every year”, “Energy production does not fit the demand curve” or “Computers start too slow”.

Analyze

More detailed look at the problem at hand. Find a root cause of the problem before you try to find a solution. A nice example of this step is shown in the documentary “Freakonomics” looking at root causes of high crime rate in the USA. Result of this step should be a definition of the problem in a way that is solvable, for example “The level of greenhouse gasses is too high”, “Produced energy needs to be efficiently stored until required”, or “Computer boot stage needs to be parallelized for speed”.

Evaluation

The investigation and evaluation of problem solutions is at the center of engineering. It is the most important, and the most neglected part of the engineering process. It is unlikely for anyone to implement a better solution, without knowing everything about all existing applicable ones. For complex problems, proper investigation might become extremely hard. This boring, lengthy, non-productive investigation/evaluation process ist the root cause of failed projects, when not done properly.

Computer software, as any other result of engineering, is made to solve a particular problem. Something that is an issue for many people, for a company, or for you personally. What parameters for new solutions are usually evaluated by engineers? When comparing software, that usually means:

This implies a high level of knowledge for all evaluated systems. Since it is hard to learn and compare everything this way, non-engineers often use much simpler reasoning when choosing technology, like:

Unfortunately, these reasons are not good enough to make a meaningful choice. Not even all of them together. While this way you might end up with a sufficient solution, it would probably be far from optimal.

Selection

Based on previous evaluation, one solution has to be selected for implementation. Alternatively, a new solution can be proposed. Given the complexity of the problem, analysis and evaluation, this might be a real challenge. While perfect or optimal solution is hard to achieve, it is always important to strive for perfection:

“Perfection is attained not when there is nothing more to add, but when there is nothing more to remove.”

- Antoine de Saint Exupéry, Terre des Hommes (1939) quote

Simply put, perfection is a process of stripping away the unnecessary stuff. In software engineering, this process is closely bound to the main Unix rule: “do one thing, do it well”. That translates to the smallest possible amount of functionality in a single program so that the result can independently perform only a single task. Keeping pieces of software clearly separated and independent of each other helps other engineers and developers to understand it and reuse it. This modularity has the effect of stability, reliability, code readability and maintainability.

To select and properly describe a software solution is in essence represented by requirements engineering. Ideally, the result of the analyze, investigation, evaluation and selection steps should form a base of a requirement specification as an ideal form of input for implementers/developers.

Implementation

The implementation process is the least relevant to engineering. While “implementing” usually means doing something new, exciting and cool, sometimes even animals (or Edison) can implement a solution by trial and error given the proper boundaries defined by previous engineering steps. This creative, fun, “hands-on” part of engineering can be seen as a separate process. It does not necessarily involve engineers, but is often done by experts specifically trained to develop and implement solutions. In the world of IT, these specialists are commonly referred to as software developers.

Skills

Developer education

In a software developer training, you learn how to use a programming language to implement a solution: make a “hello world”, sort an array, process user input, handle errors, use a repository etc. This learning process is important to know your tools, work efficiently, get a feeling for what is actually possible. Talented developers learn to write cleaner, readable, modular, better documented code, implement efficient algorithms, master more programming languages etc. Without this knowledge, creating high quality software would be impossible.

Engineer education

Some developer training is always a part of software engineering education. Later it’s expanded. Instead of just implementing a given solution, you are given a problem to solve. You are expected to analyze the problem and find a solution first, before you proceed with implementation. If you studied engineering of any sort, or wrote any academic paper, you know that there are certain questions you have to answer first. What is the problem you are trying to solve? What other solutions are available? What properties do they have? Why is your solution necessary, wouldn’t something else be better? To proceed with implementation without answering these questions will surely make you fail. It is not good enough to say what path did you choose, you have to explain why that makes sense.

Conclusion

Wikipedia defines development as:

“Software development in a broader sense may include any activities that result in software products”(1)

and also engineering as:

“Software engineering is an engineering discipline that is concerned with all aspects of software production”(2)

The usual definitions are rather confusing, blending development and engineering together. While both engineers and developers play equally important role, and big part of their knowledge base overlap, every group is better suited for a specific task. Many times the difference in training and skills is not properly recognized by product owners and team leaders, not just by Wikipedia. Consequently, engineering jobs are wrongly assigned to developers, resulting in whole teams working on solutions that are not optimal, or obsolete before they even started.

To illustrate the main difference between software engineering and development: when faced with general problems like “What programming language should I choose?”, “What init system should I use?”, or “What webserver is the best?”, a software engineer follows a simple process to get an optimal answer. Software developer immediately implements a very smart, cool, even elegant solution, possibly solving things that were never a problem, introducing new issues on the way (see systemd as an example). This creates frustration on both developer’s and user’s side completely unnecessarily.


cloux@rote.ch (2018-02)