Software development is the process of creating a piece of software designed to perform a particular task. It encompasses the entire software development life cycle. This resource is designed to give you an overview understanding of what software development is and the most important components, terms, and trends in software development today.

If you want to delve deeper into any of the software development sub-topics covered here you’ll find links to dedicated blogs so you can explore further. We hope you find this a great starting point for learning more about software development and we’d love to hear from you if you think we can improve it further!

What is software?

Software is a computer program – a structured compilation of commands computers and other hardware understand and act on because they are written in computer programming languages consisting of binary code. Software is developed to complete a specific task like automating a previously manual process to make it more efficient, organizing and processing information (data) in a certain way, or providing entertainment, like a video game.

The software has a front end and a back end

Most software has a front end and a back end. The front end is the part users see and interact with. It is considered the ‘presentation layer. The back end is the plumbing behind the scenes that supports the front end and allows it to work the way it does. It is the ‘data access layer’ or ‘infrastructure layer’.

Front-end and back-end development involve different skill sets and technologies and software developers are usually classified as front-end, back end or full-stack developers. Full-stack developers have the skills and experience to work on both the front and back ends of software being developed.

What’s the difference between software development and software engineering?

The terms software development and software engineering are often used interchangeably. But is there a difference? Some might argue differently but, technically, there isn’t any officially defined difference.

The Merriam-Webster dictionary defines software engineering as:

“A branch of computer science that deals with the design, implementation, and maintenance of complex computer programs.”

But the dictionary does not define software development, suggesting software engineering instead.

The Cambridge dictionary defines both software development and software engineering in the same way:

“the activity of creating computer programs”

Is there a difference between software development and computer programming or coding?

Computer programming and coding are other terms sometimes practically used as similes for software development. The distinction that could be made is software development refers specifically to the end-to-end process of creating a software product. Computer programming and coding is part of that process. A good analogy would be the distinction between ‘writing’ and ‘writing a book’.

The different types of software

There are many different types of software and the broad category can be broken down into different groupings, like use cases or technical qualities. At a macro level we can break software down into four main groups:

  1. Application software
  2. Utility software
  3. System software
  4. Embedded software

Application software

Application software is the most familiar category and refers to software that runs on a computer or mobile device. A computer game, Microsoft Office, accountancy software, Slack, Skype, Google Maps, or a web browser like Chrome are all examples of application software.

Application software can be native – designed to run on either a particular operating system like Windows or macOS (desktop and laptop) and Android or iOS (mobile devices).

Or it can be cross-platform. Cross-platform application software runs on web browsers and the process of creating it is web development. Some cross-platform applications, usually mobile apps, are referred to as ‘hybrid’ because they run on a browser but are deployed in a native container. The approach gives cross-platform apps a ‘native’ feel and can allow for better integration with the device’s hardware.

Application software is usually designed to help its users complete a particular task, including entertainment. The four main sub-categories of application software are:

Custom software

Custom software is developed as a tailor-made solution to the needs of a particular user. For example, a solar plant operator might want to develop software that will track and record yields across its installations. It has to be custom developed to the unique requirements of the specific conditions and commercial priorities.

Custom software is usually developed to provide a competitive advantage in a way a one-size-fits-all commercial software can’t.

Commercial software

Commercial software is developed as a solution to a particular market or consumer need and designed to be resold to a theoretically unlimited number of users with the same need. Video games are a good example of commercial software, as are mobile apps and SaaS products like accountancy or booking software designed for restaurants, hotels, or sports facilities.

Sometimes commercial software is also designed to be customizable to specific customer needs. Hybrid customizable commercial software usually offers a standard version of the product designed to fit the most general needs. Customers who want to customize that to their requirements then pay for the additional development work in addition to the usual license fee. Salesforce is one of the best-known examples of commercial software that can be customized.

Open-source application software

Open source software is developed and released by the copyright holder under a license that gives users the right to use, analyze, customize, develop on, and distribute it to anyone for any purpose. The source code of open-source software is often developed collaboratively by a public community of developers.

Improvements suggested by the open source community are integrated into the software by consensus decision. WordPress, the Mozilla Firefox browser, and VLC Media are all popular examples of open-source application software. Other kinds of software, like operating systems and programming languages, frameworks, and software development tools can also be open source.

The term open source was introduced by The Open Source Initiative (OSI) in the late 1990s.

‘Free source’ is a term sometimes used for open-source software distributed for free without a license fee.

Cloud-native software

Cloud-native software development means building software in a way that is specifically customized to run on a cloud computing infrastructure, including public, private, and hybrid clouds.

Cloud-native apps consist of “a collection of small, independent, and loosely coupled services”, referred to as microservices.

Cloud-native software can either be developed as ‘cloud agnostic’, which means it can run on any kind of cloud, public or private, with minimum reconfiguration or as native to a particular cloud computing platform like AWS, Microsoft Azure or Google Cloud Platform (GCP).

System software

System software is an umbrella term for software designed to start and run computer hardware and allow application software to run on it. It acts as a facilitator for the optimal interaction between hardware and application software running on it.

There are several different categories of system software:

Operating system (OS)

A device’s operating system is probably the kind of system software we are most familiar with. It is a kind of software application that sits between a computer’s hardware and the application software that is installed on it. Applications then make use of the hardware by sending requests for particular services, like processing or visual representation, go through the OS, communicating with it by API.

Users of the device an OS is installed use it to communicate with the hardware without having to know a programming language. For example, when you turn off your laptop’s power by selecting ‘shut down’ via Windows or macOS.

The operating systems we are most familiar with (Windows, macOS, Android, and iOS) as users of mainstream tech like laptops, smartphones, and tablets use a graphical UI (GUI) user interface. Linux, popular with software developers, has a command-line interface (CLI), which means you don’t tap a button with a cursor but type in a command.

The most common operating systems in 2021 (used by almost 92% of all devices) are:

  • Windows
  • macOS
  • Linux
  • Android
  • iOS

Utility Software

Utility software is also designed to perform particular tasks but in the background and related to hardware performance and security. Examples of utility software are antivirus programs, hard disk optimization tools, and device backup.

Drivers

Device drivers are a kind of software extension, or component, that facilitates communication between an operating system and a device. If an application needs to access data from the device it is running on the driver communicates that request to the hardware, which then returns the requested data to the operating system which, in turn, passes it over to the application.

Diagram of how driver software extensions site between a device and operating system

Source: Microsoft

Middleware

Middleware is a category of software that augments the services and capabilities the operating system provides and is often used to handle things like API management, data management, authentication, messaging, and more advanced application services.

RedHat defines the role of middleware succinctly:

“Middleware helps developers build applications more efficiently. It acts like the connective tissue between applications, data, and users”.

Middleware has become increasingly important with the rise of the DevOps software development methodology and cloud-native development because it helps manage the complexity of applications deployed across hybrid and multi-cloud infrastructures.

As part of a DevOps approach to application software development, middleware is used to ensure software developed in one environment works in the same way when run in another.

Windowing systems and operating system shells are also examples of narrower system software categories.

Embedded software

Embedded software (also referred to as firmware) is a kind of software designed to run on hardware that wouldn’t normally be defined as a computer. For example, the software in a car or a robot vacuum cleaner is an example of embedded software. As is a software developed for robotics like those used to automate production lines or IoT (internet of things) devices like smart doorbells or virtual assistants (Alexa, Apple HomePod, Google Assistant, etc.).

Embedded software engineers need specific skills and knowledge application software specialists often don’t like being able to read schematics and component data sheets, communication protocols between processors and other components, the ability to convert between decimal, hexadecimal and binary, and use bit manipulation.

Developing embedded software involves the use of a cross-compiler that converts code developed on a computer into a format executable on the target device. Debugging is also different and requires the use of an in-circuit emulator and specialist hardware.

Artificial Intelligence (AI)

Artificial intelligence algorithms are a kind of software that is usually a component of a larger piece of software that could be application software, system software or even embedded software.

AI software is an algorithm, or collection of algorithms, that are sets of rules, or step-by-step procedures for calculations. The algorithm applies the rules it has been given to a data set, which allows it to learn on its own.

Algorithm example

For example, a simple algorithm to find the largest among three different numbers would look like:

Step 1: Start
Step 2: Declare variables a,b and c.
Step 3: Read variables a,b and c.
Step 4: If a > b
           If a > c
              Display a is the largest number.
           Else
              Display c is the largest number.
        Else
           If b > c
              Display b is the largest number.
           Else
              Display c is the greatest number.  
Step 5: Stop

Machine Learning and Deep Learning are the two types of AI most used in software development.

Machine learning

Machine learning algorithms parse data, isolate patterns that exist, ‘learn’ from these and decide on or make a prediction about something in reality connected to the database learned from or similar data. ML algos are based on decision tree learning, inductive logic programming, clustering, reinforcement learning, Bayesian networks, and other processes that resemble some of the ways we learn as humans.

Deep learning

Deep learning is an approach to AI that is more like how human intelligence works. Neural networks break the decision-making process down into chunks and pass it through a system of checks and balances. If we see a Stop sign partially obscured by a branch, we may not see the whole sign but will be confident it is a stop sign because we can see what shape, size, and color it is and part of the word written.

A deep learning algorithm assigns weightings to the information it does have in the context of the database it has learned from and reaches a ‘probability vector’, or, in other words, an educated guess.

Software development lifecycle

Software is created through a development life cycle that typically includes phases of ideation, concept validation, specifying requirements, design, prototyping, coding (programming), documentation, testing, bug fixing, increasingly operations – deployment into the live environment and making sure the software runs as expected (DevOps), maintenance and the release of new iterations. The number and how the different phases are named vary between development methodologies and frameworks.

 

Phases of the software development life cycle infographic-1

Software development methodologies and frameworks

While every software development process goes through a similar process or life cycle, there are important variations in the methodologies (sometimes also referred to as software development life cycle models) used throughout the process.

Software development methodologies can be switched between depending on the qualities and priorities of a project and their perceived strengths and weaknesses in different scenarios. Agile software development, a flexible, iterative approach to the software development lifecycle, is the dominant methodology in 2021 but far from the only approach taken by software development teams and their managers.

Some of the main methodologies and methodology frameworks for software development are:

  • Waterfall
  • Agile
  • Lean
  • Scrum
  • Kanban
  • DevOps
  • GitOps

What is programming?

As already covered, software development refers to the whole lifecycle that culminates in a live piece of software doing its job for users and not just the programming or coding phase. But writing the structured computer code that makes up a piece of software is at the core of software development.

Codeacademy explains programming with a useful metaphor:

“Put simply, programming is giving a set of instructions to a computer to execute. If you’ve ever cooked using a recipe before, you can think of yourself as the computer and the recipe’s author as a programmer. The recipe author provides you with a set of instructions that you read and then follow. The more complex the instructions, the more complex the result!”

Programming languages

Programming, or code, is written in programming language computers that think in binary can understand. Humans, however, can’t typically understand these strings of 1s and 0s. Programming languages act like a translator which converts binary strings of 1s and 0s into a formatted human can understand and write.

When software developer learns a programming language they can write the structured sets of instructions software consists of. The computer then converts the programming language into the strings of binary it understands and executes on the instructions given.

Low-level and high-level programming languages

Programming languages are classified as being either low-level or high-level. Low-level languages are closer to binary. That makes them harder for humans to read and write but easier for computers. A software engineer coding in a low-level language can give a computer more precise instructions understood and actioned on faster than if they were written in a high-level language more abstracted from binary.

High-level languages use more words we also use in human-to-human communication, like object, class, request, run, etc., which makes it easier for humans to learn them.

As computers have become more powerful, the difference in the time needed to convert low and high-level languages into binary has been reduced to milliseconds. That means high-level languages can be used in most modern software products without any meaningful impact on performance.

Programming paradigms

Programming paradigms classify programming languages based on their features. The main paradigms are:

Object-oriented programming

Groups coded instructions by the part of the state they operate on.

Functional programming

The intended result is declared as the value of a series of function applications.

Declarative programming

The coder declares the properties required for the desired result but not how it is computed.

Imperative programming

The coder tells the machine how to alter its state.

Procedural programming

Coded instructions are grouped by the procedure they form part of.

Logical programming

Written in a way that is very similar to formal logic, expressing facts and rules. Rules are written as clauses and read declaratively as logical implications eg.

H if B1 and … and Bn.

Mathematical programming

The desired result is the optimal element from a range of choices based on given criteria. For example, if the optimization problem was arriving at the number 500 by using the numbers 5, 100, 10, 9, 8, 2, and 2, the desired result would be 5×100 and not 9×2-8/2*100.

Reactive programming

The desired result is declared with data streams and change propagation.

The most commonly used contemporary programming languages usually don’t fall neatly into a single paradigm but combine features from several.

Popular programming languages

As of July 2021, the three most popular programming languages, based on analysis of Google searches for tutorials are Python, Java, and JavaScript, followed by C#, C/C++, and PHP.

PYPL Popularity of Programming Language Index

Source: Github

Frameworks and libraries

Frameworks and libraries are reusable pieces of code written by someone else in a particular programming language, like JavaScript. They are designed to make the software development process more efficient by allowing software to be built from reusable components that don’t need to be built from scratch every time.

The terms framework and library are sometimes used interchangeably but there is a difference with libraries offering greater flexibility. A good analogy for the difference between a library and a framework is that the former is like designing an IKEA kitchen. You can create a kitchen of almost any shape or size with a lot of choice between elements and how they can be combined.

A framework is more like building a modular house. You have choices in the architectural layout and design but they are limited. The kitchen and bathrooms, for example, should be positioned close to where the main water pipes run top to bottom and the kitchen should be on the ground floor. You have a limited amount of space to divide up into rooms etc.

More technically defined, the difference between a library and a framework is something called inversion of control. When a software developer uses a library they control the flow of the application – when and where the library is called.

When a framework is used, it is in charge of the flow and tells the developer where they can plug in their code to customize it. And it calls the code plugged in as it needs.

Popular libraries and frameworks

There are a large number of frameworks and libraries, especially for the most popular programming languages. However, a much smaller number are regularly used commercially. As of 2021, some of the most popular frameworks and libraries in use are:

Popular JavaScript libraries

React.js and React Native (for mobile applications) are by far the most used JavaScript libraries in general use. Others used regularly in commercial software development include Axios, Lodash, Moment, date-fns, RxJS, jQuery, Underscore.js, and Day.js.

Popular JavaScript frameworks

Angular is the most used frontend JavaScript framework followed by Vue.js. Others include Svelte, Preact, and Ember.

There are also backend JavaScript frameworks with Express the most commonly used by some margin, followed by Next.js and Gatsby.

Node.js is a popular back-end run-time environment also sometimes referred to as a framework though it technically isn’t.

Popular Python frameworks and libraries

The most used python frameworks are Django, Flask, and Bottle, while the most used libraries are TensorFlow, SciKit Learn, Numpty, Keras, and Pytorch.

Popular Java frameworks and libraries

The most used Java frameworks are Spring, Scala, GWT (Google Web ToolKit), JSF, and Struts.

The most used Java libraries are Lombok, Guava, Hibernate, Feign, and Spring Boot.

K&C – Creating Beautiful Technology Solutions For 20+ Years. Can We Be Your Competitive Edge?

Drop us a line to discuss your needs or next project Contact US

Software development tools

As well as frameworks and libraries, software developers make use of a large collection of software development tools designed to make their job easier. Tools are used throughout the software development process to help develop, debug, maintain, or otherwise support software.

Popular examples of software development tools are Integrated Development Environments (IDEs), which are really a collection of tools, Git for version control, Selenium and Cucumber for integrated testing, Stack Overflow an online community of developers where knowledge and resources are shared, the continuous integration server Jenkins, and many more.

Even Docker, the containerization platform, and Kubernetes, the container orchestration platform, used in cloud-native development, could be described as software development tools.

Software Development Kit (SDK)

A software development kit (SDK) is a toolkit that comes in a single installable package and typically includes a compiler, debugger, often a framework, and other tools. SDKs are usually made to help software development for a particular combination of hardware platforms and OS.

For example, the iOS SDK is designed to help developers create iPhone and iPad applications and AWS provides SDKs that provide language-specific APIs for AWS services.

Software development team roles

A small app can be built in its entirety by a single full-stack developer with both backend and frontend experience. But most commercial software products are built by a software development team consisting of several different roles.

Some roles, like Scrum Master, are specific to a particular software development methodology (Agile), while others are more general. Not every development team will consist of the same collection of roles and the number of specialists performing each role will vary with a project’s nature and scale.

Some of the most common roles found in a modern software development team are:

Project Manager

Oversees a software development project from a bird’s eye view with a strategic focus on budget, risk, schedule, and resource management. Project managers often don’t have a software engineering background themselves.

Product Owner

Product owners also an analytical not technical role and is focused on a product’s functionalities and features. Their main responsibility is that a software product is fit for purpose and its business value optimized. In an Agile development model, the product owner also prioritizes the order in which new features and functionalities are developed and released after the launch of the MVP.

Tech Lead

The tech lead of a software development project is responsible for selecting the best technical solutions to meet business requirements. This role used to often be referred to as the software architect.

In consultation with other stakeholders, the tech lead makes decisions on things like what programming languages or frameworks the software is to be written in, infrastructure choices, perhaps what methodology the team will follow, etc.

Full stack developer

Full stack developers are generalist software engineers that have a working knowledge of a wide range of frontend and backend skills and technologies. A full-stack developer can theoretically build a whole software solution alone. But full-stack developers usually aren’t specialists so are often seen as most valuable in teams working on less complex software.

Front end developer

Front-end developers are specialized in and work exclusively on the front end of a piece of software being created by a development team. They use front-end languages, libraries, and frameworks to create the visual interface users see and interact with.

Back end developer

Back-end developers are specialized in and work exclusively on the back end of a piece of software being built by a software development team. The back-end team is responsible for a software’s CRM, databases, and server-side web application logic and integration.

QA Lead

The quality assurance (QA) lead is accountable for the overall quality of deployed software. They manage the QA team itself and make sure its findings are acted upon by those responsible for different areas of the software development process.

QA engineer

The QA team is made up of QA engineers who run software testing throughout the development process, identifying potential issues and reporting them back to the responsible teams. Most testing and QA assurance is now automated as part of a DevOps approach, especially for more complex software products, and consists of regression error testing ie. have new features resulted in errors or bugs in already existing parts of the software?

Software Tester

Testers are usually responsible for manual verification software works as it is supposed to for end users. Standard user journeys can usually be tested by automated tools but manual testers take an analogical approach to make sure bugs and errors don’t crop up in alternative user paths and exceptions. For example, in airline ticket booking software, can the system handle the exception of a customer booking outgoing and incoming flights on the same day?

UI designer

A UI (user interface) designer is responsible for the user-facing appearance of the software. The UI designer designs all the screens a user might see while interacting with software, including all the visual elements, and their interactive qualities, that take a user from one screen to another.

UX designer

A user experience (UX) designer is responsible for software being as intuitive as possible for users. Nobody wants a kitchen that looks beautiful but in which elements are inconveniently positioned. The sink should be adjacent to a work surface and not too far from the hobs etc.

A UX designer makes sure software doesn’t just look nice and is convenient to use.