Home Programs Software quality characteristics. Methods for assessing software properties Quality software

Software quality characteristics. Methods for assessing software properties Quality software

The software must perform its functions and meet the specified criteria of quality, safety, and reliability. Assessing the product, its requirements, and design documentation is the task of quality assurance engineers, or QA engineers.

Software quality assurance includes activities that are carried out at each stage of its development. The purpose is to provide assurance that the product meets functional and non-functional requirements.

Concept of software quality

At first glance, “software quality” may seem like an abstract concept. However, for project managers, programmers, testers, QA engineers and other participants in the product development process, quality criteria are transparent and measurable. Let's look at the general definition first.

Software quality – a set of characteristics software product, determining the ability to perform the functions assigned to it.

Currently, this indicator is regulated by the international standard ISO/IEC 25010:2011. This standard establishes a multi-level software quality assessment system based on eight basic characteristics.

Software quality parameters

Main quality characteristics software according to ISO/IEC 25010:2011 standard:

Functionality. Software is considered functional if it performs the tasks assigned to it and meets the specified user needs. This aspect presupposes correct and accurate operation and compatibility of all components included.
Reliability. Software reliability refers to the uninterrupted performance of the tasks assigned to it under specified conditions within a specified time.
Usability (ease of use). This parameter characterizes the degree of convenience of the software for users, its clarity, ease of operation and study.
Efficiency. The parameter corresponds to the degree to which the product provides the required performance under given conditions.
Ease of maintenance. This indicator characterizes the ease of analysis, testing, correction of software components, its maintenance, as well as the degree of adaptation to new conditions.
Portability. The degree to which it can be easily transferred to another platform. Ensuring software quality involves checking it for each of the listed parameters, identifying weaknesses and troubleshooting.
Compatibility. The ability of software components to interact with each other.
Security, i.e. minimizing threats associated with unauthorized reading, changing information, etc. Threats may also be associated with incorrect use BY, external influence from unauthorized persons, failure of technical means.

Quality Assurance and Testing

The terms “testing” and “quality assurance” are certainly related, but not identical. What's the difference?

Quality Assurance is responsible for the entire development process and is integrated into all its stages: from creating requirements for a future solution to testing, product release and its post-release maintenance.

The tasks of QA specialists include:

formation of quality criteria;
planning measures to comply with criteria at each stage of product development;
selection of testing tools;
product testing;
KPI calculation;
preventing errors and improving the process.

Testing– checking software for compliance with requirements.

So you see that quality assurance is a broader concept that includes testing activities.

Testing can be automated or done manually; can be a full cycle or aimed at checking a specific aspect of quality (safety, performance, usability, etc.).

Testing engineers prepare testing strategies and a plan based on the project features and solution requirements, create and in the future optimize a set of test cases, search for defects, create and send reports on detected defects to developers, and verify the elimination of the defect.

The quality assurance function can be performed by an internal department of the company, or it can be delegated to an independent contractor who will objectively evaluate the solution itself, set up quality assurance processes and thereby allow the product to be released to the market high quality, meeting business requirements and user expectations.

Currently, there are several definitions of quality that are generally compatible with each other. Here are the most common ones:

ISO Definition: Quality is the completeness of the properties and characteristics of a product, process or service that ensure the ability to satisfy stated or implied needs.

IEEE Definition: Software quality is the degree to which it possesses a required combination of properties.

The main quality standard in the field of software engineering is currently ISO/IEC 9126:1-4:2002 (GOST R ISO/IEC 9126-93). In addition to it, a set of standards ISO/IEC 14598 has been released, which regulates methods for assessing quality characteristics. Together they form a quality model known as SQuaRE (Software Quality Requirements and Evaluation).

In accordance with the ISO 9126 standard, a general vision of quality software tool(PS) is recommended to be described by three interacting and interdependent metrics of quality characteristics, reflecting:

· external quality, specified by the customer’s requirements in the specifications and reflected in the characteristics of the final product;
· internal quality, manifested in the development process and other intermediate stages of the software life cycle;
· quality when used during normal operation and the effectiveness of achieving user needs, taking into account resource costs.

External and internal quality characteristics relate to the properties of the software system and reflect the customer’s and developer’s view of it. However, the end user expects to achieve the maximum cumulative effect from using the software - increased productivity and overall satisfaction with the software product. This view of the quality of a software system is referred to as “quality in use” or “operational quality” of the software.

Attributes of a software system that characterize its quality are measured using quality metrics. A metric is a combination of a specific measurement method (a way of obtaining values), an entity attribute, and a measurement scale (a means used to structure the resulting values). A metric defines a measure of an attribute—a variable that is assigned a value as a result of the measurement.

Determining quality requirements usually begins with a listing of external quality characteristics that reflect the requirements for a functioning software product. Further, in order to quantitatively determine the quality criteria by which the software system will be checked and confirmed for compliance with the requirements imposed on it, suitable external measurable properties (external attributes) of the software and associated metrics are specified, which are attribute assessment models, as well as acceptable ranges of changes in the values (measures) of the corresponding attributes.

Metrics, the definition and application of which is possible only for software running on a computer, which is at the stage of testing or functioning as part of the system, are called external metrics. External metrics provide customers, users, and developers with the ability to monitor and analyze the quality of a software product during testing or pilot operation.

After determining the requirements for external metrics, internal quality characteristics and internal attributes of the software are specified. They are used to plan the achievement of the required external quality characteristics of the final software product and their integration into intermediate (working) software products during development. Next, internal quality metrics are determined. The concepts of internal quality characteristics, attributes and metrics are associated with software work products (documents, code texts, tests, etc.) that do not run on a computer, obtained at the development stages preceding testing (requirements definition, design, coding).

Quality measurement system

Internal metrics enable developers, testers and customers to predict the quality of the software life cycle and deal with technological quality assurance issues before the software becomes a ready-to-use product. Internal metrics can be applied during design and programming to components of a software system, such as specifications, source code, or documentation. The main purpose of using internal metrics is to ensure that the required external quality is achieved.

Quality in use metrics reflect the extent to which a product meets the needs of specific users to achieve specified goals. These metrics are not included in the six basic characteristics regulated by the ISO 9126-1 standard due to their generality, but are recommended for an integral assessment of the results of the operation and application of software packages in the ISO 9126-4 standard.

A common approach to modeling software quality is to first identify a small set of quality attributes (characteristics) at the highest level of abstraction and then, in a top-down fashion, decompose these attributes into sets of subordinate attributes. The ISO/IEC 9126 standard is a typical example of this approach.

To assess the quality of a software tool, six groups of basic indicators are used, each of which is detailed by several normative subcharacteristics. The characteristics and subcharacteristics in the standard are defined briefly, without comments or detailed recommendations for their application to specific systems and projects. The presentation is conceptual in nature and does not contain recommendations for the selection and ordering of priorities, as well as the required minimum criteria depending on the characteristics of the object, development environment, maintenance and application.

The first part of the standard, ISO 9126-1, categorizes software quality attributes into six characteristics used in the remaining parts of the standard. Based on the fundamental possibilities of their measurement, all characteristics can be combined into three groups, to which different categories of metrics are applicable:

· categorical or descriptive (nominal) metrics are used to evaluate the functionality of software;
· quantitative metrics are applicable to measure the reliability and effectiveness of complex software packages;
· quality metrics are most consistent with the usability, maintainability and portability of software.

The second and third parts of the standard - ISO 9126-2 and ISO 9126-3 - are devoted to formalizing, respectively, external and internal metrics for the quality characteristics of complex software. All tables contain a unified heading, which reflects the name and purpose of the metric; method of its application; method of measurement, type of metric scale; type of measured quantity; baseline data for measurement and comparison; as well as the stages of the software life cycle (according to ISO 12207) to which the metric is applicable.

The fourth part of the standard - ISO 9126-4 - is intended for buyers, suppliers, developers, software support services, users and quality managers. It justifies and comments on the selected indicators of the scope of software use and the groups of selected metrics for users.

Software Quality Characteristics

Functionality	The ability of software to meet stated or anticipated user needs
Suitability for intended use	Availability and correspondence of a set of functions to specific tasks
Correctness/correctness of requirements implementation (Accuracy)	The ability of the software to ensure that the results are correct (or consistent)
The ability to interact with components and the environment (Interoperability)	The ability of software to interact with specific systems
Compliance	The ability of the software to adhere to relevant standards or conventions, or detailed guidelines
Security/functional safety (Security)	Software's ability to prevent unauthorized access(accidental or intentional) to programs and data
Reliability	The ability of software to maintain its level of performance quality when established conditions for a set period of time
Stability/level of completion (Maturity)	Characterized by the frequency of failures caused by errors in the software
Fault tolerance	The ability of software to maintain a certain level of quality of operation in cases of software errors or violation of a specific interface
Recoverability after occurrence of defects (Recoverability)	The ability of software to restore the level of performance and data directly damaged in the event of a failure. Characterized by the effort and time required for this
Usability	Characterized by the amount of work required to use the software by a specific or intended group of users
Understandability of functions and documentation	Describes the user's efforts to understand the overall logical concept of the software and its applicability
Learnability of processes of operation and application (Learnability)	Describes the user's efforts to learn how to use the software (e.g., operational control, input, output)
Ease of Use (Operability)	Characterizes the user's efforts to operate and operationally manage the software
Efficiency	Determined by the relationship between the level of quality of software operation and the amount of resources used under established conditions
Temporary efficiency of implementation of a set of programs (Time behavior)	Characterized by response time and speed of function execution
Resource behavior	Characterized by the amount of resources used and the duration of software use when performing a function
Maintainability	Characterizes the amount of work required to carry out specific changes (modifications)
Analysability	Describes the effort required to diagnose deficiencies or failure cases or determine components for modernization
Changeability of components and software packages (Changeability)	Characterizes the effort required to modify, eliminate a failure, or change operating conditions.
Stability	Characterizes the risk from unforeseen modification effects
Testability of changes during maintenance (Testability)	Describes the effort required to verify modified software
Portability	The ability of software to be transferred from one environment to another
Adaptability to environmental changes (Adaptability)	Characterizes the convenience of adapting software to various specific operating conditions, without using other actions or methods other than those intended for this in the software in question
Ease of installation / implementation / installation after migration (Installability)	Describes the effort required to implement software in a specific environment
Conformance	Ability of software to comply with mobility-related standards or conventions
Interchangeability of components when adjusting the software package (Replaceability)	Characterizes the simplicity and complexity of using this software instead of another specific software tool in the environment of this tool

Before considering standards governing aspects of software quality, it is necessary to first discuss general issues related to the quality of any type of product. Common issues include definitions and terminology in the field, basic quality concepts, the role of documentation in ensuring product quality, and the selection and application of international quality standards. Of course, the main standards in the field of quality have become the international standards of the ISO 9000 series, developed by the International Organization for Standardization. The following subsection addresses the general issues listed above in light of the ISO 9000 series of standards.

1.1 Fundamentals of the ISO 9000 series of standards

First, the ISO 9000 series of standards refers to all international standards developed by Technical Committee 176, Quality Management and Quality Assurance, of the International Organization for Standardization (ISO). The series currently contains all international standards numbered 9000 to 9004 (including all parts of ISO 9000 and ISO 9004), 10001 to 10020 (including all parts), and ISO 8402. The names of the main standards that make up this series are given below.

ISO 9000-1-94 Standards for quality management and quality assurance. Part 1. Guidelines for selection for use.

ISO 9000-2-93 Standards in the field of quality management and quality assurance. Part 2: General guidance for the application of ISO 9001, ISO 9002 and ISO 9003.

ISO 9000-3-91 Standards for quality management and quality assurance. Part 3: Guidelines for the application of ISO 9001 in software development, delivery and maintenance.

ISO 9000-4-93 Standards for quality management and quality assurance. Part 4: General Reliability Program Administration Guidelines.

ISO 9001-94 Quality systems. A model for quality assurance in design, development, production, installation and maintenance.

ISO 9002-94 Quality systems. Model for quality assurance in production, installation and maintenance.

ISO 9003-94 Quality systems. Model for quality assurance during finished product inspection and final testing.

ISO 9004-1-94 Quality management and elements of a quality system. Part 1: Guidelines.

ISO 9004-2-91 Quality management and elements of a quality system. Part 2: Service Guidelines.

ISO 9004-3-93 Quality management and elements of a quality system. Part 3: Guidelines for processed materials.

ISO 9004-4-93 Quality management and elements of a quality system. Part 4: Quality Improvement Guidelines.

ISO 10011-1-90 Quality systems. Inspection Guidelines. Part 1. Checks.

ISO 10011-2-91 Quality systems. Inspection Guidelines. Part 2. Qualification criteria for expert auditors of quality systems.

ISO 10011-3-91 Quality systems. Inspection Guidelines. Part 3: Administrative management of inspection programs.

ISO 10012-1-92 Quality assurance of measuring equipment. Requirements. Part 1. Systems for metrological support of measuring equipment.

ISO 10013 Quality guidelines. Development provisions. (At the publishing stage).

ISO 8402-94 Quality management and quality assurance. Dictionary.

The currently increased competition between organizations producing products, including software, leads to the establishment of more stringent requirements for the quality of these products. To be competitive, organizations must implement effective systems that lead to improved product quality and better satisfaction of their customers. Correctly formulated and complete customer requirements included in the technical specifications do not guarantee that these requirements will be fully satisfied, since there are shortcomings in the organization’s supply and support system. This consideration led to the development of standards related to quality systems and complementing customer requirements for products. The ISO 9000 series of international standards is intended to provide a common framework for quality system standards. A quality system is understood, according to ISO 8402, as a set of organizational structure, methods, processes and resources necessary for the overall management of the quality of products produced by an organization.

An organization's quality management system is those aspects of the overall management function used by an organization that define the product quality policy, the organization's objectives and responsibilities, and implement them through the means of planning, managing, ensuring and improving quality within the quality system. In addition to the purpose of the organization, the quality management system is influenced by the products it produces and the production methods characteristic of this organization. Due to the fact that the production methods of organizations operating even in the same field are different, and the goals of the organization are not always the same, the quality systems of these organizations do not coincide. The main objective of the quality management system is to improve systems and processes to improve product quality.

The ISO 9000 series of standards specify which elements should be included in a quality system, while the organization itself must implement them, taking into account the specific objectives, products and processes, as well as the specific methods used by the organization.

In addition, the guidelines and requirements of the ISO 9000 series of standards are expressed in terms of quality system objectives that must be achieved and do not prescribe how to achieve these objectives, leaving the choice of these methods to the management of the organization. The standards in this series distinguish requirements for quality systems from customer requirements for products. Requirements for quality systems are additional to the technical requirements for products. For example, ISO 12207 specifies the software development life cycle. Processes and quality models corresponding to the quality assurance process (2.3 of ISO 12207) are established by the ISO 9000 series of standards.

ISO 9000-1 identifies four general product categories covering all types of products supplied by any organization:

Technical means.

Software.

Processed materials.

The quality system requirements set out in the ISO 9000 series of international standards apply to all four general product categories, but the terminology and some provisions and aspects of quality management systems may vary. This is evident from the names of the standards ISO 9004 - 2 and ISO 9004 - 3. It should be noted that any organization offers products in at least two categories. For example, an organization engaged in software development additionally provides its customers with maintenance services for the developed software.

The purpose of the guidelines and requirements of the ISO 9000 series of international standards is to meet the requirements from the perspective of four aspects that are key to product quality.

1. Quality through identifying customer needs for products. The first aspect is quality through defining and upgrading products to meet market requirements and opportunities.

2. Quality through design. The second aspect is quality by building into products features that help them meet market demands and opportunities. In other words, quality by design is those design properties that affect the uninterrupted operation of a product under variable conditions of production and use.

3. Quality through consistent design. The third aspect is quality due to maintaining constant compliance with the design, realizing the characteristics included in the design.

4. Quality through maintenance. The fourth aspect is quality through maintenance of products during use as necessary to maintain the desired characteristics.

The ISO 9000 series of standards comprehensively provides general management guidance and external quality assurance requirements in four areas.

The ISO 9000 series of international standards are based on the understanding that all work is accomplished through processes (see Figure 1). Every process has input factors. The output of the process is the result - products, tangible and intangible. The process itself is (or should be) a value-adding transformation. Every process involves people and/or other resources to some extent. The output can be, for example, a program, a banking service, a finished (or intermediate) product of any major product category. There are possibilities to make measurements at the input, at various stages of the process, as well as at the output.

As shown in Fig. 2, inputs and outputs can be of several types: product-related (solid lines in Fig. 2) (for example, raw materials, finished product) and information-related (dashed lines) (for example, product requirements, information characteristics). This figure represents supplier processes with subsupplier and customer processes in a supply network. In the structure of this network, various input and output factors move in different directions. The term “products” here refers to all four main product categories.

Administrative quality management is carried out through process management in the organization. Process control has two sides:

management of the structure and functioning of the process itself, within which products or information are moved;

managing the quality of products or information within the structure.

Given the complex structure of most organizations, it is important to identify the main processes and to simplify and rank processes depending on the objectives of quality management. An example of a complex network of processes would be an organization developing software according to ISO/IEC 12207 and DO-178.

Fig.1.1 All work is performed using processes.

Processes

supplier

consumer

requirements

Input factors

Output factors

Status and characteristics

products

Status and characteristics

products

requirements

Feedback

subsupplier

Fig. 1.2 Interrelation of processes in the supply network in the presence of flows associated with products and information.

Every organization must define, establish, and manage its network of processes and interfaces. An organization creates, improves and ensures a consistent level of quality in its products through a network of processes. This is the conceptual basis of the ISO 9000 series of standards. Processes and their interfaces must be subject to analysis and continuous improvement in order to ensure the quality of the products produced.

When assessing any organization's quality systems, ISO 9000-1 recommends asking three important questions regarding each network process being assessed.

Are these processes defined and their procedures documented?

Are these processes fully applied and followed as documented?

Are these processes effective in achieving the expected results?

The outcome of the assessment is the sum of the answers to these questions, related respectively to the approach, application and result. Quality system assessments may vary in scope and involve different types of activities.

One of the most important types of such activities, carried out systematically, is the assessment of the status and adequacy of the quality system, carried out by the management of the organization in accordance with ISO 9001, 9002, 9003 standards. The conclusions made in the process of assessing the quality system should lead to an increase in its efficiency and economy. The source of information for such conclusions is also the results of internal and external audits of the quality system.

Internal quality audits conducted by the organization itself (the first party) provide information for effective management review and corrective, preventive and improvement actions.

External audits carried out by customers of products (second party) and independent bodies (third party) respectively ensure customer confidence in the supplier and obtaining a certificate, thereby ensuring confidence in a number of potential consumers of the organization's products.

Consideration must also be given to the situations in which the ISO 9000 series of standards may be applicable and how the supplier uses the series.

The ISO 9000 series of international standards are intended to apply to the following four situations.

1. As guidelines for quality administration. The quality system in this situation must improve its own efficiency in order to fulfill product quality requirements in an economical and optimal way.

2. In terms of concluding a contract between the first and second parties. In this situation, the consumer demands that certain elements and processes of the quality system become part of the supplier's quality system, while specifying a specific quality assurance model.

3. Upon approval or registration by a second party. This is the situation in which the quality system is assessed by the customer. The supplier can obtain official recognition that its products comply with the standard.

4. When certified or registered by a third party. In this situation, the quality system is assessed by the certification body, and the organization agrees to maintain such a quality system for all consumers of its products.

A supplier can choose either of two ways to use the ISO 9000 series of standards: the “management-motivated way” and the “stakeholder-motivated way”. The second method is considered the most common.

When using the stakeholder-motivated method, the supplier initially introduces a quality system as a response to direct customer requirements. The quality system must meet the requirements of ISO 9001, 9002, 9003 standards. The management of the organization plays a leading role in this method, but the driving force is the external stakeholder (customers).

In the management-driven approach, it is the management of the organization that begins to make efforts to determine future market needs and trends. Guidance for the initial establishment of a quality system that improves product quality is ISO 9004-1 (and other parts of ISO 9004). The supplier may then apply ISO 9001, 9002 or 9003 standards as a quality assurance model to demonstrate the adequacy of the quality system in order to obtain certification. The quality system implemented in this way is more capacious and fruitful than that implemented in the first way.

The ISO 9000 series of standards pays close attention to the preparation and use of documentation as a value-adding activity. Appropriate documentation plays a significant role in the following quality assurance activities:

in achieving the required product quality;

assessment of quality systems;

in improving quality;

in maintaining the achieved level of quality.

During internal and external audits, documentation of procedures indicates that processes are defined, procedures are approved and are under control. Only in these circumstances will audits ensure a full assessment of the adequacy of the application and execution of the organization's process network.

In addition, documentation plays an important role in improving product quality. If procedures are documented, applied and followed, it is possible to determine how they are performed.

Next, we will consider the ISO 9001 standard in more detail, which defines a model for ensuring quality in the design, development, production, installation and maintenance of all types of products, including software.

The quality of software (software) is determined based on the study of external and internal features of the product. External quality is determined by how it works in real time, how productive it is for users. The second feature focuses on internal aspects that depend on the quality of the code written. The user focuses more on how the software works at the external level, the quality of which can only be maintained if the specialist has written good program code.

Software quality

ISO 9126 product requirements

ISO 9126 is an international standard for software evaluation. It is divided into four parts, covering the following topics:

External indicators.
Internal indicators.
Quality model.
Software quality indicators.

The first part of ISO 9126 is an extension of the previous standard carried out by McCall (1977), Boehm (1978) and FURPS in defining a set of quality characteristics.

Functionality.
Reliability.
Usability.
Maintainability.
Portability.

Product functionality

Cost of analysis processes

The cost of quality is calculated by analyzing the costs of conformance and nonconformity. The price of the first indicator is related to:

Prevention costs. This is the amount spent to ensure that all methods are followed correctly. It includes training teams, code reviews and any other activities related to QA.

Assessment costs. This is the amount of money spent on planning all test tasks, and then on their implementation, for example, on developing test cases.

Non-compliance costs. These are costs that arise due to internal and external failures.

Internal failures are costs that occur when test cases are executed internally for the first time, with some failing. Costs arise when a programmer must correct all defects identified in his piece during unit or component testing.

External failures are costs that occur when a defect is installed by the customer rather than the tester. These costs are much greater than those that appear internally. This is especially true if a software glitch intensifies.

Disciplined Process Analysis

This is a processor evaluation of a software product. It involves identifying and characterizing current practices, identifying strengths and weaknesses, and the ability to control or avoid significant causes of poor product quality. Program audits can be of three types:

Self-esteem. Conducted within the organization's own personnel.

Third party assessment.

The software process audit is performed in an open general environment in order to improve its performance and using software quality assurance programs. The results of such an audit are confidential to the organization.

Setting up and optimizing operating systems