software quality in the context of software engineering software quality refers to two related but distinct notions that exist wherever quality is defined in a business context software functional quality reflects how well it complies with or conforms to a given design based on functional requirements or specifications that attribute can also be described as the fitness for purpose of a piece of software or how it compares to competitors in the marketplace as a worthwhile product it is the degree to which the correct software was produced software structural quality refers to how it meets nonfunctional requirements that support the delivery of the functional requirements such as robustness or maintainability it has a lot more to do with the degree to which the software works as needed many aspects of structural quality can be evaluated only statically through the analysis of the software inner structure its source code at the unit level the technology level and the system level which is in effect how its architecture adheres to sound principles of software architecture outlined in a paper on the topic by omg but some structural qualities such as usability can be assessed only dynamically users or others acting in their behalf interact with the software or at least some prototype or partial implementation even the interaction with a mock version made in cardboard represents a dynamic test because such version can be considered a prototype other aspects such as reliability might involve not only the software but also the underlying hardware therefore it can be assessed both statically and dynamically stress test functional quality is typically assessed dynamically but it is also possible to use static tests such as software reviews historically the structure classification and terminology of attributes and metrics applicable to software quality management have been derived or extracted from the iso 91263 and the subsequent iso 250002005 based on these models the consortium for it software quality cisq has defined five major desirable structural characteristics needed for a piece of software to provide business value reliability efficiency security maintainability and adequate size software quality measurement quantifies to what extent a software program or system rates along each of these five dimensions an aggregated measure of software quality can be computed through a qualitative or a quantitative scoring scheme or a mix of both and then a weighting system reflecting the priorities this view of software quality being positioned on a linear continuum is supplemented by the analysis of critical programming errors that under specific circumstances can lead to catastrophic outages or performance degradations that make a given system unsuitable for use regardless of rating based on aggregated measurements such programming errors found at the system level represent up to 90 of production issues whilst at the unitlevel even if far more numerous programming errors account for less than 10 of production issues as a consequence code quality without the context of the whole system as w edwards deming described it has limited value to view explore analyze and communicate software quality measurements concepts and techniques of information visualization provide visual interactive means useful in particular if several software quality measures have to be related to each other or to components of a software or system for example software maps represent a specialized approach that can express and combine information about software development software quality and system dynamics motivation a science is as mature as its measurement tools louis pasteur in ebert dumke p91 measuring software quality is motivated by at least two reasons risk management software failure has caused more than inconvenience software errors have caused human fatalities the causes have ranged from poorly designed user interfaces to direct programming errors an example of a programming error that led to multiple deaths is discussed in dr levesons paper cost management as in any other fields of engineering an application with good structural software quality costs less to maintain and is easier to understand and change in response to pressing business needs industry data demonstrate that poor application structural quality in core business applications such as enterprise resource planning erp customer relationship management crm or large transaction processing systems in financial services results in cost and schedule overruns and creates waste in the form of rework up to 45 of development time in some organizations moreover poor structural quality is strongly correlated with highimpact business disruptions due to corrupted data application outages security breaches and performance problems however the distinction between measuring and improving software quality in an embedded system with emphasis on risk management and software quality in business software with emphasis on cost and maintainability management is becoming somewhat irrelevant embedded systems now often include a user interface and their designers are as much concerned with issues affecting usability and user productivity as their counterparts who focus on business applications the latter are in turn looking at erp or crm system as a corporate nervous system whose uptime and performance are vital to the wellbeing of the enterprise this convergence is most visible in mobile computing a user who accesses an erp application on their smartphone is depending on the quality of software across all types of software layers both types of software now use multilayered technology stacks and complex architecture so software quality analysis and measurement have to be managed in a comprehensive and consistent manner decoupled from the softwares ultimate purpose or use in both cases engineers and management need to be able to make rational decisions based on measurement and factbased analysis in adherence to the precept in god we trust all others bring data misattributed to w edwards deming and others definitions there are many different definitions of quality for some it is the capability of a software product to conform to requirements isoiec 9001 while for others it can be synonymous with customer value highsmith 2002 or even defect level the first definition of quality history remembers is from shewhart in the beginning of 20th century there are two common aspects of quality one of them has to do with the consideration of the quality of a thing as an objective reality independent of the existence of man the other has to do with what we think feel or sense as a result of the objective reality in other words there is a subjective side of quality shewhart kitchenham pfleeger and garvins five perspectives on quality kitchenham and pfleeger identify five different perspectives on quality the transcendental perspective deals with the metaphysical aspect of quality in this view of quality it is something toward which we strive as an ideal but may never implement completely the user perspective is concerned with the appropriateness of the product for a given context of use whereas the transcendental view is ethereal the user view is more concrete grounded in the product characteristics that meet users needs the manufacturing perspective represents quality as conformance to requirements this aspect of quality is stressed by standards such as iso 9001 which defines quality as the degree to which a set of inherent characteristics fulfills requirements isoiec 9001 the product perspective implies that quality can be appreciated by measuring the inherent characteristics of the product the final perspective of quality is valuebased this perspective recognises that the different perspectives of quality may have different importance or value to various stakeholders software quality according to deming the problem inherent in attempts to define the quality of a product almost any product were stated by the master walter a shewhart the difficulty in defining quality is to translate future needs of the user into measurable characteristics so that a product can be designed and turned out to give satisfaction at a price that the user will pay this is not easy and as soon as one feels fairly successful in the endeavor he finds that the needs of the consumer have changed competitors have moved in etcw edwards deming software quality according to feigenbaum quality is a customer determination not an engineers determination not a marketing determination nor a general management determination it is based on the customers actual experience with the product or service measured against his or her requirements stated or unstated conscious or merely sensed technically operational or entirely subjective and always representing a moving target in a competitive market software quality according to juran the word quality has multiple meanings two of these meanings dominate the use of the word 1 quality consists of those product features which meet the need of customers and thereby provide product satisfaction 2 quality consists of freedom from deficiencies nevertheless in a handbook such as this it is convenient to standardize on a short definition of the word quality as fitness for use cisqs quality model even though quality is a perceptual conditional and somewhat subjective attribute and may be understood differently by different people as noted in the article on quality in business software structural quality characteristics have been clearly defined by the consortium for it software quality cisq under the guidance of bill curtis coauthor of the capability maturity model framework and cisqs first director and capers jones cisqs distinguished advisor cisq has defined five major desirable characteristics of a piece of software needed to provide business value in the house of quality model these are whats that need to be achieved reliability an attribute of resiliency and structural solidity reliability measures the level of risk and the likelihood of potential application failures it also measures the defects injected due to modifications made to the software its stability as termed by iso the goal for checking and monitoring reliability is to reduce and prevent application downtime application outages and errors that directly affect users and enhance the image of it and its impact on a companys business performance efficiency the source code and software architecture attributes are the elements that ensure high performance once the application is in runtime mode efficiency is especially important for applications in high execution speed environments such as algorithmic or transactional processing where performance and scalability are paramount an analysis of source code efficiency and scalability provides a clear picture of the latent business risks and the harm they can cause to customer satisfaction due to responsetime degradation security a measure of the likelihood of potential security breaches due to poor coding practices and architecture this quantifies the risk of encountering critical vulnerabilities that damage the business maintainability maintainability includes the notion of adaptability portability and transferability from one development team to another measuring and monitoring maintainability is a must for missioncritical applications where change is driven by tight timetomarket schedules and where it is important for it to remain responsive to businessdriven changes it is also essential to keep maintenance costs under control size while not a quality attribute per se the sizing of source code is a software characteristic that obviously impacts maintainability combined with the above quality characteristics software size can be used to assess the amount of work produced and to be done by teams as well as their productivity through correlation with timesheet data and other sdlcrelated metrics software functional quality is defined as conformance to explicitly stated functional requirements identified for example using voice of the customer analysis part of the design for six sigma toolkit andor documented through use cases and the level of satisfaction experienced by endusers the latter is referred as to as usability and is concerned with how intuitive and responsive the user interface is how easily simple and complex operations can be performed and how useful error messages are typically software testing practices and tools ensure that a piece of software behaves in compliance with the original design planned user experience and desired testability ie a piece of softwares disposition to support acceptance criteria the dual structuralfunctional dimension of software quality is consistent with the model proposed in steve mcconnells code complete which divides software characteristics into two pieces internal and external quality characteristics external quality characteristics are those parts of a product that face its users where internal quality characteristics are those that do not alternative approaches one of the challenges in defining quality is that everyone feels they understand it and other definitions of software quality could be based on extending the various descriptions of the concept of quality used in business dr tom demarco has proposed that a products quality is a function of how much it changes the world for the better this can be interpreted as meaning that functional quality and user satisfaction are more important than structural quality in determining software quality another definition coined by gerald weinberg in quality software management systems thinking is quality is value to some person this definition stresses that quality is inherently subjectivedifferent people will experience the quality of the same software differently one strength of this definition is the questions it invites software teams to consider such as who are the people we want to value our software and what will be valuable to them measurement although the concepts presented in this section are applicable to both structural and functional software quality measurement of the latter is essentially performed through testing introduction relationship between software desirable characteristics right and measurable attributes left software quality measurement is about quantifying to what extent a system or software possesses desirable characteristics this can be performed through qualitative or quantitative means or a mix of both in both cases for each desirable characteristic there are a set of measurable attributes the existence of which in a piece of software or system tend to be correlated and associated with this characteristic for example an attribute associated with portability is the number of targetdependent statements in a program more precisely using the quality function deployment approach these measurable attributes are the hows that need to be enforced to enable the whats in the software quality definition above the structure classification and terminology of attributes and metrics applicable to software quality management have been derived or extracted from the iso 91263 and the subsequent isoiec 250002005 quality model the main focus is on internal structural quality subcategories have been created to handle specific areas like business application architecture and technical characteristics such as data access and manipulation or the notion of transactions the dependence tree between software quality characteristics and their measurable attributes is represented in the diagram on the right where each of the 5 characteristics that matter for the user right or owner of the business system depends on measurable attributes left application architecture practices coding practices application complexity documentation portability technical and functional volume correlations between programming errors and production defects unveil that basic code errors account for 92 of the total errors in the source code these numerous codelevel issues eventually count for only 10 of the defects in production bad software engineering practices at the architecture levels account for only 8 of total defects but consume over half the effort spent on fixing problems and lead to 90 of the serious reliability security and efficiency issues in production codebased analysis many of the existing software measures count structural elements of the application that result from parsing the source code for such individual instructions park 1992 software quality measurement is about quantifying to what extent a system or software rates along these dimensions the analysis can be performed using a qualitative or quantitative approach or a mix of both to provide an aggregate view this view of software quality on a linear continuum has to be supplemented by the identification of discrete critical programming errors these vulnerabilities may not fail a test case but they are the result of bad practices that under specific circumstances can lead to catastrophic outages performance degradations security breaches corrupted data and myriad other problems nygard 2007 a repository of vulnerabilities in the source code that make applications exposed to security breaches the measurement of critical application characteristics involves measuring structural attributes of the applications architecture coding and inline documentation as displayed in the picture above thus each characteristic is affected by attributes at numerous levels of abstraction in the application and all of which must be included calculating the characteristics measure if it is to be a valuable predictor of quality outcomes that affect the business the layered approach to calculating characteristic measures displayed in the figure above was first proposed by boehm and his colleagues at trw boehm 1978 and is the approach taken in the iso 9126 and 25000 series standards these attributes can be measured from the parsed results of a static analysis of the application source code even dynamic characteristics of applications such as reliability and performance efficiency have their causal roots in the static structure of the application structural quality analysis and measurement is performed through the analysis of the source code the architecture software framework database schema in relationship to principles and standards that together define the conceptual and logical architecture of a system this is distinct from the basic local componentlevel code analysis typically performed by development tools which are mostly concerned with implementation considerations and are crucial during debugging and testing activities reliability the root causes of poor reliability are found in a combination of noncompliance with good architectural and coding practices this noncompliance can be detected by measuring the static quality attributes of an application assessing the static attributes underlying an applications reliability provides an estimate of the level of business risk and the likelihood of potential application failures and defects the application will experience when placed in operation assessing reliability requires checks of at least the following software engineering best practices and technical attributes application architecture practices coding practices complexity of algorithms complexity of programming practices compliance with objectoriented and structured programming best practices when applicable component or pattern reuse ratio dirty programming error exception handling for all layers gui logic data multilayer design compliance resource bounds management software avoids patterns that will lead to unexpected behaviors software manages data integrity and consistency transaction complexity level depending on the application architecture and the thirdparty components used such as external libraries or frameworks custom checks should be defined along the lines drawn by the above list of best practices to ensure a better assessment of the reliability of the delivered software efficiency as with reliability the causes of performance inefficiency are often found in violations of good architectural and coding practice which can be detected by measuring the static quality attributes of an application these static attributes predict potential operational performance bottlenecks and future scalability problems especially for applications requiring high execution speed for handling complex algorithms or huge volumes of data assessing performance efficiency requires checking at least the following software engineering best practices and technical attributes application architecture practices appropriate interactions with expensive andor remote resources data access performance and data management memory network and disk space management coding practices compliance with objectoriented and structured programming best practices as appropriate compliance with sql programming best practices security most security vulnerabilities result from poor coding and architectural practices such as sql injection or crosssite scripting these are well documented in lists maintained by cwe and the seicomputer emergency center cert at carnegie mellon university assessing security requires at least checking the following software engineering best practices and technical attributes application architecture practices multilayer design compliance security best practices input validation sql injection crosssite scripting etc programming practices code level error exception handling security best practices system functions access access control to programs maintainability maintainability includes concepts of modularity understandability changeability testability reusability and transferability from one development team to another these do not take the form of critical issues at the code level rather poor maintainability is typically the result of thousands of minor violations with best practices in documentation complexity avoidance strategy and basic programming practices that make the difference between clean and easytoread code vs unorganized and difficulttoread code assessing maintainability requires checking the following software engineering best practices and technical attributes application architecture practices architecture programs and code documentation embedded in source code code readability complexity level of transactions complexity of algorithms complexity of programming practices compliance with objectoriented and structured programming best practices when applicable component or pattern reuse ratio controlled level of dynamic coding coupling ratio dirty programming documentation hardware os middleware software components and database independence multilayer design compliance portability programming practices code level reduced duplicate code and functions source code file organization cleanliness maintainability is closely related to ward cunninghams concept of technical debt which is an expression of the costs resulting of a lack of maintainability reasons for why maintainability is low can be classified as reckless vs prudent and deliberate vs inadvertent size measuring software size requires that the whole source code be correctly gathered including database structure scripts data manipulation source code component headers configuration files etc there are essentially two types of software sizes to be measured the technical size footprint and the functional size there are several software technical sizing methods that have been widely described the most common technical sizing method is number of lines of code loc per technology number of files functions classes tables etc from which backfiring function points can be computed the most common for measuring functional size is function point analysis function point analysis measures the size of the software deliverable from a users perspective function point sizing is done based on user requirements and provides an accurate representation of both size for the developerestimator and value functionality to be delivered and reflects the business functionality being delivered to the customer the method includes the identification and weighting of user recognizable inputs outputs and data stores the size value is then available for use in conjunction with numerous measures to quantify and to evaluate software delivery and performance development cost per function point delivered defects per function point function points per staff month the function point analysis sizing standard is supported by the international function point users group ifpug it can be applied early in the software development lifecycle and it is not dependent on lines of code like the somewhat inaccurate backfiring method the method is technology agnostic and can be used for comparative analysis across organizations and across industries since the inception of function point analysis several variations have evolved and the family of functional sizing techniques has broadened to include such sizing measures as cosmic nesma use case points fp lite early and quick fps and most recently story points however function points has a history of statistical accuracy and has been used as a common unit of work measurement in numerous application development management adm or outsourcing engagements serving as the currency by which services are delivered and performance is measured one common limitation to the function point methodology is that it is a manual process and therefore it can be laborintensive and costly in large scale initiatives such as application development or outsourcing engagements this negative aspect of applying the methodology may be what motivated industry it leaders to form the consortium for it software quality focused on introducing a computable metrics standard for automating the measuring of software size while the ifpug keep promoting a manual approach as most of its activity rely on fp counters certifications cisq announced the availability of its first metric standard automated function pointsto the cisq membership in cisq technical these recommendations have been developed in omgs request for comment format and submitted to omgs process for standardization identifying critical programming errors critical programming errors are specific architectural andor coding bad practices that result in the highest immediate or long term business disruption risk these are quite often technologyrelated and depend heavily on the context business objectives and risks some may consider respect for naming conventions while others those preparing the ground for a knowledge transfer for example will consider it as absolutely critical critical programming errors can also be classified per cisq characteristics basic example below reliability avoid software patterns that will lead to unexpected behavior uninitialized variable null pointers etc methods procedures and functions doing insert update delete create table or select must include error management multithread functions should be made thread safe for instance servlets or struts action classes must not have instancenonfinal static fields efficiency ensure centralization of client requests incoming and data to reduce network traffic avoid sql queries that dont use an index against large tables in a loop security avoid fields in servlet classes that are not final static avoid data access without including error management check control return codes and implement error handling mechanisms ensure input validation to avoid crosssite scripting flaws or sql injections flaws maintainability deep inheritance trees and nesting should be avoided to improve comprehensibility modules should be loosely coupled fanout intermediaries to avoid propagation of modifications enforce homogeneous naming conventions operationalized quality models newer proposals for quality models such as squale and quamoco propagate a direct integration of the definition of quality attributes and measurement by breaking down quality attributes or even defining additional layers the complex abstract quality attributes such as reliability or maintainability become more manageable and measurable those quality models have been applied in industrial contexts but have not received widespread adoption isoiec 9126 software process improvement and capability determination isoiec 15504 software testing quality quality control total quality management software quality assurance programming style coding conventions software architecture software quality control software metrics cyclomatic complexity cohesion and coupling software reusability software standard accessibility availability dependability testability security security engineering computer bug best coding practices anomaly in software international organization for standardization software engineeringproduct qualitypart 1 quality model iso geneva switzerland 2001 isoiec 912612001e diomidis spinellis code quality the open source perspective addison wesley boston ma 2006 howon jung seunggweon kim and changsin chung measuring software product quality a survey of isoiec 9126 ieee software 2151013 septemberoctober 2004 stephen h kan metrics and models in software quality engineering addisonwesley boston ma second edition 2002 omar alshathry helge janicke optimizing software quality assurance compsacw pp8792 2010 ieee 34th annual computer software and applications conference workshops 2010 robert l glass building quality software prentice hall upper saddle river nj 1992 roland petrasch the definition of software quality a practical approach issre 1999 capers jones and olivier bonsignour the economics of software quality addisonwesley professional 1st edition december 31 2011 isbn9780132582209 measuring software product quality the iso 25000 series and cmmi sei site msqf a measurement based software quality framework cornell university library stefan wagner software product quality control springer 2013 girish suryanarayana software process versus design quality tug of war association of maritime managers in information technology communications ammitec maritime software quality guidelines september 2017 notes bibliography albrecht a j 1979 measuring application development productivity in proceedings of the joint shareguide ibm applications development symposium ibm benmenachem m marliss g s 1997 software quality producing practical and consistent software thomson computer press boehm b brown jr kaspar h lipow m macleod gj merritt mj 1978 characteristics of software quality northholland chidamber s kemerer c 1994 a metrics suite for object oriented design ieee transactions on software engineering 20 6 pp476493 ebert christof dumke reiner software measurement establish extract evaluate execute kindle edition p91 garmus d herron d 2001 function point analysis addison wesley halstead me 1977 elements of software science elsevier northholland hamill m gosevapopstojanova k 2009 common faults in software fault and failure data ieee transactions of software engineering 35 4 pp484496 jackson dj 2009 a direct path to dependable software communications of the acm 52 4 martin r 2001 managing vulnerabilities in networked systems ieee computer mccabe t december 1976 a complexity measure ieee transactions on software engineering mcconnell steve 1993 code complete first ed microsoft press nygard mt 2007 release it design and deploy production ready software the pragmatic programmers park re 1992 software size measurement a framework for counting source statements cmusei92tr020 software engineering institute carnegie mellon university pressman roger s 2005 software engineering a practitioners approach sixth international ed mcgrawhill education isbn0071267824 spinellis d 2006 code quality addison wesley wikimedia commons has media related to software quality linux fewer bugs than rivals wired magazine 2004 automated function points beta 1 by omg vtesoftware engineeringfields computer programming requirements engineering software deployment software design software maintenance software testing systems analysis formal methods concepts data modeling enterprise architecture functional specification modeling language orthogonality programming paradigm software software archaeology software architecture software configuration management software development methodology software development process software quality software quality assurance software verification and validation structured analysis orientations agile aspectoriented object orientation ontology service orientation sdlc modelsdevelopmental agile eup executable uml incremental model iterative model prototype model rad up scrum spiral model vmodel waterfall model xp other spice cmmi data model er model function model information model metamodeling object model systems model view model languages idef uml usl sysml softwareengineers victor basili kent beck grady booch fred brooks barry boehm peter chen danese cooper ward cunningham tom demarco edsger w dijkstra delores m etter martin fowler adele goldstine margaret hamilton c a r hoare lois haibt mary jean harrold grace hopper watts humphrey michael a jackson ivar jacobson alan kay nancy leveson stephen j mellor bertrand meyer david parnas trygve reenskaug winston w royce james rumbaugh mary shaw peri tarr elaine weyuker niklaus wirth edward yourdon related fields computer science computer engineering project management risk management systems engineering category commons vtemajor fields of computer sciencenote this template roughly follows the 2012 acm computing classification systemhardware printed circuit board peripheral integrated circuit very large scale integration systems on chip socs energy consumption green computing electronic design automation hardware acceleration computer systemsorganization computer architecture embedded system realtime computing dependability networks network architecture network protocol network components network scheduler network performance evaluation network service software organization interpreter middleware virtual machine operating system software quality software notationsand tools programming paradigm programming language compiler domainspecific language modeling language software framework integrated development environment software configuration management software library software repository software development software development process requirements analysis software design software construction software deployment software maintenance programming team opensource model theory of computation model of computation formal language automata theory computational complexity theory logic semantics algorithms algorithm design analysis of algorithms algorithmic efficiency randomized algorithm computational geometry mathematicsof computing discrete mathematics probability statistics mathematical software information theory mathematical analysis numerical analysis informationsystems database management system information storage systems enterprise information system social information systems geographic information system decision support system process control system multimedia information system data mining digital library computing platform digital marketing world wide web information retrieval security cryptography formal methods security services intrusion detection system hardware security network security information security application security humancomputerinteraction interaction design social computing ubiquitous computing visualization accessibility concurrency concurrent computing parallel computing distributed computing multithreading multiprocessing artificialintelligence natural language processing knowledge representation and reasoning computer vision automated planning and scheduling search methodology control method philosophy of artificial intelligence distributed artificial intelligence machine learning supervised learning unsupervised learning reinforcement learning multitask learning crossvalidation graphics animation rendering image manipulation graphics processing unit mixed reality virtual reality image compression solid modeling appliedcomputing ecommerce enterprise software computational mathematics computational physics computational chemistry computational biology computational social science computational engineering computational healthcare digital art electronic publishing cyberwarfare electronic voting video games word processing operations research educational technology document management book category portal wikiproject commons 