reliability engineering reliability engineering is a subdiscipline of systems engineering that emphasizes dependability in the lifecycle management of a product dependability or reliability describes the ability of a system or component to function under stated conditions for a specified period of time reliability is closely related to availability which is typically described as the ability of a component or system to function at a specified moment or interval of time reliability is theoretically defined as the probability of success reliability 1 probability of failure displaystyle textreliability1textprobability of failure as the frequency of failures or in terms of availability as a probability derived from reliability testability and maintainability testability maintainability and maintenance are often defined as a part of reliability engineering in reliability programs reliability plays a key role in the costeffectiveness of systems reliability engineering deals with the estimation prevention and management of high levels of lifetime engineering uncertainty and risks of failure although stochastic parameters define and affect reliability reliability is not solely achieved by mathematics and statistics for example it is easy to represent probability of failure as a symbol or value in an equation but it is almost impossible to predict its true magnitude in practice which is massively multivariate so having the equation for reliability does not begin to equal having an accurate predictive measurement of reliability reliability engineering relates closely to safety engineering and to system safety in that they use common methods for their analysis and may require input from each other reliability engineering focuses on costs of failure caused by system downtime cost of spares repair equipment personnel and cost of warranty claims safety engineering normally focuses more on preserving life and nature than on cost and therefore deals only with particularly dangerous systemfailure modes high reliability safety factor levels also result from good engineering and from attention to detail and almost never from only reactive failure management using reliability accounting and statistics history the word reliability can be traced back to 1816 and is first attested to the poet samuel taylor coleridge around the time that waloddi weibull was working on statistical models for fatigue the development of reliability engineering was here on a parallel path with quality the modern use of the word reliability was defined by the us military in the 1940s characterizing a product that would operate when expected and for a specified period of time in world war ii many reliability issues were due to the inherent unreliability of electronic equipment available at the time and to fatigue issues in 1945 ma miner published the seminal paper titled cumulative damage in fatigue in an asme journal a main application for reliability engineering in the military was for the vacuum tube as used in radar systems and other electronics for which reliability proved to be very problematic and costly the ieee formed the reliability society in 1948 in 1950 the united states department of defense formed group called the advisory group on the reliability of electronic equipment agree to investigate reliability methods for military equipment this group recommended three main ways of working improve component reliability establish quality and reliability requirements for suppliers collect field data and find root causes of failures in the 1960s more emphasis was given to reliability testing on component and system level the famous military standard 781 was created at that time around this period also the muchused and also muchdebated military handbook 217 was published by rca radio corporation of america and was used for the prediction of failure rates of components the emphasis on component reliability and empirical research eg mil std 217 alone slowly decreases more pragmatic approaches as used in the consumer industries are being used in the 1980s televisions were increasingly made up of solidstate semiconductors automobiles rapidly increased their use of semiconductors with a variety of microcomputers under the hood and in the dash large air conditioning systems developed electronic controllers as had microwave ovens and a variety of other appliances communications systems began to adopt electronics to replace older mechanical switching systems bellcore issued the first consumer prediction methodology for telecommunications and sae developed a similar document sae870050 for automotive applications the nature of predictions evolved during the decade and it became apparent that die complexity wasnt the only factor that determined failure rates for integrated circuits ics kam wong published a paper questioning the bathtub curvesee also reliabilitycentered maintenance during this decade the failure rate of many components dropped by a factor of 10 software became important to the reliability of systems by the 1990s the pace of ic development was picking up wider use of standalone microcomputers was common and the pc market helped keep ic densities following moores law and doubling about every 18 months reliability engineering now was more changing towards understanding the physics of failure failure rates for components kept on dropping but systemlevel issues became more prominent systems thinking became more and more important for software the ccm model capability maturity model was developed which gave a more qualitative approach to reliability iso 9000 added reliability measures as part of the design and development portion of certification the expansion of the worldwide web created new challenges of security and trust the older problem of too little reliability information available had now been replaced by too much information of questionable value consumer reliability problems could now have data and be discussed online in real time new technologies such as microelectromechanical systems mems handheld gps and handheld devices that combined cell phones and computers all represent challenges to maintain reliability product development time continued to shorten through this decade and what had been done in three years was being done in 18 months this meant that reliability tools and tasks must be more closely tied to the development process itself in many ways reliability became part of everyday life and consumer expectations overview objective the objectives of reliability engineering in decreasing order of priority are to apply engineering knowledge and specialist techniques to prevent or to reduce the likelihood or frequency of failures to identify and correct the causes of failures that do occur despite the efforts to prevent them to determine ways of coping with failures that do occur if their causes have not been corrected to apply methods for estimating the likely reliability of new designs and for analysing reliability data the reason for the priority emphasis is that it is by far the most effective way of working in terms of minimizing costs and generating reliable products the primary skills that are required therefore are the ability to understand and anticipate the possible causes of failures and knowledge of how to prevent them it is also necessary to have knowledge of the methods that can be used for analysing designs and data scope and techniques reliability engineering for complex systems requires a different more elaborate systems approach than for noncomplex systems reliability engineering may in that case involve system availability and mission readiness analysis and related reliability and maintenance requirement allocation functional system failure analysis and derived requirements specification inherent system design reliability analysis and derived requirements specification for both hardware and software design system diagnostics design fault tolerant systems eg by redundancy predictive and preventive maintenance eg reliabilitycentered maintenance human factors human interaction human errors manufacturing and assemblyinduced failures effect on the detected 0hour quality and reliability maintenanceinduced failures transportinduced failures storageinduced failures use load studies component stress analysis and derived requirements specification software systematic failures failure reliability testing and derived requirements field failure monitoring and corrective actions spare parts stocking availability control technical documentation caution and warning analysis data and information acquisitionorganisation creation of a general reliability development hazard log and fracas system chaos engineering effective reliability engineering requires understanding of the basics of failure mechanisms for which experience broad engineering skills and good knowledge from many different special fields of engineering for example tribology stress mechanics fracture mechanics fatigue thermal engineering fluid mechanics shockloading engineering electrical engineering chemical engineering eg corrosion material science definitions reliability may be defined in the following ways the idea that an item is fit for a purpose with respect to time the capacity of a designed produced or maintained item to perform as required over time the capacity of a population of designed produced or maintained items to perform as required over specified time the resistance to failure of an item over time the probability of an item to perform a required function under stated conditions for a specified period of time the durability of an object basics of a reliability assessment many engineering techniques are used in reliability risk assessments such as reliability hazard analysis failure mode and effects analysis fmea fault tree analysis fta reliability centered maintenance probabilistic load and material stress and wear calculations probabilistic fatigue and creep analysis human error analysis manufacturing defect analysis reliability testing etc it is crucial that these analysis are done properly and with much attention to detail to be effective because of the large number of reliability techniques their expense and the varying degrees of reliability required for different situations most projects develop a reliability program plan to specify the reliability tasks statement of work sow requirements that will be performed for that specific system consistent with the creation of a safety cases for example arp4761 the goal of reliability assessments is to provide a robust set of qualitative and quantitative evidence that use of a component or system will not be associated with unacceptable risk the basic steps to take are to first thoroughly identify relevant unreliability hazards eg potential conditions events human errors failure modes interactions failure mechanisms and root causes by specific analysis or tests assess the associated system risk by specific analysis or testing propose mitigation eg requirements design changes detection logic maintenance training by which the risks may be lowered and controlled for at an acceptable level determine the best mitigation and get agreement on final acceptable risk levels possibly based on costbenefit analysis risk here is the combination of probability and severity of the failure incident scenario occurring in a de minimis definition severity of failures includes the cost of spare parts manhours logistics damage secondary failures and downtime of machines which may cause production loss a more complete definition of failure also can mean injury dismemberment and death of people within the system witness mine accidents industrial accidents space shuttle failures and the same to innocent bystanders witness the citizenry of cities like bhopal love canal chernobyl or sendai and other victims of the 2011 thoku earthquake and tsunamiin this case reliability engineering becomes system safety what is acceptable is determined by the managing authority or customers or the affected communities residual risk is the risk that is left over after all reliability activities have finished and includes the unidentified riskand is therefore not completely quantifiable risk vs costcomplexity the complexity of the technical systems such as improvements of design and materials planned inspections foolproof design and backup redundancy decreases risk and increases the cost the risk can be decreased to alara as low as reasonably achievable or alapa as low as practically achievable levels reliability and availability program plan implementing a reliability program is not simply a software purchase its not just a checklist of items that must be completed that will ensure one has reliable products and processes a reliability program is a complex learning and knowledgebased system unique to ones products and processes it is supported by leadership built on the skills that one develops within a team integrated into business processes and executed by following proven standard work practices a reliability program plan is used to document exactly what best practices tasks methods tools analysis and tests are required for a particular subsystem as well as clarify customer requirements for reliability assessment for largescale complex systems the reliability program plan should be a separate document resource determination for manpower and budgets for testing and other tasks is critical for a successful program in general the amount of work required for an effective program for complex systems is large a reliability program plan is essential for achieving high levels of reliability testability maintainability and the resulting system availability and is developed early during system development and refined over the systems lifecycle it specifies not only what the reliability engineer does but also the tasks performed by other stakeholders a reliability program plan is approved by top program management which is responsible for allocation of sufficient resources for its implementation a reliability program plan may also be used to evaluate and improve the availability of a system by the strategy of focusing on increasing testability maintainability and not on reliability improving maintainability is generally easier than improving reliability maintainability estimates repair rates are also generally more accurate however because the uncertainties in the reliability estimates are in most cases very large they are likely to dominate the availability calculation prediction uncertainty problem even when maintainability levels are very high when reliability is not under control more complicated issues may arise like manpower maintainers customer service capability shortages spare part availability logistic delays lack of repair facilities extensive retrofit and complex configuration management costs and others the problem of unreliability may be increased also due to the domino effect of maintenanceinduced failures after repairs focusing only on maintainability is therefore not enough if failures are prevented none of the other issues are of any importance and therefore reliability is generally regarded as the most important part of availability reliability needs to be evaluated and improved related to both availability and the total cost of ownership tco due to cost of spare parts maintenance manhours transport costs storage cost part obsolete risks etc but as gm and toyota have belatedly discovered tco also includes the downstream liability costs when reliability calculations have not sufficiently or accurately addressed customers personal bodily risks often a tradeoff is needed between the two there might be a maximum ratio between availability and cost of ownership testability of a system should also be addressed in the plan as this is the link between reliability and maintainability the maintenance strategy can influence the reliability of a system eg by preventive andor predictive maintenance although it can never bring it above the inherent reliability the reliability plan should clearly provide a strategy for availability control whether only availability or also cost of ownership is more important depends on the use of the system for example a system that is a critical link in a production systemeg a big oil platformis normally allowed to have a very high cost of ownership if that cost translates to even a minor increase in availability as the unavailability of the platform results in a massive loss of revenue which can easily exceed the high cost of ownership a proper reliability plan should always address ramt analysis in its total context ramt stands for reliability availability maintainabilitymaintenance and testability in the context of the customers needs reliability requirements for any system one of the first tasks of reliability engineering is to adequately specify the reliability and maintainability requirements allocated from the overall availability needs and more importantly derived from proper design failure analysis or preliminary prototype test results clear requirements able to designed to should constrain the designers from designing particular unreliable items constructions interfaces systems setting only availability reliability testability or maintainability targets eg max failure rates is not appropriate this is a broad misunderstanding about reliability requirements engineering reliability requirements address the system itself including test and assessment requirements and associated tasks and documentation reliability requirements are included in the appropriate system or subsystem requirements specifications test plans and contract statements creation of proper lowerlevel requirements is critical provision of only quantitative minimum targets eg mtbf values or failure rates is not sufficient for different reasons one reason is that a full validation related to correctness and verifiability in time of a quantitative reliability allocation requirement spec on lower levels for complex systems can often not be made as a consequence of 1 the fact that the requirements are probabilistic 2 the extremely high level of uncertainties involved for showing compliance with all these probabilistic requirements and because 3 reliability is a function of time and accurate estimates of a probabilistic reliability number per item are available only very late in the project sometimes even after many years of inservice use compare this problem with the continuous rebalancing of for example lowerlevelsystem mass requirements in the development of an aircraft which is already often a big undertaking notice that in this case masses do only differ in terms of only some are not a function of time the data is nonprobabilistic and available already in cad models in case of reliability the levels of unreliability failure rates may change with factors of decades multiples of 10 as result of very minor deviations in design process or anything else the information is often not available without huge uncertainties within the development phase this makes this allocation problem almost impossible to do in a useful practical valid manner that does not result in massive over or underspecification a pragmatic approach is therefore neededfor example the use of general levels classes of quantitative requirements depending only on severity of failure effects also the validation of results is a far more subjective task than for any other type of requirement quantitative reliability parametersin terms of mtbfare by far the most uncertain design parameters in any design furthermore reliability design requirements should drive a system or part design to incorporate features that prevent failures from occurring or limit consequences from failure in the first place not only would it aid in some predictions this effort would keep from distracting the engineering effort into a kind of accounting work a design requirement should be precise enough so that a designer can design to it and can also provethrough analysis or testingthat the requirement has been achieved and if possible within some a stated confidence any type of reliability requirement should be detailed and could be derived from failure analysis finiteelement stress and fatigue analysis reliability hazard analysis fta fmea human factor analysis functional hazard analysis etc or any type of reliability testing also requirements are needed for verification tests eg required overload stresses and test time needed to derive these requirements in an effective manner a systems engineeringbased risk assessment and mitigation logic should be used robust hazard log systems must be created that contain detailed information on why and how systems could or have failed requirements are to be derived and tracked in this way these practical design requirements shall drive the design and not be used only for verification purposes these requirements often design constraints are in this way derived from failure analysis or preliminary tests understanding of this difference compared to only purely quantitative logistic requirement specification eg failure rate mtbf target is paramount in the development of successful complex systems the maintainability requirements address the costs of repairs as well as repair time testability not to be confused with test requirements requirements provide the link between reliability and maintainability and should address detectability of failure modes on a particular system level isolation levels and the creation of diagnostics procedures as indicated above reliability engineers should also address requirements for various reliability tasks and documentation during system development testing production and operation these requirements are generally specified in the contract statement of work and depend on how much leeway the customer wishes to provide to the contractor reliability tasks include various analyses planning and failure reporting task selection depends on the criticality of the system as well as cost a safetycritical system may require a formal failure reporting and review process throughout development whereas a noncritical system may rely on final test reports the most common reliability program tasks are documented in reliability program standards such as milstd785 and ieee 1332 failure reporting analysis and corrective action systems are a common approach for productprocess reliability monitoring reliability culture human errors human factors in practice most failures can be traced back to some type of human error for example in management decisions eg in budgeting timing and required tasks systems engineering use studies load cases systems engineering requirement analysis setting systems engineering configuration control assumptions calculations simulations fem analysis design design drawings testing eg incorrect load settings or failure measurement statistical analysis manufacturing quality control maintenance maintenance manuals training classifying and ordering of information feedback of field information eg incorrect or too vague etc however humans are also very good at detecting such failures correcting for them and improvising when abnormal situations occur therefore policies that completely rule out human actions in design and production processes to improve reliability may not be effective some tasks are better performed by humans and some are better performed by machines furthermore human errors in management the organization of data and information or the misuse or abuse of items may also contribute to unreliability this is the core reason why high levels of reliability for complex systems can only be achieved by following a robust systems engineering process with proper planning and execution of the validation and verification tasks this also includes careful organization of data and information sharing and creating a reliability culture in the same way that having a safety culture is paramount in the development of safety critical systems reliability prediction and improvement see also risk assessment quantitative risk assessment reliability prediction combines creation of a proper reliability model see further on this page estimation and justification of input parameters for this model eg failure rates for a particular failure mode or event and the mean time to repair the system for a particular failure estimation of output reliability parameters at system or part level ie system availability or frequency of a particular functional failure some recognized reliability engineering specialists eg patrick oconnor r barnard have argued that too much emphasis is often given to the prediction of reliability parameters instead more effort should be devoted to the prevention of failure reliability improvement for existing systems it is arguable that any attempt by a responsible program to correct the root cause of discovered failures may render the initial mtbf estimate invalid as new assumptions themselves subject to high error levels of the effect of this correction must be made another practical issue is the general unavailability of detailed failure data with those available often featuring inconsistent filtering of failure feedback data and ignoring statistical errors which are very high for rare events like reliability related failures very clear guidelines must be present to count and compare failures related to different type of rootcauses eg manufacturing maintenance transport systeminduced or inherent design failures comparing different types of causes may lead to incorrect estimations and incorrect business decisions about the focus of improvement to perform a proper quantitative reliability prediction for systems may be difficult and very expensive if done by testing at the individual partlevel reliability results can often be obtained with comparatively high confidence as testing of many sample parts might be possible using the available testing budget however unfortunately these tests may lack validity at a systemlevel due to assumptions made at partlevel testing these authors emphasized the importance of initial part or systemlevel testing until failure and to learn from such failures to improve the system or part the general conclusion is drawn that an accurate and absolute prediction by either fielddata comparison or testing of reliability is in most cases not possible an exception might be failures due to wearout problems such as fatigue failures in the introduction of milstd785 it is written that reliability prediction should be used with great caution if not used solely for comparison in tradeoff studies design for reliability design for reliability dfr is a process that encompasses tools and procedures to ensure that a product meets its reliability requirements under its use environment for the duration of its lifetime dfr is implemented in the design stage of a product to proactively improve product reliability dfr is often used as part of an overall design for excellence dfx strategy statisticsbased approach ie mtbf reliability design begins with the development of a system model reliability and availability models use block diagrams and fault tree analysis to provide a graphical means of evaluating the relationships between different parts of the system these models may incorporate predictions based on failure rates taken from historical data while the input data predictions are often not accurate in an absolute sense they are valuable to assess relative differences in design alternatives maintainability parameters for example mean time to repair mttr can also be used as inputs for such models the most important fundamental initiating causes and failure mechanisms are to be identified and analyzed with engineering tools a diverse set of practical guidance as to performance and reliability should be provided to designers so that they can generate lowstressed designs and products that protect or are protected against damage and excessive wear proper validation of input loads requirements may be needed in addition to verification for reliability performance by testing a fault tree diagram one of the most important design techniques is redundancy this means that if one part of the system fails there is an alternate success path such as a backup system the reason why this is the ultimate design choice is related to the fact that highconfidence reliability evidence for new parts or systems is often not available or is extremely expensive to obtain by combining redundancy together with a high level of failure monitoring and the avoidance of common cause failures even a system with relatively poor singlechannel part reliability can be made highly reliable at a system level up to mission critical reliability no testing of reliability has to be required for this in conjunction with redundancy the use of dissimilar designs or manufacturing processes eg via different suppliers of similar parts for single independent channels can provide less sensitivity to quality issues eg early childhood failures at a single supplier allowing veryhigh levels of reliability to be achieved at all moments of the development cycle from early life to longterm redundancy can also be applied in systems engineering by double checking requirements data designs calculations software and tests to overcome systematic failures another effective way to deal with reliability issues is to perform analysis that predicts degradation enabling the prevention of unscheduled downtime events failures rcm reliability centered maintenance programs can be used for this physicsoffailurebased approach for electronic assemblies there has been an increasing shift towards a different approach called physics of failure this technique relies on understanding the physical static and dynamic failure mechanisms it accounts for variation in load strength and stress that lead to failure with a high level of detail made possible with the use of modern finite element method fem software programs that can handle complex geometries and mechanisms such as creep stress relaxation fatigue and probabilistic design monte carlo simulationsdoe the material or component can be redesigned to reduce the probability of failure and to make it more robust against such variations another common design technique is component derating ie selecting components whose specifications significantly exceed the expected stress levels such as using heavier gauge electrical wire than might normally be specified for the expected electric current common tools and techniques many of the tasks techniques and analyses used in reliability engineering are specific to particular industries and applications but can commonly include physics of failure pof builtin selftest bit testability analysis failure mode and effects analysis fmea reliability hazard analysis reliability blockdiagram analysis dynamic reliability blockdiagram analysis fault tree analysis root cause analysis statistical engineering design of experiments eg on simulations fem models or with testing sneak circuit analysis accelerated testing reliability growth analysis reactive reliability weibull analysis for testing or mainly reactive reliability thermal analysis by finite element analysis fea and or measurement thermal induced shock and vibration fatigue analysis by fea and or measurement electromagnetic analysis avoidance of single point of failure spof functional analysis and functional failure analysis eg function fmea fha or ffa predictive and preventive maintenance reliability centered maintenance rcm analysis testability analysis failure diagnostics analysis normally also incorporated in fmea human error analysis operational hazard analysis preventativeplanned maintenance optimization pmo manual screening integrated logistics support results from these methods are presented during reviews of part or system design and logistics reliability is just one requirement among many for a complex part or system engineering tradeoff studies are used to determine the optimum balance between reliability requirements and other constraints the importance of language reliability engineers whether using quantitative or qualitative methods to describe a failure or hazard rely on language to pinpoint the risks and enable issues to be solved the language used must help create an orderly description of the functionitemsystem and its complex surrounding as it relates to the failure of these functionsitemssystems systems engineering is very much about finding the correct words to describe the problem and related risks so that they can be readily solved via engineering solutions jack ring said that a systems engineers job is to language the project ring et al 2000 than quantifying when a failure is likely to occur eg via determining mtbf to do this first the reliability hazards relating to the partsystem need to be classified and ordered based on some form of qualitative and quantitative logic if possible to allow for more efficient assessment and eventual improvement this is partly done in pure language and proposition logic but also based on experience with similar items this can for example be seen in descriptions of events in fault tree analysis fmea analysis and hazard tracking logs in this sense language and proper grammar part of qualitative analysis plays an important role in reliability engineering just like it does in safety engineering or ingeneral within systems engineering correct use of language can also be key to identifying or reducing the risks of human error which are often the root cause of many failures this can include proper instructions in maintenance manuals operation manuals emergency procedures and others to prevent systematic human errors that may result in system failures these should be written by trained or experienced technical authors using socalled simplified english or simplified technical english where words and structure are specifically chosen and created so as to reduce ambiguity or risk of confusion eg an replace the old part could ambiguously refer to a swapping a wornout part with a non wornout part or replacing a part with one using a more recent and hopefully improved design reliability modeling reliability modeling is the process of predicting or understanding the reliability of a component or system prior to its implementation two types of analysis that are often used to model a complete systems availability behavior including effects from logistics issues like spare part provisioning transport and manpower are fault tree analysis and reliability block diagrams at a component level the same types of analyses can be used together with others the input for the models can come from many sources including testing prior operational experience field data as well as data handbooks from similar or related industries regardless of source all model input data must be used with great caution as predictions are only valid in cases where the same product was used in the same context as such predictions are often only used to help compare alternatives a reliability block diagram showing a 1oo3 1 out of 3 redundant designed subsystem for part level predictions two separate fields of investigation are common the physics of failure approach uses an understanding of physical failure mechanisms involved such as mechanical crack propagation or chemical corrosion degradation or failure the parts stress modeling approach is an empirical method for prediction based on counting the number and type of components of the system and the stress they undergo during operation software reliability is a more challenging area that must be considered when computer code provides a considerable component of a systems functionality reliability theory reliability theory failure rate and survival analysis reliability is defined as the probability that a device will perform its intended function during a specified period of time under stated conditions mathematically this may be expressed as r t p r t t t f x d x displaystyle rtprttint tinfty fxdx where f x displaystyle fx is the failure probability density function and t displaystyle t is the length of the period of time which is assumed to start from time zero there are a few key elements of this definition reliability is predicated on intended function generally this is taken to mean operation without failure however even if no individual part of the system fails but the system as a whole does not do what was intended then it is still charged against the system reliability the system requirements specification is the criterion against which reliability is measured reliability applies to a specified period of time in practical terms this means that a system has a specified chance that it will operate without failure before time t displaystyle t reliability engineering ensures that components and materials will meet the requirements during the specified time note that units other than time may sometimes be used eg a mission operation cycles reliability is restricted to operation under stated or explicitly defined conditions this constraint is necessary because it is impossible to design a system for unlimited conditions a mars rover will have different specified conditions than a family car the operating environment must be addressed during design and testing that same rover may be required to operate in varying conditions requiring additional scrutiny two notable references on reliability theory and its mathematical and statistical foundations are barlow r e and proschan f 1982 and samaniego f j 2007 quantitative system reliability parameterstheory quantitative requirements are specified using reliability parameters the most common reliability parameter is the mean time to failure mttf which can also be specified as the failure rate this is expressed as a frequency or conditional probability density function pdf or the number of failures during a given period these parameters may be useful for higher system levels and systems that are operated frequently ie vehicles machinery and electronic equipment reliability increases as the mttf increases the mttf is usually specified in hours but can also be used with other units of measurement such as miles or cycles using mttf values on lower system levels can be very misleading especially if they do not specify the associated failures modes and mechanisms the f in mttf in other cases reliability is specified as the probability of mission success for example reliability of a scheduled aircraft flight can be specified as a dimensionless probability or a percentage as often used in system safety engineering a special case of mission success is the singleshot device or system these are devices or systems that remain relatively dormant and only operate once examples include automobile airbags thermal batteries and missiles singleshot reliability is specified as a probability of onetime success or is subsumed into a related parameter singleshot missile reliability may be specified as a requirement for the probability of a hit for such systems the probability of failure on demand pfd is the reliability measure this is actually an unavailability number the pfd is derived from failure rate a frequency of occurrence and mission time for nonrepairable systems for repairable systems it is obtained from failure rate meantimetorepair mttr and test interval this measure may not be unique for a given system as this measure depends on the kind of demand in addition to system level requirements reliability requirements may be specified for critical subsystems in most cases reliability parameters are specified with appropriate statistical confidence intervals reliability testing the purpose of reliability testing is to discover potential problems with the design as early as possible and ultimately provide confidence that the system meets its reliability requirements reliability testing may be performed at several levels and there are different types of testing complex systems may be tested at component circuit board unit assembly subsystem and system levels the test level nomenclature varies among applications for example performing environmental stress screening tests at lower levels such as piece parts or small assemblies catches problems before they cause failures at higher levels testing proceeds during each level of integration through fullup system testing developmental testing and operational testing thereby reducing program risk however testing does not mitigate unreliability risk with each test both a statistical type 1 and type 2 error could be made and depends on sample size test time assumptions and the needed discrimination ratio there is risk of incorrectly accepting a bad design type 1 error and the risk of incorrectly rejecting a good design type 2 error it is not always feasible to test all system requirements some systems are prohibitively expensive to test some failure modes may take years to observe some complex interactions result in a huge number of possible test cases and some tests require the use of limited test ranges or other resources in such cases different approaches to testing can be used such as highly accelerated life testing design of experiments and simulations the desired level of statistical confidence also plays a role in reliability testing statistical confidence is increased by increasing either the test time or the number of items tested reliability test plans are designed to achieve the specified reliability at the specified confidence level with the minimum number of test units and test time different test plans result in different levels of risk to the producer and consumer the desired reliability statistical confidence and risk levels for each side influence the ultimate test plan the customer and developer should agree in advance on how reliability requirements will be tested a key aspect of reliability testing is to define failure although this may seem obvious there are many situations where it is not clear whether a failure is really the fault of the system variations in test conditions operator differences weather and unexpected situations create differences between the customer and the system developer one strategy to address this issue is to use a scoring conference process a scoring conference includes representatives from the customer the developer the test organization the reliability organization and sometimes independent observers the scoring conference process is defined in the statement of work each test case is considered by the group and scored as a success or failure this scoring is the official result used by the reliability engineer as part of the requirements phase the reliability engineer develops a test strategy with the customer the test strategy makes tradeoffs between the needs of the reliability organization which wants as much data as possible and constraints such as cost schedule and available resources test plans and procedures are developed for each reliability test and results are documented reliability testing is common in the photonics industry examples of reliability tests of lasers are life test and burnin these tests consist of the highly accelerated aging under controlled conditions of a group of lasers the data collected from these life tests are used to predict laser life expectancy under the intended operating characteristics reliability test requirements reliability test requirements can follow from any analysis for which the first estimate of failure probability failure mode or effect needs to be justified evidence can be generated with some level of confidence by testing with softwarebased systems the probability is a mix of software and hardwarebased failures testing reliability requirements is problematic for several reasons a single test is in most cases insufficient to generate enough statistical data multiple tests or longduration tests are usually very expensive some tests are simply impractical and environmental conditions can be hard to predict over a systems lifecycle reliability engineering is used to design a realistic and affordable test program that provides empirical evidence that the system meets its reliability requirements statistical confidence levels are used to address some of these concerns a certain parameter is expressed along with a corresponding confidence level for example an mtbf of 1000 hours at 90 confidence level from this specification the reliability engineer can for example design a test with explicit criteria for the number of hours and number of failures until the requirement is met or failed different sorts of tests are possible the combination of required reliability level and required confidence level greatly affects the development cost and the risk to both the customer and producer care is needed to select the best combination of requirementseg costeffectiveness reliability testing may be performed at various levels such as component subsystem and system also many factors must be addressed during testing and operation such as extreme temperature and humidity shock vibration or other environmental factors like loss of signal cooling or power or other catastrophes such as fire floods excessive heat physical or security violations or other myriad forms of damage or degradation for systems that must last many years accelerated life tests may be needed accelerated testing the purpose of accelerated life testing alt test is to induce field failure in the laboratory at a much faster rate by providing a harsher but nonetheless representative environment in such a test the product is expected to fail in the lab just as it would have failed in the fieldbut in much less time the main objective of an accelerated test is either of the following to discover failure modes to predict the normal field life from the high stress lab life an accelerated testing program can be broken down into the following steps define objective and scope of the test collect required information about the product identify the stresses determine level of stresses conduct the accelerated test and analyze the collected data common ways to determine a life stress relationship are arrhenius model eyring model inverse power law model temperaturehumidity model temperature nonthermal model software reliability further information software reliability and site reliability engineering software reliability is a special aspect of reliability engineering system reliability by definition includes all parts of the system including hardware software supporting infrastructure including critical external interfaces operators and procedures traditionally reliability engineering focuses on critical hardware parts of the system since the widespread use of digital integrated circuit technology software has become an increasingly critical part of most electronics and hence nearly all present day systems there are significant differences however in how software and hardware behave most hardware unreliability is the result of a component or material failure that results in the system not performing its intended function repairing or replacing the hardware component restores the system to its original operating state however software does not fail in the same sense that hardware fails instead software unreliability is the result of unanticipated results of software operations even relatively small software programs can have astronomically large combinations of inputs and states that are infeasible to exhaustively test restoring software to its original state only works until the same combination of inputs and states results in the same unintended result software reliability engineering must take this into account despite this difference in the source of failure between software and hardware several software reliability models based on statistics have been proposed to quantify what we experience with software the longer software is run the higher the probability that it will eventually be used in an untested manner and exhibit a latent defect that results in a failure shooman 1987 musa 2005 denney 2005 as with hardware software reliability depends on good requirements design and implementation software reliability engineering relies heavily on a disciplined software engineering process to anticipate and design against unintended consequences there is more overlap between software quality engineering and software reliability engineering than between hardware quality and reliability a good software development plan is a key aspect of the software reliability program the software development plan describes the design and coding standards peer reviews unit tests configuration management software metrics and software models to be used during software development a common reliability metric is the number of software faults usually expressed as faults per thousand lines of code this metric along with software execution time is key to most software reliability models and estimates the theory is that the software reliability increases as the number of faults or fault density decreases or goes down establishing a direct connection between fault density and meantimebetweenfailure is difficult however because of the way software faults are distributed in the code their severity and the probability of the combination of inputs necessary to encounter the fault nevertheless fault density serves as a useful indicator for the reliability engineer other software metrics such as complexity are also used this metric remains controversial since changes in software development and verification practices can have dramatic impact on overall defect rates testing is even more important for software than hardware even the best software development process results in some software faults that are nearly undetectable until tested as with hardware software is tested at several levels starting with individual units through integration and fullup system testing unlike hardware it is inadvisable to skip levels of software testing during all phases of testing software faults are discovered corrected and retested reliability estimates are updated based on the fault density and other metrics at a system level meantimebetweenfailure data can be collected and used to estimate reliability unlike hardware performing exactly the same test on exactly the same software configuration does not provide increased statistical confidence instead software reliability uses different metrics such as code coverage eventually the software is integrated with the hardware in the toplevel system and software reliability is subsumed by system reliability the software engineering institutes capability maturity model is a common means of assessing the overall software development process for reliability and quality purposes comparison to safety engineering reliability engineering is concerned with overall minimisation of failures that could lead to financial losses for the responsible entity whereas safety engineering focuses on minimising a specific set of failure types that in general could lead to large scale widespread issues beyond the responsible entity reliability hazards could transform into incidents leading to a loss of revenue for the company or the customer for example due to direct and indirect costs associated with loss of production due to system unavailability unexpected high or low demands for spares repair costs manhours multiple redesigns interruptions to normal production etc safety engineering is often highly specific relating only to certain tightly regulated industries applications or areas it primarily focuses on system safety hazards that could lead to severe accidents including loss of life destruction of equipment or environmental damage as such the related system functional reliability requirements are often extremely high although it deals with unwanted failures in the same sense as reliability engineering it however has less of a focus on direct costs and is not concerned with postfailure repair actions another difference is the level of impact of failures on society leading to a tendency for strict control by governments or regulatory bodies eg nuclear aerospace defense rail and oil industries this can occasionally lead to safety engineering and reliability engineering having contradictory requirements or conflicting choices at a system architecture level for example in train signal control systems it is common practice to use a failsafe system design concept in this example a wrongside failure needs an extremely low failure rate as such failures can lead to such severe effects like frontal collisions of two trains where a signalling failure leads to two oncoming trains on the same track being given green lights such systems should be and thankfully are designed in a way that the vast majority of failures eg temporary or total loss of signals or open contacts of relays will generate red lights for all trains this is the safe state this means in the event of a failure all trains are stopped immediately this failsafe logic might unfortunately lower the reliability of the system the reason for this is the higher risk of false tripping as any failure whether temporary or not may be trigger such a safe but costly shutdown state different solutions can be applied for similar issues see the section on fault tolerance below fault tolerance fault tolerance reliability can be increased by using 1oo2 1 out of 2 redundancy at a part or system level however if both redundant elements disagree it can be difficult to know which is to be relied upon in the previous train signalling example this could lead to lower safety levels as there are more possibilities for allowing wrong side or other undetected dangerous failures faulttolerant systems often rely on additional redundancy eg 2oo3 voting logic where multiple redundant elements must agree on a potentially unsage action before it is performed this increases both reliability and safety at a system level and is often used for socalled operational or mission systems this is common practice in aerospace systems that need continued availability and do not have a failsafe mode for example aircraft may use triple modular redundancy for flight computers and control surfaces including occasionally different modes of operation eg electricalmechanicalhydraulic as these need to always be operational due to the fact that there are no safe default positions for control surfaces such as rudders or ailerons when the aircraft is flying basic reliability and mission operational reliability the above example of a 2oo3 fault tolerant system increases both mission reliability as well as safety however the basic reliability of the system will in this case still be lower than a nonredundant 1oo1 or 2oo2 system basic reliability engineering covers all failures including those that might not result in system failure but do result in additional cost due to maintenance repair actions logistics spare parts etc for example replacement or repair of 1 faulty channel in a 2oo3 voting system the system is still operating although with one failed channel it has actually become a 2oo2 system is contributing to basic unreliability but not mission unreliability as an example the failure of the taillight of an aircraft will not prevent the plane from flying and so is not considered a mission failure but it does need to be remedied with a related cost and so does contribute to the basic unreliability levels detectability and common cause failures when using fault tolerant redundant architectures systems or systems that are equipped with protection functions detectability of failures and avoidance of common cause failures becomes paramount for safe functioning andor mission reliability reliability versus quality six sigma six sigma has its roots in manufacturing reliability engineering is a specialty engineering part of systems engineering the systems engineering process is a discovery process that is quite unlike a manufacturing process a manufacturing process is focused on repetitive activities that achieve high quality outputs with minimum cost and time the systems engineering process must begin by discovering a real potential problem that needs to be solved the biggest failure that can be made in systems engineering is finding an elegant solution to the wrong problem or in terms of reliability providing elegant solutions to the wrong root causes of system failures the everyday usage term quality of a product is loosely taken to mean its inherent degree of excellence in industry a more precise definition of quality as conformance to requirements or specifications at the start of use is used assuming the final product specification adequately captures the original requirements and customersystem needs the quality level can be measured as the fraction of product units shipped that meet specifications variation of this static output may affect quality and reliability but this is not the total picture more inherent aspects may play a role and in some cases these may not be readily measured or controlled by any means at a part level microscopic material variations such as unavoidable microcracks and chemical impurities may over time due to physical or chemical loading become macroscopic defects at a system level systematic failures may play a dominant role eg requirement errors or software or software compiler or design flaws furthermore for more complex systems it should be questioned if derived or lowerlevel requirements and related product specifications are truly valid and correct will these result in premature failure due to excessive wear fatigue corrosion and debris accumulation or other issues such as maintenance induced failures are there any interactions at a system level as investigated by for example fault tree analysis how many of these systems still meet function and fulfill the needs after a week of operation what performance losses occurred did full system failure occur what happens after the end of a oneyear warranty period and what happens after 50 years a common lifetime for aircraft trains nuclear systems etc that is where reliability comes in these issues are far more complex and can not be controlled only by a standard quality six sigma way of working they need a systems engineering approach quality is a snapshot at the start of life and mainly related to control of lowerlevel product specifications this includes timezero defects ie where manufacturing mistakes escaped final quality control in theory the quality level might be described by a single fraction of defective products reliability as a part of systems engineering acts as more of an ongoing account of operational capabilities often over many years theoretically all items will fail over an infinite period of time some of these reliability issues may be due to inherent design issues which may exist even though the product conforms to specifications even items that are produced perfectly may fail over time due to one or more failure mechanisms eg due to human error or mechanical electrical and chemical factors these reliability issues can also be influenced by acceptable levels of variation during initial production quality is therefore related to manufacturing and reliability is more related to the validation of subsystem or lower item requirements system or part inherent design and life cycle solutions items that do not conform to any product specification will generally do worse in terms of reliability having a lower mttf but this does not always have to be the case the full mathematical quantification in statistical models of this combined relation is in general very difficult or even practically impossible in cases where manufacturing variances can be effectively reduced six sigma tools may be useful to find optimal process solutions which can increase reliability six sigma may also help to design products that are more robust to manufacturing induced failures in contrast with six sigma reliability engineering solutions are generally found by focusing on a system design and not on the manufacturing process solutions are found in different ways such as by simplifying a system to allow more of the mechanisms of failure involved to be understood performing detailed calculations of material stress levels allowing suitable safety factors to be determined finding possible abnormal system load conditions and using this to increase robustness of a design to manufacturing variance related failure mechanisms furthermore reliability engineering uses systemlevel solutions like designing redundant and faulttolerant systems for situations with high availability needs see reliability engineering vs safety engineering above sixsigma is also more quantified measurementbased the core of sixsigma is built on empirical research and statistical analysis eg to find transfer functions of directly measurable parameters this can not be translated practically to most reliability issues as reliability is not easily measurable due to being very much a function of time large times may be involved especially during the requirementsspecification and design phases where reliability engineering is the most efficient full quantification of reliability is in this phase extremely difficult or costly due to the amount of testing required it also may foster reactive management waiting for system failures to be measured before a decision can be taken furthermore as explained on this page reliability problems are likely to come from many different causes eg inherent failures human error systematic failures besides manufacturing induced defects note a defect in sixsigmaquality literature is not the same as a failure field failure eg fractured item in reliability a sixsigmaquality defect refers generally to nonconformance with a requirement eg basic functionality or a key dimension items can however fail over time even if these requirements are all fulfilled quality is generally not concerned with asking the crucial question are the requirements actually correct whereas reliability is within an entity departments related to quality ie concerning manufacturing six sigma ie concerning process control and reliability product design should provide input to each other to cover the complete risks more efficiently reliability operational assessment once systems or parts are being produced reliability engineering attempts to monitor assess and correct deficiencies monitoring includes electronic and visual surveillance of critical parameters identified during the fault tree analysis design stage data collection is highly dependent on the nature of the system most large organizations have quality control groups that collect failure data on vehicles equipment and machinery consumer product failures are often tracked by the number of returns for systems in dormant storage or on standby it is necessary to establish a formal surveillance program to inspect and test random samples any changes to the system such as field upgrades or recall repairs require additional reliability testing to ensure the reliability of the modification since it is not possible to anticipate all the failure modes of a given system especially ones with a human element failures will occur the reliability program also includes a systematic root cause analysis that identifies the causal relationships involved in the failure such that effective corrective actions may be implemented when possible system failures and corrective actions are reported to the reliability engineering organization some of the most common methods to apply to a reliability operational assessment are failure reporting analysis and corrective action systems fracas this systematic approach develops a reliability safety and logistics assessment based on failureincident reporting management analysis and correctivepreventive actions organizations today are adopting this method and utilizing commercial systems such as webbased fracas applications that enable them to create a failureincident data repository from which statistics can be derived to view accurate and genuine reliability safety and quality metrics it is extremely important for an organization to adopt a common fracas system for all end items also it should allow test results to be captured in a practical way failure to adopt one easytouse in terms of ease of dataentry for field engineers and repair shop engineers and easytomaintain integrated system is likely to result in a failure of the fracas program itself some of the common outputs from a fracas system include field mtbf mttr spares consumption reliability growth failureincidents distribution by type location part no serial no and symptom the use of past data to predict the reliability of new comparable systemsitems can be misleading as reliability is a function of the context of use and can be affected by small changes in designmanufacturing reliability organizations systems of any significant complexity are developed by organizations of people such as a commercial company or a government agency the reliability engineering organization must be consistent with the companys organizational structure for small noncritical systems reliability engineering may be informal as complexity grows the need arises for a formal reliability function because reliability is important to the customer the customer may even specify certain aspects of the reliability organization there are several common types of reliability organizations the project manager or chief engineer may employ one or more reliability engineers directly in larger organizations there is usually a product assurance or specialty engineering organization which may include reliability maintainability quality safety human factors logistics etc in such case the reliability engineer reports to the product assurance manager or specialty engineering manager in some cases a company may wish to establish an independent reliability organization this is desirable to ensure that the system reliability which is often expensive and timeconsuming is not unduly slighted due to budget and schedule pressures in such cases the reliability engineer works for the project daytoday but is actually employed and paid by a separate organization within the company because reliability engineering is critical to early system design it has become common for reliability engineers however the organization is structured to work as part of an integrated product team education some universities offer graduate degrees in reliability engineering other reliability engineers typically have an engineering degree which can be in any field of engineering from an accredited university or college program many engineering programs offer reliability courses and some universities have entire reliability engineering programs a reliability engineer may be registered as a professional engineer by the state but this is not required by most employers there are many professional conferences and industry training programs available for reliability engineers several professional organizations exist for reliability engineers including the american society for quality reliability division asqrd a group of engineers have provided a list of useful tools for reliability engineering these include relcalc software military handbook 217 milhdbk217 and the navmat p48551a manual analyzing failures and successes coupled with a quality standards process also provides systemized information to making informed engineering designs engineering portal dependability factor of safety failing badly failure mode and effects analysis fmea fracture mechanics highly accelerated life test highly accelerated stress test human reliability industrial engineering institute of industrial engineers logistic engineering performance engineering product qualification rams reliability availability and serviceability reliability theory of aging and longevity riskbased inspection security engineering software reliability testing solid mechanics spurious trip level strength of materials structural fracture mechanics temperature cycling n diaz r pascual f ruggeri e lpez droguett 2017 modeling age replacement policy under multiple time scales and stochastic usage profiles international journal of production economics 188 2228 doi101016jijpe201703009cs1 maint multiple names authors list link barlow r e and proscan f 1981 statistical theory of reliability and life testing to begin with press silver springs md blanchard benjamin s 1992 logistics engineering and management fourth ed prenticehall inc englewood cliffs new jersey breitler alan l and sloan c 2005 proceedings of the american institute of aeronautics and astronautics aiaa air force te days conference nashville tn december 2005 system reliability prediction towards a general approach using a neural network ebeling charles e 1997 an introduction to reliability and maintainability engineering mcgrawhill companies inc boston denney richard 2005 succeeding with use cases working smart to deliver quality addisonwesley professional publishing isbn discusses the use of software reliability engineering in use case driven software development gano dean l 2007 apollo root cause analysis third edition apollonian publications llc richland washington holmes oliver wendell sr the deacons masterpiece kapur kc and lamberson lr 1977 reliability in engineering design john wiley sons new york kececioglu dimitri 1991 reliability engineering handbook prenticehall englewood cliffs new jersey trevor kletz 1998 process plants a handbook for inherently safer design crc isbn1560326190 leemis lawrence 1995 reliability probabilistic models and statistical methods 1995 prenticehall isbn0137205171 frank lees 2005 loss prevention in the process industries 3rdedition ed elsevier isbn9780750675550 macdiarmid preston morris seymour et al 1995 reliability toolkit commercial practices edition reliability analysis center and rome laboratory rome new york modarres mohammad kaminskiy mark krivtsov vasiliy 1999 reliability engineering and risk analysis a practical guide crc press isbn0824720008 musa john 2005 software reliability engineering more reliable software faster and cheaper 2nd edition authorhouse isbn neubeck ken 2004 practical reliability analysis prentice hall new jersey neufelder ann marie 1993 ensuring software reliability marcel dekker inc new york oconnor patrick d t 2002 practical reliability engineering fourth ed john wiley sons new york isbn9780470844625 samaniego francisco j 2007 system signatures and their applications in engineering reliability spinger international series in operations research and management science new york shooman martin 1987 software engineering design reliability and management mcgrawhill new york tobias trindade 1995 applied reliability chapman hallcrc isbn0442004699 springer series in reliability engineering nelson wayne b 2004 accelerated testingstatistical models test plans and data analysis john wiley sons new york isbn0471697362 bagdonavicius v nikulin m 2002 accelerated life models modeling and statistical analysis chapmanhallcrc boca raton isbn1584881860 todinov m 2016 reliability and risk models setting reliability requirements wiley 9781118873328 us standards specifications and handbooks aerospace report number tor200785836889 reliability program requirements for space systems the aerospace corporation 10 jul 2007 dod 32351h 3rd ed test and evaluation of system reliability availability and maintainability a primer us department of defense march 1982 nasa gsfc 431ref000370 flight assurance procedure performing a failure mode and effects analysis national aeronautics and space administration goddard space flight center 10 aug 1996 ieee 13321998 ieee standard reliability program for the development and production of electronic systems and equipment institute of electrical and electronics engineers 1998 jpl d5703 reliability analysis handbook national aeronautics and space administration jet propulsion laboratory july 1990 milstd785b reliability program for systems and equipment development and production us department of defense 15 sep 1980 obsolete superseded by ansigeiastd00092008 titled reliability program standard for systems design development and manufacturing 13 nov 2008 milhdbk217f reliability prediction of electronic equipment us department of defense 2 dec 1991 milhdbk217f notice 1 reliability prediction of electronic equipment us department of defense 10 jul 1992 milhdbk217f notice 2 reliability prediction of electronic equipment us department of defense 28 feb 1995 milstd690d failure rate sampling plans and procedures us department of defense 10 jun 2005 milhdbk338b electronic reliability design handbook us department of defense 1 oct 1998 milhdbk2173 reliabilitycentered maintenance rcm requirements for naval aircraft weapon systems and support equipment us department of defense 30 jan 1998 superseded by navair 0025403 milstd1543b reliability program requirements for space and launch vehicles us department of defense 25 oct 1988 milstd1629a procedures for performing a failure mode effects and criticality analysis us department of defense 24 nov 1980 milhdbk781a reliability test methods plans and environments for engineering development qualification and production us department of defense 1 apr 1996 nswc06 part a b handbook of reliability prediction procedures for mechanical equipment naval surface warfare center 10 jan 2006 sr332 reliability prediction procedure for electronic equipment telcordia technologies january 2011 fdarpp01 automated reliability prediction procedure telcordia technologies january 2011 gr357 generic requirements for assuring the reliability of components used in telecommunications equipment telcordia technologies march 2001 httpstandardssaeorgja10001199903 sae ja10001 reliability program standard implementation guide uk standards in the uk there are more up to date standards maintained under the sponsorship of uk mod as defence standards the relevant standards include def stan 0040 reliability and maintainability rm part 1 issue 5 management responsibilities and requirements for programmes and plans part 4 armp4issue 2 guidance for writing nato rm requirements documents part 6 issue 1 inservice r m part 7 armp7 issue 1 nato rm terminology applicable to armps def stan 0042 reliability and maintainability assurance guides part 1 issue 1 oneshot devicessystems part 2 issue 1 software part 3 issue 2 rm case part 4 issue 1 testability part 5 issue 1 inservice reliability demonstrations def stan 0043 reliability and maintainability assurance activity part 2 issue 1 inservice maintainability demonstrations def stan 0044 reliability and maintainability data collection and classification part 1 issue 2 maintenance data defect reporting in the royal navy the army and the royal air force part 2 issue 1 data classification and incident sentencinggeneral part 3 issue 1 incident sentencingsea part 4 issue 1 incident sentencingland def stan 0045 issue 1 reliability centered maintenance def stan 0049 issue 1 reliability and maintainability mod guide to terminology definitions these can be obtained from dstan there are also many commercial standards produced by many organisations including the sae msg arp and iee french standards fides the fides methodology utec 80811 is based on the physics of failures and supported by the analysis of test data field returns and existing modelling utec 80810 or rdf2000 the rdf2000 methodology is based on the french telecom experience international standards tc 56 standards dependability media related to reliability engineering at wikimedia commons vtesystems engineeringsubfields aerospace engineering biological systems engineering configuration management earth systems engineering and management electrical engineering enterprise systems engineering performance engineering reliability engineering safety engineering processes requirements engineering functional specification system integration verification and validation design review concepts business process system system lifecycle vmodel systems development life cycle tools decisionmaking function modelling idef optimization quality function deployment system dynamics systems modeling language systems analysis systems modeling work breakdown structure people james s albus ruzena bajcsy benjamin s blanchard wernher von braun kathleen carley harold chestnut wolt fabrycky barbara grosz arthur david hall iii derek hitchins robert e machol radhika nagpal simon ramo joseph francis shea katia sycara manuela m veloso john n warfield related fields control engineering computer engineering industrial engineering operations research project management quality management risk management software engineering category vtesoftware engineeringfields computer programming requirements engineering software deployment software design software maintenance software testing systems analysis formal methods concepts data modeling enterprise architecture functional specification modeling language orthogonality programming paradigm software software archaeology software architecture software configuration management software development methodology software development process software quality software quality assurance software verification and validation structured analysis orientations agile aspectoriented object orientation ontology service orientation sdlc modelsdevelopmental agile eup executable uml incremental model iterative model prototype model rad up scrum spiral model vmodel waterfall model xp other spice cmmi data model er model function model information model metamodeling object model systems model view model languages idef uml usl sysml softwareengineers victor basili kent beck grady booch fred brooks barry boehm peter chen danese cooper ward cunningham tom demarco edsger w dijkstra delores m etter martin fowler adele goldstine margaret hamilton c a r hoare lois haibt mary jean harrold grace hopper watts humphrey michael a jackson ivar jacobson alan kay nancy leveson stephen j mellor bertrand meyer david parnas trygve reenskaug winston w royce james rumbaugh mary shaw peri tarr elaine weyuker niklaus wirth edward yourdon related fields computer science computer engineering project management risk management systems engineering category commons vteengineeringcivil architectural construction environmental earthquake geotechnical hydraulic mining structural transportation mechanical acoustical aerospace automotive marine mechatronics railway thermal electrical computer control electromechanics electronics microwaves power radio frequency telecommunications chemical biochemical biological molecular petroleum process reaction thermodynamics transport phenomena interdisciplinary audio biomedical ceramics engineering mathematics engineering mechanics engineering physics engineering science fire industrial information materials science metallurgy military nanotechnology nuclear optical photonics privacy robotics security systems glossaries engineering aerospace engineering civil engineering electrical and electronics engineering mechanical engineering structural engineering list of engineering branches categoryengineering engineering portal vtesystems sciencesystems types anatomical art biological complex complex adaptive conceptual coupled humanenvironment database dynamical ecological economic energy formal holarchic information legal measurement metric multiagent nervous nonlinear operating physical planetary political sensory social star writing concepts doubling time leverage points limiting factor negative feedback positive feedback theoretical fields chaos theory complex systems control theory cybernetics earth system science living systems sociotechnical system systemics urban metabolism worldsystems theory analysis biology dynamics ecology engineering neuroscience pharmacology psychology theory thinking systems scientists alexander bogdanov russell l ackoff william ross ashby ruzena bajcsy bla h bnthy gregory bateson anthony stafford beer richard e bellman ludwig von bertalanffy margaret boden kenneth e boulding murray bowen kathleen carley mary cartwright c west churchman manfred clynes george dantzig edsger w dijkstra heinz von foerster stephanie forrest jay wright forrester barbara grosz charles a s hall lydia kavraki james j kay faina m kirillova george klir allenna leonard edward norton lorenz niklas luhmann humberto maturana margaret mead donella meadows mihajlo d mesarovic james grier miller radhika nagpal howard t odum talcott parsons ilya prigogine qian xuesen anatol rapoport peter senge claude shannon katia sycara francisco varela manuela m veloso kevin warwick norbert wiener jennifer wilby anthony wilden applications systems theory in anthropology systems theory in archaeology systems theory in political science organizations list principia cybernetica category portal commons vtedesign outline portal designer disciplinescommunicationdesign advertising book design corporate design exhibit design film title design graphic design motion postage stamp design print design illustration information design instructional design news design photography retail design signage traffic sign design typography type design video design visual merchandising environmentaldesign architecture architectural lighting design building design passive solar ecological design environmental impact design garden design computeraided healthy community design hotel design interior architecture interior design eid keyline design landscape architecture sustainable landscape design spatial design urban design industrialdesign automotive design automotive suspension design cmf design corrugated box design electric guitar design furniture design sustainable hardware interface design motorcycle design packaging and labeling photographic lens design product design production design sensory design service design interactiondesign experience design eed game design level design video game design hardware interface design icon design immersive design information design sonic interaction design user experience design user interface design web design otherapplied arts public art design ceramic glass design fashion design costume design jewelry design floral design game art design property design scenic design sound design stageset lighting design textile design otherdesign engineering algorithm design boiler design conceptual design database design drug design electrical system design experimental design filter design job design integrated circuit design circuit design physical design power network design mechanism design nuclear weapon design nucleic acid design organization design process design processor design protein design research design social design software design spacecraft design strategic design systems design approaches activitycentered adaptive web affective brainstorming by committee by contract ck theory closure codesign conceptoriented configuration contextual continuous cradletocradle creative problemsolving creativity techniques critical design fiction defensive designbidbuild designbuild architectled domaindriven ecodesign energy neutral engineering design process probabilistic design errortolerant faulttolerant frameworkoriented for assembly for behaviour change for manufacturability for six sigma for testing for x functional generative geodesign highlevel integrated integrated topside intelligencebased iterative kiss principle lowlevel metadesign mind mapping modular new wave objectoriented open parametric participatory platformbased policybased processcentered public interest rational regenerative reliability engineering researchbased responsibilitydriven rwd safelife sustainable systemic sod tableless web theory of constraints topdown and bottomup transformation transgenerational triz universal design for all usagecentered usecentered usercentered empathic user innovation valuedriven value sensitive privacy by design choice computing controls flow leadership management marker methods pattern research science strategy theory thinking toolsintellectual propertyorganizationsawardstools aad architectural model blueprint comprehensive layout cad caid virtual home design software cautod design quality indicator electronic design automation flowchart mockup product design specification prototype sketch storyboard technical drawing web design program website wireframe intellectualproperty community design design around design patent fashion design copyright geschmacksmuster industrial design rights european union organizations aiga chartered society of designers design and industries association design council international forum design the design society design research society awards european design award german design award good design award chicago good design award japan graphex if product design award james dyson award prince philip designers prize related topics aesthetics agile concept art creative industries cultural icon design enterprise architecture futures studies innovation management intelligent design lean startup new product development ooda loop philosophy of design process simulation slow design steam fields unintelligent design visualization wicked problem design brief change classic director education elements and principles engineer firm history knowledge language life load museum paradigm rationale review specification studio technology commons wikibooks wikinews wikiquote wikisource wiktionary vtestatistics outline index descriptive statisticscontinuous datacenter mean arithmetic geometric harmonic median mode dispersion variance standard deviation coefficient of variation percentile range interquartile range shape central limit theorem moments skewness kurtosis lmoments count data index of dispersion summary tables grouped data frequency distribution contingency table dependence pearson productmoment correlation rank correlation spearmans rho kendalls tau partial correlation scatter plot graphics bar chart biplot box plot control chart correlogram fan chart forest plot histogram pie chart qq plot run chart scatter plot stemandleaf display radar chart data collectionstudy design population statistic effect size statistical power sample size determination missing data survey methodology sampling stratified cluster standard error opinion poll questionnaire controlled experiments design control optimal controlled trial randomized random assignment replication blocking interaction factorial experiment uncontrolled studies observational study natural experiment quasiexperiment statistical inferencestatistical theory population statistic probability distribution sampling distribution order statistic empirical distribution density estimation statistical model lp space parameter location scale shape parametric family likelihoodmonotone locationscale family exponential family completeness sufficiency statistical functional bootstrap u v optimal decision loss function efficiency statistical distance divergence asymptotics robustness frequentist inferencepoint estimation estimating equations maximum likelihood method of moments mestimator minimum distance unbiased estimators meanunbiased minimumvariance raoblackwellization lehmannscheff theorem median unbiased plugin interval estimation confidence interval pivot likelihood interval prediction interval tolerance interval resampling bootstrap jackknife testing hypotheses 1 2tails power uniformly most powerful test permutation test randomization test multiple comparisons parametric tests likelihoodratio wald score specific tests ztest normal students ttest ftest goodness of fit chisquared gtest kolmogorovsmirnov andersondarling lilliefors jarquebera normality shapirowilk likelihoodratio test model selection cross validation aic bic rank statistics sign sample median signed rank wilcoxon hodgeslehmann estimator rank sum mannwhitney nonparametric anova 1way kruskalwallis 2way friedman ordered alternative jonckheereterpstra bayesian inference bayesian probability prior posterior credible interval bayes factor bayesian estimator maximum posterior estimator correlationregression analysiscorrelation pearson productmoment partial correlation confounding variable coefficient of determination regression analysis errors and residuals regression model validation mixed effects models simultaneous equations models multivariate adaptive regression splines mars linear regression simple linear regression ordinary least squares general linear model bayesian regression nonstandard predictors nonlinear regression nonparametric semiparametric isotonic robust heteroscedasticity homoscedasticity generalized linear model exponential families logistic bernoulli binomial poisson regressions partition of variance analysis of variance anova anova analysis of covariance multivariate anova degrees of freedom categorical multivariate timeseries survival analysiscategorical cohens kappa contingency table graphical model loglinear model mcnemars test multivariate regression manova principal components canonical correlation discriminant analysis cluster analysis classification structural equation model factor analysis multivariate distributions elliptical distributions normal timeseriesgeneral decomposition trend stationarity seasonal adjustment exponential smoothing cointegration structural break granger causality specific tests dickeyfuller johansen qstatistic ljungbox durbinwatson breuschgodfrey time domain autocorrelation acf partial pacf crosscorrelation xcf arma model arima model boxjenkins autoregressive conditional heteroskedasticity arch vector autoregression var frequency domain spectral density estimation fourier analysis wavelet whittle likelihood survivalsurvival function kaplanmeier estimator product limit proportional hazards models accelerated failure time aft model first hitting time hazard function nelsonaalen estimator test logrank test applicationsbiostatistics bioinformatics clinical trials studies epidemiology medical statistics engineering statistics chemometrics methods engineering probabilistic design process quality control reliability system identification social statistics actuarial science census crime statistics demography econometrics national accounts official statistics population statistics psychometrics spatial statistics cartography environmental statistics geographic information system geostatistics kriging category portal commons wikiproject 