Quality Assurance ----------------------------------------- slide number 1 Quality Assurance ______________________________ Outline Motivation Background Broad Picture of QA Software Quality Assurance QA Standards Quality Assurance Plan Quality Assurance ----------------------------------------- slide number 2 What Is Quality 1qual-i-ty 'kwa^:l-et-e^- pl -ties [ME qualite, fr. OF qualite', fr. L qualitat-, qualitas, fr. qualis of what kind ] 2a: degree of excellence: GRADE 2b: superiority in kind "Quality is the relationship between what I expected and what I got." "[Q]uality has much in common with sex. Everyone is for it. (Under certain conditions, of course.) Everyone feels they understand it. (Even though they wouldn't want to explain it.) Everyone thinks it is only a matter of following natural inclinations. (After all, we do get along somehow.) And, of course, most people feel that problems in these areas are caused by other people. (If only they would take the time to do things right.)" - P. Crosby, Quality is Free, 1979 Quality Assurance ----------------------------------------- slide number 3 What Is Quality Assurance Reexamine preceding quote: "Quality is the relationship between what I expected and what I got." That is, how can I -a-s-s-u-r-e-that I get what I expect. What QA is there for: * slot machine * cooking * manufactured item * software Quality Assurance ----------------------------------------- slide number 4 Why Q.A. ? External Factors * competitive pressure * product evaluation commonplace * poor quality destroys an organization's reputation * customer demand * regulatory requirement Internal Factors * fixing a problem after completion of a project is 100 times more expensive than catching the problem during specification and design * for every dollar spent for development of software, two dollars are spend on maintenance Professionalism * professional obligation to produce quality software Quality Assurance ----------------------------------------- slide number 5 Increasing Cost of (Low) Quality defect found at impact origin step very minor next step minor delay end of line rework rescheduled work final inspection significant rework delay in delivery additional inspection end user's hand liabilities warranty cost administrative cost damage to reputation loss of market share Quality Assurance ----------------------------------------- slide number 6 QA Yields Improvements Gain control: OMITTED figure Increase capability: OMITTED figure Quality Assurance ----------------------------------------- slide number 7 Uses of Quality Measurement diagram Quality Assurance ----------------------------------------- slide number 8 Nature of Faults Error (cause) V Defect (location) V Fault (occurrence) V Failure (appearance) V Accident (result) Quality Assurance ----------------------------------------- slide number 9 Reliability Terminology MTTF = mean time to failure MTTR = mean time to repair Availability = MTTF / (MTTF + MTTR) Quality Assurance ---------------------------------------- slide number 10 Early Quality Assurance Efforts * applied to manufacturing in the early 20th Century through production monitoring. * used statistical methodologies to predict undiagnosed failure rates and locate defect "hot spots." Quality Assurance ---------------------------------------- slide number 11 QA for Hardware Hardware in the sense of manufactured objects Steps in assuring quality of a toaster: 1. test toaster at the end of the assembly line V many throw-aways or re-works V 2. test toaster components during assembly V assembly line bottle-necks at some failure-prone step V 3. monitor & measure assembly process itself hence QA plan is mostly concerned with process, not product Quality Assurance ---------------------------------------- slide number 12 QA Agents Traditional QA relies on: Technical Interpersonal Measurement Objectives Monitoring Evaluation Feedback Empowerment Management Support Quality Assurance ---------------------------------------- slide number 13 Industrial Quality Continuum * Deming Prize * Baldrige Award + QS 9000 (US auto industry) + ISO 9000 * Military standards o Meets specifications - No quality system Quality Assurance ---------------------------------------- slide number 14 ISO 9001/9002 Elements 4.1 Management Responsibility 4.2 Quality System 4.3 Contract Review 4.4 Design Control 4.5 Document & Data Control 4.6 Purchasing 4.7 Customer Supplied Components 4.8 Product Identification & Traceability 4.9 Process Control 4.10 Inspection & Testing 4.11 Control of Test Equipment 4.12 Inspection & Test Status 4.13 Control of Non-conforming Product 4.14 Corrective & Preventive Action 4.15 Handling, Packaging, etc. 4.16 Control of Quality Records 4.17 Internal Quality Audits 4.18 Training 4.19 Servicing 4.20 Statistical Techniques Quality Assurance ---------------------------------------- slide number 15 Software versus Hardware QA Comparing manufactured products with software: manufactured products software internal variability variability in external environment continuous behavior discrete basically additive highly non-linear scalability issue manufacturability complexity issue issue Quality Assurance ---------------------------------------- slide number 16 Characterizations "[S]oftware quality is the whole issue of software engineering." - A. Macro, 1990 "[Software quality is] conformance to explicitly stated functional and performance requirements, explicitly documented development standards, and implicit characteristics that are expected of all professionally developed software." - Pressman, 1987 Quality Assurance ---------------------------------------- slide number 17 Special Characteristics of SQA Process * methods and tools * configuration management and change control * standards and standard compliance mechanisms * reporting mechanisms Verification and Validation * formal technical reviews * multi-tiered testing strategy * measurement o symbolic execution => formal verification techniques Quality Assurance ---------------------------------------- slide number 18 Software Quality Factors/Goals Utility/Correctness: extent to which program satisfies specifications (presence of function) Reliability: precision with which program performs as expected (absence of failure) Modifiability: effort required to modify Maintainability: effort required to locate and fix errors Security: extent to which access is controlled Efficiency: amount of computing resources required Usability: effort required to learn, operate, and interpret behavior Testability: effort required to test Interoperability: effort required to couple to another system Portability: effort required to transfer to different environment Reusability: extent to which program can be used in other applications - after McCall, 1977 Quality Assurance ---------------------------------------- slide number 19 Sources of Non-Clerical Errors 81% design 11% requirements 8% language and environment 2% other - Weiss & Basili, 1985 Quality Assurance ---------------------------------------- slide number 20 Program Complexity Metrics "Software Science" * begins with count of operations and distinct operands, combined using various formulas * has both strong proponents and strong detractors Cyclomatic Complexity * counts number of simple cycles in flow graph * applies at design stage * relates to testing complexity Higher-Order Measures * length * f(data flow) Quality Assurance ---------------------------------------- slide number 21 Quality Models Specification focus on parts meet any inspection product specifications system ISO 9000 focus on process QS 9000 process under control Capability Maturity Model focus on organization SEI culture under control certification Quality Assurance ---------------------------------------- slide number 22 Capability Maturity Model "Levels of Maturity" 1. Initial V disciplined process 2. Repeatable V standard, consistent process 3. Defined V predictable process 4. Managed V continuously improving process 5. Optimizing Quality Assurance ---------------------------------------- slide number 23 IEEE Standard Outline of IEEE Standard for Quality Assurance Plans (STD 730-1984) 1. Purpose and Scope 2. References 3. Organization, Tasks, and Responsibilities * 4. Documentation Required 5. Standards, Practices, and Conventions * 6. Reviews and Audits * 7. Configuration Management * 8. Problem Reporting and Corrective Action * 9. Tools, Techniques, and Methodologies 10. Code Control * 11. Media Control 12. Supplier Control * 13. Records Collection, Maintenance, and Retention * important * not applicable to InfoSys Quality Assurance ---------------------------------------- slide number 24 Quality is NOT Optional Management will require QA Management will specify overall QA plan Management will indicates what variations are * allowed * required Copyright 2108-10-20, Edward L. Robertson