VALUEBASEDMANAGEMENTOFSOFTWARETESTING.doc

外文资料名称： Value-Based Management of Software Testing 外文资料出处： Institute of Software Technology & Interactive System 附 件： 1.外文资料翻译译文 2.外文原文 基于价值的软件测试管理鲁道夫,斯蒂芬,保罗沈青松译摘要: 根据研究表明测试已经成为软件开发过程中一个很重要的环节，它占据了整个软件开发成本的百分之三十到五十。测试通常不是用来组织商业价值的最大化，也不是肩负着项目的使命。路径测试、分支测试、指导测试、变换测试、场景测试以及需求测试等对于软件的所有方面都是同等重要的。然而在实践中百分之八十的价值往往来自百分之二十的软件。为了从软件测试中得到最大的投资回报，测试管理需要最大化它的价值贡献。在本章，我们将更加促进对基于价值的测试的需要，描述支持基于价值的测试管理的实践，勾画出基于价值的测试管理的框架，并举例说明该框架。关键词: 基于价值的软件测试,基于价值的测试,测试成本,测试利益,测试管理11.1 前言测试是软件质量保证过程中最重要和最广泛使用的方法。校验和验证旨在通过综合分析, 测试软件确保其正确运行功能，确保软件的质量和软件的可靠性。在IEEE610.12(1990)中,测试被定义为在规定条件下对执行的系统或者组件进行观察和记录,并对系统或者组件进行评价的活动。测试在实践过程中被广泛的使用，在保证质量策略的诸多组织中扮演着重要的角色。软件影响着成千上万人的日常生活，担负着艰巨的任务。因此软件在不久的将来将显得尤其的重要。研究表明，测试通常消耗软件开发成本的30%至50%。对于安全危急系统，甚至更高的比例也不足为奇。因此软件测试具有挑战的就是寻找更多的有效途径进行有效的测试。软件测试管理的价值在于努力减少测试成本和满足需求。有价值的测试管理对于项目目标和商业价值也能有很好的向导。在第一章，Boehm 列举了很多方面的潜在测试成本。该例子说明了利用客户结账类型的7%的成本来完成50%的软件测试利益。尽管百分百测试是一个不太切实际的目标, 然而通过调整测试方法, 仍有很大的空间来改进和节省达到预期的价值。基于软件工程的价值动力在于目前软件工程的实践研究都是把需求, 测试案例, 测试对象和产品缺陷看的同等重要。这对测试显然是正确的，它对产品价值间接的做出贡献。开发和测试的分离使得这问题显得更为突出。测试往往是一个纯粹的技术问题，将使得测试和商业决策之间曾解链的关系变的更加紧密。本章主要描述了提高基于价值的测试管理的需求, 解释其基本要素, 讨论现有的实例来支持基于价值的测试, 以及基于价值的测试管理的基本框架。本章接下来部分的基本结构如下：11.2节讨论测试管理的贡献价值； 11.3节讨论已测案例对于测试管理的支持； 11.4节用例子描述基于价值的测试管理的框架。本章最后将对具体的再作更进一步的研究。11.2 基于价值测试的描述基于价值的校验和验证的目标被定义为确保软件的实施能够满足其预期的目标价值利益。如果我们从价值的角度去考虑，那么测试的贡献应该在哪呢？从根本上来说, 可以从两个方面来考虑: 内部方面包括测试成本以及测试效益, 外部方面强调未来系统的威胁以及机遇。对于基于价值的测试，其关键就是要把这两个方面结合起来,也就是说通过客户和市场需求调整内部测试的过程。为了使内部和外部两方面因素相结合，只专注于技术方面的测试显然是不恰当的。相反， 测试管理需要有个全局的把握。例41描述了测试管理内外双方面之间的依赖关系。内部方面的就如同测试经理对整个项目的控制。这方面的费用主要来自于软件测试实践以及短期长期的测试。而外部方面被认为是测试主管所能控制以外的一些利益和参数数据.基于价值的软件测试管理使得测试能够满足以利益为重的价值主张，以及使整个小组聚焦在有价值的测试方向上。对于软件测试的外部观点的首要问题是“我们如何确保软件系统的价值目标？” 这目标就是通过协调价值主张来测试软件集中的有价值部分，最重要的品质以及项目风险的及时调整等.回答这样的问题包括市场机遇，项目的价值主张以及成本效益。参考第一章关于机遇和风险的详细介绍以及参考第七章的价值主张的引出与调和。内部观点是建立在价值主张的利益之上，以及测试的预算代表着整个项目的一个大概水平。这主要的问题就是如何把测试作为一项投资活动。为了能够高效迅速的测试及降低开发预算。适当的内外部交流协调能够满足测试的利益价值。测试的价值贡献测试与其他的开发环节诸如代码和用户界面设计相比，它不能立即对产品创造价值。然而测试提供和支持软件开发过程中产生的有价值的任务活动。理解测试贡献价值的关键点在于测试的贡献效应。测试的贡献建立了测试与最终产品价值利益之间的关系。最直接的客户是直接与测试小组有密切联系的程序开发者和项目主管。在基于价值的软件工程测试过程中的集中力量是顾客和用户（见第七章）。顾客和用户通过设定语境和范围来进行测试达到测试的价值目标。测试的客户开发人员,项目主管,质量主管,顾客,分析者,最终用户或者维修人员们都得益于软件系统的分析,依靠反馈来检测问题,降低其不确定性,做出相关的决定来加快产品进程。 下面的例子显示了不同组对于测试需求的反馈信息：· 顾客和用户关于多大程度上需求一致是否满意以及在多大程度上满足软件的价值利益.测试还对项目的进程提供可见性和洞察力.通过测试的结果可以了解已通过的测试案例.当验收测试时出现不实用的或者失败的显示在实际环境中才能出现的问题，和测试提供了一个更加坚实的基础来验证结果。· 销售和产品主管从测试计划、定价、促销和分配方面获取相关的信息。产品实际的质量与顾客和用户所期望的质量之间的差异很容易会导致误解和错误的设想以至于降低或者阻止了真正价值的实现。为了能够成功地达到这些期望以及满足个人或组织的目标，通过客户需求来调整产品设计满足某些功能。· 对于项目主管，测试支持了风险管理和项目进程的估计。重点是识别和排除潜在的价值破坏和抑制价值功绩的风险。早期的大幅降低项目绩效的严重缺陷,是一个主要的目的。测试降低了不确定性和帮助项目主管对于清除缺陷、系统稳定性以及产品更新发布能够作出更好、更明智的决定。· 质量主管对于问题的识别以及对特定问题的动态趋势较为感兴趣。测试结果对于项目评估、对于质量策略的保证以及进程的改进提供了帮助。Rosenberg讨论了测试如何对确保质量作出贡献并展示了测试问题如何验证如何修正以此来提升项目进程。开发者和用户了解相关问题的当前状态，并且提供相关数据来衡量及预测软件的质量和可靠性。· 开发人员通常需要获取反馈信息来验证测试实施是否完整，是否符合标准，是否满足质量要求。为了保证稳定性，测试提供了相关缺陷的详细信息，提示测试失败的原因。除此之外，测试对于项目缺陷的改进作出反馈。例如，通过相关的修改以后需要测试其是否随着相关的改动使原先的功能有所改动或者出现衰退的情况，这些都是需要注意的。· 对于需求工程师来说，测试对于验证和确认需求是很有价值的。Weinberg曾指出“最有效的方法之一就是通过测试案例就象测试一个完整的系统来形成测试需求”。黑盒测试通过其需求帮助能够保证它们的完整性，准确性，透明性及简明性等。因此测试能够提升要求且向着测试驱动的方向发展。 简言之，测试能够通过降低计划的不确定性和风险性来提升利益，作出相关的决定，努力控制把不必要的消耗减少到最低程度(内部原因)。尤为重要的是，它有助于实现预期的价值利益。这些利益的得来也并不是免费的，测试的代价通常也是有意义的(外部原因).测试可以被理解为购买信息，也可被认为降低成本风险，减少不确定性的投资活动。在成本和利益的投资上需要对测试需求作出相关的决定。因此接下来的两个问题是：什么是测试成本？什么是价值活动的测试利益？Value-Based Management of Software Testingby Rudolf Ramler, Stefan Biffl, Paul GrünbacherValue-Based Management of Software TestingRudolf Ramler, Stefan Biffl and Paul GrünbacherAbstract: Testing is one of the most resource-intensive activities in software development and consumes between 30 and 50% of total development costs according to many studies. Testing is however often not organized to maximize business value and not aligned with a projects mission. Path, branch, instruction, mutation, scenario, or requirement testing usually treat all aspects of software as equally important, while in practice 80% of the value often comes from 20% of the software. In order to maximize the return of investment gained from software testing, the management of testing needs to maximize its value contribution. In this chapter we motivate the need for value-based testing, describe practices supporting the management of value-based testing, outline a framework for value-based test management, and illustrate the framework with an example. Keywords: Value-based software engineering, value-based testing, cost of testing, benefits of testing, test management. 11.1 Introduction Testing is one of the most important and most widely used approaches for validation and verification (V&V). V&V aims at comprehensively analyzing and testing software to determine that it performs the intended functions correctly, to ensure that it performs no unintended functions, and to measure its quality and reliability (Wallace and Fujii, 1989). According to IEEE 610.12 (1990) testing is defined as “an activity in which a system or component is executed under specified conditions, the results are observed or recorded, and an evaluation is made of some aspect of the system or component.”Testing is widely used in practice and plays a central role in the quality assurance strategies of many organizations. As software pervades more and more critical tasks and affects everyday life, security, and well being of millions of people (Ferscha and Mattern, 2004), the importance of testing will increase in the future. Studies show that testing already consumes between 30 and 50% of software development costs (Beizer, 1990). Even higher percentages are not uncommon for safety-critical systems. Finding more efficient ways to perform effective testing is therefore a key challenge in testing (Harrold, 2000). Managing software testing based on value considerations promises to tackle increasing testing costs and required effort. Value-based test management could also provide guidance to better align testing investments with project objectives and business value. In Chapter 1, Boehm presents an impressive example of potential test cost savings (on project level as well as on global scale) by focusing testing on the most valuable aspects. The example illustrates that with an investment-oriented focus on testing 7% of the customer billing types (1 in 15) achieved 50% of the benefits of testing the software. Completely testing the system requires a constantly increasing effort and, due to decreasing marginal benefits, results in a negative return on investment. Although a “100% tested” status is not a practical goal, there is still room for a considerable amount of improvement and savings by better adjusting testing to its value contribution. The motivation for value-based software engineering comes from the fact that “much of current software engineering practice and research is done in a value neutral setting, in which every requirement, use case, object, and defect is treated as equally important” (Boehm, 2003). This is especially true for testing, where its indirect contribution to product value leads to a value-neutral perception of testing. The common separation of concerns between development and testing exacerbates the problem. Testing is often reduced to a purely technical issue leaving the close relationship between testing and business decisions unlinked and the potential value contribution of testing unexploited. The objectives of this chapter are to motivate the need for value-based management of testing, to explain its underlying elements, to discuss existing practices that support value-based testing, and to outline a general framework for value-based test management. The remainder of this chapter is thus structured as follows. In Section 11.2 we discuss test management under the light of its value contribution. In Section 11.3 we describe existing practices that support value-based testing. Section 11.4 depicts a value-based test management framework using an example for illustration. An outlook on further research directions closes the chapter.11.2 Taking a Value-Based Perspective on Testing The objectives of value-based verification and validation are defined as “ensuring that a software solution satisfies its value objectives” and organizing V&V tasks to operate as an investment activity” (Boehm and Huang, 2003). What are the contributions of testing if we look at it from a value-based perspective? Fundamentally, we can consider two dimensions: The internal dimension of testing covers costs and benefits of testing. The external dimension emphasizes the opportunities and risks of the future system that have to be addressed. The key challenge in value-based testing is to integrate these two dimensions, i.e., align the internal test process with the value objectives coming from the customers and the market. It becomes clear that a pure focus on the technical aspects of testing (e.g., the testing methods and tools) is inappropriate to align the internal and external dimensions. Instead, test management activities need to adopt a value-based perspective.Figure 41 illustrates the external and internal dimensions of test management and their interdependencies. The internal dimension is similar to the scope of control of the test manager in the project. This dimension addresses costs from software testing practice as well as short-term and long-term benefits of testing. The external dimension considers stakeholders and parameters outside the scope of control of the test manager. Value-based test management organizes testing to satisfy value propositions of the stakeholders and to focus the team on the most worthwhile testing targets.The key question coming from the external view of software testing is: “How can we ensure the value objectives of the software system?” The goal is to reconcile stakeholder value propositions by focusing testing efforts on the most worthwhile parts of the software, the most important quality characteristics, and the most urgent symptoms of risks that threaten the value contribution of the project. Answering this question involves market opportunities and threats, project-specific customer value propositions, as well as costs and benefits. Please refer to Chapter 1 for details about opportunities and risks and to Chapter 7 for elicitation and reconciliation of stakeholder value propositions. The internal view builds on the stakeholder value propositions and the test budget that represents the possible level of testing effort in a project. The key question in this view is: “How can we organize testing as an investment activity?”The goal is to achieve effective and efficient testing considering changes in development and budget reductions. Internal project stakeholders consider how plans for software development and associated testing activities can contribute to stakeholder value propositions by supplying system functionality and performance, but also by limiting the impact of project-relevant risks. Appropriate communication is necessary to balance the external and internal dimensions of testing to assure the consistency of testing objectives with stakeholder value propositions. Value Contribution of Testing Compared to other development activities such as coding or user interface design, testing does not create immediate product value. Instead, testing informs and supports other value generating tasks in software development. A key to understanding the value contribution of testing is the contribution chain of testing (see the benefits realization approach described in Chapter 1). The contribution chain establishes the relation of testing to the final product that ultimately creates value for the stakeholders. Usually, the contribution chain of testing is complex and involves several different “clients,” who benefit from testing.Direct clients of testing are developers and project managers, who directly interact with the testing team (representing the internal dimension). However, in the spirit of value-based software engineering important parties for testing are customers and users (representing the external view). Customers and users are the source of value objectives (see Chapter 7), which set the context and scope of testing. Within this context testing informs developers and project managers to what extent value objectives are met and where improvement is required. Clients of Testing Developers, project managers, quality managers, customers, analysts, end users, or maintenance staff benefit from a thorough analysis of the software system and rely on feedback for detecting problems, reducing uncertainty, making decisions, or improving products and processes. The following examples show the kind of feedback from testing required by different groups: · Customers and users get information as to what extent mutually agreed requirements are satisfied and to what extent the software meets their value propositions. Testing also provides visibility and insights about project progress. Passed tests reduce the odds of misbehavior and acceptance decisions are thus frequently based on the results of tests. When acceptance tests are impractical or fail to reveal hidden problems that become visible only in real-world conditions, alpha and beta testing provide a more solid foundation for acceptance decisions. · Marketing and product managers require information from testing for planning releases, pricing, promotion, and distribution. A gap between the actual quality and the quality expected by customers and users most certainly leads to misleading expectations and wrong assumptions that diminish or prevent value realization (Boehm, 2000b). In order to successfully manage these expectations and to satisfy individual and organizational objectives, reconciling customer needs with product design has to consider quality in addition to functionality. · For project managers testing supports risk management and progress estimation. The focus is on identifying and eliminating risks that are potential value breakers and inhibit value achievements. Early detection of severe defects that significantly reduce project performance is a major objective. Ideally, testing reduces uncertainty and helps project managers to take better, more informed decisions, e.g., for defect removal, system stabilization, and release decisions. · Quality managers are interested in the identification of problems and in particular problem trends. Results from testing are the input for the assessment of development performance and provide the basis for quality assurance strategies and process improvement. Rosenberg (2003) discusses how testing contributes to quality assurance and shows that problems need to be documented, corrected, and can then be used for process improve