Lean Software Development.doc

Lean Software DevelopmentNathan DeckerDepartment of Software EngineeringUniversity of Wisconsin Plattevillenathandecker1Abstract    Lean development focuses on creating high-value products by removing waste from the development cycle. Lean software development is a software development model adapted from lean manufacturing and agile development principles. The lean model focuses on customer feedback and waste reduction as well as a number of other useful principles that implement the basics of lean development in a software environment. Lean software development promotes an adaptable, incremental model that creates a quality product faster and more predictably than other development processes.Introduction        Agile programmers follow a manifesto originally written in 2001 10. The Agile Manifesto states that developers shall value:· Individuals and interactions over processes and tools· Working software over comprehensive documentation· Customer collaboration over customer negotiation· Responding to change over following a planWhat makes this manifesto so interesting is that some companies in the manufacturing industry had been following a similar model years before the Agile Manifesto was ever conceived. In manufacturing environments, this development model is called lean development and produces faster, customer-valued products with lower costs than conventional development models. Lean development focused on reducing any waste present in the production of a product and maximizing the value of the product. In order to reduce waste, lean development provides a manufacturing model that relies on a flowing process to rapidly produce quality products that provide exactly what a customer wants exactly when the customer wants it. In an effort to achieve these same results in a software environment, lean software development uses the principles of industrial lean development to provide a basis for implementation of the agile manifesto.HistoryThe concept of lean software development originated from the work of Taiichi Ohno, a Toyota executive 2. Originating from Taiichi's 1950 system of Just-in-Time, Taiichi wanted to cut down on the waste he saw present in the organization, and to that end he developed the lean development strategy. Lean development was not applied to the software industry until Mary Poppendieck saw the use of lean development in a Toyota production plant. She and her husband, Tom Poppendieck, published a book titled Lean Software Development: An Agile Toolkit that addressed the use of lean development for software development. However, before the principles of lean software development can be discussed, an understanding of the basics of lean development is necessary.The Basics of Lean DevelopmentLean development revolves around the reduction of muda in a product while maximizing the value of a product. Muda is the Japanese word for waste, and is any activity in business that uses up resources, but creates no value 2 Examples of muda in the software industry include partially done work, extra processes and functions, and defects. Muda is the enemy of value in a system. Value is the basis of any product, and is defined by the final customer: simply put, value is what the customer wants the product to do. Oftentimes software developers will add many features that customers do not value or even need, as shown in Standish group research where 45% of all software features go unused by customers. 1 These unused features are not value, and all the time spent developing, coding, and testing these features amount to pure muda.The creation of value is determined by the value stream. The value stream includes all the actions required to bring a project from creation to completion. Lean development is especially concerned with removing steps from the value stream that add no value to the final product and are avoidable. All other steps are made to flow, insofar as that the product is in motion at all times. A non-lean strategy in production is to use large batches of product that are shipped from facility to facility to wait in a queue before going through the next step of a development process. While common sense might suggest that this large batch and queue system is more efficient, Lean Development argues that a continuous flow promotes a more efficient process that produces quicker and higher quality results. 2 After a flow for the product is created, the time taken to produce each product falls, and the next principle of lean can be used, that of pull.The pull is the beginning of any lean development cycle: pull is the concept that no product will be made until it is requested by the customer. An excellent example of pull can be seen at any Culver's restaurant, a chain of fast-food restaurants based in the Midwest. Unlike many other fast-food restaurants, the food preparation starts with the customer asking for an item on the menu. The food is then cooked and assembled on the spot. This can be contrasted with a non-pull system, such as McDonalds. McDonalds will prepare a batch of multiple menu items ahead of time, so that they are ready to be consumed upon order. While this may seem to be more efficient and faster, the amount of waste produced by non-ordered food should be a warning that a non-pull system often produces too much waste. However, a pull system will produce the correct number of the product every time, and can easily be tailored to handle custom orders better than a non-pull system.The final principle of lean development is the continuous pursuit of perfection. After the initial definition of the value stream and implementation of a pull system, all remaining steps are continually examined for possible optimization. This furthers the effects of switching to lean development from one large improvement in efficiency to a long-term process and product improvement. Lean development supposes that all processes can be improved to get closer to a perfect process, so the pursuit of perfection is a lifetime goal for a company.How does Lean Development apply to software?Historically, software has been designed by planning out the system almost completely up front, using some variation of the waterfall model. Requirements are gathered first, and then continuously changed throughout the product lifecycle as customers change their mind. The software design is carefully planned out, and then the design is scrapped as the system changes, or difficulties are encountered that make previous plans irrelevant. The essential cause of these problems is a reliance on a forecast model, where everything is planned ahead of time, based on assumptions created very close to the start of the program. However, these problems can be avoided by switching to a feedback model. 8 This model focuses on small increments of change, where each increment rapidly adds useful features created from the final customers feedback. This incremental style is at the heart of lean software development.   Mary and Tom Poppendieck based Lean Software Development on the Lean Development used in Toyota's plants, as well as a firm basis in agile programming. They created a set of principles that a lean software development process should follow. Many of these steps are implemented in other programming processes, such as extreme programming.1Eliminate wasteThe first principle is based off the elimination of muda that is defined in the original lean development. The principles simple goal is to remove all parts of product design that add no value to the final product. According to Mary Poppendieck, there are seven causes of muda in software 7. These wastes can be sorted into three types of waste: wastes in code development, wastes in project management, and wastes in workforce potential.The first example of code development waste can be found in partially completed work. Partially completed work is prone to becoming outdated and unusable. Whenever code expires, the time spent working on it in the first place is wasted. Another code development waste is when defects are found that must be corrected and retested. Both of these wastes are combated by the lean development system of iterative releases. In an iterative cycle system, only small, fully coded portions of the system are developed at a time, removing the need for partially completed code. The working releases of the iterative cycle also have the benefit of allowing for test regression suites that can monitor code, so that errors can be fixed immediately after they are detected with a lower complexity than would be found in a larger system.Project management wastes are often the most difficult to diagnose, especially in the case of extra processes. 1 These processes introduce extra documentation that provides no real value to the final project, but are done because "that's the way software is made". To combat this waste, every document must be reviewed for its necessity in a project. However, developers also contribute to the problems in project management. Code or request hand-offs will almost always result in a loss of knowledge in the transition, even with comprehensive documentation. In the end, the true expert on what is needed for a requirement is the customer, therefore a way to correct the problem of hand-offs is to have the developer work directly with the customer. 11 This dovetails into another waste of project management, that of extra functions not needed by the customer. Studies have shown that over 45% of any application will never be used by customers 1. As shown in Figure 1, even more features in any application are rarely or only sometimes used. While an iterative method is useful in implementing customer requests quickly, the best way to make sure that developers only implement what customers need is to have the developers and customers communicate frequently and extensively.Figure 1: Feature usage in enterprise software 4Workforce potential wastes are perhaps the simplest wastes to see in a project. The first waste is when a developer is required to switch tasks repeatedly. Multitasking is conventionally thought to be more productive, but research has shown that multitasking reduces a person's ability to focus on a single task 12. This is especially true in software companies where developers check their email many times an hour. Lean development suggests that by focusing on a single task, developers will be able to devote more of their intellect to a request. This also works well with the short iterations, where developers are able to focus completely on a single task for one of the short increments, instead of splitting their focus with multiple requirements at a time.The last and perhaps most prevalent waste of workforce potential is when a developer has to wait for information or instructions. A waiting developer is wasting both their time and often highly expensive skill set while adding no value to the project. The best way to stop this waste is to allow developers to make their own decisions, and to allow easy access to any information a developer requires. This is discussed more under the topic of empowering the development team.The iterative development model at the heart of lean development is a key to removing most of the waste in an organization. After these wastes have been addressed, the following principles can help to promote a continual drive for added value and perfect processes.Amplify Learning/ Create KnowledgeIn any reasonably complex software project it is impossible to have full knowledge of the full system at the start of the project. This can create problems with the classic software design plans, where large portions of the final system are mapped out before code is written. This method causes problems when major parts of the system have to be redesigned when an unforeseen change to the system is required, either because of technical issues or changing requirements. Lean Software Development methods handle this problem by promoting short, full-cycle iterations 7. Each of these cycles results in working, tested, and deployable code based on the requirements of a customer at the time. This method handles changing requirements and adapts to technical issues better than the classic "once and done" method. Lean development also relies heavily on interaction with customers to solve problems and provide information. By gathering customer requirements and using customers to better understand the problem domain, lean developers can increase their knowledge of the system and provide a product that better meets the needs of customers. This will reduce the wastes of extra functions that the customer does not care about. By involving the customer into the decision process they can also begin to see possibilities for what the product can do, and they can make informed requests for the development team.Delay CommitmentAs described in the fast food example previously, a forecast model driven methodology often creates incorrect assumptions. Changing requirements are so common in software that it becomes an expectation for almost any project. This is an essential problem for any fully specified project, which is why lean developers will delay committing to a decision until the last responsible moment. This allows for a couple of benefits. First, delaying decisions allow developers to fully research a problem, and developers can apply a larger amount of knowledge to the decision than if it were decided at the beginning of a project. Secondly, delaying decisions keeps options open, allowing the developer to handle uncertainty, even that created by an indecisive customer. Leaving options open is another way to begin removing dependencies in the project, with the end goal of a system that can have any feature added at any time 1. Empower the TeamWhile it is important to make decisions as late as possible, it is also important to decide as "low" as possible 9. In other words, decisions should be made by people doing the work. In software engineering this will usually be the developer who is writing the code. The iterative model is useful only so long as developers are able to make quick decisions on how to design and implement the customers requests. Instead of creating models and documentation that specify exactly how each part of the project should be designed and coded, the documentation should be closer to a set of guidelines and goals for the specific part of the project. There are a couple of benefits to allowing low-level decisions. By allowing developers to make decisions themselves based on this set of guidelines and goals, developers closest to the problems of implementation can make informed decisions for their work. Since there are short cycles and rapid feedback in lean development, developers are ab