To become a better software company is invaluable for almost every company these days. In his Wall Street Journal article “Why Software is Eating The World” from 2011, Marc Andreessen laid out the basic reasons and examples why software is becoming the major driver in almost every industry. Over four years later, the article has not lost any of its relevance, quite the contrary. It is truer than ever as the following quote by Tom Goodwin reveals.
Uber, the world’s largest taxi company, owns no vehicles.
Facebook, the world’s most popular media owner, creates no content.
Alibaba, the most valuable retailer, has no inventory.
And Airbnb, the world’s largest accommodation provider, owns no real estate.
Something interesting is happening.
Every company in formerly software-free industries has to learn how to do software or, in other words, how to develop innovative software-intensive systems. In the last five years, I had the opportunity to develop several mobile applications and web-based client-server applications within our former startup and develop embedded safety-critical advanced driver’s assistance systems for BMW at the same time. One of the things that struck me most was that the basic rules for how to develop any innovative software-intensive system are always the same.
These basic rules can be your 9 concrete steps to become a better software company, if you want to reach the following goals:
- developing innovative software-intensive systems fast and shipping them continuously (on time),
- developing it while being cost saving (on budget),
- satisfying the customer (with high functional or outer quality),
- being fit for the future (with high inner or structural quality),
- becoming better over time (continuous improvement),
- and having fun in doing so.
The 9 concrete steps I am proposing are posed as questions, just like in the well-known Joel Test.
Please be reminded: this list focuses on efficiency (doing things right) rather than effectiveness (doing the right things). Also, most of these questions do not have a discrete “yes” or “no” answer. By evaluating your company or team, you will most likely find yourself on a continuous spectrum ranging from “Yes, we’re doing it 100% right” to “No, we’re not doing that at all“.
0x1. Autonomy: Do you work in cross-functional self-organizing teams with shared artifacts?
In order to be able to develop small- and large-scale systems, your whole business should evolve around cross-functional and self-organizational teams. Cross-functional means that your team has all the abilities that it takes to develop the system from end-to-end. This usually means that you have your requirements analysts, your software developers, your software architects and your testers in the same team. Within the team, all created artifacts should be reviewed and shared (common ownership), resulting in higher quality artifacts, fewer workarounds and a higher bus factor. Of course there are limits to this approach and scaling this approach is not easy, however putting a lot of effort into making this possible is worth it. Developing innovative software-intensive systems requires very creative work and is neither outsourceable nor a conveyor belt job that can be done by the rules of Taylorism. The team should be self-organized, meaning they can make their own decisions without asking for permission, are responsible for how they work and for the outcome. Also they should pull their work without pressure from the outside and share useful information with each other and the whole company (think wiki, task boards, information radiators). Most importantly, the team should be co-located and have the opportunities to collaborate closely together or work quietly and undisturbed when needed.
0x2. Mastery: Is technical excellence and technical progress part of your business?
The single most important factor for good software engineering is simply to have good software engineers. Every member of your teams should be selected carefully with regard to his (learning) abilities and his willingness to learn more. Testing these skills should be part of your interview process and advancing these skills should be part of your company culture (think technical and personal training, onboarding programs, mentorships, reading groups, brown bags, conferences, community meetings, coding dojos, fedex days). A tester should know what a good test case looks like, a requirements analyst knows that requirements and testing should go hand in hand, an architect knows how to build the right software abstractions, a software engineer knows what maintainable code looks like, and everybody tries to be on track with new developments in the field. Notice that the mentioned roles do not necessarily have to be filled by different people. The basis for mastering the development of complex software-intensive systems independent of your role very often lies in the principles of KISS, DRY, The big difference is that making it work is not at all the same as making it good. Making it good is an essential investment in your company’s future.
0x3. Purpose: Do people know what they are working on and is everybody involved in the development loop?
It is essential that people on the development team understand what they are working on and what the benefits for the involved stakeholders (e.g. customers) are. The best way to do that is eating your own dog food, meaning that everybody should be able to experience the product in development. Other means to do that include (good!) mission statements, a clear product vision, release goals, sprint goals and user stories. Requirements should not just be handed to the development team, the development team should be included in the process of gathering requirements, understanding the problem and engineering the solution. The key to success is more communication and less documentation. The best way to document requirements is in the form of executable requirements. That way the specification stays up to date, is a living document and has few chances to become obsolete or inconsistent with the code. Also, this method automatically ensures that requirements are documented in a lean manner. Still, always make sure to include rationales in your artifacts. In addition to requirements engineering going hand in hand with testing, operations shoud go hand in hand with development (DevOps). Only by closing the gap between development and operations you can deploy faster, fix bugs faster and close your customer feedback loop.
Please note how the first three items resemble Daniel Pink’s key factors of motivation autonomy, mastery and purpose. Coincidence?
0x4. Fast Feedback: Do you have short round-trip cycles between idea, implementation, testing, deployment, operations and feedback?
If you want to measure how efficient your product development process is, this is the single most important metric you can get: How long does it take your team to get from an idea to a working new feature in your end product to customer feedback about that new feature? We all know that we’re dealing with highly complex software systems with changing requirements that cannot be governed by waterfall-like processes, so you have to enable your team to get fast feedback. A prerequisite to having short round-trip cycles is that you work in small prioritized batches with value for the customer. This gives you the opportunity of getting feedback faster. The smaller your batches are, the faster you can test them, the faster you can ship them and the faster you get feedback. Also, smaller batches are way easier to handle. If you’re doing too much at once, you will get more integration problems and bugs, it’ll be hard to figure out where the problem is buried and you will fail to have running software. At the end of the day you will fall behind your schedule. When prioritizing the batches, don’t forget Joel Spolsky’s rule of first fixing important defects before implementing new features. Before starting to work on the batches, make yourself aware of when the batch is considered to be done (acceptance criteria) and when working on the batches, don’t forget Kanban’s rule of limiting your work in progress: stop starting, start finishing!
Remark: When you think about short round-trip cycles, also think about the first round-trip cycle of a new developer. How easily is she able to set up her development machine? This task can be automated.
0x5. Likable Assistance: Do your tools, processes and workflows support these short round-trip cycles?
Joel Spolsky coined it that way: “Do you have the best tools money can buy?”. I want to get a little bit further by expanding this rule not only to tools, but also to processes and workflows. The question here is basically: Do your tools, processes and workflow support your team members in doing a good job and being able to concentrate on valuable tasks instead of supervising them? A requirements management tool that takes several minutes to start is not an eligible tool that supports short round-trip cycles. The Apple App Store review process is not an eligible process that supports short round-trip cycles. Processes and workflows that are made by externals (not part of the development team) are very often not tailored to those who use them. Therefore always make sure that the team itself owns its processes, workflows and tools and when changing workflows and processes follow the same rule of using small batch sizes like for changing your software.
Another way to coin this question is simply to ask if your employees like to use the tools, workflows and processes or do they feel like they just have to use them. Also you should appreciate the special needs of software engineers like choosing their own hardware.
0x6. Continuous Integration: Do you have a continuous integration system with dependency management and a high coverage of automated test cases?
Fortunately, nowadays the term continuous integration (CI) cannot be overlooked any more if you’re doing serious software development. Continuous integration means that every developer commits her changes on a regular basis (think daily) and a continuous integration system builds your whole software product, runs quality checks and provides fast feedback to the developer. More sophisticated CI systems are aware of dependencies within your software and can decide by themselves not only which builds are necessary to rerun but also which test suites and test cases. CI systems need to be able to reproduce all builds and your build managements needs to be able to tell you exactly which features, bug fixes and known issues are in your software release (release notes). The more complex your software-intensive system is, the higher the need for a lot of automated testing on all levels (unit, integration and functional testing). Advanced continuous integration systems keep your application in a state where it is always able to deploy into production. This is called continuous delivery and when you’ve automated continuous delivery, you have reached the highest level called continuous deployment.
0x7. Building Quality In: Do you build quality in and follow the Boy Scout Rule? Do you refactor on a regular basis?
Building in quality is essential in today’s complex software systems, because quality cannot simply be checked after the fact. Use test-driven-development on both unit and functional levels and make sure that testing is not only done by testers but by everybody on the team. And of course automated testing is just one part of your quality assurance method (think peer reviews, static code analysis, coding conventions, design principles and manual testing). Make sure to use an appropriate mixture of them and use them as early as possible in your development process. Furthermore, quality should not only be built in but also improved every time you’re working on a part of the system. Following the Boy Scout Rule not only on a small scale, but also on larger scales should be common among all your engineers. Make sure that your architecture, tools and processes supports refactorings and responding quickly to changing requirements and conditions and that refactoring is an ongoing activity in your development process.
Remark 1: Big architecture refactorings should also be split into small incremental tasks, otherwise you’re running into the same risks already mentioned in 0x4 Fast Feedback: introducing hard-to-find bugs, integration risks and falling behind the schedule.
Remark 2: Don’t fall into the trap of premature optimization. Only refactor when it is really needed.
Remark 3: This rule can and should be bent, if you’re working on a system whose future is uncertain. Refactoring and building a lot of quality into a product without a user base (think prototypes, MVPs, startup products, etc.) is one form of premature optimization and can be a serious waste of your resources. The appropriate mixture for an early phase product, where the product risk is very high, can be only manual testing. Once you know you’re building the right product you can still catch up with your quality assurance methods. Still, make sure to do so. See Do Things That Don’t Scale.
0x8. Automation: Is automation and continuous improvement part of your daily business?
Good software engineers get bored quickly if they have to do tasks that can be automated. Don’t waste precious engineering time doing things manually, which can be automated. Give your employees the time to automate and improve the work they are doing on a small basis. That gives your teams more time to work on the actual product in the long run. The same applies for continuous improvement of the company on bigger scales. Do your development team and the supporting organization have an established continuous improvement process? Does everybody care about how to improve? Are development obstacles (impediments) removed in a quick manner?
0x9. Metrics: Can you measure the outcome and reproduce the behavior of your software product?
Last but not least, without being able to reproduce the behavior of your software and without being able to objectively define and measure if you’ve solved your stakeholders’ problems or fulfilled your stakeholders’ needs, you’re flying blindfolded. Ask yourself if you can express the success of your software product in numbers? Do you have measurements in place on how to easily reproduce and visualize the behavior of your software product? Use instrumentation in your software to get the numbers you need.
0x2D: Is software development (considered to be) a key factor to your company’s success?
Summing it up, in the end it all comes down to the question of whether you and your company appreciate software development as an essential part of your core business or in other words if you have a good software engineering culture. Make sure that people who do not have traditional software engineering roles but work closely together with software engineering (think project managers, management in general) have hands-on experience in software development.
Please be advised: These rules apply to every team and every company involved in the development of your product. A chain is only as strong as its weakest link. If you have any comments or questions concerning these rules, please do not hesitate to contact me or leave your comments.
- The Agile Manifesto
- The Effective Engineer, Edmund Lau
- Team Geek: A Software Developer’s Guide to Working Well With Others, Brian W. Fitzpatrick and Ben Collins-Sussman
- Drive: The Surprising Truth About What Motivates Us, Daniel H. Pink
Many thanks go to Brian Berenbach, Andreas Budde, Daniel Hoheisel, Tim Lantz, Moritz Prinz, Stefan Puchner and Joachim Schmidt for their useful feedback.