Some of you may have seen the classic Star Trek episode, "The Trouble with Tribbles". In the episode, Lt. Uhura gets a gift of a cute little ball of fur from a passing trader. It's a huge hit with the crew, and when the tribble has babies, everyone who wants one can enjoy these little fuzzy creatures. The trouble begins when they discover that the tribbles are born pregnant and quickly give birth, increasing the tribble population exponentially until they are taking over the entire ship.
Today I would like to talk about the trouble with toggles, and what they have in common with tribbles. Toggles can be helpful to developers in many ways. They make it easier for development of a new feature to continue in the main branch, without disturbing the work of others. They enable us to do beta testing, where a handful of customers try out a feature or layout and provide their feedback while most customers continue using the software without the changes. And they enable us to quickly remove a feature if it's discovered that it has a critical error. But toggles can be problematic for two very important reasons, which I will explain using our tribble friends.
First, toggles make more work for QA engineers. Just as the tribbles multiplied exponentially, the number of test passes we need to do when using toggles also multiplies exponentially. Consider the following scenario: Team One adds a new feature with a toggle. In order to make sure that the feature hasn't broken any existing functionality, they need to do two passes of smoke tests: one with the toggle on, and one with the toggle off. Next they add a second feature with a toggle. Now there are four different combinations of scenarios that their software can have: both toggles on, toggle A on and toggle B off, toggle B on and toggle A off, and both toggles off. So they'll need to do four different passes of smoke tests to make sure that all their features are working correctly. Now let's add in Team Two. They have two toggles as well, so they'll need to do four passes of smoke tests for all the permutations of toggle A and toggle B.
As release day approaches, Team One and Team Two are merging their code into the main branch. Now there are EIGHT different possible combinations for toggle states. Even if the teams decide that it's a low risk to skip smoke testing in all eight scenarios, there will still be a lot of investigative work involved for every bug found. Let's say someone finds a bug in an instance where Toggle A and C are on and Toggle B and D are off. They'll need to answer questions like "What if just Toggle C is on? Do we still see the bug? What about if Toggle D is turned on as well? Does the bug go away?" Research like this can be really time-consuming. This is time that could be better spent doing exploratory testing. If all the features were enabled in all scenarios, testers would only need to do one pass of smoke tests and spend the rest of their time digging deeper into each new feature.
Secondly, toggles can give software teams a false sense of security, which can lead to buggy software and tech debt. In "The Trouble with Tribbles", the tribbles' purring sound had a soothing effect on human's nervous systems, which made the crew pay less attention to the fact that in a couple of days the tribble population would fill the Enterprise from deck to ceiling. Similarly, engineers can be soothed by the knowledge that they can always turn the feature off, which means that they can test the feature less carefully. If bugs are found, they can be dismissed with the statement, "Well, it's Beta, and the Beta users will understand. We'll log it and take a look at it later." Bugs like these can compound, especially when mixed with all the other bugs in all the other Beta features. And the bugs can take a lot longer to fix six months later when the original developers don't remember what they did or may not even be with the company any more. Alternatively, if the teams tested every feature with the knowledge that all users would be seeing it on first release, they would test more carefully and fix bugs right away, ensuring much greater code quality from the outset.
Like tribbles, toggles can be fun, in that they enable us to add more pizazz more quickly to an application. But as the crew of the Enterprise found, too much of a good thing has its consequences!
Today I would like to talk about the trouble with toggles, and what they have in common with tribbles. Toggles can be helpful to developers in many ways. They make it easier for development of a new feature to continue in the main branch, without disturbing the work of others. They enable us to do beta testing, where a handful of customers try out a feature or layout and provide their feedback while most customers continue using the software without the changes. And they enable us to quickly remove a feature if it's discovered that it has a critical error. But toggles can be problematic for two very important reasons, which I will explain using our tribble friends.
First, toggles make more work for QA engineers. Just as the tribbles multiplied exponentially, the number of test passes we need to do when using toggles also multiplies exponentially. Consider the following scenario: Team One adds a new feature with a toggle. In order to make sure that the feature hasn't broken any existing functionality, they need to do two passes of smoke tests: one with the toggle on, and one with the toggle off. Next they add a second feature with a toggle. Now there are four different combinations of scenarios that their software can have: both toggles on, toggle A on and toggle B off, toggle B on and toggle A off, and both toggles off. So they'll need to do four different passes of smoke tests to make sure that all their features are working correctly. Now let's add in Team Two. They have two toggles as well, so they'll need to do four passes of smoke tests for all the permutations of toggle A and toggle B.
As release day approaches, Team One and Team Two are merging their code into the main branch. Now there are EIGHT different possible combinations for toggle states. Even if the teams decide that it's a low risk to skip smoke testing in all eight scenarios, there will still be a lot of investigative work involved for every bug found. Let's say someone finds a bug in an instance where Toggle A and C are on and Toggle B and D are off. They'll need to answer questions like "What if just Toggle C is on? Do we still see the bug? What about if Toggle D is turned on as well? Does the bug go away?" Research like this can be really time-consuming. This is time that could be better spent doing exploratory testing. If all the features were enabled in all scenarios, testers would only need to do one pass of smoke tests and spend the rest of their time digging deeper into each new feature.
Secondly, toggles can give software teams a false sense of security, which can lead to buggy software and tech debt. In "The Trouble with Tribbles", the tribbles' purring sound had a soothing effect on human's nervous systems, which made the crew pay less attention to the fact that in a couple of days the tribble population would fill the Enterprise from deck to ceiling. Similarly, engineers can be soothed by the knowledge that they can always turn the feature off, which means that they can test the feature less carefully. If bugs are found, they can be dismissed with the statement, "Well, it's Beta, and the Beta users will understand. We'll log it and take a look at it later." Bugs like these can compound, especially when mixed with all the other bugs in all the other Beta features. And the bugs can take a lot longer to fix six months later when the original developers don't remember what they did or may not even be with the company any more. Alternatively, if the teams tested every feature with the knowledge that all users would be seeing it on first release, they would test more carefully and fix bugs right away, ensuring much greater code quality from the outset.
Like tribbles, toggles can be fun, in that they enable us to add more pizazz more quickly to an application. But as the crew of the Enterprise found, too much of a good thing has its consequences!
No comments:
Post a Comment