What is dark launching and how do you use feature flags?
Dark launching means silently launching a new feature. This lets you test the feature with a small subset of users before releasing it widely.
Modern web applications are extremely feature rich. Indeed, they have largely replaced the traditional desktop applications of the past.
Nowadays, Google Workspace offers all the functionality of Microsoft Office via the web. Delivering applications via the web allows you to be far more agile and release faster, but it comes at a cost. Users are now able to shop around much more easily. So, an application’s success is often dependent on the product team getting the application features perfect. A key part of that is being able to test them with real-world users. Dark launching allows you to do just that without the need to recruit a panel of users to actively engage with.
What is dark launching?
Dark launching involves releasing a new feature without actually enabling it to all users in the frontend. This is a great technique for two reasons. Firstly, it allows you to try out a new feature on a small set of users. This means you can do A/B testing to see if the feature works as expected and is well-received. Secondly, it lets you validate the performance of the feature in the real world. Regardless of how good your testing is, nothing beats letting real users loose in the application.
The idea is that rather than launch a new feature for all users, you instead release it to a small subset. Usually, these users aren’t aware they are testing the new feature. Typically, nothing highlights the new functionality—hence the term dark launching.
How to do dark launches
To run a test, the product team selects a feature to turn on for a set of users. As new user sessions arrive at your load balancer, they are assigned one of two profiles. The first has the feature under test disabled. The second enables the feature. Typically, you would test the feature on a relatively small number of users. But this approach can also be used for A/B testing against a larger user group.
The product team needs to determine what metrics matter for the feature. Is it about improving UX, driving up revenues, improving accessibility, or something else entirely? Some things are easy to measure, such as whether the new feature is actually being seen by the users. Others are harder to gauge. For instance, does the new feature drive up revenue by improving product recommendations in your eCommerce store? Does the new feature create an onslaught of angry customer support tickets? Or are your customers so impressed by the update that they submit rave online reviews?
Pre-requisites
Before the development team decides to adopt a dark launching approach for feature testing, ensure that every application feature can be toggled on or off. So, the prerequisite is to adopt the use of feature flags. Effectively, these allow features of the application to be enabled or disabled at runtime. So, you first need to ensure the deployment system can use feature flags. This may require a new approach from your developers. But most modern applications are inherently modular like this.
You also need to enable UX instrumentation so you can monitor how users respond to the new feature. Fortunately, there are lots of libraries to help you do just this. But be aware of any potential issues relating to data protection and privacy, for instance the GDPR or CCPA. Of course, you should be checking for this anyway…
Dark launching in practice
You can see how commonplace dark launching is by using a popular online application, such as Google Maps. You frequently see new features that may later disappear. This is because the Google team is constantly trying things out to decide what works best. Things that work are rolled out across the entire user base; other features are quietly dropped. Yet, other features may be only available in certain geographies. For example, Austrian users had over a decade without Google Street View, and even now only see a limited version.
The above example shows how feature flags can also be used to assist with complying with local regulations and requirements. Rather than create an entire version of an application for a given jurisdiction, you can just create a specific profile. This can definitely simplify life when it comes to complex legislation, such as the upcoming EU ePrivacy directive.
What next?
Dark launching is a great tool to add to your testing arsenal. It allows you to safely test out new features on real users. This is true shift right testing. What’s even better is that it relies on tools and functionality you likely already have in your application. That means it is one of the easiest things you can do to improve your application quality.
