The basics of data-driven working: 5 steps to success

There are various ways we can make decisions – based on data, using our own experience and intuition, or by consulting others. Computers, on the other hand, can only make decisions based on data. In principle, this is an objective process and computer can make decisions much faster than humans. So data-driven decision-making has unprecedented potential in, for example, marketing, process automation, risk management and many other areas.

Well, so much for the theory… In practice, introducing data-driven working is much less straightforward. And systems that have not been set up properly can lead to completely the wrong decisions – and nobody noticing them in time. The consequences can be serious. A good example of the way things can go wrong is the SyRi anti-fraud system, which was trialled by the Dutch government a few years ago

Making your organization more data-driven involves more than just technology. So we would like to run through the basics for you, broken down into five smaller steps:

• Step 1: Data
• Step 2: Rules
• Step 3: Processes
• Step 4: Organization
• … and (only when the first four steps have been completed) step 5: Technology

Step 1: Data

This might seem like a no brainer, but in practice this is where things often go wrong. The data has to be reliable, because it is going to be the sole basis on which decisions are made. So it is important that you know what your data means, how it relates to other datasets and whether the data is completely reliable.

For each data type, you need to know the following:

• The definition of the data. This applies both implicitly (no ambiguity is possible) and explicitly (to prevent confusion). A good definition means one sentence that follows a fixed structure. The context in which this definition applies must also be known.
• What form does the data take? Is it numbers, characters, dates, monetary units, files?
• How does one dataset relate to other datasets?
• Where does the data come from? Is it generated by users, is it taken from an external source, is it calculated using other data? How many sources are there?
• When is the data created? How long does it stay relevant?
• Who owns the data? What purposes can it be used for?

Step 2: Rules

When you make decisions based on data, those decisions need to be based on rules. What choices are you going to make, and have the implications of those choices been fully thought through? There are various types of rules that can be applied to your data:

• Integrity rules relate to the requirements that the data must meet in order to be accepted.
• Inference rules calculate new data based on existing data.
• Exchange rules determine the requirements under which data can be sent or received. This can be from task to task, from application to application, between databases or between users.
• Queries relate to retrieving and presenting data.
• Process rules or decision-making rules.

All these rules affect the data, and therefore they also affect each other. If the definition of the data type changes, the rules may also change as a result. These rules are also known as algorithms, particularly inference rules and rules about decision-making.

Step 3: Processes

The whole reason for adopting data-driven working methods is to make a process better, faster, more reliable and/or cheaper. But in order to do that, you need to know exactly how your processes work. Abstract overviews or descriptions are not enough. In automated systems, rules apply at the task level, or even below that. It is important to eliminate any ‘black boxes’ – steps in the process where it’s impossible to say exactly what is going on.

It is crucial that every part of the process that plays a role in achieving the end goal is identified and analysed in detail. Right down to the level of individual variables and the rules applied to the process. This is the only way to control the final results.

Step 4: Organization

Organization is an area that sometimes receives too little attention when it comes to data-driven working. We are talking not only about specific roles that need to be created, but also about routine activities, responsibilities and skills:

• Make sure that data is managed properly. This can be challenging, because data quality often leaves a lot to be desired. It’s essential to take data management seriously and to help data owners ensure that the quality is up to the required standard. Indeed, you need to enforce this quality.
• Make sure that your modelers and data analysts are also good business analysts. If you have one team focusing on process models and the other team creating data models, then the right hand won’t know what the left hand is doing. The process models will often be too abstract to be very useful, and the connection between process, data, rules and definition will be missing. A good business analyst can create an integrated model, with consistency between process, data, rules and definitions. This is an absolute must for successful data-driven working.
• Make sure you have a good lead architect to keep all the sub-architectures in synch with adjacent components.
• Appoint an algorithm manager. When you develop algorithms using AI, unintended situations can arise that may be inappropriate or even dangerous. The algorithm manager is responsible for ensuring that the algorithms used meet social and ethical standards, and that they achieve the desired result.
• Prioritize transparency. Be open and clear about what you are creating, what materials you are using, and what the goals are.

Step 5: Technology

Whichever technical solution you choose, first make sure that you have completed steps 1 to 4 as fully as you can. The knowledge that you gain by doing this will be invaluable, and will help you choose the right product or supplier.

The supplier will usually also insist that you have completed steps 1 to 4 as fully as you can, simply because otherwise the technical solution provided will not work properly. For your organization, it is especially important that the solution produces the desired result, and that means that you need to be in control of all the factors described above.

Getting started!

Which factors play the main role will depend entirely on the ambitions of your organization and which steps you take. If you are not yet using data-driven methods, and you would like to keep the process easy to implement, then start off small, without using any special software. Create a small team to analyse a task or sub-process fully, in one sprint session. The way you approach the implementation will depend on your existing infrastructure and how much freedom you have to make changes to it.

Make a prototype (in Excel for example) to learn how the process works. Once you have done this a few times, you should start to see results and then you can decide if you want to scale up further. The next logical step would be to recruit or train a business analyst and a data manager.

Want to learn more about the opportunities for data-driven working in your organization? Would you like to improve your existing data-driven processes, or take the first steps in this direction? Then why not get in touch with Gert Veldhuis at gert.veldhuis@transitionexperts.nl or on +31 (0)85 – 487 29 01? He will be happy to share his expertise with you.

Data-driven working is just one of the themes that we cover as part of our Modular Digital Transformation. It’s our way of making digital transformation clearer for everybody. The various modules give you a useful way into digital transformation within your organization. You can focus on a single subject or combine different modules of your choice.