When we start to look at what we are doing across the breadth of the DataVaults project – i.e. developing technology to enable citizens share their data and use some of its value for their own personal gain, as well as for improving services or other uses in the data economy – the question on the evaluation of the DataVaults platform becomes daunting. Add to this, the fact that we have five demonstration sites, with citizen’s personal data being shared for very diverse purposes and you can see how the problem is exasperated.
Evaluating Software
There are some common questions to answer in the evaluation of any software solution. Does the platform do what it is intended to do? Are the interfaces suitable for all end users, both those that provide the data and those that consume them? Can the underlying design support data from citizens acting in a variety of capacities- such as tourists, travellers, attendees at sports events, athletes, energy consumers and consumers of healthcare and municipal services?
In these respects, there is precedence and the evaluation of the project’s success in terms of technological and demonstration goals can be covered through widely adopted methodologies and evaluation frameworks.
Evaluating High-Level Goals
But the evaluation task becomes more complex when our higher level, strategic goals are scrutinised – How are we satisfying the European Commission as a stakeholder? How are we contributing to the evolution of a Data economy shaped by European values? How are we sharing our knowledge or contributing to standardisation?
To tackle these questions, in addition to the processes put in place to evaluate the deployment of the technology and its operation at the five demonstration sites, it was also necessary to simultaneously investigate aspects such as:
- Does the consortium share a common understanding of what the project is trying to achieve?
- What is the logic behind what we are trying to achieve?
- What is the theory underpinning the changes we imagine happening as a result of our project?
Essentially it all sums up to defining the underlying story beneath the surface of the project. To get to that point, we utilised tools such as logic models and the Theory of Change. After much deliberation, the concept was distilled down to the premise that, if we succeed in persuading citizens to share their personal data more widely, for example with the wide uptake of personal data marketplaces, we will grow the “personal data lake”, which in turn will provide a boost to the European data economy.
These were some highlights from the work performed to shape the DataVaults Evaluation Framework. If you want to find out more, you can read our deliverable D6.1 – Project and Pilots Evaluation and Impact Measurement Plan
