How do we evaluate our new prototype?
Evaluate your new design
What is the result of this step?
This last step will lead to an evaluation of your value sensitive design or prototype.
Why?
It is important to check whether all values that were identified in previous steps have been sufficiently taken into account in the design and/or whether this had led to a design that is acceptable and appreciated by end-users.
How?
There are different ways to do this. First, you can look back to the values identified in step 3 and 4, the value conflicts and the solutions you developed for the conflicts in step 5, as well as the design requirements developed in step 6. Based on all these previous steps you can assess whether the eventually resulting design or prototype is acceptable.
But you can also do the evaluation with stakeholders. If the previous steps 3, 4 and 5 have been carried out without the participation of stakeholders, it is recommended to involve them in step 8. This is the best way to ‘check’ whether the values that you think are important for the stakeholders, conform to the values that these stakeholders consider important.
Note that Value Sensitive Design always includes an empirical phase. Therefore, it is important to actually include stakeholders and ask for their opinion at some point in the process. Below you will find an example of an interview protocol to evaluate the design of the product or service with users (see box 10).
Box 10: Example
An interview protocol
for users
Box 10
Example of an interview protocol for users
After presenting the concept for your product or service, you could ask the envisioned users to use the application. You could evaluate through the following questions:
- At first glance: would you be interested in buying this kind of product or service?
- If yes: why d you appreciate the product? What particular aspect you like most? Why?
- If n: why would you not be interested? Is there any particular reason why?
- Show a list of values that were identified before as important for the design (or use the list in the glossary):
- Which three values d you consider most important in relation to this application? Why do you think they are important?
- Which three values are clearly addressed by this prduct or service? In what way?
- Which three values are at risk when this prduct is introduced and used? In what way? How big are these risks according to you? Are there any ways to mitigate these risks, what are these ways? What needs to be done? By whom? Please explain.
- If preconditions to realize those values are accomplished and risks are mitigated: would this change your interest in the product or service?
- Given these considerations, what would your opinion be about this technology application? What are its merits/flaws?