Holistic Evaluations in Design Science Research: A Framework for Systematic Selection of Evaluation Method and Effective Artifact Evaluation

Design Science Research (DSR) has established itself in business informatics as a key method for developing and evaluating innovative artifacts. Nevertheless, there is often no systematic basis for the precise selection of evaluation methods that meet different artifact types, criteria, and practical contexts. This paper closes this gap by identifying six superordinate artifact types, ten evaluation criteria, and eight evaluation methods, as well as bringing them together in a framework. In line with a concept-centered literature review, the evaluation methods used in the analyzed studies are systematically assigned to each combination of artifact type and criterion. The results show that a “one-size-fits-all” approach is not expedient: depending on the maturity of the artifact, available resources, and contextual requirements, different evaluation methods must often be used in combination. In addition, our paper highlights the growing need for innovative forms of evaluation—such as participatory approaches or real-time analyses—especially in the field of AI-based artifacts. The presented framework provides researchers and practitioners with a sound basis for decision-making on DSR evaluation designs while simultaneously highlighting open research gaps, for example regarding feasibility or validity for certain types of artifacts. Future studies should empirically validate the applicability of the framework in different industries and take emerging criteria (e.g., ethical accountability, sustainability) even more into account.