Design Research in Information Systems

[Overview of Design Research]  [Can Design be Research]  [Outputs of Design Research]
[ An Example of Community Determined Outputs] The Philosophical Grounding of Design Research]
[ Design Research Methodology]  [A Design Research Example]
[Design Research vs. Design] [Citatations On Design Research]  [Resources For Design Research]

Welcome to the page on Design Research in Information Systems (IS). The intent of the page is to provide design researchers in IS as well as others interested in design research with useful information on understanding, conducting, evaluating, and publishing design research.

  Introduction

This page is dedicated to design research in Information Systems (IS). Design research is yet another "lens" or set of analytical techniques and perspectives (complementing the Positivist and Interpretive perspectives) for performing research in IS. Design research involves the analysis of the use and performance of designed artifacts to understand, explain and very frequently to improve on the behavior of aspects of Information Systems. Such artifacts include - but certainly are not limited to - algorithms (e.g. for information retrieval), human/computer interfaces and system design methodologies or languages. Design researchers can be found in many disciplines and fields, notably Engineering and Computer Science, using a variety of approaches, methods and techniques. In Information Systems, following a number of years of a general shift in IS research away from technological to managerial and organizational issues, an increasing number of observers are calling for a return to an exploration of the "IT" that underlies all IS research (Orlikowski and Iacono, 2001).

The page is organized as follows. We begin with a general overview of design research, provide its philosophical and epistemological underpinnings, and contrast design research in IS with traditional positivist and qualitative research in IS. This is followed by a section on design research methods that includes an extended discussion of a published example of design research in IS. Through the example we hope to make concrete all phases of the design research method: artifact design, construction, analysis and evaluation. This is then followed by a number of resource sub-sections that relate to design research in general as well as to design research in IS: citation lists, links to resources on the Internet for design researchers, calls for papers and links to conferences, workshops, journals and communities of practice for IS design research.

The goal is to provide the IS community with useful information on design research both in and outside of IS. The page contains numerous citations permitting the interested reader to easily access original material on and examples of this unique and dynamic IS research paradigm.

If you wish to cite this work, the complete citation information is included below. Please send suggestions for improvement to the Section Editors at: vvaishna@gsu.edu or kuechler@unr.edu   

         Research

Drawing heavily from Kuhn (1996; first published in 1962) and Lakatos (1978), research can be very generally defined as an activity that contributes to the understanding of a phenomenon. In the case of design research, all or part of the phenomenon may be created as opposed to naturally occurring. The phenomenon is typically a set of behaviors of some entity(ies) that is found interesting by the researcher or by a group - a research community. Understanding in most western research communities is knowledge that allows prediction of the behavior of some aspect of the phenomenon. The set of activities a research community considers appropriate to the production of understanding (knowledge) are its research methods or techniques. Historically, some research communities have been observed to have nearly universal agreement on the phenomenon of interest and the research methods for investigating it; in this page we term these paradigmatic communities. Other research communities are bound into a nominal community by overlap in sets of phenomena of interest and/or overlap in methods of investigation. We term these pre-paradigmatic or multi-paradigmatic research communities. As of the writing of this page Information Systems is an excellent example of a multi-paradigmatic community.

          Design

Design means "to invent and bring into being" [Webster's Dictionary and Thesaurus, 1992]. Thus, design deals with creating something new that does not exist in nature. The design of artifacts is an activity that has been carried out for centuries. This activity is also what distinguishes the professions from the sciences.  "Schools of architecture, business, education, law, and medicine, are all centrally concerned with the process of design" [Simon, 1996; first published in 1969]. However, in this century natural sciences almost drove out the design from professional school curricula in all professions, including business, with exceptions for management science, computer science, and chemical engineering -- an activity that peaked two or three decades after the Second World War [Simon, 1996].

Simon sets out a prescription for schools of business (in which most IS departments are housed) that has motivated this page to a considerable degree: ". . . The professional schools will reassume their . . . responsibilities just to the degree that they can discover a science of design, a body of intellectually tough, analytic, partly formalizable, partly empirical teachable doctrine about the design process . . . "

To bring the design activity into focus at an intellectual level, Simon [1996] makes a clear distinction between "natural science" and "science of the artificial" (also known as design science):

A natural science is a body of knowledge about some class of things -- objects or phenomenon -- in the world (nature or society) that describes and explains how they behave and interact with each other. A science of the artificial, on the other hand, is a body of knowledge about artificial (man made) objects and phenomena designed to meet certain desired goals.

Simon further frames sciences of the artificial in terms of an inner environment, an outer environment, and the interface between the two that meets certain desired goals. The outer environment is the total set of external forces and effects that act on the artifact. The inner environment is the set of components that make up the artifact and their relationships – the organization – of the artifact. The behavior of the artifact is constrained by both its organization and its outer environment. The bringing-to-be of an artifact, components and their organization, which interfaces in a desired manner with its outer environment, is the design activity. The artifact is “structurally coupled” to its environment and many of the concepts of structural coupling that Valera (1988) and Maturana (1987) have developed for biological entities are applicable to designed artifacts.

In a perspective analogous to considering design as the crafting of an interface between inner and outer environment, design can be thought of as a mapping from function space - a functional requirement constituting a point in this multidimensional space - to attribute space, where an artifact satisfying the mapping constitutes a point in that space (Takeda, et al, 1990). Design then, is knowledge in the form of techniques and methods for performing this mapping – the know-how for implementing an artifact that satisfies a set of functional requirements.

    Can Design Be Research

The question this page intends to answer in the affirmative is: can design (i.e. artifact construction) ever be considered an appropriate technique for conducting research into Information Systems? We will pursue the specific question in the next section. For the remainder of this section we discuss the question in the abstract – can design be research? – using as exemplars communities other than IS where the question of whether or not design is a valid research technique has for many years been a resounding Yes!

Owen (1997) discusses the relation of design to research with reference to a conceptual map of disciplines (Figure 1) with two axes: Symbolic/Real and Analytic/Synthetic. The horizontal axis of the map position disciplines according to their defining activities: disciplines on the left side of the map are more concerned with exploration and discovery. Disciplines on the right side of the map are characterized more by invention and making. The map’s vertical division, the symbolic/real axis, characterizes the nature of the subjects of interest to the disciplines – the nature of the phenomena that concerns the research community. Both axes are continua and no discipline is exclusively concerned with synthesis to the exclusion of analytic activities. Likewise, no activity is exclusively concerned with the real to the exclusion of the symbolic although the strong contrast along this axis between the physical science of chemistry (real) and the abstract discipline of mathematics (symbolic) is strongly and accurately indicated in the diagram.

The disciplines that lie predominantly on the synthetic side of the map are either design disciplines or the design components of multi-paradigmatic disciplines. Design disciplines have a long history of building their knowledge base through making – the construction of artifacts and the evaluation of artifact performance following construction. Architecture is a strongly construction-oriented discipline with a history extending over thousands of years. The architectural knowledge base consists of a pool of structural designs that effectively encourage the wide variety of human activities and has been accumulated largely through the post-hoc observation of successful constructions (Alexander, 1964). Aeronautical engineering provides a more recent example. From the Montigolfer balloon through WWI, the aeronautical engineering knowledge base was built almost exclusively by analyzing the results of intuitively guided designs – experimentation at essentially full scale.  

Owen(1997) further presents a general model for generating and accumulating knowledge (above) that is helpful in understanding design disciplines and the design research process: “Knowledge is generated and accumulated through action. Doing something and judging the results is the general model . . . the process is shown as a cycle in which knowledge is used to create works, and works are evaluated to build knowledge." While knowledge building through construction is sometimes considered to lack rigor, the process is not unstructured. The channels in the diagram of the general model are the “systems of conventions and rules under which the discipline operates." They embody the measures and values that have been empirically developed as “ways of knowing” as the discipline has matured. They may borrow from or emulate aspects of other disciplines channels, but, in the end, they are special to the discipline and are products of its evolution."

Takeda, et al. (1990) have analyzed the reasoning that occurs in the course of a general design cycle illustrated in Figure 3. This diagram can be interpreted as an elaboration of the Knowledge Using Process arrow in Figure 2. In following the flow of creative effort through this diagram the types of new knowledge that arise from design activities and the reason that this knowledge is most readily found during a design effort will become apparent.

* An operational principle can be defined as “any technique or frame of reference about a class of artifacts or its
characteristics that facilitates creation, manipulation and modification of artifactual forms” (Dasgupta, 1996; Purao, 2002)

In this model all design begins with Awareness of a problem. Design research is sometimes called “Improvement Research” and this designation emphasizes the problem- solving/performance-improving nature of the activity. Suggestions for a problem solution are abductively drawn from the existing knowledge/theory base for the problem area (Pierce, 1931). An attempt at implementing an artifact according to the suggested solution is performed next. This stage is shown as Development in the diagram. Partially or fully successful implementations are then Evaluated (according to the functional specification implicit or explicit in the suggestion). Development, Evaluation and further Suggestion are frequently iteratively performed in the course of the research (design) effort. The basis of the iteration, the flow from partial completion of the cycle back to Awareness of the Problem, is indicated by the Circumscription arrow. Conclusion indicates termination of a specific design project.

New knowledge production is indicated in Figure 3 by the arrows labeled Circumscription and Operation and Goal Knowledge.  The Circumscription process is especially important to understanding design research because it generates understanding that could only be gained from the specific act of construction. Circumscription is a formal logical method (McCarthy, 1980) that assumes that every fragment of knowledge is valid only in certain situations. Further, the applicability of knowledge can only be determined through the detection and analysis of contradictions – in common language, the design researcher learns or discovers when things don’t work “according to theory." This happens many times not due to a misunderstanding of the theory, but due to the necessarily incomplete nature of ANY knowledge base. The design process, when interrupted and forced back to Awareness of Problem in this way, contributes valuable constraint knowledge to the understanding of the always-incomplete-theories that abductively motivated the original design.  

  The Outputs of Design Research

Even within design research communities there is lack of consensus as to the precise objective – and therefore the desired outputs - of design research. In this page we present a broad perspective that explicates the types and levels of knowledge that can be derived from design research while reserving judgment on whether a narrower goal of design research should be held within any specific research community.

March and Smith (1995) in a widely cited paper contrasting design research with natural science research, propose four general outputs for design research: constructs, models, methods, and instantiations. Constructs are the conceptual vocabulary of a problem/solution domain. Constructs arise during the conceptualization of the problem and are refined throughout the design cycle. Since a working design (artifact) consists of a large number of entities and their relationships, the construct set for a design research experiment may be larger than the equivalent set for a descriptive (empirical) experiment. 

A model  is “a set of propositions or statements expressing relationships among constructs.” March and Smith identify models with problem and solution statements. They are proposals for how things are. Models differ from natural science theories primarily in intent: natural science has a traditional focus on truth whereas design research focuses more on (situated) utility. Thus a model is presented in terms of what it does and a theory described in terms of construct relationships. However a theory can always be extrapolated to what can be done with the implicit knowledge and a set of entities and proposed relationships can always be expressed as a theoretical statement of how or why the output occurs.

A method is a set of steps (an algorithm or guideline) used to perform a task. "Methods are goal directed plans for manipulating constructs so that the solution statement model is realized." Implicit in a design research method then is the problem and solution statement expressed in the construct vocabulary. In contrast to natural science research, a method may well be the object of the research program in design research. Since the axiology of design research stresses problem solving, a more effective way of accomplishing an end result – even or sometimes especially a familiar or previously achieved end result – is valued.

The final output from a design research effort in March and Smith’s explication is an instantiation which “operationalizes constructs, models and methods.”  It is the realization of the artifact in an environment. Emphasizing the proactive nature of design research, they point out that an instantiation sometimes precedes a complete articulation of the conceptual vocabulary and the models (or theories) that it embodies. We emphasize this further by referring to the aeronautical engineering example given earlier in this page: aircraft flew decades before a full understanding of how such flight was accomplished. And, it is unlikely the understanding would ever have occurred in the absence of the working artifacts.

Rossi and Sein (2003) and Purao (2002) in an ongoing collaborative effort to promote design research have set forth their own list of design research outputs. All but one of these can be mapped directly to March and Smith’s list. Their fifth output, better theories, is highly significant and merits inclusion in our general list of design research outputs. Design research can contribute to better theories (or theory building) in at least two distinct ways, both of which may be interpreted as analogous to experimental scientific investigation in the natural science sense. First, since the methodological construction of an artifact is an object of theorizing for many communities (e.g. how to build more maintainable software), the construction phase of a design research effort can be an experimental proof of method or an experimental exploration of method or both.

Second, the artifact can expose relationships between its elements. It is tautological to say that an artifact functions as it does because the relationships between its elements enable certain behaviors and constrain others. However if the relationships between artifact (or system) elements are less than fully understood and if the relationship is made more visible than previously during either the construction or evaluation phase of the artifact, then the understanding of the elements has been increased, potentially falsifying or elaborating on previously theorized relationships. (Theoretical relationships enter the design effort during the abductive reasoning phase of Figure 3). For some types of research, artifact construction is highly valued precisely for its contribution to theory. Human-Computer Interface (HCI) researchers Carroll and Kellogg (1989) state that “. . . HCI artifacts themselves are perhaps the most effective medium for theory development in HCI.” Walls, Widmeyer and El Sawy (1992) elaborate the theory building potential of design and construction in the specific context of IS. Table 1 summarizes the outputs that can be obtained from a design research effort.

A different perspective on the output of design research is developed in Purao (2002) following Gregg, et al. (2001). In Figure 4 the multiple outputs of design research are classified by level of abstraction. 

Explicitly the upper level of Figure 4 and implicitly the middle level, knowledge about operational principles, are theories about the emergent properties of the inner environment of the artifact (Simon, 1996). However, in any complex artifact, at either level of abstraction, multiple principles may be invoked simultaneously to explain aspects of the artifacts behavior. In this sense, the behavior of the artifact in any single design research project is overdetermined (Carroll and Kellogg, 1989). This inevitable aspect of design research has consequences discussed further in the section on Philosophical Grounding of Design Research.

  An example of community determined outputs

Precisely what is obtained from a design research effort is determined by (1) the phase of research on which reflection and analysis focuses (from Figure 3) and (2) the level of abstraction to which the reflection and analysis generalize (from Figure 4). These factors in turn are strongly influenced by the community performing the research.

To illustrate the different outputs that are commonly seen as the desired result for design research, consider the same artifact development as carried out by different IS research sub-communities: database, software engineering, Human-Computer interface (HCI), decision sciences, and IS Cognitive Researchers (IS Cognitive Research Exchange -- IS CORE): the construction of a data visualization interface for complex queries against large relational databases. For all of the communities, the research is motivated by a common problem awareness: that a better interface can be developed that will allow users to more quickly and effectively obtain answers to questions about the performance of their business operations.

The theoretical impetus for the prospective improvement would vary between research communities. For the software engineering or database communities the motivation could be new knowledge of faster access techniques or visual rendering techniques.  For the decision sciences community and the HCI and cognitive research communities the impetus could be new research in reference disciplines on visual impacts on cognition and/or on decision-making. The resulting artifact would be quite similar for all communities, as would the construction mechanics – the computer languages used in development, the deployment platforms, etc. However the stages of development on which observation and reflection centered and the measures used to evaluate the resultant artifact (cf. Figure 3) would be considerably different for each community.  Table 2 lists the communities that might construct a data visualization artifact, the primary perspective with which they would view the artifact and the different knowledge that would emerge from the research effort as a result of the differing perspectives.

Some explications of design research in IS have stated that the primary focus is always on the finished artifact and how well it works rather than its component interactions i.e. why it works (Hevner, et al, 2004). Other writers and our example present a broader view. The apparent contradiction may simply be in how wide the net of IS Research is cast and the selection of sub-communities it is considered to contain.

General References on Design Research

  The Philosophical Grounding of Design Research

Ontology is the study that describes the nature of reality: for example, what is real and what is not, what is fundamental and what is derivative?

Epistemology is the study that explores the nature of knowledge: for example, on what does knowledge depend and how can we be certain of what we know?

Axiology is the study of values: what values does an individual or group hold and why?

The definitions of these terms are worth reviewing because although assumptions about reality, knowledge and value underlie any intellectual endeavor, they are implicit most of the time for most people, including researchers. Indeed, as historians and philosophers of science have noted, in “tightly” paradigmatic communities, people may conduct research for an entire career without considering the philosophical implications of their passively received areas of interest and research methods (Kuhn, 1996; first published in 1962). It is typically only in multi-paradigmatic or pre-paradigmatic communities - such as IS - that researchers are forced to consider the most fundamental bases of the socially constructed realities (Berger and Luckman, 1966; Searle, 1995) in which they operate.

The contrasting ontological and epistemological assumptions implicit in natural science and social science research approaches have been authoritatively explicated in a number of widely cited works (Bunge, 1984; Guba and Lincoln, 1994). Gregg et al. (2001) add the meta-level assumptions of design research (which they term the Socio-technologist / developmentalist approach) to earlier work contrasting positivist and interpretive approaches to research. We have drawn from Gregg, et al. in compiling Table 3 which summarizes the philosophical assumptions of those three “ways of knowing” and have added several insights from our combined 40+ years of design research experience. Our first addition is the stress on iterative circumscription (cf. Figure 3) and how this essential part of the design research methodology iteratively determines (or reveals) the reality and the knowledge that emerge from the research effort. The second addition to Table 3 is the row labeled Axiology – the study of values. We believe it is the shared valuing of what researchers hope to find in the pursuit of their efforts that binds them into a community. Certainly the self and community valuation of their efforts and findings is a highly significant motivator for any researcher, and we were surprised to find how little stress this topic has received in the literature, especially given the significant differences in what each community values.

The metaphysical assumptions of design research are unique.  First, neither the ontology, epistemology, nor axiology of the paradigm is derivable from any other. Second, ontological and epistemological viewpoints shift in design research as the project runs through circumscription cycles depicted in Figure 3. This iteration is similar to but more radical than the hermeneutic processes used in some interpretive research. 

Design research by definition changes the state-of-the-world through the introduction of novel artifacts. Thus, design researchers are comfortable with alternative world-states. The obvious contrast is with positivist ontology where a single, given composite socio-technical system is the typical unit of analysis; even the problem statement is subject to revision as a design research effort proceeds. However, the multiple world-states of the design researcher are not the same as the multiple realities of the interpretive researcher: many if not most design researchers believe in a single, stable underlying physical reality that constrains the multiplicity of world-states. The abductive phase of design research (Figure 3) in which physical laws are tentatively composed into a configuration that will produce an artifact with the intended problem solving functionality virtually demands a natural-science-like belief in a single, fixed grounding reality.

Epistemologically, the design researcher knows that a piece of information is factual and knows further what that information means through the process of construction/circumscription. An artifact is constructed. Its behavior is the result of interactions between components. Descriptions of the interactions are information and to the degree the artifact behaves predictably the information is true. Its meaning is precisely the functionality it enables in the composite system (artifact and user). What it means is what it does. The design researcher is thus a pragmatist (Pierce, 1931). There is also a flavor of instrumentalism (Hendry, 2004) in design research.  The dependence on a predictably functioning artifact (instrument) gives design research an epistemology that resembles that of natural-science research more closely than that of either positivist or interpretive research.

Axiologically, the design researcher values creative manipulation and control of the environment in addition to (if not over) more traditional research values such as the pursuit of truth or understanding. Certainly the design researcher must have a far higher tolerance for ambiguity than is generally acceptable in the positivist research stance. As many authors have pointed out, the end result of a design research effort may be very poorly understood and still be considered a success by the community (Hevner, et al, 2004). A practical or functional addition to an area body of knowledge, codified and transmitted to the community where it can provide the basis for further exploration, may be all that is required of a successful project. Indeed, it is precisely in the exploration of “wicked problems” for which conflicting or sparse theoretical bases exist that design research excels (March and Smith, 1995; Carroll and Kellogg, 1989). 

Finally, the philosophical perspective of the design researcher changes as progress is iteratively made through the phases of Figure 3. In some sense it is as if the design researcher creates a reality through constructive intervention, then reflectively becomes a positivist observer, recording the behavior of the system and comparing it to the predictions (theory) set out during the abductive phase. The observations are interpreted, become the basis for new theorizing and a new abductive, interventionist cycle begins. In this sense design research is very similar to the action research methodology of the interpretive paradigm, however, the time frame of design research construction is enormously foreshortened relative to the social group interactions typical of action research.

Bunge (1984) implies that design research is most effective when its practitioners shift between pragmatic and critical realist perspectives, guided by a pragmatic assessment of progress in the design cycle.  Purao (2002) presents a very rich elaboration on the perspective shifts that accompany any iterative design cycle. His analysis is grounded in semiotics and describes in detail how “the design researcher arrives at an interpretation (understanding) of the phenomenon and the design of the artifact simultaneously.”

References on Philosophical Grounding of Design Research

  Proceed to Part 2: Design Research Methodology (by Example)
and Design Research Resources

  Help Build This Page

You are invited to contribute links to design research material. Additionally, we are soliciting critiques of the page and short abstracts of items in the references pages (maximum 100 words). Please contact the Section Editors by email Vijay Vaishnavi or Bill Kuechler to see how you can help

  Citation Information

The complete citation for this work is as follows:

Vaishnavi, V. and Kuechler, W. (2004). “Design Research in Information Systems” January 20, 2004, last updated August 16, 2009. URL: http://ais.affiniscape.com/displaycommon.cfm?an=1&subarticlenbr=279