A follow-up to the Mission to Mars post, which describes the experimental work. This one is about the methodology layer underneath it — specifically, what I got wrong.


The Mistake Was a Category Error

My background is in physics. I entered physics education research through the astro-lab and chose a framework that felt close to the work: empathise, define, ideate, prototype, test. I was building and revising classroom experiments. The design-thinking vocabulary described that activity neatly.

It did not specify how observations became defensible research claims. I had a design workflow and treated it as though it supplied sampling, data collection, analysis, reflexivity, and standards of inference. Colleagues advised me to use grounded theory because the material I most needed to understand was in student responses, including responses the design did not anticipate.

For my thesis, they were right.

That sentence needs a boundary. Design thinking is a family of innovation and design practices. Design-based research (DBR) is an educational research tradition that deliberately joins theory, intervention design, empirical study, and revision in authentic settings [[1]]. They overlap, but they are not synonyms. The earlier version of this essay criticised the first and repeatedly assigned the criticism to the second.

The five steps above are set out, for example, in the Stanford d.school’s process guide, which calls them “one suggestion of a framework” [[2]]. The guide counts surprises among the benefits of testing, which “often yields unexpected insights,” and the purposes it gives for testing are design purposes: “to refine prototypes and solutions,” to learn about the user, to refine the problem statement [[2]]. In the guide’s account, an insight serves the next design. It does not set out how an observation becomes a claim that someone else could check.


What Design-Based Research Can Do

The Design-Based Research Collective describes DBR as intertwining the design of learning environments with theoretical claims about teaching and learning [[1]]. Its cycles are not simply “try it and make it better.” The Collective’s phrase is “continuous cycles of design, enactment, analysis, and redesign” [[1]], and analysis is one of the four. A rigorous study can document the conjecture embodied in a design, collect systematic evidence, explain revisions, test consequences, and produce context-sensitive design principles or theory.

That can be a doctoral contribution. It need not pretend that an intervention has a context-free effect [[3]], and it need not reduce to a practitioner manual. Transferability comes from specifying mechanism, context, and design principle, not from erasing the setting.

DBR can also fail. A researcher who designed the intervention may privilege expected outcomes, revise several things at once, use an instrument aligned too tightly with the design, or leave the analytical chain implicit. The Collective names the tension itself: design-based researchers “regularly find themselves in the dual intellectual roles of advocate and critic” [[1]]. Those are risks to manage through study design, triangulation, negative cases, documentation, and appropriate analysis. They are not proof that DBR has “no systematic answer.”

My work had too much of the workflow and too little of that chain.


What Grounded-Theory Techniques Offered

Grounded theory is not one unchanging recipe. Glaser and Strauss’s original programme [[4]], Strauss and Corbin’s later procedures [[5]] [[6]], and constructivist variants such as Kathy Charmaz’s [[7]] disagree about coding, prior literature, researcher position, and what it means for theory to be “grounded” [[8]]. In Rieger’s account, the first edition of Strauss and Corbin’s textbook “caused a rift between Glaser and Strauss,” and Glaser’s reply, a book subtitled Emergence vs. Forcing [[9]], was “a scathing critique” of its prescribed approach [[8]]. His objection was that such procedures force a preconceived framework onto the data instead of letting theory emerge [[8]]. Charmaz took a third position. She treats data as constructed by researcher and participants together rather than found, which makes the researcher’s reflexivity, kept in memos, part of the method [[7]] [[8]].

Shared techniques include constant comparison, memo writing, category development, and theoretical sampling directed by the emerging analysis [[8]].

Theoretical sampling does not mean walking into the field with no question and no practical boundary. It means that analytical needs influence which data are sought next. Saturation is likewise a methodological judgment about category development, not an automatic counter that announces when a sample is complete.

Those techniques suited the weakness in my project. They would have forced me to record how I coded a response, compare it with disconfirming cases, write down why a category changed, and show how the next observation followed from the analysis. Grounded theory would not automatically remove bias or validate the intervention. It would make a different kind of claim possible: a theory built through a traceable comparison of student accounts.


The Observation I Let Pass

In the Mission to Mars activity, a push-fit lid sometimes came off under a pressure difference. I expected the event to clarify the direction of the net force. Students did not all interpret it that way. Some described internal air pushing outward. Some retained the language of an external vacuum pulling. Some remained unsure.

I noted this, but I did not build a systematic corpus, coding procedure, or theoretical sample around it (Fig. 1). The observation is therefore an anecdote about my implementation, not evidence for a transferable typology of pressure reasoning.

Figure 1. The lid response was noted but not systematically coded; the grounded-theory route starts by collecting such responses as data. (a) A design-based research cycle, for comparison only: my design-thinking study was not run as one. The four steps are the Design-Based Research Collective’s [[1]]; the return from redesign to enactment is my simplification of its “continuous cycles.” The response arose in the classroom, the step DBR calls enactment. (b) Grounded theory as summarised from [[8]], drawn as a sequence although data collection and analysis run together, and repeated until the analyst judges the categories saturated. The dashed line marks where the observation could have entered such a study, not what the study would have produced.

That is the missed opportunity. A grounded-theory-informed study might have developed such a typology; it might instead have shown that my categories were wrong or that the data were too thin. The counterfactual cannot be promoted to a result.

DBR would not have excused the omission. In the Collective’s description, design-based research “typically triangulates multiple sources and kinds of data to connect intended and unintended outcomes to processes of enactment,” and in one of its examples, micro-analyses of student interactions let “emergent behaviors of students” drive the redesign [[1]].

The design did not force me to discard the unexpected response as noise. My framing and analysis practice made it easy to do so.


The Postmortem

The choice was never “make an artefact or produce theory.” DBR can do both. Grounded theory can analyse a phenomenon without evaluating whether an intervention caused improvement. Other questions might require case study, interaction analysis, thematic analysis, experiment, or mixed methods. The research question and claim should govern the methodology, not the comfort of the researcher or the prestige of a label.

What I needed was an explicit chain:

  1. What claim am I trying to make?
  2. What data could support or contradict it?
  3. How will those data be sampled, analysed, and compared?
  4. What role did I play in producing both intervention and interpretation?
  5. Which part is local observation, design principle, theoretical proposal, or measured effect?

I chose a useful way to design and mistook it for a complete answer to those questions. A more rigorous DBR design might have answered them. Grounded-theory techniques might have changed the question toward the unexpected student responses. In the thesis I was actually writing, the latter was the correction people were trying to give me.

I would have understood more if I had taken it earlier.


The experimental work is described in Mission to Mars. For a later use of qualitative methodology in a related context, see AI Transcription and Grounded Theory.

Methodology literature checked through 2026-10-28.


References

[1] The Design-Based Research Collective. (2003). Design-based research: An emerging paradigm for educational inquiry. Educational Researcher, 32(1), 5–8. DOI: 10.3102/0013189X032001005

[2] Hasso Plattner Institute of Design at Stanford. (2010). An Introduction to Design Thinking: Process Guide. d.school. Archived PDF

[3] Barab, S., & Squire, K. (2004). Design-based research: Putting a stake in the ground. Journal of the Learning Sciences, 13(1), 1–14. DOI: 10.1207/S15327809JLS1301_1

[4] Glaser, B. G., & Strauss, A. L. (1967). The Discovery of Grounded Theory: Strategies for Qualitative Research. Aldine.

[5] Strauss, A., & Corbin, J. (1990). Basics of Qualitative Research: Grounded Theory Procedures and Techniques. SAGE.

[6] Strauss, A., & Corbin, J. (1998). Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory (2nd ed.). SAGE.

[7] Charmaz, K. (2006). Constructing Grounded Theory: A Practical Guide Through Qualitative Analysis. SAGE. 2nd ed., Constructing Grounded Theory, 2014. Rieger [8] cites both editions for the co-construction of data.

[8] Rieger, K. L. (2019). Discriminating among grounded theory approaches. Nursing Inquiry, 26(1), e12261. DOI: 10.1111/nin.12261

[9] Glaser, B. G. (1992). Basics of Grounded Theory Analysis: Emergence vs. Forcing. Sociology Press.


Changelog

  • 2026-07-11: Separated design thinking from design-based research, removed universal claims that DBR cannot be systematic or doctoral, bounded grounded-theory sampling and saturation, and recast the missed student-response analysis as a counterfactual rather than a result.
  • 2026-10-28: Named the grounded-theory positions the text had only alluded to, so that a reader can trace them: Glaser’s critique of the first edition of Strauss and Corbin (Emergence vs. Forcing) and Charmaz’s constructivist version, both checked against Rieger’s (2019) comparison of the traditions, with the references now numbered and cited inline. Quoted the Design-Based Research Collective’s wording for the DBR cycle and for the designer-researcher’s tension between “advocate and critic,” and added a paragraph saying that DBR would not have excused my omission, because the Collective’s description uses unintended outcomes and emergent student behaviour; the point is to keep the postmortem from reading as a verdict on DBR. Cited the Stanford d.school process guide as one statement of the five design-thinking steps and added that, in the guide’s account, an insight serves the next design, which supports the point that the workflow did not say how observations become research claims. Added Fig. 1: a DBR cycle for comparison beside a simplified grounded-theory loop, marking where the lid response arose and the grounded-theory route, collecting it as data, that it did not take.