Two analysts read the identical set of open-ended responses and walk away with genuinely different themes - not because one of them is careless, but because each person's own background, assumptions, and stake in the outcome shaped what they noticed, what they weighted heavily, and how they interpreted an ambiguous comment. This isn't a flaw to be trained out of existence. It's a basic, unavoidable feature of interpretation, and reflexivity is the practice of examining that influence deliberately, in the open, rather than pretending the analyst is a neutral, interchangeable instrument standing entirely outside the data.
Table of Contents¶
- Positionality Is a Snapshot, Reflexivity Is a Practice
- Where This Actually Shows Up in Survey Analysis
- Bracketing: Naming It Doesn't Make It Disappear
- A Practical Version for Applied, Non-Academic Work
- A Worked Example
- FAQ
Positionality Is a Snapshot, Reflexivity Is a Practice¶
Positionality describes who a researcher is in relation to the study - their role, their background, their relationship to the people or organization being studied, at a fixed point in time. Reflexivity is the ongoing practice of examining what that position actually does to the research as it unfolds - not a one-time disclosure, but a continuing habit of checking how the analyst's own vantage point is shaping what gets noticed, coded, and reported. A product manager analyzing feedback about a feature they personally championed has a specific positionality worth naming; reflexivity is the deliberate practice of repeatedly asking, throughout the analysis, whether that personal investment is quietly pulling interpretation in a favorable direction.
Where This Actually Shows Up in Survey Analysis¶
Reflexivity isn't an abstract academic concern once real coding starts - it shows up in specific, concrete decisions. An analyst who's personally frustrated with a particular team's performance may read an ambiguous comment as a criticism of that team more readily than a neutral reader would. An analyst deeply familiar with a product's internal roadmap may unconsciously interpret vague customer language through the lens of features they already know are coming, in a way a customer reading their own words back would never have intended. An analyst under pressure to justify a decision that's already been made may notice confirming comments more readily than disconfirming ones - a version of the confirmation bias covered in our guide on cognitive biases in survey analysis, specifically rooted in the analyst's own position and stake in the outcome rather than a general cognitive tendency.
Bracketing: Naming It Doesn't Make It Disappear¶
Bracketing - the practice of explicitly naming a researcher's own assumptions and expectations before diving into analysis, in order to set them aside as consciously as possible - is a commonly recommended reflexive practice, and it's worth being honest about its real limits. Naming a bias doesn't erase it; a researcher who writes down "I expect to find that onboarding is the main complaint" before coding hasn't thereby become immune to noticing onboarding-related comments more readily than they otherwise would have. What bracketing genuinely accomplishes is making the expectation visible and checkable - both to the researcher themselves during analysis, and to anyone else reviewing the findings afterward, who can now weigh the conclusions against a stated starting expectation rather than being asked to simply trust that no such expectation existed.
A Practical Version for Applied, Non-Academic Work¶
Full reflexive practice, as taught in academic qualitative methods training, often involves keeping an ongoing reflexive journal throughout a study - a real, valuable practice for a formal research project, and more overhead than most applied business analysis genuinely needs or has time for. A lighter, practical version still captures most of the value: before coding a batch of open-ended responses, spend two or three minutes writing down what result you expect to find and why, and any personal stake you have in a particular outcome. After coding, briefly compare your actual findings against that stated expectation - not to prove yourself wrong, but to notice whether the two suspiciously match more closely than the underlying ambiguity of the data would predict. For any analysis feeding a significant decision, having a second person - ideally someone without the same stake in the outcome - independently review a sample of the coded data functions as an external reflexivity check, catching an individual analyst's blind spot in a way self-reflection alone often can't.
A Worked Example¶
A CX analyst who was personally involved in designing a recent checkout flow redesign is assigned to code open-ended feedback about the new checkout experience. Before starting, they write a short note: "I expect this feedback to be mostly positive, since I believe the redesign solved real problems, and I have a personal stake in that being true." Midway through coding, they notice they've been consistently coding ambiguous comments as neutral-to-positive and flagging clearly negative comments as one-off edge cases rather than a pattern - exactly the tendency their pre-written note had predicted. Recognizing this, they ask a colleague uninvolved in the redesign to independently code a random sample of twenty responses from the same dataset. The colleague's coding surfaces a more consistent negative theme around a specific new step in the flow - a pattern the original analyst's own coding had been quietly under-weighting, caught specifically because the reflexive note made the risk visible enough to check against rather than something to discover much later, if at all.
FAQ¶
Is reflexivity only relevant for academic qualitative research?
No - it applies anywhere a person is interpreting qualitative data, including everyday business analysis. The formal journaling practice common in academic training is optional; the underlying discipline of checking your own stake in the outcome is broadly useful regardless of context.
Doesn't a completely neutral, disinterested analyst avoid this problem entirely?
In practice, true neutrality is rare - even an analyst with no obvious personal stake still brings prior assumptions, professional training, and cultural background to how they interpret ambiguous language. Reflexivity assumes some perspective is always present rather than assuming a genuinely blank, neutral reader exists.
How is this different from the confirmation bias covered in cognitive biases guides?
Confirmation bias is the general cognitive tendency to notice confirming evidence more readily; reflexivity specifically addresses how a researcher's particular position, background, and stake in a specific project shapes that tendency in that instance. They're related, overlapping concepts approached from slightly different angles.
Is a second, independent coder always necessary?
Not for every analysis, but it's worth reserving for higher-stakes projects, or specifically when the primary analyst has an unusually direct personal or professional stake in a particular outcome - exactly the situations where reflexive self-checking alone is least reliable.
Sources: Positioning Positionality and Reflecting on Reflexivity · A practical guide to reflexivity in qualitative research: AMEE Guide No. 149
For related practical guidance, see Cognitive Biases That Quietly Distort Survey Analysis and Training Two Coders to Agree.