Josh Freeze, Senior Design Researcher at Walgreens, recounts six months of conducting design research to support the COVID-19 vaccine rollout. He describes how a well-structured four-phase plan rapidly became unworkable as the rollout accelerated and stakeholder complexity exploded, and how confronting his own anxiety, rather than suppressing it, led to a fundamentally different and more effective research approach. The talk draws on philosopher Sherry Turkle's work on emotional self-awareness in qualitative research and Kierkegaard's framing of anxiety as a signal of unmet values.
Key Takeaways
- When the environment shifts faster than a fixed research plan can accommodate, restructure the cadence rather than push through. Spreading interviews across four weekly waves of five to six sessions each, with a weekly alignment meeting, kept findings timely enough to influence decisions.
- Go to where stakeholders already are. Embedding in existing product and business meetings, rather than convening new research touchpoints, improved participation and kept the researcher current with rapidly shifting priorities.
- Treat your own anxiety as a diagnostic tool. Unpacking what makes you anxious reveals the values being violated, and reframing the problem around those values opens up solutions that pure stoicism would obscure.
- Distinguish empathy from compassion. Taking on participants' emotions wholesale is unsustainable in high-volume, high-stakes research. Compassion, without full emotional simulation, protects researcher well-being without sacrificing care.
- In fast-moving research, a smaller number of interviews per cycle is not necessarily a weakness. Two or three participants independently encountering the same problem is statistically meaningful and sufficient to flag a scalable issue for stakeholders.
- Vulnerability builds trust faster than professionalism masks it. Openly acknowledging uncertainty with colleagues and stakeholders created the psychological safety needed to propose a major plan change four days before fieldwork was scheduled to begin.
Questions & Answers
- How did the gravity of the vaccine rollout affect your anxiety and your interactions with participants?
- Josh noted that anxiety is a social phenomenon and spreads between people, making it important to avoid absorbing others' distress wholesale. He described interviews with patients and frontline team members as the most emotionally taxing of his career, and said he had to consciously shift from empathy, simulating what others feel, to compassion, caring about them without taking on their emotions.
- How did you translate qualitative findings into actionable recommendations when you had so few interviews each week?
- Josh argued that two or three participants independently encountering the same problem is statistically unlikely by chance, and therefore meaningful. For single interviews with underrepresented groups such as seniors with technology difficulties, he relied on accumulated experience with those demographics to judge whether a finding was idiosyncratic or indicative of a broader pattern. He also emphasized trusting intuition as one input among several, examined with a critical lens.
- Was the shift from inviting stakeholders into research meetings to embedding in their meetings something you had done before at Walgreens?
- No. Josh said the team had operated on a fairly standard consultancy model before the pandemic, with structured kickoff calls, proposal reviews, and formal reports. The vaccine rollout environment made that model unworkable and forced a new approach.
- What will you carry forward from this experience in terms of how Walgreens conducts research?
- Josh highlighted two things: staying more embedded in ongoing business and product stakeholder meetings rather than operating separately, and a deeper conviction that design research contributes to strategy, not just tactical execution. He also noted he has clearer views on what he will not do in future projects, though the rollout was still ongoing at the time of the talk.
- How has this experience changed you personally?
- Josh said he no longer has the energy or inclination to maintain a purely professional mask. He believes more strongly in honesty, in treating every person as a full and complex human being, and in the value of vulnerability as a trust-building tool, both in research and in professional relationships generally.
Session Notes
Context: Planning Research for an Uncertain Rollout
In late October 2020, a Walgreens pharmacy product manager approached the design research team for foundational research on the end-to-end immunizations experience. The goal was to surface patient pain points and generate a roadmap that would integrate COVID-19 scheduling into Walgreens' existing immunization tools. At that point, most observers expected vaccine approval in early 2021, so the team planned to deliver findings by December.
The Original Four-Phase Plan
- Audit of existing research combined with customer satisfaction and analytics data
- Stakeholder interviews across product, business, and operations
- Workshops with pharmacists and pharmacy technicians to surface concerns and ideas
- Patient interviews, timed to coincide with peak flu shot season
How the Plan Broke Down
By early November, media reports placed vaccine approval as early as late November. The entire organization began mobilizing ahead of schedule. Stakeholder interviews expanded rapidly as each new contact referred the team to additional decision makers across product, business, and operations. The scope quickly exceeded what a single week of interviews could address, and each new conversation revealed information that the previous interviewee had not yet received.
The pharmacist and pharmacy tech workshops were redirected toward vaccine rollout concerns rather than the existing immunizations experience. The report from those workshops landed in early December to limited traction. Stakeholder priorities had already shifted multiple times since the workshops were planned. Patient interviews were postponed first to late January, then to early February.
The Role of Emotional Self-Awareness in Research
Josh grounded this section in the work of Sherry Turkle, professor of social studies of science at MIT, who argues that a researcher's own emotional reactions must be carefully examined and incorporated into the research process itself. Turkle underwent years of therapy and became a licensed clinical psychologist specifically to understand the relationship between inner life and the social worlds researchers inhabit.
I really want to use this time to emphasize how important it is that we honestly confront what we're seeing inside of ourselves when we're developing research insights.
By late January, Josh could no longer suppress his anxiety about the research plan. Rather than continuing to push through, he turned inward and identified three specific sources of anxiety.
Three Sources of Anxiety and the Values Behind Them
- Too many stakeholders: Difficult to get everyone the insights they need or to secure participation. Underlying value: stakeholder involvement.
- Constant pivoting: Priorities shifted so fast it was hard to keep track. Underlying value: timeliness of research results.
- No time to meet: Alignment was nearly impossible given everyone's workload. Underlying value: collaborative alignment before and after fieldwork.
Drawing on philosopher Samir Chopra's reading of Kierkegaard, Josh reframed anxiety not as a problem to suppress but as a signal pointing toward unmet values. Naming the values behind each anxiety made it possible to design concrete solutions.
The Pivot: A Guerrilla Research Model
Solving for Stakeholder Involvement
- Stopped creating new meetings and inviting stakeholders to join. Instead, embedded directly in the meetings stakeholders were already running.
- Created a Microsoft Teams channel dedicated to design research so any stakeholder could follow updates and contribute questions on a voluntary basis.
Solving for Timeliness
The original plan of 20 patient interviews in a single week was replaced with a rolling model: five to six interviews per week across four weeks. This required a tightly scheduled weekly rhythm.
- Monday and Tuesday: interviews
- Wednesday: planning meeting to align on the most pressing questions surfaced from stakeholder meetings
- Thursday morning: analysis of Monday and Tuesday interviews; Thursday afternoon: survey design
- Friday: debrief and delivery of findings
Most weeks included six interviews, two to three surveys, and a Friday debrief. Research topics ranged from the vaccine scheduling tool and COVID-19 testing appointment tool to patient expectations about in-store interactions. The rolling format allowed the team to adjust focus each week based on emerging priorities.
Analysis Under Constraint
With no time to review recordings and limited capacity for note-taking, Josh relied on deep in-session attention followed by an immediate post-interview brain dump. He tracked patterns across interviews manually, then used OneNote with a per-interview page and a synthesis page for cluster analysis when bandwidth allowed.
On the question of statistical weight in qualitative work, Josh argued that two or three participants independently encountering the same problem is statistically unlikely to happen by chance, making it a meaningful signal even in small weekly samples. For single interviews with underrepresented demographics, years of experience with those groups informed how much weight to assign an individual finding.
You should be with people, not with paper. It's really important to devote all of your attention to the conversation.
Personal and Professional Takeaways
- Anxiety is a social phenomenon. It spreads between people, and part of the work was learning not to absorb the anxiety of interviewees and stakeholders wholesale.
- Empathy and compassion are not the same thing. Empathy, simulating another person's feelings, is unsustainable at high volume. Compassion, caring without full emotional simulation, is more protective for the researcher.
- Being embedded in stakeholder meetings earlier and more continuously is a practice worth keeping beyond any crisis.
- Design research has strategic value, not just tactical value. Qualitative researchers' proximity to end users gives them a sensibility about human behavior that quantitative measures alone cannot replicate. 'The map is not the territory.'
- Vulnerability builds trust more reliably than a professional mask. Openly acknowledging uncertainty led colleagues and stakeholders to reciprocate, and made it possible to propose a major plan change just four days before scheduled fieldwork.
Transcript
Read the full transcript
hello hey josh how are you i'm doing very well how are you excellent thank you very much for asking i am going to go ahead and get this party started okay well and for everyone out there who um is just joining us for the first time today oh welcome welcome and for all of you who have been with us all day welcome back um this is our final uh presentation of the day um and it's gonna be a really good one i've been looking forward to this one all day um as all of you know that have sat through this throughout today and they've been in these with us in the past uh first of all my name is billy sorry i jumped in got really excited i'm with accelerant research and i'm going to be hosting the call um but as you all know kind of like the general ground rules of today um is that we are at the mercy of tech we all know that by now we've all been living in the zoo zoomosphere um just please be respectful of each other in the comments and presenters and then also we're going to have an opportunity to ask josh questions throughout the process but we're also going to have an opportunity after the presentations to be able to continue the dialogue and to network and engage and interact and so forth okay um and so i get to introduce josh to freeze am i saying that right josh yes okay wonderful wonderful um and he's going to be presenting a topic that we are all very close to um he's going to be talking from the perspective of walgreens about vaccine roll out so knowing when to pivot design research and the code 19 vaccine roll out so this should be fascinating for all of us josh is a senior design researcher over at walgreens so josh i'm going to go ahead and turn it over to you so if you want to go ahead and and start yours start sharing throughout if you want to go ahead and just put questions uh in the chat box and i'll be kind of monitoring that and i'll be sending those over your way okay josh okay sounds good thank you so i'm gonna stop sharing and there you go there we go you can start okay so uh first off you might notice that i changed the title um of this presentation from knowing when to pivot to research and anxiety um it's still the same topic but i think this one is actually a little bit more indicative of the the themes that i want to talk about um as i'm sure everyone can imagine uh the vaccine rollout uh has been chaotic and and um there have been a lot of learnings and a lot of problems that have come up along the way that have required us to um you know make changes on the fly and so i have learned a lot in the past six months about um adjusting to to change and i'd like to share some of those those insights that i've learned with all of you um so i guess before jumping into it i'll just introduce myself a little bit more uh i grew up in rural utah and then moved to chicago to do a master's degree where i actually studied chinese history i've been kind of academically promiscuous i've always switched between subjects so interdisciplinary thinking is very important to me and after graduating i worked in marketing research for a couple of years and then switched over to design research at walgreens um and one more thing uh i guess before i get into it i just want to say thank you to bill mcdowell for inviting me to participate in the session and i also want to give a big thanks to the design research team at walgreens they've given me constant support and feedback throughout the ups and downs and pivots of conducting design research during the coveted vaccine rollout and nothing makes a bigger difference as i'm sure we all know than working with thoughtful earnest caring colleagues um so with that let's get into it anyone who has worked in applied research had has had the experience of finishing a project only to wish at some point that they had done something slightly different or gone in a different direction in retrospect we often talk about having seen red flags along the way and then after we do that reflective thinking then later on in the future we're able to notice red flags when they pop up the first time but it's more difficult to notice red flags along the way especially when you're charting new territory um so today i'm i'm going to share the story of the last six months of conducting uh research to support the the vaccine rollout um and i hope that it's going to serve as a helpful case study for learning to identify and respond to new like red flags as you experience them rather than waiting to to reflect on them in retrospect but before we get into that i want to set the stage a little bit by introducing you to dr sherry turkle a social scientist at mit who has really deeply influenced the way that i think uh dr turkle is a professor of social studies of science at mit she joined the faculty there at the dawn of the computational revolution and she was drawn to studying the impact that computers have on how people describe their own minds and how they relate to one another i really admire dr turkle's ongoing insistence of the importance of emotional self-awareness as of as a really important variable in qualitative methodologies as part of her training she decided to undergo years of therapy herself and became a licensed clinical psychologist so that she could learn to unpack the relationship um that our inner lives have with the social worlds that we inhabit and she's argued persuasively that one's own emotional reactions to their research must be carefully unpacked and incorporated into the research process itself and for anyone interested in her work i highly recommend her recently published memoir the empathy diaries which is a really great showcase of the relationship between working in qualitative research and developing self-understanding so following in dr turple's footsteps i really want to use this time to emphasize how important it is that we honestly confront what we're seeing inside of ourselves when we're developing research insights uh so without any further ado let's let's jump into the covet story josh if you are i don't know if you're if you have a presentation you're a great storyteller do you have is there is it not sharing no and you were such a great storyteller that i didn't know if you were just going oh thank you no i have clicked share screen and oh you know what i didn't click the share button though but my screen was highlighted in blue so you all missed just it's okay we didn't need visual aids you painted a beautiful picture thank you no i was i i was clicking through slides i'll show you real quick that's what dr charcoal it's like isn't she great and then i had these red flags and whatever we'll move on thank you for letting me know um so i'd like everyone to to pause for a moment and think back to how you were feeling in october of last year um it's it's almost kind of hard to think back that far because so much has happened in the world since then uh but it was a really messy time as i'm sure everyone can recall the election had everyone on their toes rising coronavirus cases sent many of us back into social isolation and there was a general mood of anxiety around the question of how the covet 19 pandemic would change the global economic system additionally there was increasing discussion in october of last year about the number of vaccine candidates that were in trial at the time and around mid-october at walgreens our product and business stakeholders you know had been strategizing for a while about how to enable online scheduling for the eventual vaccine rollout one of our pharmacy product managers approached the design research team and requested that we help prepare by conducting foundational research on the current end-to-end immunizations experience because walgreens as you may or may not know offers flu shots every year and excuse me also a variety of other vaccines so we wanted to to dig into that experience uncover what pain points patients were currently having and then also include a generative component to help formulate a roadmap for the ideal immunizations experience at walgreens that would integrate covid19 appointments with the pre-existing tools for other immunizations now keep in mind that as we were planning this research in late october and early november most people expected a vaccine to be approved in early 2021 so we were intending to deliver these results by december uh so that they could be there in time uh you know for the vaccine rollout um and then based on that assumption we planned a four-phased approach that started with an audit of existing research and combined with an analysis of customer satisfaction and data analytics and then after that holding stakeholder interviews and then workshops with pharmacists and pharmacy techs to uncover what concerns they have and what ideas they had for the vaccine rollout in general and for integrating these various tools and then the fourth phase was going to be patient interviews and in the middle of the winter it's peak flu shot season so we were like perfect we can get patients to go and get a flu shot at walgreens and then come into the interview it's going to be lovely so um and we finished that first phase pretty quickly and delivered it in early november and then transitioned on to the second phase of stakeholder interviews and that is when things started to change uh in early november the media started reporting that vaccine approvals were on the horizon perhaps as early as late november and our stakeholder interviews began just as the entire organization began to mobilize in preparation for the impending rollout that was now slated to happen earlier than many had expected so during these stakeholder interviews we started out with just that core product team that had approached us and then they referred us to some others who were also very important on the business side or the operation side um and as we spoke to more stakeholders they referred us to more and more and more stakeholders because it turns out that in order to execute a vaccine rollout nationwide you need a lot of collaboration between a lot of different people and so as we realized the sheer number of decision makers here it became clear that the scope of research was really beyond the needs of just those initial few stakeholders that had approached us and in fact each of these additional stakeholders were also really needing research insights to make their decisions but all of them had slightly different questions or were coming at it from from different angles um and so to be quite honest at this point i started getting really anxious it was difficult just to fit these interviews in and to schedule them because everyone was so busy um and beyond just the scheduling there's the question of how will we account for the diversity of interests in this research how can we keep all of these different people who are so busy involved how will we make sure the right people receive the right insights at the right time it became clear that this was going to be a very complex endeavor and given the importance of the project i decided that i needed to be stoic about it and just grin and bear it unfortunately though my anxieties only grew each progressive conversation with stakeholders revealed new layers of information constant updates that the stakeholders we'd spoken to a day or two ago hadn't even known yet and as the updates filtered through the organization priorities shifted at an alarming pace it felt like a whirlwind and to be clear it still feels like a whirlwind um and each new wind of information or updates would cause someone to pivot and switch into a new direction uh with their workflow as their priorities shifted our list of research questions ballooned really beyond the scope of just these this one week of interviews that we had planned um so given those changes we pivoted our plan for the workshops with pharmacists and pharmacy technicians instead of focusing on the existing immunizations experience we focused a lot more on their concerns about the vaccine rollout and the ideas that they had and so here are some some of the the artifacts from from those workshop sessions uh we delivered the report on those workshops in early december and we were honestly really disappointed with the lack of traction that it gained with our stakeholders so many of their priorities had already shifted so many times since we planned the workshops that by the time we delivered the results they were already on to something new so in december we decided to postpone the patient interviews until late january to allow some more time for alignment but when january rolled around our stakeholders were so busy with urgent directives that there was rarely time to meet as a group and the lack of alignment on priorities led us to push the research back even further to the first week of february in the last weeks of january as i was reviewing all the research questions we'd collected the stimulus we put together and tried shaping these into a discussion guide i couldn't just grin and bear it anymore i started feeling really down about the project i knew that by the time we delivered the results priorities would have shifted again and you know we weren't certain that the results we had would even be impactful by the time they were delivered as the vaccine rollout commenced in january my ux and product colleagues were working on such specific elements uh of the scheduling flow that the holistic end-to-end insights we had planned to gather really wouldn't meet their urgent demands and additionally the generative component wouldn't be possible because no one had time to come up with ideas to test in the research and so my anxieties about the constraints of the research plan really left me feeling powerless to provide my colleagues the kind of information that they really needed for the critical decisions they were making for a while i found myself avoiding working on the discussion guide just to push away these these feelings uh but i remembered sherry turkle and the importance of analyzing your own emotions when conducting the research so i turned inward and i asked myself how am i really feeling and then as happens with me from time to time they're popped into my head a pseudo profound piece of guidance from my childhood hero albus percival world frick brian dumbledore order of merlin first class at the end of harry potter and the prisoner of azkaban dumbledore says to harry happiness can be found even in the darkest times if only one remembers to turn on the light now let me be real i actually really hate this quote a lot even when i first read it at the age of 14 i thought wow what a dumb thing to say there's no there's never any simple switch for the darkness of anxiety and powerlessness and so for years now whenever i get to that space of you know feeling very anxious or just feeling dark and down this quote pops into my mind in kind of a sardonic tone like it's coming back and mocking me with its simplistic world view it's empty gestures towards hope so here i was the week before the interviews were supposed to start with a whole bunch of anxieties and one really dumb wizard quote and at that point i stepped back and asked myself okay what's really going down here what are the reasons that i'm feeling anxious and moreover is albus percival wrong here or does he have a point should i you know question my lifelong hatred of this quote um and so i i wondered are there actually light switches here somewhere that i'm just missing right around that time i also read a really beautiful article by the philosopher samir chopra about the importance of anxiety pulling from kierkegaard's classic work the concept of anxiety chopra argues that our anxieties represent our unease about something that has been unrealized so if we unpack our anxieties we discover that they only exist because of something because there's something we value that isn't being realized according to kierkegaard our anxieties can become a sort of school for self-development by illuminating what we value thinking of anxieties as the flip side of our values can help us reframe negative situations in positive pragmatic terms and can open our eyes to solutions we hadn't even thought of yet when i pondered these anxieties after reading that article i discovered to my chagrin that dumbledore was right there was a light switch after all i could switch on the light by using my anxieties to identify my underlying values and then reframe the problems in positive terms that are coming from what i value rather than what i'm afraid of so as i sat down to unpack it and ask myself why do i feel anxious well i feel anxious because there are too many cooks in the kitchen there are so many people involved that it is difficult to get all of them the insights that they need it's difficult to get buy-in for participation for the research and moreover it's coming at a time when we desperately need research second i my second anxiety is the peace at which everything was shifting you know it was just so hard to keep track of all the changes and then the third anxiety was that there was no time to meet with anyone so even if i you know wanted to align with people i really wasn't able to um and i did not do these animations correctly please forgive me um and here's the seams okay so then i started wondering okay what values does this reveal about me and i thought about it well the number of stakeholders is causing me anxiety because i value stakeholder involvement um the pivoting and the busyness was causing anxiety because i value timeliness for research results um i'm going to have to pause here and say that unfortunately the vaccine rollout is still a whirlwind and there's been just tons of work to do on it and so i unfortunately was not able to finish all the slides as i had intended to i hope that everyone will forgive me but i will continue with just a voiceover from here with a nice little calendar match to how many vaccines have been delivered at each given time so when i thought about stakeholder involvement i realized that the problem was that our process for research at walgreens tended to be creating a new space and then asking stakeholders to come there and join it you know a kick-off call a research meeting you know things that are that are set up by us that then they have to come do and so given the constraint that there were this many stakeholders involved if i value stakeholder involvement i realized i need to go to where they're at instead of inviting them into new meetings on top of everything else i need to start joining the meetings they're already in where all of these pivots and changes and updates are happening and then the second thing is i needed a forum to be able to communicate to any and all stakeholders who were interested in the research so we created a microsoft teams channel specifically for design research and used that really on a voluntary basis just giving people updates asking questions in there and then people could participate as they would and meanwhile um i started to embed myself more in the ongoing meetings that our stakeholders were having now the second piece was timeliness we had originally planned to have 20 interviews in one week and if you are assessing the end-to-end experience of you know immunizations at walgreens i think that's a great plan but if you're trying to provide insights for an ongoing uh you know rollout of a vaccine that has never happened before then it turns out that one week of 20 interviews just is not timely the planning it takes to do it takes so much time that by the time you deliver the results the results are already too old and so instead of having 20 interviews in one week we reached out to the recruiters and asked them to reschedule those 20 interviews across four weeks and so we had five to six interviews set up each week for four weeks and then instead of planning what we were going to do in each interview or sorry in each week ahead of time we held a weekly weekly meeting um to align on uh you know what we gathered were the most important things based on the other meetings that that i had now become embedded in and so in order to do this we really had to create a well-oiled machine every single hour of every day was planned out to the tea we had interviews on monday and tuesday and then on wednesday we had a planning meeting for the next day and thursday was spent half of it in analysis of the findings on monday and tuesday and usually the other half of it creating a survey because we also needed um a lot of quantitative insights and so most and then delivering the results on friday and so most weeks there were six interviews two to three surveys and then a debrief on that friday and it it was very difficult as i'm sure you can imagine i think that um the the thing that got me through it was just knowing how important you know the vaccine rollout was and feeling that sense of of meaning but i found that even though my workload doubled i was so much more um emotionally stable and confident and invested in the work than i had been before because i had gone through that process of really uh you know examining how the situation made me feel and how the research problems made me feel and then really digging deep to question what underlying values were causing those feelings to emerge now in the process of these four weeks of research we covered all sorts of different topics from the vaccine scheduler tool to the tool that people use to set up appointments for covet testing to patient expectations about in-store interactions etc etc and by the end of it we were able to gather a much more holistic perspective than we ever could have just in that in doing one week of research um and since then uh things have continued not um at not exactly that pace but sometimes very close to it um and we've applied the same approach with doing research uh among our pharmacy technicians and pharmacists to identify your pain points that that they're experiencing um and we've done a lot of work to [Music] align with stakeholders on how to implement these findings as fast as possible and something else that i have learned in that process um and i'm kind of going off script here i've been off script for a while if i'm honest but um i think that that something really important that i've learned in that process is that you can't always just follow the regular way of doing things and that when you break away from the regular way of doing things people will often surprise you with how supportive they are when i went to my manager and said you know like i i don't think that this one of research is a good idea i think we should do something else i was nervous uh because my proposal felt really big um like a huge shift and it was literally four days before the research was supposed to start um and i was just blown away by how supportive she was and how supportive the entire organization was um and so another i think key takeaway uh for me from this whole experience is that developing trust trusting relationships with your colleagues is a really critical component to managing your own well-being in the midst of a hurricane um i will i guess i will stop there and open it up for any questions okay josh i'm going to get the party started here with questions because what you've shared with us is absolutely fascinating to have this inside look at the walgreens apparatus as all of this was going on and and for you to share so intimately how it affected you thank you um that is very refreshing um and with that said you know one of the questions that i have is is part of the anxiety that you felt to you know i i understand like all the stakeholders and everything changing in the world is just but but the gravity of what you were working on how did that play into it and how did how did that change the dynamic amongst you know everyone that you touched i think that um i mean in in in this talk i've really framed it as my anxieties uh and i think that's actually inaccurate anxiety is a social phenomenon you know it spreads through people um and so a lot of the process was also you know learning to not take on other people's anxieties and the the gravity of it all has honestly been really difficult i um the the sorts of interviews that i've had with patients and also with team members on the ground have been the most emotionally taxing interview experiences of my career and something that i've taken away from that is that there's a critical difference between empathy and compassion and that sometimes in order to protect ourselves we should not exercise empathy empathy you know is the attempt to simulate in your own head what the other person is going through to feel the feelings they're going through and when you're encountering so many people with so many feelings you just can't hold that all and so it's been really important for me to to sometimes let go of that emotional connection be a little bit more analytical and focus instead on just trying to exercise and feel compassion for the people i'm interacting with instead of taking their emotions on wow and the the gravity of of the work that you were doing too part of it is like you said just being able to have these really deep conversations with people and and to kind of understand their pain but not make it you know overwhelm you but then there's also the analysis part you know qualitative is already difficult because it's not uh you know you're not looking at numbers right you're looking at people's thoughts and you're translating those thoughts and so forth so how did that play into it because again the work that you're doing was very critical in developing this roadmap for your team and the translation of the information definitely so um in terms of the analysis well so the first thing is that that when we switched to this more guerrilla research approach we knew that there was a trade-off that that when you're doing this volume of of research each week there just does not exist enough time in the day to do as rigorous an analysis as you know would be ideal um having said that i think that the the critical thing um in in translating the qualitative findings into actionable recommendations was to be able to clearly articulate the reasons why what a couple people said in interviews can translate into a scalable problem so one thing that i will often uh talk about with stakeholders uh when they say okay yeah this only came up in in one sorry in like two or three interviews you know out of like the six um then often the way i think about it is well statistically speaking how likely is it that any given two or three people will encounter the exact same problem and the answer is that it's actually statistically very unlikely and so as soon as you have a couple people encountering the same problem and when you're moving at such high speeds with you know that low number of interviews each week then that's the basis you have on which to judge and statistically speaking you know it is more probable that if you know something came up with two or three people it's more likely than if it came up with one or on the flip side um there were often times when we couldn't get a good enough demographic mix as we would like and so maybe one week we were only talking to one senior um who struggles with technology uh and so you know the the issues that that came up with them we would have to take more seriously even though it was only a single person and i think that a lot of that interpretation and like how to interpret how to interpret the findings from like a single interview with a senior about this uh really just comes down to having um a few years of experience working with these demographics on these issues that that over time if you remain in the same subject area you start to see patterns and you start to intuit what is and is not idiosyncratic in in what people are telling you um and i think it's it's really important as qualitative researchers that we listen to our intuition and that we listen to it with a critical lens uh because i do think that that intuition and emotion play critical roles in cognition and that's part of you know how we think and how we process things um so so i think it's important to like allow yourself to take that into account um as one piece of data i hope that answered the question i feel like i'm no you're not it's fascinating and you also spoke to the importance of having a team around you that's established trust so i'm going to assume too that in the scenarios that you talk to again if you're if you don't have quantitative data to support your analysis in the translation of what you've heard that level of in your using your intuition and and the experience you've gained over time that that level of trust has to be there with your teams so that when you do say that they're like i trust josh that this is a fair analysis of the situation and tactically speaking because i think i think that um maybe a more pragmatic answer to your question would be i did not have time to review any of the recordings no one else had bandwidth to take notes for me uh usually every now and again they did and so i like the way that i did the analysis was i really deeply believe in qualitative research um you should be with people not with paper i don't know if anyone's heard that one before um but i i believe it's really important to devote all of your attention to the conversation and so i would do that and then immediately after the conversation just do a really big brain dump and then as i started seeing trends between people i would start you know tracking like what came up where but it was really just after the conversation uh for the most part and luckily uh in in subsequent rounds of research we've had a lot more availability for note-taking and so we've been able to um you know build out uh i i like to use onenote so i'll build out a notebook with you know a single page for each interview and then put a template in there for the note takers um and so when when we have the luxury of of taking notes or have enough time to like review recordings uh and take notes on them then then that's usually my go-to is putting things in one note and then uh creating a synthesis page and you know dragging quotes from individual pages there and doing what we all do you know the cluster analysis sort of stuff so is there any questions i have more but are there any questions uh from any of our part uh any of our members who are joining us they're just fascinating they're in awe of your process um i i guess one of the one of the things i thought was very interesting to josh is when you took us through that in a very planful way you had this all constructed you know the whole framework put laid out and at some point you realized that you you had a problem and it in engaging the stakeholders because their world was spinning too and there were so many of them and then at that point you reframed the problem right and you went from how do we get them to interact with us to how do we find a way to interact with them like i think that was fascinating is that something that you've encountered at walgreens before or was that kind of a unique situation here that was kind of a unique situation here i think prior to the pandemic we had a fairly standard operating model for research um we so our team kind of works as an internal consultancy you know we have a team of researchers we we often switch around between subjects and projects and it's pretty templated you know you have your kickoff call where you align on the research questions and review the research proposal and then you meet again later to review all the screening criteria and then the discussion guide um and then you do the research and then have a formal report and and then you move on to the next project um and that's kind of you know just been our our the way of operating that we have found to be most successful in this organization and then the the context change when when the rollout happened um you know just it totally shifted the the environment and you know when you're in a new environment you have to adapt and find new ways to to do things what have you i'm curious and because you've had to change a lot about the way that you operate um what will you take with you what will you keep with you and and apply moving forward even though you know it won't be like this this is like kind of a once-in-a-lifetime type of problem that you had to tackle but are there things that you learn that you're like we should implement those moving forward in the way we work i have to take a second because like this isn't over right i gave you like chapter one and so it's like i don't know that i've had the time to like sit back yet and think about like in the long term how does this you know change what we'll do i think that that the um i i feel like i i have a lot more thoughts on what we won't do in the future which is which is really really good yeah um but uh hmm i think that being more embedded in those uh conversations between uh business stakeholders and product stakeholders uh is really important and i'm really happy to have experienced that now because prior to working at walgreens i was at a consultancy and so you know you're also like external there not really in the embedded in the internal meetings usually um and working at walgreens felt pretty similar to that for you know until until the pandemic and then since then i've been a lot more integrated into other meetings and i think that's something important that i'll continue and then i think a second really really big takeaway that's now occurring to me is the importance of design research for strategy i think that people often tend to think of user research as being tactical you know very in the weeds and i've i've found sometimes that the the sort of pushback you get from from uh some leadership about usability problems is well that's just a small detail you know it's just like a small detail we have bigger fish to fry um but unfortunately we live in a world composed of small details everything is just like the composition of many many details and um i think that that very very small tactical decisions have can have huge strategic impacts and i also think that um qualitative researchers in general but then especially user and like design researchers are so close to the people actually using the end product that we just develop um a a more acute sensibility for how people tend to react to things in general um and i think that there is a large bias among people who have a business background or other backgrounds for quantitative measures um when and i absolutely love quantitative measures don't get me wrong but the and i'm sure i'm preaching to the choir here but um i i always come back to the idea that the map is not the territory you know quantitative measures kind of show you the map but the map of a place will never tell you like how the flowers smell there you know or you know any other little details that actually inform people's behavior um i've forgotten what the question was at this point forgive me it's okay so so i guess the the other side to this to this coin is you were sharing like how it will potentially impact you know walgreens moving forward in regards to research and design research but what about you personally i mean you shared with us some some pretty personal you know feelings that you had around anxiety and going through this whole process how will this change you what you've gone through oh i think that one thing already has changed in me and that's evidenced by the kind of presentation i gave is that i don't i just feel like i don't have time to put on a professional mask all the time anymore i'm just like so sorry this this is what you've got this is what you're dealing with and um i think it's really i think i think that i believe more than ever in the importance of being honest with people and the importance of treating everyone you interact with as a full complex human being and being willing to be vulnerable uh usually opens other people up to also be vulnerable and can really help to build trust and so uh i think that's that's been one big takeaway of mine oh that's amazing personally that's amazing that you know just that um ability to connect with i mean it's so important to qualitative research right that you have to have that ability to very quickly connect with somebody so that they will you know they'll open up to you and i'm sure that you know like you said with the patients that you had to talk to and the stakeholders and so forth that wasn't an easy place to go with them um but thank you so much for being the person that could go there with them yeah um i don't know are we what's time uh we have ten more minutes so there's anything else you'd like to share with us oh i just i just had something pop into my head related to what you said that i that i really wanted to share which was that i watched a beautiful documentary the other day about friendships between animals of different species on youtube highly recommended but one thing i took away from it that was really interesting is that that in play between animals of all species but then also intraspecies you have this flip between aggressive behavior and vulnerable behavior that um and the aggressive behavior um helps animals to learn because it turns on their anxiety which helps them remember more but then the vulnerable behavior of you know like like a dog laying down and showing its belly or something like that is an indication that like i'm friends with you you know we can trust each other and so i think that that it's like like something so similar like something so simple that we see all over the animal kingdom you know that that we really need to reflect on in our own professional relationships as well that that the way that you build rapport with people is sometimes through showing your soft side or showing your weak side that that's not something that should be hidden oh i won that's a wonderful way to end it um i am a uh a big um reader of brene brown and all of the work that she does around empathy and vulnerability and so forth and and how she's applied that to her own research practice and then just helping people i think that's a beautiful story that you shared with us thank you so much um for everything you've done um at walgreens um it's it's amazing work um and you know such such valuable work that you've done so thank you i know it cannot be easy and i know that you're like you said the road is still ahead of you that we are not out of this yet so you have a lot of work to do so thank you um i'm sure i will uncover so many more values our values will be uncovered you will learn so much about yourself but you know after you've gone through this process and i also want to thank everyone else who's um who's attended today's today's presentations uh we have learned so much we are so blessed in our industry to have so many smart people who are willing to share with each other um and so that is wonderful please we will be sending out presentations so these will all be made available to everybody so you can always reach out to presenters and to us if you want more information also keep a lookout in your inbox there'll be more details to come for next month's events so thank you everybody and i hope you all have a wonderful day thanks josh thank you have a wonderful thank you you
