Starting a UX Research Function from the Ground Up

Our first presenter today is going to be Mary Curran and she's from Warner Music Group. She's going to be talking to us about starting a UX research function from the ground up. Uh something that I'm sure a lot of the folks in the audience can can get behind and have experience from from one standpoint or another. So, um that's it. That's the intro. I'm going to let Mary do her thing because that's really all we're here for. So, I'm gonna stop sharing. I'm gonna let you jump in, Mary, and then I will circle back, you know, toward the end, lead some Q&A, and we'll see what's what. Great. Um, ah, there we go. I was going to say I can't present, but now I'm up. Can everybody see my screen? Okay, right now. All right. Awesome. Thank you, Bill, and thank you for the warm welcome and inviting me today to present. I'm really excited to be here. and the lineup for the rest of the day sounds awesome. So, I'll definitely be back to be an observer as well. So, yeah. Um, I'm Mary. I am currently the senior manager of UX research at Warner Music Group. It's a really cool, awesome place to work as a UX researcher. And I have been in the industry for about 10 years. And eight of those 10 years have involved me building a research function from the ground up at a company. Um I've done so as a research team of one so lonely within a small collaborative group of researchers which was a lot of fun and again now at Warner Music where I work alongside a really fantastic staff level UX researcher. Um so yeah, what can I say? I'm kind of a glutton for punishment. I seem to always be drawn to this like mission of scaling research in these misssy ambiguous spaces and um I think it's fun. It's challenging but it's a lot of fun. Um and my gluttony is to your benefit today because I'll be sharing advice for scaling research in your organization based on my own experience. We will have a Q&A for the last 10 or so minutes. So feel free to drop questions in the chat as they arise or wait until the Q&A to jump in. So, let's get started. So, when I first begin a role like this, um, I start with what any researcher would do, research. Of course, that's what we're good at, right? So, when I first begin a role, I usually take about 30 maybe 60 days depending on the size of the organization to conduct what I think of as internal research. So that entails meeting with different crossf functional stakeholders and leaders to learn about the culture and the business. It also helps you get out in front of people and become visible right away. Um and also just kind of allows you to connect with people on a human level. So often I'll you know bake in a couple of minutes at the front of that conversation to just get to know each other a little bit better. So in these crossf functional stakeholder interviews um I look for a couple of different things. I want to learn about the different organizational or cultural constraints that people have bumped up against when implementing user centered design or user research in the product development process. What do I need to watch out for right away? I also like to sus out um the research literacy within the organization. Um, how much do people know about research? How much have they worked with researchers in the past? And of course, I really want to find out what are like the big problems, user problems we want to solve. What are the things that are keeping product managers up at night? Um, and I also look for people who are research enthusiasts versus maybe someone who's a bit more resistant to research. Find out who your fans are. uh some folks are probably even taking up the mantle and practicing research already within the organization in lie of a dedicated researcher. Find out who those folks are and then who are the people that are a little bit more apprehensive or skeptical of research. Those are the folks that you might want to devote more time and care to educating, evangelizing with those particular individuals or even the entire team. And when you go through these conversations and do this research, don't feel discouraged by what you're hearing or seeing in the organization, it's important to remember that change like this is going to take time. So, I also try to lead these conversations with empathy. You know, we're already skilled empa empaths for our users. Um, but I also like to apply that skill into my stakeholder conversations as well. I try to meet them where they're at. um you know there's folks that may not have had the best experience at prior companies with researchers or may be resistant for one reason or another. So I like to also exercise the research skill probing for why kind of getting to the root cause behind any apprehension um and really actively listen and learn. But I also use this face-to-face time, you know, to listen, to learn, to empathize with the stakeholders, but also clear up any misconceptions they might have about research. So, in order to do that, I always have a couple back pocket responses um to any kind of resistance. um two great sources for these back pocket um talking points. Um Erikica Hall's book, Just Enough Research, has some great um talking points for the the most obvious resistance to research. And then more recently, a researcher named Carrie Boyd wrote a fantastic article that's circulating in our community right now that breaks down the most common objections to doing research using empirical data and really does a fantastic job of illustrating the positive ROI of research. So, I'll be sure to link those in chat. It's something that I like to bookmark, even memorize a couple of these talking points of statistics so I can weave them into even the most informal stakeholder conversation. Next, in conjunction with sort of eliciting stakeholder feedback, doing internal research, I also utilized the eight pillars of user research framework. So, this was created by team reops. Um, and they took this on so they could define the eight major areas of focus that go into operationalizing research within an organization. So, there are eight and they look like this. The first one is environment. It's aligning the or on why user research is important and who should be involved. Scope is another huge one. All the various processes, methods, frameworks by which you conduct research. Recruitment of course is hugely important. You can't do research if you don't even have access to your users. It's often the first hurdle to jump over. um gaining access to research participants, making sure we're tackling the right users and getting them involved. Knowledge management also involves not just a research repository, but how we communicate research out within the organization. People, so you know, if there's anyone that's been doing research or advocating for research in the organization, how can you bring them in? um can we create a community of practice with those folks to support and mature the craft as well. Next is organizational context. So something I also like to assess when I come into an organization is what is the UX maturity level kind of based on the maturity model that Neielson Norman Group has created. Where do they fall? Are they way down at the bottom? Are we making a little progress? And how can I get us a little bit higher up that ladder? Um, next of course is governance. Making sure we're doing legal and ethical research and of course tools and infrastructure. So our process of procurement, budget for tools, finding and procuring the right tools that will support our research and then of course prioritizing problems to solve. So I'll plug the the sort of internal research findings into this eight pillars framework and then I look across all the pillars to determine immediate needs things that are holding back teams right away. So for example at Warner Music there's certain groups of end users that historically that haven't been able to um attract those are artists. Um, so you know, it's not exactly easy to get Ed Sheeran or Dual Lipa or Lizo on the call, but what other opportunities do we have on how can I unblock the teams that need to be engaging with Warner Music artists on a regular basis? And then there's sort of the big heavier weight efforts, something that's certainly going to require more time, staff, definitely more buyin. So I often find that things like building out a robust tech stack often fun like falls into the longtail objectives because it requires a hardier budget and more buyin. So here you can see how I've leveraged the reops pillar for Warner Music. Um so the at the top are all the different pillars. Um and under each pillar my colleague and I have brainstormed re research op needs based on the internal research findings. And then my next step after this was to break out into sort of a research ops roadmap of sorts. So really defining what are the immediate needs I need to tackle. What are more medium weight things that can maybe hold off for a couple quarters? And then what's more of a long-term objective, something that I probably can't get buy in or budget or time for. Um so anything that's like a year or beyond. And so this helps me tackle or lay down a foundation for research that I can then scale so I can determine and prioritize all the research operation needs that need to be handled first. Next, set achievable goals. So by this point, uh this is the point in my role where I'm looking at the enormous volume of work that lays ahead of me and feeling a little bit stressed. Wow, I certainly have my work cut out for me. And one of my early career pitfalls is coming in hot to a space with no dedicated research function and just feeling like I needed to do all the things to start showing value right away. So, you know, this approach of course made me feel paralyzed by overwhelm and didn't really help me get stuff done. So, something that has really helped me in my work is a book called Atomic Habits by James Clear. You've probably seen it on the bestseller shelves at your local bookstore. has nothing to do with research, but I definitely like to apply those learnings in my work and in my personal life. So, I approach the atomic habits thinking when trying to scale research by really assigning myself just like weekly micro or mini goals that really ultimately ladder up to this overarching huge objective of scaling research. I go for the easy wins first, those lowhanging fruits um that help us deliver value but don't have me working on the weekends, right? So, and then the third bullet is starting to look around the organization for my first research project to lead. So, finding easy achievable research that also drives a lot of impact too. So I'm looking for the small achievable project and I have a few criteria that I look at. I'm looking at the right team. So are they research enthusiasts and as a result will the insights be actionable? Is the timeline right? Do I have time to do this project while managing other responsibilities? Does the team have enough time to iterate on designs and make sure the findings are implemented? And of course the right impact is this solving like a really huge gnarly problem for our users. And I find for me that typically looks like a project that's impactful but low smaller in scope and complexity. My colleagues and I joke that we find usability testing to be the absolute gateway drug for getting value buying for research. The reason why is stakeholders tend to be most familiar with that method and there really is nothing more powerful than observing uh an end user struggling with your product. It's very humbling and really helps us increase buyin. So, uh, while I'm partnering with the first team I'm conducting research with, I definitely like to use this collaboration as a learning experience for them so they know what to expect when working with a researcher in the future. So, along the way, I really clearly articulate my process of planning and conducting research, the different deliverables that are created within this process, and really stress why these things are important. So, they don't think that this is just busy work. We know as researchers how important it is to create a research plan, how important it is to synthesize our findings, but our stakeholders may not know. So, it's really important within the context of the project to really help them understand what's going on and why I'm doing things a certain way. And lastly, um, socialize the heck out of the research findings. You know, you did all that great work. It's time to brag about it a little bit. I usually socialize within a more intimate setting with the immediate team. So we kind of workshop how to make those findings impactful but also set up a broader insights within the uh insight share within the organization through like a research readout baking plenty of time for Q&A and that helps me increase my own visibility in the org and kind of drums up a little bit more interest and appetite for research too. Next, change begins by building relationships with other internal advocates. So, start to think about and assemble your research allies, especially if you're a team of one. You can't do this alone. So, find people who are equally committed to baking user needs into the DNA or the fabric of your environment. So, you know, it's funny. We can know all the methods and have years of experience, but I find that relationship building more and more is really half the battle in research. And you're going to find people inside your company and outside of your department that really do feel your pain. And they're not maybe defining it exactly as you use a research culture, but they might be seeing time and time again features or products fail and just know something needs to change. And that's where you can come in and start delivering value. So I also find that assembling user advocates helps me bust down silos and foster collaboration when we can align around a common problem to solve. For me in my experience that looks like a couple different things. I often find my closest partnerships, no surprise, is with the UX design team since I often roll up to that department. definitely design leadership, data analytics or data science, marketing and product managers. So often I find when I join an organization, I start working with design leaders that share some of the same frustrations of working in a low UX maturity or So we'll find ways to collaborate on things like educating and evangelizing for the entire end-to-end user centered design process, research and iterating on designs and creating user flows and personas. We're all very closely intertwined. Um, and do that to just really show the benefits of the endto-end user center design process. Um, often I like to partner with data science or data analytics to show how our two functions can work collaboratively collaboratively together uh to help the business obtain a holistic understanding of our users. It's just another way to build relationships and to increase visibility within the company. It also serves me well to break out of the research echo chamber. I have a really fantastic community of research enthusiasts both in my role and outside of my role. Um, but I find most of our deliverables as researchers are external facing. So I think of it just like conducting user research, right? Like we conduct user research, usability studies, concept testing with users. So we should also apply the same idea to capture feedback on our work on our deliverables from stakeholders. So that's another way uh place where our allies can be helpful and sometimes that looks like getting feedback in say a design chair critique. Um I find designers are really warm and welcoming when I crash their design chair for to work through like a challenging problem on the research side. They're always giddy and excited to jump in and help out or even asking colleagues to pilot a new idea with me to see how it works in the real world. So next is democratization. So if you're active on LinkedIn or any research communities, I think we all know democratization of research is a somewhat contentious topic in our community. There are some researchers that passionately advocate for it and then others who are fiercely opposed to it. And I've been working in companies for 10 years that do some level of democratization. And I feel like I fall somewhere in the middle. I agree with both points, but I think where I land is democratization is okay in my book, but with careful guardrails in place. I also like to make it clear with executive leadership upfront that I'm rolling out or democratized research program as a band-aid solution to a lack of dedicated research roles. Um, and just let them know what the trade-offs here are. So, if you're not investing in research at this time, just know that I have to delegate this project to a designer. Um, and that will be taking away more time for them to do design work. Uh, and secondly, you're going to have a higher quality of data if you have someone who's 100% devoted just to doing research and is highly skilled and trained and experienced in that. So, that gets them thinking early about the potential cost benefit of investing in research as its own function. So, what do my guard rails look like? Um, through trial and error over the years, I've established a somewhat careful approach. One of the first things I do within my 90 days is start rolling out a lightweight democratized research program. So, most of my experience, this work ends up getting delegated to UX designers because they're usually the other role besides researchers that have prior experience in doing this work. Sometimes it's product managers as well. And what I do is roll out plug-and-play templates for deliverables like research plans or research findings so they don't really have to reinvent the wheel in a space they're less familiar with. And then at Warner Music, for example, my colleague and I offer one-to-one coaching and mentoring with a researcher. We do office hours twice a week where people can pop in and get advice on how to spin up a research opportunity into a project. um how to do synthesis, how to build a research plan. Really anything that goes into planning, conducting, and synthesizing research. Um if time allows, I'll even moderate one or two sessions to show best practices or conversely observe one or two of their sessions if they are looking for some feedback on how to improve. So something I do when I'm doing the internal research at the forefront uh is sus out research skills, interest level and appetite among different roles to try to pair democratize research to their interests to their strengths but also kind of assess where maybe we need to do some upleveling of their research knowledge as well. And I play it safe. I definitely like to delegate smaller, less complicated projects to non-ressearchers so they don't run the risk of scope creep or dealing with a method they're not super familiar with. Keep it simple. And I found overall like the the benefits um outweigh the cons. I think it's a really effective method for delivering research insights broadly to achieve additional buyin. For example, at Warner Music over the past year, we've had two different designers that partnered with really resistant um product managers who just did not want to invest time and effort in conducting research. So, they both ran different usability studies and both had the surprisingly the exact same outcome. They ran one or two and then they got a call from their PM who had been observing and almost word for word said the same thing like, "Oh my god, I can't believe I've been resisting this. This is so fascinating. It's so helpful to see in the product where we're going wrong, where people are struggling." And as a result, like they've actually become much more research enthusiasts and bought in on the process. So that was a really exciting win for us and for our design team. Yeah. So anyways, I think it, you know, overall it really helps um having additional people helping us show the value and positive impact research can have on product development. So once I've been in a company for a while and build up trust, do a lot of collaborative projects with folks and people know me and recognize me a little bit better. Then I start to introduce the product teams to the concept of research strategy and how researchers can be thought key thought partners and helping them develop their road maps. So research strategy is really just about ensuring that we're doing the right research. So when it comes to that we're asking what are the different research opportunities within the organization? What within the squad level are people seeing as the most immediate important things to solve for their end users? And even at the executive leadership level, what are sort of the big blue sky foundational or maybe strategic research efforts that they need help with? What are the things that are keeping them up at night where research come in and help answer those questions? And so I start my strategy development by first collecting stakeholder hypotheses and questions. And I'm a huge workshop nerd. I love a good design thinking framework to really get everybody aligned and out of their heads um in in the context of like a digital or an in-person workshop. Um, so in order to start eliciting feedback and collecting those hypotheses and questions, I'll do so um, in the context of two different frameworks that are my current go-tos. IBM's assumptions and questions framework and Chris Gson, who is a research strategist at Workday, created a fantastic research strategy framework that I also employ here. So, I merged those two together to meet the needs of our Warner Music stakeholders. And I'm happy to share links to those resources as well later. So, here's a snapshot from the workshop I ran. Each participant documented assumptions, their questions, and the dig digital whiteboard like Fig Jam or Muro, and then we grouped them by theme and keep iterating on that theme until it's uh written as a research goal or objective. And then lastly, I have the team dot vote on those. What are the research needs that need to be tackled first? What are things that maybe we can table for a future quarter? Then I take another pass at through the researcher lens to determine how to prioritize work that we lead as researchers. And we know we can't do everything right. So that requires ruthless prioritization efforts on our parts. In order to do that, I ask myself questions like how is this research going to be used and by whom? How it aligns with the overall master product roadmap? How sure we are that the action will be taken from the insights. So does this work with their timeline? Are they under a lot of pressure to deliver quickly? Um and then also start to assess where are the opportunities for democratizing some of this research? And lastly, learning to say no. There are there situations where even with democratized research, we just can't resource these projects and they just aren't as high priority as some of the other items. So right now my UX prioritization process for research looks a little bit like this. Um first and foremost I look at level of risk. So the likelihood of negative business consequences is high if research is not done or done well. Um impact is second most important to me making sure uh that the research has potential to influence critical business decisions. Complexity is also important to consider especially in a democratized environment. Does this study maybe require advanced research skills or methodology? And lastly, visibility. Um, so are executives and other leadership, do they have eyes all over this project? Are they deeply invested in the insights? That's always good thing to consider. And then if I'm working at a slightly higher UX maturity organization where research has been happening for a while, I may also weave in certainty to this list to like the degree in which reliable research already exists within the organization. Can we repurpose another study somebody has run to get those answers? So we don't do redundant research. Um can we do desk research to make this happen? It's another criteria I consider. Um so obviously if most of or all of the criteria is high that really signals to me that this would be a researcher project and projects that are tend to be lower in risk impact visibility complexity uh all or in some of these areas can most likely be delegated or pitched as democratized research projects. So here's the final deliverable that I spin up for my organization of the research strategy work. I weave it into a master research road map. Um and I aim to do this about two times a year with the team right before H1 and again before H2 to ensure research needs and product road mapaps are aligned. And I also use this road map as a way to advocate for additional headcount I need because I can have some sort of visual representation that shows executives, hey, okay, we did these workshops. This is what we surfaced from all the teams and our leadership. Um, here's the volume of projects we have coming down the pipeline for the next two quarters versus our tiny team of two. So, you know, we can only delegate so much to democratization and we can only do so much. So, how do we start this conversation about how we might be able to resource these additional projects? And lastly, you've done a lot of work in the space by now. It's time to toot your own horn a little bit and show off within the context of our research road show. So I do road shows to achieve a couple of different things. First and foremost, use it as a educational platform uh to establish really a common definition of what user research means to us within this organization. What research is, what it's not. For example, uh I work in an organization now where we have business analysts, we have designers, we have uh data analysts. and often our role gets conflated with the work they do all the time. So this is a great way to help people understand the nuances to different between our roles. I also try to mirror the language used by cross functional stakeholders to meet them where they're at. So for example, in my organization, I notice people use uh discovery research as the the core term instead of generative research, which is usually my preferred term, but because I want to mirror their language, I'll refer to it as discovery as well. And in general, we'll just try to avoid overly jargoning research terms to feel more approachable. Evangelizing obviously is a huge part of our role. So using that platform to communicate the benefits of research. So earlier on in my talk, I mentioned having some great back pocket talking points or statistics showing the benefit of research and the high cost of not investing in it. This is another place where I really like to repurpose these talking points and really drive the point home with a larger audience using a lot of like compelling visuals and stats. And lastly, and probably most importantly to our stakeholders and executives is the ability to show impact. So, another early career pitfall I made was coming in and immediately jumping to the road show, putting together a flashy presentation I was so proud of, talking about research, getting overly jargony about different methodologies, and just hearing a room full of crickets at the end of my talk. Well, I'm in New York City, so it's really more hearing sirens in the background. But nevertheless, it just did not land because I think it's more important to show that into the like show value in the context of an actual project. So now I really hold off on the road show in a new role because I want to have a few internal examples to illustrate how research has helped the teams within our organization make competent product decisions. And so at this point that you've got to the road show, you, your allies in democratized research are going to have a couple of really cool case studies to show off and you can contextualize your educational learnings and talking points within the context of a royal project. Um, I also find that having some case studies weaved in where I've partnered with other teams carry a lot of weight. You know, I can talk about the benefits of, you know, UX research with a product manager until the cows come home, but it may not land. But hearing another product manager hear about how great it's helped them, how it's helped them with their road map, how it's solved user problems carries a lot more weight than anything that I could say. So, I usually like to bring in a few talker um speakers in addition to me to talk about how they feel about the process and how it's benefited them. And of course, if you really want to speak or executive leadership love language, include metrics if you have that. I worked in a lot of SAS and enterprise spaces. So, if you're in those spaces, too, this may you may relate that sometimes it can take a long time to get those statistics or metrics. So, don't beat yourself up if you don't have them right away. But if you do, definitely leverage those. So, for example, you can say, "Hey, we did this great usability study. We found out these five major things wrong with our checkout process on the site. Uh we iterated on the designs, we made those changes and we saw a 3% conversion. Um I find that really resonates with executive leadership that wants to see value that wants to see impact. And then my final thoughts, have patience. Have patience with yourself. Have patience with your stakeholder. Change takes time. So, make sure you celebrate even the smallest accomplishments along the way and don't get overly caught up in the big picture. Um, I definitely try to focus on what I can influence or control. So, I don't think I've ever worked for a company that didn't have some certain amount of cultural constraints, resistance to UX, or have some nasty politics at play. And these are things going on way above my pay grade, but frustratingly impact my ability to do good work. So, it can kind of get tempting to be overly involved or even just ruminate on these problems, stress out about these problems, and it doesn't really serve me. Um, also setting healthy boundaries is absolutely crucial in any job. If you're a research team of one, this is imperative. Or even if you're among a small team, you're going to be stretched. You guys may feel overwhelmed with the amount of work that needs to get done. So, make sure you're not biting off more that you can chew. Set and hold firm boundaries of what you can focus on and prioritize where you need to. And then, of course, my woowoo pitch for self-care. I think if we learned anything from the pandemic, it's how important that is. Um, and as researchers, we're in a really unique position. We carry a really heavy cognitive burden in this role. you know, we have to do our day-to-day responsibilities of conducting fantastic research, but then we're in this somewhat unique position where we're constantly have to advocate for our role and convince people that it's important. And that can be really exhausting. So, make sure you self-care, you take time off, take your lunch break, and create space in your day to walk, to read a book, to meditate, whatever really like relaxes and recharges you. because burnout researchers are never going to be the effective change agent we want to be. Right? So that's my soap box. U and now we can move on to Q&A if time allows. Um great. I see a couple questions in here. Let's go through them. Oh, actually this is a great question. And I was wondering too, but if people are going to have access to the recordings at the end of the event because I want to make sure that I um you know if you want to also have a copy of this deck, I'm happy to share that out as well. Ah yeah. So answer is yes to that that question. Yes, all recordings will be archived. um those speakers that are allowed to share their decks, we will PDF and make those available for download um on the web uh on the landing page for comments. Right. Yeah. Um someone asked what are the most common objections I come across? Um yeah, so for me it's time time at Warner Music especially. Um because before sort of our new leadership came in there, it was really about shipping quantity over quality of products. Um which you know always hurts my researcher heart, right? And um so it really kind of came down to we just don't have time. So often my talking points are showing statistics around how doing discovery research and doing iterative rounds of of research will actually save you time and money on the back end. Um and that about 40 to 50% of developers time is spent on rework. And we all know engineers are pretty expensive hires, right? So imagine how much money you could be saving if you're working out most of your usability issues before you ever launch your product. Um, also I I hear this to a lesser extent. I feel like the people I work with now are a little bit more self-aware, but um that we already know everything that we need to know about users. So I definitely utilize the why framework as a researcher. Why is that? What data has shown you that that is true? Just kind of slightly kind of peel back the layers of like do you really know your users or are these rooted and unvalidated assumptions? Right. Here's another one. Have you ever been in a situation where the stakeholders are dragging their feet and actually approving the resources on a project? Yes. God, I'm so glad I'm not alone in that. You know, I so is this dubtales really perfectly into the into the problem of research takes too much time and sometimes I need to have a candid conversation with my stakeholders and say, "Hey guys, you're the bottleneck here. I'm waiting." You know, especially if you're working in a highly collaborative fashion, right? like you want to get sign up on alignment on your research plan. You want to do you want to say like set up a moderator's guide collaboratively with your stakeholders. And sometimes I find a couple days will go I go by I ping them on Slack and it's just crickets. And I think in that sense you have to be aggressive and candid with your feedback like hey you know I actually can't move forward with the next phase of research until you um answer the following question. So, I'm very clear with that. And usually it's just like, "Oh god, oh, sorry. We didn't realize we were the bottleneck here." Especially if they're really new to research. They're just not always aware that it is really highly collaborative. So, just being clear in that or often after meetings, I'll set clear action items. Here are my asks of you. These are the four or five things I need from you to move forward. Here are the things that I'll be working on. And also even setting up like a weekly or a bi-weekly um when you're um spinning up a project just so you're on their calendar, you're on their radar and you can call those um those blockers or those bottlenecks out live in a meeting. Hope that helps. Yeah, you mentioned doing the right research and working with stakeholders to determine needs and importance. Have you encountered a situation where you had a research topic you considered important that wasn't considered a priority by your stakeholders? Yeah. What do you do in that situation? Um yeah, that's a typical one and I think we come across that a lot. Um I try to bring in um my advocates that are at sort of the executive leadership level to say hey you know this is what I'm finding but I think we're also missing the mark. Often it ends up being less tactical research and more strategic. And so sometimes you know when when people are within the constraints of a squad they're siloed. They're not seeing the big picture. So, you know, I just kind of roll up the go up the ladder um until I have some facetime with executive leadership and say, you know, this is what I'm seeing. This is the importance I'm seeing here. This is the value this research would help you with. Um for example, at Warner Music, it's not a particular project, but in general, we have what a ton of what I consider research debt. um where historically no one has done any sort of discovery or evaluative research in the space. They're shipping based on assumptions and they're shipping quickly and our products are failing over and over and over again. So when we had our new leadership come in, I told him like, "Hey, we had he he asked for personas." And it's like, "We hardly have any because you need discovery research. You need these certain types of methods to capture that feedback." And historically, people have not been allowed to do that with an organization. Um, so that was my in to figure out what mattered to executive leadership and try to pair that to what that matters to them. Like, hey, I have a great research opportunity for you. We need to dig into some of this research debt. so that we can give you the persona data you're looking for. Um, how do you determine risk and prioritization? Um, I try and find out, you know, any sort of like business metrics like is it tied to potentially um increasing our revenue tremendously? Is it potentially going to hurt our revenue? Um, so for example, I had to do a product market fit project with some with marketing a couple of years ago and we saw this as a high-risisk project because they wanted to tap uh they wanted to target an untapped market and we convinced them before they did that to do a lot of research in the space to make sure that this is the right market to tap because about a year prior this company tried um that without doing any due diligence or any research on space. Um they tried to expand in Australia and it was an unmitigated disaster. So we were able to true back to where we went wrong in the past and say, "Hey, this is a really high priority thing. We don't want to lose potentially hundreds of millions of dollars by not thinking this through clearly." And as a result of the research that I did, we realized that we didn't want to attract that market. So in that way we ultimately saved the company hundreds of millions of dollars of not going after a market that we really can't build and scale for. So I I definitely try to tie it to business objectives and revenue opportunities. Um but if anything is like potentially a high revenue generating um project or conversely might cause the company to lose a lot of money which would be huge disaster. That's what I kind of think of as high risk as well. Uh do you have any tips for a new researcher team establishing a recruitment budget? Um I definitely try to use competitive research in this space. So again another Warner Music example is we are always trying to look at what Spotify is doing. Spotify in our minds is the gold standard. So, I'll put together a budget deck that says, "Hey, I anticipate needing X amount for incentives, um this amount for, you know, maybe vendors, etc." Um, and tie it back to what our competitors are doing. Um, that definitely piqus the interest of um of our executive leadership, if you can tie it back into the return on investment, um things like that, and maybe try to find easy wins. So when I worked at Etsy and now I'm working on a e-commerce project for Warner Music, something we can do is offer gift cards. So if you have that ability, that's easy because it just the revenue comes right back full circle. People get the gift cards, then they spend them that gift card within the company on their site. Um yeah, it is hard at first. I've definitely worked in environments where they're like absolutely not. People should be excited to talk to you because they want to help build a better product. And this is mostly in SAS and enterprise. And I found that actually isn't entirely untrue. I usually do am able to recruit um with just the pitch like we really want your input and we want to help create a better product. If you're doing B toC, I think it's pretty table stakes to offer incentives. So I would look across that competitive research um and try to tie it back into the return on investment. Yeah. And then let's see, parent asked, "What are some uh strategies, tactics for relationship building that's worked for you?" Um, meeting them on a human level carries more weight than anything else. Um and you know one thing that's really a bummer you know with the pandemic is we all wrote remote of us haven't gone back to the the office so it is a little bit harder but when I am in the office I ask PMs out for coffee and I treat them to coffee and chat with them and chat about their lives and their kids. Um yeah and then also just kind of seeing where we might be able to partner. Um, I had a a project that has been really political lately with an organization and um, you know, definitely shipping a lot of features that aren't delivering value. So, I reached out directly to that product lead and sort of building a relationship and I thought given all the politics and constraints that she would say no or be resistant. She was like, "Yes please." So, you never know where your your fans and enthusiasts lie. And so I'm actually getting in there and starting to help her peel back the layers of this messy ambiguous space where there's a lot of unknowns. But yeah, just um chatting with people, water cooler conversations, going out to coffee, setting up one ons um and just kind of asking a lot of questions, looking for opportunities to collaborate. Yeah. Ah, someone from agency life. Yeah. So almost all of our work is client-based. We have awareness of the need for research internally. H but it's often a hard sell to get clients to pay for it. I hear this all the time from my agency friends. Do you have any suggestions on how to convince external clients? You know, so I have to preface this with the fact that I've never worked in an agency setting. So here's some things to try, but I can't promise they'll land because it's a group I haven't been in front of and interfaced with before. But if I was in that situation, I would definitely repurpose the same talking points and the same stats that I use with my most resistant stakeholders because they kind of fit the same archetype like just like internally they're like we don't have time for that. External they're saying we don't have the budget for that. We're just trying to do this on a bare bones budget. This sounds like extra fluff we don't need. So really driving home the importance of it's actually not a nice little nice to have. It's kind of table stakes really crucial important part of our user center design process. And again tie that back to the return on investment. Tie that back to those great talking points and statistics that really drive that point home. Yeah. And on that note, you guys reminded me that I want to share some of the links that I was talking about in our chat as well. Um, yeah, and I would I would just while you're doing that, amen that last point. Um, I think your blueprint while internal team building focused and dealing with the politics of of being inside an organization, I think it applies perfectly to someone at, you know, on the provider side who would be, you know, basically working with someone in your shoe, right? Yeah, speaking of in those terms helps to make life easier for the person that you're gazing with internally, right? Um, we do have a couple of questions that are coming from YouTube as well, so I want to make sure we we hit on those. Um, sorry, I've got my mission control multiple screens going. Uh, Nicole just had a comment and you know, I'm gonna echo this one as well. Loves the term research debt. Um, just really good feedback. Um folks really enjoying I would say no crickets in this presentation so there uh let's see biggest music industry research needs um I would say you know really tracking artists progress um you know going in and making sure what tracks are are blowing up using that data to kind of estimate what are sort of the next big release you want to set set out you know um so you know we have like ANR managers that represent Lizo who's like crazy hot right now so they'll go in and look at our platform of like okay how do we think strategically about Lizo's next big move um you know she released about damn time and you know this is how it's it's um you know hitting in the what should we do next? And they use like a lot of we have we're data heavy analytics tools um inhouse um for our like internal facing colleagues or who are also our end users. Um also a lot to do with archiving and making sure assets are easily attri um attainable. So, um, sort of like a working I worked on like a archive of all of our our body of works, our assets, um, our performers, um, so people can pull data about them all the time. We have other cool um, apps that help songwriters pitch um, their work to our artists. So, the songwriter wants to pitch something to Ed Sheeran that they think would be a good fit, they'll do that. Um, yeah, there's a lot of cool stuff. And then we have the sort of the the um BTOC facing site as well where we sell merch and you know blog posts about our artists. So there's a lot going on. It's a really cool space to work in. Good. Another question we have is as you build out a UXR team, how does research ops factor into your process? Well, I do know one thing is I really need to hire a research because I am the research ops and an IC researcher and the research manager and I'm drinking from the fire hose but research ops is everything to me. That's why in addition to doing internal research is one of the first things I start to think through with that pillars. Um, so really kind of looking at what our big bottlenecks are. Um, and knowing welcome to Zoom. Enter your key or by pound. Um, so yeah, just uh looking across um, sorry I lost my train of thought with what was the question again? Uh, how does research ops factor in? Oh yeah, research ops participant ID followed by town. I love the town. Um, so research ops. So yeah, looking at um what sort of research ops work we can get off the ground to help us uh deliver insights faster because speed is still very much the love language among our executive leadership. Even though uh we want to establish more rigor practices, they still want to ship things quickly. So what are our big bottlenecks? Getting access to users, um recruiting things like that. So, those are kind of the things I'm looking at as like the immediate needs to unplug um my colleagues so they can get to access to users quickly and um speed up their their insights faster. Uh looks like we had one more question come in in the Q&A. Uh what if you don't have the ROI info yet because you're still trying to build a capability trying to bootstrap without a booked LOL. Yeah. Um that's great. Yeah. I mean when I say ROI, I'm kind of using case studies externally because I'm in the same boat. I don't have return on investment to share yet. But I think as we start so uh you know doing more projects we'll be able and shipping more stuff we'll be able to to measure um so definitely be measuring setting you know benchmarking and measuring against the the changes you made where research was highly impactful to start increasing your ROI but in the short term if you're where I'm at you know less than a year in at a new company where we don't have those metrics yet I definitely like to true back to case studies externally Um, and that's where uh, one of those articles I was referencing, it was um, sort of like the the big statistics that you tie to the the biggest talking points or or push back or objections to research um, and show, you know, how it's benefited other companies. Um, there's a lot of research studies on research that shows that improvement. Um, but then do that in tandem of starting to figure out what your own personal internal ROI strategy should be. Awesome. Um, and I know we're coming up on time, but uh, so you mentioned some some share links that that you're going to provide. Yeah, you know what? I noticed it doesn't actually allow me to share them as links in chat for some reason. Um, they're without the URL. So, what I'll do is I'll pull them into my presentation at the end after the Q&A slide and then when we share that out, everybody will have access to them. Yeah, that's perfect. That was going to be my suggestion. So, Okay cool. Good job. Great minds. Awesome. Thank you so much for having me and you can find me on LinkedIn if you want to continue this conversation folks. Yeah, Mary, great great kickoff to to the conference. This was fantastic. Uh very engaging. Um really glad you were able to share this. So and you took up the full hours. It's great. No dead air between presenters. No bio breaks for anyone. My apologies. Whatever. We we're hanging on in word. Thank you. Thank you very much. show you that.