October 2022 ARVIC

The October 2022 ARVIC virtual conference drew insights and product professionals from companies including Roku, T-Mobile, Lyft, and JM Smucker. Across four talks, a consistent thread ran through the sessions: the gap between data that exists and insight that actually drives decisions is a structural problem, and researchers can close it through smarter program design, cleaner operational habits, and sharper communication. The sessions covered behavioral data integration, mobile engagement measurement, self-serve research scaling, and presentation craft, each grounded in real programs rather than theoretical frameworks.

Connecting Behavioral Data to Survey Research

Two talks explored how brands with large first-party or proprietary datasets can turn behavioral signals into a research advantage. At Roku, appending behavioral log data to email surveys before fielding allowed the team to eliminate screener questions, reduce survey length, and avoid relying on claimed behavioral responses that respondents frequently misremember. The trade-off is operational: legal review, contact exclusion lists, sampling discipline, and privacy guardrails (including GDPR compliance) must all be in place before a single email goes out, not after.

T-Mobile Advertising Solutions demonstrated a complementary approach, using proprietary mobile app engagement data collected across millions of devices to measure brand loyalty without asking a single survey question. Metrics like time in app, login frequency, and post-install retention function as measurable loyalty proxies and can be tracked across competing apps simultaneously. The case study on music streaming apps showed how engagement patterns map closely to established brand loyalty rankings, and how co-installed app data can build audience personas precise enough to inform re-engagement targeting.

• Behavioral data belongs in the analysis layer as a filter for attitudinal findings, not as a finding in itself.
• Privacy and consent design are foundational, not afterthoughts. Both programs built opt-out and compliance mechanisms into their core architecture.
• Response rates on unincentivized email surveys run around one to two percent; operational expectations and send volumes should be set accordingly.
• Post-install retention drops sharply in week one across apps, making that window the most critical signal for loyalty and re-engagement strategy.

Scaling Research Capacity Without Scaling Headcount

Lyft's rapid research program addressed a problem many insights teams face: demand that outpaces researcher supply. The solution was a structured self-serve model that trains non-researchers to moderate interviews with real customers, supported by researcher-built templates, intake forms, and mandatory dry runs. Running up to six simultaneous project teams in a single session lets multiple product teams get usable findings in one afternoon. The researcher's role shifts from sole executor to program architect and quality gate.

A notable secondary benefit of the parallel session format is cross-pollination. Teams hear findings from adjacent projects they would never encounter in isolated studies, generating knowledge transfer that compounds over time. The program also proved self-reinforcing: trained partners improve, build a research culture, and sustain demand, which justified dedicated headcount to run the program full time.

• Rapid research is best suited to focused, time-sensitive questions answerable in 20-minute moderated interviews: design comparison, usability testing, and concept testing.
• It is not a replacement for surveys, multi-phase studies, or contextual inquiries.
• Recruit six participants per session with one to two floaters. General population recruitment is simpler and less biasing than tight persona matching.
• Five-minute breaks between 20-minute slots, rather than one long batch break, are necessary for both participants and moderators to reset.

Presenting Research So It Actually Gets Used

The conference closed with a practical look at presentation craft, a skill that determines whether good research drives decisions or gets filed away. The core argument: researchers doing DIY or vendor-managed work are already at a credibility disadvantage versus full-service agencies, and weak slide design makes that gap worse before anyone reads a word. Simple, polished visuals signal competence. Dense tables and Excel outputs do not survive a live meeting.

• Replace complex index charts with one bold graphic, one key stat, and a clear label of who over-indexes.
• Apply the log line test to every slide: if you cannot write one sentence explaining why it is in the deck, cut it.
• Build two versions of every presentation. A detailed 'scuba diving' deck for technical stakeholders, and a distilled 'snorkeling' deck of essential takeaways for leadership.
• Front-load key takeaways at the opening. If the conversation derails early, the core message is already on the table.
• Track who brings you problems and whether your research informs tactical or strategic decisions. Both are meaningful signals of growing organizational credibility.