PayPal - Rapid Research

Overview

As a Rapid Researcher at PayPal, I delivered one-week qualitative studies across various products, including Venmo, consumer and merchant experiences, and developer tools. In this role, I conducted over 80 usability and comprehension studies, helping teams make informed design decisions.


This case study highlights a study that led to a 19% increase in transaction rates and millions in potential revenue.

What I did

  • Met with stakeholders (dedicated product UXR, designers, product managers, content) to understand research questions

  • Drafted moderation guide and conducted research with six participants

  • Synthesized qualitative data

  • Created and shared report to stakeholders with recommendations



Timeline

1 week

Research Context: Cashback in PayPal

As part of PayPal’s effort to become the most rewarding way to pay, the company introduced cashback rewards in the consumer app.


Before launch, the team requested a final usability and comprehension study to identify any issues that could impact engagement or revenue.

Stakeholder Alignment & Research Planning

Each week, research leads assigned me three high-priority projects. Once assigned, I met with key stakeholders—dedicated UX researchers, designers, product managers, and content strategists—to refine research questions and set expectations on what small-sample qualitative research could and couldn’t achieve.


For this study, the team wanted to know:

  • Are the cashback offers compelling and relevant?

  • Do users understand which purchases are eligible for cashback?

  • Are there usability or comprehension barriers preventing engagement?

Setting Expectations on Small-Sample Research

A large part of my role was helping teams use qualitative insights effectively. Small-sample usability research is powerful for identifying usability issues, comprehension gaps, and user expectations, but it is not a reliable predictor of long-term engagement or revenue impact.

I often guided stakeholders on how to interpret findings within the right context. For example:

  • If a feature performs well in usability testing, it doesn’t mean users will engage with it over time. Novelty and testing conditions can create an artificial sense of enthusiasm.

  • If a feature performs poorly, it signals clear usability or comprehension issues—but doesn’t always indicate lack of interest. Users may still engage if issues are fixed.

  • Metrics like click-through rates and transaction volume require A/B testing at scale to measure real-world behavior.

For this study, I recommended usability testing to uncover comprehension issues and user hesitation, but also advised the team to validate engagement impact through an A/B test after launch.

Determining the Right Methodology

The team initially planned to test the cashback widget using generic offers, but this wouldn’t reveal whether users found the offers relevant. To get meaningful insights, I recommended a live usability test—a first for PayPal's Rapid Research program—where participants interacted with real cashback offers tailored to their purchase history.

I worked with data science to:

  • Recruit active PayPal customers instead of relying on an external panel.

  • Ensure participants’ accounts were enrolled in the cashback experiment so they could engage with actual offers.

Turning Raw Observations Into Actionable Insights

With just a week to turn research around, my analysis process had to be structured and efficient. After each session, I reviewed notes and transcripts to identify patterns in user behavior, pain points, and misunderstandings.

Here’s how I approached synthesis:

  1. Highlight key observations – I pulled direct quotes, behaviors, and moments of friction from session notes and transcripts.

  2. Group similar themes – I categorized issues based on frequency and impact, ensuring findings weren’t just surface-level but pointed to broader usability trends.

  3. Prioritize insights – Not all issues carry the same weight. I focused on those with the highest potential to affect engagement and revenue.

  4. Structure findings for clarity – I framed key issues in a way that resonated with stakeholders:

    • What users struggled with

    • Why it mattered (business impact)

    • How we could fix it

To make the insights more tangible, I incorporated direct user quotes and, when possible, short video clips highlighting usability challenges.

Key Findings & Insights

Users struggled to identify products. Some product images lacked clear labels, leaving users confused about what they were viewing.

  • Participants frequently asked what items were and which store they would be directed to.

  • Impact: Hesitation to engage, reducing transaction potential.


Misunderstanding of cashback eligibility. Users wrongly assumed some items weren’t eligible due to unclear messaging.

  • Lack of transparency led to lower user trust in cashback offers.

  • Impact: Lost revenue as users skipped purchases they mistakenly thought didn’t qualify.

Offers were sometimes irrelevant. Some users received offers that didn’t match their shopping behavior, making the feature feel less valuable.

  • Participants were shown products that didn’t align with their shopping behavior (e.g., hunting gear for users with no interest in outdoor activities).

  • Impact: Lower engagement with the cashback widget.

Driving Product Change: SKU-Level Data Implementation

To address these issues, I recommended adding SKU-level data (brand, item name, and price) to cashback tiles. The product team ran an A/B test:


  • Control: Product images only.

  • Variant: Images with SKU-level details.

Results:


  • 19% improvement in transaction rate in the variant group.

  • Users clicked on fewer items overall, but clicks were more intentional and led to higher spend.

  • Following this success, the team iterated further by adding price information, which is now the standard design in the PayPal app.

Learnings

This project reinforced the power of qualitative research, even on tight timelines. It also taught me how to quickly ramp up on unfamiliar products, ask the right questions, and deliver insights that drive meaningful change.