A process and case study in user research and outreach
My Role
I led the strategy, design and implementation of our lean user outreach projects. I performed a competitive analysis, user research and led efforts through user outreach to better understand what our users want. I used this knowledge to address customer and business pain points as well identify and leverage areas of opportunity.
Research and Ideation
I performed a competitive analysis and user research as well referred to our historical data to create a list of hypotheses and assumptions. From this list I prioritized items and created goals.
Strategy and Vision
I lead the strategy to create and implement a lean and effective user outreach process.
Planning and Project Scope
I defined and created smaller actionable projects from the larger project of user outreach. Each project was defined with actionable items, clear deliverables and metrics for success where applicable.
Design Execution
I created and tested different user outreach methods that allowed me to create the most effective and lean user outreach strategy. I created and tested variety of different surveys, email templates, and email send times. I also tested our outreach methods amongst a variety of different user cohorts.
Oversight and Coordination
I worked closely with members of our customer support and data teams, to gather valuable data from as well implement processes to allow our user outreach projects run smoothly.
Leadership
I presented process, strategy and data gathered to gain buy-in from stakeholders, and many other team members throughout the project lifecycle. Using data gathered, I pitched next steps, areas of opportunity and future projects to stakeholders and throughout the product team.
The Challenge
We are always working to provide the best experience to our users, and trying to provide the best products and features. The challenge is trying to identify what a better experience looks like, and what products and features students want. Through user research and outreach we are challenged to be able to answer these questions with the goal to identify where and how we can create a better experience, as well what products and features our users do and don’t want.
The Approach
Life’s too short to build something nobody wants.
Ash Maurya, Running Lean: Iterate from Plan A to a Plan That Works
Our approach is to better understand what our users want through user research and outreach. To allow this knowledge and data gathered to help direct future iterations, features and new products.
Research and Ideation
Before performing any user outreach, I first did some user research to dial into who college students are today. Alongside this initial user research I performed a competitive analysis to identify trends, products and features that our competitors are implementing. Here are some key findings from this research:
Who are college students today? – Millennials vs. Gen Z
Over the years we have seen a shift in college student demographics from being predominantly millennials to now being predominantly Gen Z. This is important because there are some key differences between millenials and Gen Z, that if overlooked can negatively impact the experience of our users. Here are a few key differences that I found most interesting:
Speed and reliability
Videos, picture messaging and emojis are the most popular forms of digital communication with gen Z. They are most popular because they are much faster and reliable than traditional text messaging.
Immediate on demand consumption is expected
Gen z are the first cohort to have always had the immediacy and mobility internet and cell phones, and this directly impacts their consumption habits. They have always known and expected what is provided in our current on demand economy
Internet is used as an extension of how they interact with people in the real world
Gen z spends a lot of time online in private communicate with people they already interact with outside the internet on a regular basis. Online communication is viewed the same way as having an in person conversation.
Increased online usage has meant an increase in loneliness for gen z
Online usage patterns with gen z may be associated with loneliness, anxiety and fragility.
College education is highly valued
Gen z are entering college at higher rates than previous generations
Gen z are realists, and are much more concerned about acquiring debt in college than previous generations (childhood in recession)
The primary concern for gen z who are preparing to go to college is paying for college without acquiring debt
Gen z are much more entrepreneurial (childhood in recession)
The newly developing high tech and highly networked world has resulted in an entire generation thinking and acting more entrepreneurial than millenials. Generation Z desires more independent work environments. As a matter of fact, 72% of teens say they want to start a business someday
Gen Z does not represent 100% of the college student body, but it does represent the majority at roughly 80%. 20% of today’s college student body are considered “non traditional”. This means they may be:
- Between 24-35 years of age
- Financially independent from their parents
- Having a child or other dependent
- Being a single caregiver
- Lacking a traditional high school diploma
- Delaying postsecondary enrollment
- Attending school part time
- Being employed full time
Being that I am working through a lean user outreach strategy, I chose to focus on the 80% Gen Z student body. The non traditional student body is growing, and there may opportunity to further explore this cohort in future projects.
Competitive analysis – how are things changing in the market
Now that we have a better understanding of who college students are today, I decided to analyze what our competitors are doing. What I am looking for with this analysis are new products and features. Here are a few key findings:
Self test tools like quizzes and sample questions
Our competitors have implemented tools to encourage self studying, things like quizzes and sample questions that students can complete.
Gamified and interactive elements
I identified new gamified and interactive elements, offering engaging unique ways to study.
Audio features
An increase in audio support has been identified amongst our competitors, allowing content to become more accessible as well catering to different learning styles.
Structured content that is easy to digest
Our competitors are moving towards different ways of structuring their content. Things like flashcards, and wikipedia style outlined content with not only what our competitors are implementing but what google is choosing to rank.
Step-by-step solutions
Content structured in a way where it includes not only the answer to the problem, but the steps to get there. Again a feature to encourage self studying.
New products and features like Tutoring and Q&A
New interactive and on demand ways to study like access to tutors and q&a features
Summary – User research and competitive analysis
Now that we have a better understanding of who college students are today, and what our competitors are up to; we can identify some patterns between the two and some areas of opportunity. From this research I was able to create a list of key takeaways:
- Students today are interested in a variety of engaging and interactive ways to study. Things like flashcards, videos, games and audio features.
- Interest in tools to support self studying, like quizzes, sample questions and step-by-step solutions.
- Students want on demand services like tutoring and q&a
- Students are cautious of acquiring debt, but see the value in education. They are looking for more value in study subscriptions services; this means high quality content and a variety of features
- Opportunity to combine the need for on demand services with the interest that college students have in earning more income. If our students are interested in tutoring, do we have students who are also interested in becoming tutors?
Building the hypotheses
Now that I’ve done some user research and a competitive analysis, I’m going to use the data gathered to start creating hypotheses that we will attempt to prove out through user outreach. These hypotheses will drive the direction of the questions that we will be asking our users, and the goals of our outreach projects. The listed hypotheses could be categorized under one of two goals:
Identify opportunity to grow current and new products and features
Identify how we can improve subscribed user experience
Identify opportunity to grow current and new products and features
Under this goal category, I created a list of hypotheses that could be proven out through user outreach:Students want a tutoring feature
- Our students are self studiers
- Our students want tools to encourage self studying
- Our students want quizzes
- Our students want step by step solutions
- Our students are social studiers
- Our Students want tools to encourage social studying
- Our students want a peer tutoring feature
- Our students want interactive ways to study
- Our students want games
- Our students want flashcards
Identify how we can improve subscribed user experience
- Students want better quality content
- Students want more content
- Students want more visibility to relevant content
- Students want more features
- Students are/ are not satisfied with their subscriptions
- Students are/ are not interested in a refer-a-friend program
Vision and Strategy
Now that we’ve done some research, we’re ready to start building our outreach strategy. The vision for our strategy is through user outreach, allow our users to tell us what they want; with the goal of allowing this feedback to help direct future iterations and features. I decided to build a strategy that supports two different outreach goals : Identify opportunity to grow current and new products and features, and Identify how we can improve subscribed user experience. The strategy for how we are reaching out to users will be very similar for each goal, but each goal will be given it’s own unique outreach project. One outreach project with the specific goal of : Identify opportunity to grow current and new products and features, and another outreach project with the specific goal to : Identify how we can improve subscribed user experience. For each of of these outreach projects, the strategy is as follows:
Define the user outreach groups
Determine what groups of users we would like to be gathering feedback from. Gathering email lists that represent these user groups.
Create user outreach questionnaires
Building different user outreach questionnaires specific to the goals we defined above.
Create email templates that the questionnaires will be linked from
Building email templates that we will be emailing our users. These email templates will include links to our different questionnaires.
Determine if incentives will be necessary, and what they should be
Sometimes it can be challenging trying to gather feedback from users, depending on the user group and what level of feedback you’re requesting. It’s always a nice gesture to offer users an incentive for their time and feedback, especially in cases where it can be difficult to gather feedback.
Send emails with questionnaires to our users
We’re ready to start sending outreach emails to our users, and start gathering valuable feedback.
Analyse user feedback data, and report on findings
Once we have sent outreach emails and have generated quality questionnaire responses, it’s time to analyze that data gathered and report on the findings.
Determine future projects and next steps based on user feedback gathered
Lastly, after we have analyzed our feedback we can determine future projects and next steps
Planning & Project Scope
Once a vision and strategy have been defined, I broke things down into smaller actionable projects. Projects were placed in priority order by
Define user outreach groups, and generate email lists for outreach
Determine what user groups we will be reaching out to for each outreach project, and generate email lists of these users so we know who we should be sending emails to.
Create questions for questionnaires, and build questionnaires using 3rd party tools
Create lists of questions for each outreach project, and build questionnaires using 3rd party tools.
Create email templates to be sent to our users, linking to our questionnaires
Create email templates that we will be sending to our users, that link to our now built questionnaires
Create email campaigns that will trigger our outreach email sends
Build email campaigns using the new email templates and the user email lists. This will allow us to easily send outreach emails to a large list of users.
Create an A/B test using different 3rd party questionnaire tools
Test different 3rd party questionnaire tools to be able to determine which tool performs best – based on open, click and response rate
Create an A/B test using different outreach email templates and subject lines
Test different email template variants to determine what language performs best – based on open, click and response rate
Create an A/B test using different incentives
Test using incentives, and a variety of different incentives to determine which performs best – based on open, click and response rate
Test email send times
Test sending emails at a variety of different times during the day, to determine which time of day is most effective.
Design Execution
Now that our projects have been defined and scoped out, I started to define our user outreach lists, build questionnaires and start sending emails
Define user outreach groups, and generate email lists for outreach
Now that we’ve done research and defined our tasks, we’re ready to start defining our user outreach groups and generating email lists. As I mentioned earlier, I decided to build two separate outreach projects to represent our two goals: Identify opportunity to grow current and new products and features, and Identify how we can improve subscribed user experience. Each of these outreach projects require us to reach out to different groups of users.
Identify opportunity to grow current and new products and features:
I created a list including users who are subscribed, non subscribed, cancelled, and our power users elite notetakers.
Identify how we can improve subscribed user experience:
I created a list specifically of users who are subscribed.
Create questions for questionnaires, and build questionnaires using 3rd party tools
Now that we have our outreach groups defined, it’s time to create the questionnaires that we will be emailing our users. Before deciding on a 3rd party tool and questionnaire structure, I actually did some testing to make sure that we are using the most effective approach