Capstone – Project
Generating And Testing Hypotheses About User Retention Using Site Analytics
A Collaboration Between
The objective of this capstone project is to generate hypotheses about which demographic and behavioral segments are correlated with user retention, to test these hypotheses using the tools available within Google Analytics, and to produce actionable reports useful to Marketing and/or Product departments aiming to increase user retention and ultimately grow the user base. Students will gain facility with Google Analytics, will exercise their data analysis and visualization skills, and will find ways to effectively, succinctly, and creatively communicate results to other departments within a firm.
The first milestone entails gaining familiarity with a community forecasting platform before producing a table of variables hypothesized to be positively or negatively correlated with user retention across a 12-week period in the life of the platform, along with brief explanations of why these relationships might exist.
The second milestone requires a dive into the different types of user and session-level segmentation made possible by Google Analytics, followed by creation of a more refined table of variables to be tested, as well as plausible means for creating segments able to test these variables.
The third milestone calls for creating these segments within Google Analytics and analyzing the resulting retention data while keeping careful record of which variables did or did not correlate with retention, and while using these results to fine-tune the first set of segments to run a second set of analyses.
The fourth milestone is a final report to include visualizations of data pulled directly from Google Analytics, as well as plots of data generated from the segmentation analyses. This report will home in on the most salient, surprising, and actionable results from the segmentation analyses, will consider why these variables were or were not correlated with retention, and will offer suggestions to both Marketing and Product departments on ways they might make use of the results from the analyses.
Project Topics
Marketing
Product Design & Development
Reporting, Financial Planning & Analysis
Company Information
| Company | Metaculus |
| HQ | Santa Cruz, CA |
| Revenue | Unlisted |
| Employees | Unlisted |
| Stage | High-Growth Startup |
| Hiring Potential | N/A |
| Website |
Company Overview
Metaculus is a community forecasting platform that delivers ML-optimized collective intelligence on topics of global importance. The platform brings together a global reasoning community, and keeps score for these thousands of forecasters via explicit, standardized track records. Built on data, it generates scalable ensemble forecasts while delivering consistent feedback on the quality of the reasoning and analysis of its participants. The collective intelligence generated by Metaculus connects rigorous modeling and the scientific method to complex real-world challenges, thereby improving decision-making and fostering transparency and accountability in the public domain. Its open access ensemble forecasts provide a sense-making utility for hundreds of thousands of readers, and are trusted by the scientific community, government agencies, effective charities, and the Fortune 500.
Experiential Learning Program Details
| School | University of South Carolina – Upstate |
| Engagement Format | - |
| Course | Summer Capstone in Business Analytics |
| Level |
|
| Students Enrolled | 12 - 4 students / project |
| Meeting Day & Time | Online |
| Student Time Commitment | 4-7 Hours Per Week |
| Company Time Commitment | 1 Hour |
| Duration | 8.29 Weeks |
Program Timeline
| Touchpoints & Assignments | Due Date | Type | |
|---|---|---|---|
|
REMINDER OF CONFIDENTIALITY
REMINDER OF CONFIDENTIALITY
During this program, you'll be working on a real project posed by a real industry partner. By registering for the CapSource system and participating in this Program, you are bound by the Rules of Engagement, including strict confidentiality of information shared with you. Please be respectful and professional.
|
June 4th, 2021 | Event na | |
|
Complete Pre-Kickoff Evaluation
Complete Pre-Kickoff Evaluation
Survey link: https://forms.gle/M6s5n9whydzqPP5W9
|
June 6th, 2021 | Event na | |
|
OFFICIAL PROJECT KICKOFF
OFFICIAL PROJECT KICKOFF
Find time to web conference and discuss project goals and timeline
|
June 14th, 2021 | Event na | |
|
Complete Temp Check Evaluation
Complete Temp Check Evaluation
Survey link: https://forms.gle/AVy1Xdq3pzskeN5d9
|
June 28th, 2021 | Event na | |
|
Complete Temp Check Evaluation
Complete Temp Check Evaluation
Survey link: https://forms.gle/AVy1Xdq3pzskeN5d9
|
July 14th, 2021 | Event na | |
|
OFFICIAL PROJECT END
OFFICIAL PROJECT END |
August 4th, 2021 | Event na | |
|
Complete Final Peer Evaluation Survey
Complete Final Peer Evaluation Survey
Survey link: https://forms.gle/om5ss8f1jHZEXWyd8
|
August 7th, 2021 | Event na | |
|
Complete Final Self Evaluation Survey
Complete Final Self Evaluation Survey
Survey link: https://forms.gle/Hb87qoaLq43stpZ19
|
August 7th, 2021 | Event na |
Key Project Milestones
-
June 28, 2021 - Product Analysis and Hypothesis Building
The first milestone entails gaining familiarity with a community forecasting platform before producing a table of variables hypothesized to be positively or negatively correlated with user retention across a 12-week period in the life of the platform, along with brief explanations of why these relationships might exist.
Suggested Deliverable:
A Word or Google doc detailing at least ten hypotheses of relationships that might exist between a particular variable and user-retention, as well as whether that relationship is positive or negative. There should also be a link to a relevant page of the site and an explanation of what on that page prompts a given hypothesis. If there are other offsite links that provide more general rationales for looking at this or that variable in light of user retention, these can be provided.
-
July 12, 2021 - Skill Up in Google Analytics Toolkit and Refine Hypotheses
The second milestone requires a dive into the different types of user and session-level segmentation made possible by Google Analytics, followed by creation of a more refined table of variables to be tested, as well as plausible means for creating segments able to test these variables.
Suggested Deliverable:
A Word or Google Doc updating on the first milestone deliverable to explain for each of the first document’s proposed hypotheses why it is or is not practical to be able to test a given hypothesis. For those that can be tested, a description should be provided of several segments that the student will generate in order to analyze the resulting cohort data. For any hypotheses that are no longer viable, the student should generate a new hypothesis to take its place until the student has at least 10 different hypotheses to test, each with at least one proposed segment to divide the data on.
-
July 23, 2021 - User and Session-Level Segment Creation, Logging, and Additional Segmentation
The third milestone calls for creating these segments within Google Analytics and analyzing the resulting retention data while keeping careful record of which variables did or did not correlate with retention, and while using these results to fine-tune the first set of segments to run a second set of analyses.
Suggested Deliverable:
An Excel workbook or Google spreadsheet with a row for each segment created. One column indicates the hypothesis to be tested. Another column indicates the segment used. Another column indicates the result of the analysis.
A Word or Google doc writing up the results of the analyses and proposing additional segments to test the hypotheses as needed, in light of information the student gained in the first set of analyses. These additional segments are added as rows in the Excel workbook or Google spreadsheet as a separate tab or sheet.
-
August 4, 2021 - Final Report with High-Level Writeup of Results of Analyses, Data Visualizations, and Actionable Next Steps
The fourth milestone is a final report to include visualizations of data pulled directly from Google Analytics, as well as plots of data generated from the segmentation analyses. This report will home in on the most salient, surprising, and actionable results from the segmentation analyses, will consider why these variables were or were not correlated with retention, and will offer suggestions to both Marketing and Product departments on ways they might make use of the results from the analyses.
Suggested Deliverable:
A Word or Google doc at least 1000 words in length and with at least two plots or data visualizations. This report will detail the results of the analyses conducted in a high-level manner to focus on the most salient, surprising, and useful results. The report will also provide actionable next steps for Marketing and/or Product departments in light of the results of the analyses. That is, what change to the product would likely increase retention and why? What might an effective marketing campaign look like and why?
Project Resources
There are no resources currently available
Industry Mentors
Academic Mentors
Assigned Students
There are currently no students assigned.