The White House
Office of Digital Strategy Intern
As an intern in the Obama White House’s Office of Digital Strategy, I conducted user research for product rollouts. One of my major projects was conducting user research for the launch of the White House’s Facebook Messenger bot.
Scoping the project
To outline research goals for the launch, I participated in multiple meetings with the product team to identify opportunities for getting feedback from research participants. Based on these meetings and conversations with the product manager, I decided on the following central goals for the project:
Measure response to the bot’s greeting response
Record the user’s overall response to the bot experience
Once I determined the project's direction, I focused on determining what type of user testing I would conduct to fulfill the research goals.
I decided to design an A/B test with different variations of the bot’s greeting to determine which greeting was the most compelling. I choose an A/B test because the product manager was considering two separate options for the bot’s greeting, and I determined that quantitative data would provide the clearest distinction in this instance.
To record the user’s overall response to the bot, I opted for a qualitative approach. I recruited research participants to participate in an informal interview regarding their impressions of the bot.
SYNTHESIZING and presenting the deliverables
After conducting the research, I mapped out my findings and designed a presentation to share with the product team. These were my findings:
With regards to the A/B test, 65% of users continued the conversation with the bot following “Greeting 1,” whereas 78% of users continued the conversation following “Greeting 2.” Therefore, I concluded that “Greeting 2” was a better option to conclude in the bot.
To make sense of the qualitative data regarding the users’ overall response, I coded and developed themes based on the responses I received. Two major themes that I found were that users felt that the bot was impersonal, while it was also informative. Therefore, I concluded that the text of the bot should be updated to include more elements of personalization where possible (for example, including the user’s name in the greeting.)
To present this research to the product team, I delivered a live presentation that outlined my methodology and findings.