A Centralized portal for users
Refining Pulse's template Content Management Portal
COMPANY
Microsoft
ROLE
Lead Designer
Duration
4 months
YEAR
2024
Summary
Pulse's CMS is an admin-level tool that helps HR teams manage templated survey content across their organization.
As the design lead, I built out an existing MVP with new core features to improve users time-on-task, reduce screen switching, and centralizing content creation and management.
Results
60% (10 minute) reduction of time-on-task for users & a 15% uptick in key customer adoption
Constraints
A tough-to-change legacy backend framework
Process
Lightweight testing with users; Synthesize data
Narrow down on problem
Alignment workshop to develop hypothesis
Design explorations & feedback sessions
Usability testing
Iterating on feedback; Collaborating with engineering
Initial product rollout
(Fast-follow) Iterating on feedback to add needed feature
01. Data reveals an underlying problem
Users were spending almost 20 minutes to make or edit a survey template
A key metric indicator was flagging that users were spending 3x as long making custom survey templates as we had targeted.
To narrow down on the cause, I ran a moderated usability study of the current design with users, having them think aloud as they went through their task of creating a template.
A moderated usability study reveals an underlying issue
So why were users taking so long in the template creation process?
Well, users relied on external programs or searched through existing templates to find survey question content.
This task-switching likely caused the slow task completion times and could become a blocker for customers looking to adopt Pulse.
How the journey faltered under the Existing design approach…
An HR team that manages Pulse content wants to update some a survey template with an additional question
The user first needs to find the question that they had written and saved into an existing survey template
Here the user enters the template management portal to look for that existing survey template
Once found, the user clicks into the template and enters the template editing screen
They can now look through the existing questions on the template to see if they can find the question content they need
If the user is unable to find the content they need they have to…
Check if it's part of Pulse's library of existing questions, or…
Hope the content is stored in an external word doc and manually copy the question and its format verbatim
03. Cultivating collaboration
After synthesizing findings from the usability study, I structured a problem framing workshop with my project team (PM, eng, content, and PM lead)
The goal of the workshop was to align the team around a mutually understood problem and shared strategy that bridged user needs and business goals.
I first ran through the current user journey to create a visual idea of user pain points and areas for improvement.
I then guided participants to identify business goals and metrics followed by dot voting to prioritize both.
Key business goals:
1
Increase customer adoption
2
Decrease time-on task metric
Problem statement created from workshop
Users spend too much time retrieving survey content across scattered sources, slowing template creation, reducing satisfaction and hurting adoption. Streamlining content management into a cohesive experience will improve efficiency and drive adoption.
04. Experiments in design
Exploring design options to test and validate
With alignment on a problem statement, I started to explore various approaches that we could eventually bring to users for testing. I held weekly syncs with the project PM and engineering team to discuss ideas and understand feasibility and use of existing patterns.
Different design approaches
Search for questions by templates that contain them
Users can find templates that contain specific questions, using a design that would require minimal engineering effort
Combined table with all content
Questions are displayed in-line with templates, leveraging the existing UI and centralizing content
Questions and templates in separate tabs
Question content is assigned its own tab within the portal, minimizing cog load when searching and supporting scalability
The separate tabs version won out for its focus on cognitive load reduction and scalability
05. Feeling Validated
Testing the designs
In partnership with the PM on the product, we developed a research/testing plan to validate the design approach. We tested a functional prototype with 7 participants over two days.
Results
Clear improvement
6/7 people said they felt the process was more connected and they loved being able to stay in the app
Request to save work
5/7 brought up an ask to have the ability to save content that they're not finished editing
Searching by topic
4/7 users mentioned that they were used to categorizing and searching for content by assigned topic
06. slight hang up
Working through Legacy framework issues
The biggest finding to come out of usability testing was that users really wanted a way to save in progress work and come back to it at a later time.
As I reviewed our test findings with product and engineering partners and proposed that we implement some type of drafting feature, I found out adding this would not be as simple as I first imagined.
The issue
A legacy backend framework couldn't support drafting of content shared among multiple users
The approach
Use competitive analysis to evaluate how other cloud-based programs manage storing and saving content
The solution — Saving locally
We noticed that these products all had one thing in common that we could leverage, the ability to save content locally on the user's computer. While this is normally meant for offline saving, it worked well as a compromise for not being able to implement a full drafting feature.
After confirming feasibility, and checking with our privacy team for any security risks, I drafted up a solution that was incorporated into our final design!
07. A good start
Rolling out the product showed positive results!
After 2.5 months of work, cross-collaboration, and research, we were able to ship this feature!
Shipped design flow (v1)
08. A quick-ish Fix
POST-LAUNCH DATA LED TO A NEEDED Change
Initial results
After the rollout of the updated portal, we saw a successful drop of over 10 minutes in the time users spent both creating new templates and editing existing templates.
A looming problem
While data was positive on our main success metric, a separate metric that tracked deleted content was on the rise. As usage increased, so did the amount of questions deleted, while the number of templates deleted by users remained relatively flat, but why was that?
The answer as to what was going on
After conducting short interviews with customers, I found that many users wanted to delist questions, but couldn't, so they ended up documenting them externally and then completely deleting the question
09. Rerouting
Updates to the product roadmap
Using data to advocate for a change
After discussing with my project PM and eng lead, I met with product leads to find a way we could work to implement a "deactivate" feature, noting that this problem could compound as our usage increased and as users did this more often and eventually cause users to rely on external programs to store content.
Successful negotiation
After making a case for this new feature by highlighting those key business impacts, it was successfully added to the roadmap! This gave me time to work out the design and plan with other designers to map downstream effects in other parts of Pulse.
10. The grand finale
Showcasing final design updates
After an additional 2 months of work to collaborate with engineering, product, and designers of other product areas, the final design based on the updated roadmap was rolled out!
Updates for content deactivation
Prototype embed
Want to click around a more functional version? Feel free to use the prototype below! (PW is included in my resume)
Key Impacts
60%
Decrease in time-on-task
15%
Uptick in key customer adoption
Multiple key customers who were unsure about adopting Pulse, adopted the product after previewing this feature.
Project takeaways
Areas to improve on
We had the option to run a private preview before full rollout of the project, however opted not to due to time constraints. In the future, I would want to budget our time better and negotiate for this step to avoid having to design a larger fix in a fully launched product.
I realized my usability test plan was affected by tunnel vision. I focused too much on confirming narrow assumptions, which kept me from seeing how users naturally moved through the workflow. A more open approach would have revealed unexpected behaviors and surfaced the need for disabling content earlier.
VLL (Valuable Lessons learned)
Working through the drafting issue wouldn't have been possible without partnering with engineers on the project. For me, this reinforced the positive outcomes of collaborating cross-functionally.
Framing problems in a different lens for different stakeholders can go along way in helping to get buy in.
For example in this project, I approached a new user pain point of having to use a work around to hide question content as a long-term business problem that could grow quickly and scare off current or future customers.
