Postscript, an SMS marketing platform, allows merchants to send automated messages (called automation flows) to their customers. Recently, new functionality has been added that allows merchants to see previous versions of their flows.
On average, 178 SMS merchants edit their automation flows every week. However, sometimes these edits result in a negative impact to their revenue, so merchants end up wanting to revert their changes. Up until recently, there was no way for merchants to revert their changes and also no easy way to compare analytics to previous automation versions. This lack of functionality not only led to frustration for Postscript customers, but prevented prospects from switching to Postscript from competitor platforms (that did provide this fucntionality).
The original experience for automation flows only showcased the most up-to-date version of a flow along with its unique analytics.
As the lead Product Designer on the project, I worked closely with a Product Manager, digging into analogous research and customer feedback. I created several different mock-ups and shared them with my team for feedback. Upon receiving feedback from Product, Engineering, and other designers, I continued to iterate until I felt the designs were in a polished enough state for usability testing.
I conducted an unmoderated usability test using UsabilityHub, where 8 participants walked through a Figma prototype. The results are as follows:
(Above) A heatmap displaying first clicks for the prototype
Initially, I was concerned with how many participants would struggle to locate the version history drawer. (Version history could be accessed via an overflow button in the top right corner. This placement seemed to be a standard pattern across other versioning functionality in applications like Google Docs.) Most misclicks did occur during this initial step; however, all participants were able to discover the version history drawer and successfully complete the task.
(Above) Prototype from external usability test
In addition to the external test, I had 4 internal Messaging Strategists walk through a prototype, identifying key differences in the multiple automation versions. All 4 participants easily navigated through the prototype and correctly accounted for the flow changes.
(Above) Prototype from internal usability test
One interesting point discussed in the internal feedback was around the “Changes” in the version history drawer. One strategist asked if the changes could be automated, so the system could record the changes and populate them in the section without an individual having to manually record them. (This idea unfortunately didn't make the cut for the initial feature release, but it's something the team will be looking into at a later time.)
When a merchant makes changes to their automation, they are prompted to confirm their changes and add details regarding what changed.
(Above) A feature where a merchant can document their flow changes
(Above) When a merchant selects a previous version and the preview updates
While the mock-ups above were what was originally envisioned, there were some limitations our team had to address concerning the drawer UI. Working closely with a FE Engineer, we discovered that a drawer spanning the full height of the screen was not possible due to the page layout component our engineers leverage.
(Above) The currently implemented version history drawer design
We went back and forth on how to solve for this limitation and we eventually landed on the decision to reuse a pre-existing UI container component that sits within the flow grid view. While this does significantly limit the space of the version history drawer, we were able to reduce some of the initial vertical space in the data cards to maximize what a merchant could see while the drawer was opened.
Post-release, we also received feedback from our CTO. Despite the average number of versions for a single automation being under 30, our CTO wanted us to account for scalability in order to not overload the database in the future. After mocking up several different options and getting feedback from the design team, I landed on a skeleton loading state in order to decrease "users' perception of a long loading time" (Nielsen Norman Group, 2023). Once 20 automation flows are loaded, we seamlessly load another 20 automation flows into the drawer as a merchant is scrolling.
(Above) Different iterations for how to display the drawer's loading state
After the new version history functionality was released to 100% of customers, we saw the version history drawer accessed 50+ times on average in any given week. We've also seen that ~5% of merchants who access the drawer use the "Restore Version" functionality in order to optimize their automations based on success metrics (like revenue and orders).
The next steps for expanding on the version history functionality include adding a duplication action for an automation flow and automating the description creation when a change is made to an automation flow.
There were several instances in the last year where the Flow Builder team leveraged the use of workshops in order to achieve a goal. Having led several sessions, I wanted to highlight a few important ones.
Workshops are used to solve important problems and generate ideas in a highly collaborate way. While similar outcomes can sometimes be achieved by an individual, including a group or team in the process can generate more and often better ideas. It is because of this reason that I led the Flow Builder team through two important workshops in 2023.
The first workshop I led was in response to internal feedback about a navigability issue in Flow Builder. There were some known spacing issues inside of the Flow Builder tool; however, the issues were amplified after a recent feature release. These spacing issues made it extremely difficult for individuals to navigate Flow Builder.
Unfortunately, our team didn't have the bandwidth for a complex solution; therefore, I proposed that we try a post-up exercise to generate possible solutions.
(Above) A video sent from an internal stakeholder
During the post-up exercise, each team member had a set amount of time to generate ideas on their own. Next, we sorted the ideas into broader categories and took additional time to vote on the post-its or categories that resonated the most with us. After voting, we discussed the ideas with the most votes.
(Above) The Figjam our team used to brainstorm ideas
Lastly, after the brainstorming session, I placed the most-voted on ideas in an Impact Effort Matrix. I then moved post-its around according to team feedback.
(Above) A matrix that organized the team's ideas based on effort and impact
Based on the workshop outcomes, the team was able to prioritize items that would create the most impact for the least amount of effort. One of these items included introducing a mini-map in Flow Builder. The workshop exercise helped reveal that the flow react library the team leverages for Flow Builder actually has a built-in option to add a mini-map. Therefore, very little effort was required to make the update.
(Above) The implemented mini-map
After introducing the mini-map, internal and external feedback concerning the navigability of Flow Builder was reduced by 99%.
Working for a remote company doesn't introduce many opportunities to work with others in person; however, the Flow Builder team was fortunate enough to meet up in person for a few days in August, 2023. During this time, we spent an hour and a half brainstorming ideas that individual team members had to improve the Flow Builder experience.
(Above) The Figjam prompt for the workshop
Similar to the navigability workshop, this workshop focused on first generating individual ideas and then coming back together for a group discussion. One unique facet of this session was the introduction of a “won't do” section. Instead of only generating ideas that the team thought would improve Flow Builder, team members also had the opportunity to push back on ideas that were already in our backlog or roadmap. While this section didn't get overly utilized during the workshop, the team had a few concerns to share about a couple backlog items.
(Above) General ideas and ideas related to AI enhancements organized into a matrix
After generating ideas and voting on them, the team came together to discuss the ideas that received the most votes. These items ranged from ideas that were strictly BE initiatives to items that were purely related to the content we surface in-app. Once we walked through all the ideas, I again created an Impact Effort Matrix and shared it with the team. (During our in-person collaboration time, we also ideated on ways to leverage AI/ML within Flow Builder. Those items were added to the matrix as well.)
Once the Flow Builder team helped to properly organize the matrix, the Product Manager, Engineering Manager, and myself reviewed and added some of the items to our stretch goals for Q4 in 2023. One proposed project (updating the message builder 3rd party library) even made it onto the Q4 roadmap as a priority rather than a stretch goal.
Merchants spend the majority of their time on Postscript, an SMS marketing platform, creating and managing their SMS campaigns and automations. Flow Builder allows them to accomplish this more easily by introducing multi-message functionality within a single SMS campaign or automation.
There is limited functionality and support for the current tools merchants are accustomed to using; however, merchants are wary of change and have been slow to adopt Flow Builder—in turn, missing out on improved features that can help make their jobs easier.
The legacy SMS campaign experience (below left) uses outdated design system components and has limited functionality such as a single-message restriction and lack of branching capabilities. Meanwhile, the Flow Builder experience (below right) supports single-message use cases as well as more complex flows that include important functionality like A/B testing.
As the lead Product Designer on the project, I interviewed merchants about their experience using Flow Builder. In addition to running customer interviews, I created a Pendo survey soliciting asynchronous feedback and utilized Amplitude to track Flow Builder usage. I also worked closely with Product and Engineering by conducting a workship to highlight opportunities for increasing adoption.
We made the following updates to the product in hopes of boosting Flow Builder usage:
(Above) The addition of a "Try Flow Builder" button added to the legacy experience
(Above) A step in the process for migrating legacy automations to Flow Builder
In order to sunset the legacy experience, we are working towards product pairity between tthe legacy experience and Flow Builder.
Postscript, an SMS marketing platform, recently released new functionality (called Text-to-Buy) that allows customers to purchase products via the text thread.
Since Text-to-Buy is still in Beta status, it does not provide all the capabilities merchants are seeking. On average, our enterprise level merchants have 5 options per product that require their customers to make a selection (e.g. size, color, etc.), but the current implementation requires merchants to make this choice on behalf of their customers. Since merchants have a limited knowledge of their customers' personal details, they opt out of using Text-to-Buy in its current state.
Currently, the Text-to-Buy experience requires merchants to select a product and all its options before sending out a Text-to-Buy campaign.
(Above) What a merchant sees when creating a Text-to-Buy message
(Above) What a merchant's customers experience
As the lead Product Designer on the project, I iterated through several ideas and shared them with the Postscript design team for feedback. After sharing the refined designs with Product and Engineering, I used UsabilityHub to run a series of usability tests, comparing different approaches. I shared the findings with my team and led a kickoff meeting where I aligned with Engineering on requirements.
We ran two rounds of usability testing. For the first round, users ran through 2 mobile-simulated experiences on their computers. The first prototype had them place an order solely in the text thread, while the second prototype had users go back and forth between the text thread and a web checkout experience.
(Above left) The text-based prototype, (Above right) The web-based prototype
Since the results of the initial usability test were very similar, we decided to iterate on the two experiences further and test again. This time, we were able to test on mobile devices to ensure the most authentic experience for test participants.
The first prototype still took place entirely in the text thread, but the selection process was simplified furthered. Meanwhile, the second prototype was updated so that only the first step took place in the text thread and the rest of the experience was on mobile web.
(Above) The preferred text-based prototype
Once we had our kickoff meeting surrounding the project, we uncovered some possible limitations in the winning Usability Hub design. Therefore, as an MVP, we slightly shifted our approach by scaling back complexity and focusing on products that only have one customizable option. However, once we get validation from the new implementation, the team will spike on how we can leverage our data to make the winning design a reality.
(Above) What a merchant will see when creating a Text-to-Buy message
(Above) What a merchant's customers will experience
This project is currently in development, so we do not have any data surrounding our main objective around increasing Text-to-Buy usage. However, once the update is made, we will create product release materials to share with Postscript merchants and garner more feedback from the merchants that our currently using Text-to-Buy.
Homeowners that use Vrbo, a vacation rental platform, set property rules travelers must agree to prior to booking. These rules encompass items like max occupancy, minimum renter age, allowance of pets, and several other rules.
Hosts cancel on travelers due to confusion surrounding house rules. Vrbo has over 13,000 cancellations annually because of house rules. The result is unhappy customers and a loss of revenue.
There have been a lot of concerns surrounding the original house rules experience for both hosts and travelers. For example, one early study found travelers less inclined to book a property when custom notes were included in the house rules. (The outlined sections pictured above are areas for custom content.)
* A UX hueristic evaluation is an internal evaluation by subject matter experts applying the Nielsen Norman Group usability hueristics to an experience.
As the lead Product Designer on the project, I attended weekly progress meetings with the Product Manager, Technical Project Manager, Marketing Specialist, and Content Strategist. I also worked closely with the traveler designer to ensure there was consistency across the host and traveler experience. Lastly, I regularly shared my working designs at our host team design critique for transparency and feedback.
(Above) Some early iterations for a rule about pets
Our UX Researcher performed a remote moderated study where 9 participants walked through a multi-step prototype. Our team was looking for feedback on:
(Above) A brief look into the functioning prototype
After updating some of the content and syncing with the Engineering team about scope, we were able to push forward with an updated version of house rules.
The new host design utilizes components from the Vrbo design system, allowing hosts to always have the save action within view. The content in the host view is also clearly represented in the traveler view as the host intended.
The original rule included an "on/off" state as well as options for "yes" and "no". The new rule has only the "yes" and "no" options with other more specific inputs inspired by early user research.
In order to provide the Engineers with clarity, each rule has been broken down into its different states. Using progressive disclosure, we only surface relevant content when the appropriate rule choice is selected.
To improve the house rules experience even further, we plan to add more rules after implementing additional natural language processing for host custom rules and notes.
When using Vrbo, a vacation rental platform, hosts must sometimes change or cancel a guest's reservation if the need arises.
When a host cancels on a guest, everyone loses. The guest is disappointed, the host's search rank falls, and Vrbo loses out on revenue. Host cancellations contribute to 10% of reported guest pain points and current limitations of the reservations cancel and change capabilities lead to millions of dollars in operational costs—not including the revenue lost for Vrbo and hosts.
The original cancel experience has led to a lack of trust between travelers and hosts. In this experience, hosts are not required to explain to the guest why they are canceling.
The original change experience does not allow a host to add a payment request or issue a refund. These actions can be taken immediately following the flow, but there is little direction for hosts about how to handle these situations.
While we are currently working with the research team to test the newest cancellation and change flows, we have gained some insights from the swarm usability study:
(Above) A portion of the tested prototype that included changing a reservation
Though we are still waiting on the next round of testing to begin, the new designs are based off previous research and extensive feedback.
The new design adds friction against canceling at the top of the panel. We have also added a new required step of submitting additional context for every cancellation reason.
For host cancellations, we want to require 100% refund for guests since it is not their fault for these kinds of cancellations.
Though we may not be able to prevent all cancellations, we can at least help prevent future cancellations by prompting hosts to sync their calendars and avoid double bookings. This coaching is specific to the reason the host is canceling on the guest.
The options to change a reservation are now surfaced with the option to cancel. This allows for more scalability as change functionality grows.
The new design gives hosts the option to view their calendar availability when choosing to change a reservation's dates. In the past, host have provided feedback concerning how they prefer to view their availability via the calendar.
We have also included the ability to add a payment request or issue a refund within the change flow. The option presented will be based on a host's settings, but hosts are still able to override the selection.
While working on the vacation rental designs, we have also been working on designs for larger hosts like hotel owners. Our two products, vacation rentals and hotels, are headed towards convergence and the cancel/edit experiences lend themselves as an optimal place for consistency.
Vrbo is a vacation rental platform that allows homeowners to rent out their property. Homeowners/hosts can approve or decline a traveler's request to book their property. The decline flow allows hosts to explain why they cannot host a guest.
Host declines lead to disappointed travelers and a loss of revenue. Vrbo loses out on millions of dollars annually due to host declines. If hosts had an easier way to keep their settings and calendars up-to-date, a bulk of these declines could be prevented.
The initial decline experience provides friction for hosts—possibly too much with the red symbol and dramatic red text. This design attempts to scare the host out of declining, rather than persuading them to approve.
On the next step, hosts are bombarded with a confusing list of reasons. Once they select a reason, they are able to decline without any guidance on how to avoid missing out on future opportunities.
* A UX hueristic evaluation is an internal evaluation by subject matter experts applying the Nielsen Norman Group usability hueristics to an experience.
As the lead Product Designer on this project, I worked with the Product Manager and Content Strategist. We worked closely with the Engineering team to ensure our ideas were feasible from a code standpoint and received thoughtful UX feedback from the engineers in turn. I also shared the work-in-progress designs with the larger design team to ensure a variety of feedback.
(Above) A mapping of the old decline reasons to the newly proposed reasons
(Above) Different explorations prompting hosts to block their calendars if declining due to property unavailability
Based on user feedback, we were able to pare down the reasons for declining a reservation request. We also opted to add a step mid-flow to block a host's availability. Lastly, depending on the decline reason, we implemented coaching either mid-flow or post-flow to help hosts save requests or prevent future declines.
We used an A/B test to determine if the new designs had a positive impact on booking acceptance rates and a negative impact on booking decline rates.
After a few months, we were able to validate our designs and pushed the new experience live for all users.
Since the new experience has been live for all users, we have seen the following changes:
Since implementing new decline reasons, we have learned that many hosts are now selecting "Dates booked on another platform" (22%) when declining a booking request. This tells us that many hosts that list their properties on multiple websites are still not syncing their calendars between platforms.
With the new designs, we are exposing more hosts to the calendar sync functionality, but we still have a ways to go. The next step forward is to simplify the calendar sync process by exploring automation through app permissions.
While interacting with Vrbo, a vacation rental platform, hosts spend the majority of their time managing reservations and messaging guests via their inbox. The inbox page is the second most visited page on Vrbo.
The inbox is home to booking requests, guest messages, cancellations, and so much more. Therefore, it holds a lot of value. That being said, there is a lack of urgency with hosts responding to guests—especially when it comes to inquiries (messages without a booking request attached). 10% of Vrbo's revenue can be traced back to inquiries, but only 3% of inquiries turn into full-fledged reservations.
The original design treats every message with the same sense of urgency. There is no indication of unread messages or actions a host needs to take on a specific reservation.
* A UX hueristic evaluation is an internal evaluation by subject matter experts applying the Nielsen Norman Group usability hueristics to an experience.
During the aforementioned design swarm, our team was able to do a lot of competitive and analogus research pertaining to how other inbox-like designs were constructed. Utlizing the swarm artifacts, I worked as the lead Product Designer on the inbox redesign. I regularly met with the Product Manager, Engineers, and the Content Strategist to convey progress and troubleshoot issues with working designs.
(Above) Several early iterations of a mobile inbox design
Our UX Researcher performed a remote moderated study where 9 participants walked through a native prototype. We were looking for feedback on:
In the prototype pictured above, participants were able to send a payment request reminder from the inbox. Participants were also able to utilize native-specific functionality like swiping in order to perform certain tasks.
The new design not only highlights the booking status better than the original experience, but depending on the booking status, inline coaching maybe present to help hosts act more quickly when they need to. We have also added the estimated payout to inquiries and booking requests to help incentivize action.
We started designing the new mobile experience and worked up to desktop—leveraging native iOS components to make implementing more efficient for our Engineers.
Another way we hope to increase response and acceptance rates is by surfacing response and booking acceptance metrics to hosts. If a host's acceptance rate falls below a certain threshhold, we can coach them to do better.
This experience has not been implemented yet, but stakeholders are expecting a noteworthy impact in response times and acceptance rates.
Automation Flow Versioning
Flow Builder Workshops
Flow Builder Adoption
Custom Variant Support
House rules
Decline coaching
Cancel and change
Inbox redesign