Performance is taken seriously at Airbnb and it’s part of their Commitment to Craft.
They’re devoted to optimizing user experience and making continuous improvements to fulfill their mission: create a world where anyone can belong anywhere.
Airbnb has invested significant effort in developing a complex weighted metric called the Page Performance Score (PPS) to accurately reflect user engagement. A series of A/B tests demonstrated that Time to Initial Load (TTIL) contributes to 50% of the impact of all performance metrics on user engagement.
In this article, you’ll learn how Airbnb leveraged Product Science Tool (PS Tool) to identify a critical performance opportunity that reduced category switch time by 30% with zero manual instrumentation work. By shortening the content loading time by 30%, approximately 400ms on high-end devices, they achieved around a 3.5% increase in user engagement metrics for the corresponding feature.
Concurrent processing — a common problem easily overlooked
It’s common that more than one UI change happens at a time, not in parallel, but in sequential order with the main loading process. In fact, Product Science research shows that almost 70% of apps have at least one screen that is slowed down by data loading after minor interface changes. This long loading can negatively impact the user experience.
Here’s an example of Airbnb’s “I’m flexible search” category switching
“I’m flexible search” is one of the key performance features at Airbnb because it directly impacts a user’s ability to find and view listings before making a booking. Users leverage this feature to explore Airbnb’s topical categories, such as "amazing view", "treehouse", "desert", and even "OMG" without specifying travel dates or a destination simply by tapping on different categories and browsing the results.
Airbnb’s PPS (Page Performance Score) had dropped lower than the targeted value for this feature. Upon investigation, there was a 1.2 second delay for high-end devices before the results appeared after a mobile user selected a category.
This happened because the network request was scheduled after the animation completion, rather than in parallel.
Complex challenges require intelligent solutions
Like most aspiring tech startups, Airbnb strives to respond to customer needs by shipping products quickly, and that speedy approach can sometimes impact performance. When features are launched and performance issues arise, it’s also easy to lose track of how they happened in the first place.
While there are tools to measure Airbnb’s PPS and support other observability needs, catching suboptimal performance issues and their root causes is a mix of luck and hard work. Most toolings’ instrumentations mainly measure system calls that don't always capture sub-optimality. A lot of time can be spent manually instrumenting an app to capture things from the right places. It can be a lot of work and hinder feature-shipping speed. Most toolings provide sampled data with missing method calls and don’t allow developers to back track what happened and how the user experience is impacted in real life, which is critical to performance optimization.
This is where PS Tool came in! Product Science’s automatic instrumentation is smart enough to filter what is important—even if it’s the smallest function—while removing any noise that impacts the analysis. Their AI technology, called execution path (we’ll cover this in next section), also helped to backtrack the user experience and pinpoint which part of the code was negatively impacting performance. With Product Science’s instrumentation and tool, the root issue was identified, performance issues were fixed and quickly brought a better user experience to Airbnb customers!
Identifying performance opportunities
PS Tool is designed to provide a detailed view of executing functions and the threads they’re running in, all while capturing what’s appearing on a user’s screen at the same time with 0 manual instrumentation. The tool provides multi-thread visualization that shows dependencies among different threads, which helps identify bottlenecks and isolate specific processes that cause slowdowns. By understanding the critical execution path (a sequence of functions forming the process under investigation) and delays within it, the Airbnb team was able to optimize code at ease.
Here’s an example of an execution path (highlighted in yellow).
Before fix
After fix
What does PS Tool show?
Using PS Tool’s video synchronization feature, a significant delay was identified from the screen recording in the initial build and the corresponding code was immediately located. Upon investigating the critical execution path, it showed that the network request fetching the category switch results was scheduled from a function that didn’t execute until after the category bar fling animation was over (flag 2 on pic 2), causing a delay.
After a category was selected, an animation occurred by sliding the scrollable tab bar over (flags 1-2 on pic 2). For a closer zoom into the area where the network request was scheduled, see pics 4, 5 & 6.
The function waited until the end of the animation to execute and then triggered the necessary network request to load the page.
Performance optimized
Airbnb’s engineering team rescheduled the network request to fetch the category switch results directly after a user lifts their finger (the moment when the category to fetch results is known) and tested it with PS Tool. It was confirmed that on the trace recorded with the after-fix build, the function responsible for scheduling the network request MapSearchFragment onFlexDestTabClicked was prioritized and executed before the animation (in less than 3 ms after the user lifts their finger: deliverInputEvent — see pic 7).
Results
The improvement was noticeable on the trace: category bar fling animation (area between flags 1 & 2, pic 3 and 8)was no longer part of the execution path and wasn’t delaying the moment when the user sees the category switch results. The trace-video sync confirmed this observation by comparing the end of category fling animation with the network request execution. By the time the animation ended, the network request was already halfway through.
No more tedious guess work
While Airbnb’s PPS provided guidance with performance and helped define performance goals, PS Tool completed Airbnb’s performance puzzle and provided clear next steps to specifically take the PPS higher.
With the help of PS Tool, Airbnb’s team implemented a fix that reduced the wait time of user’s tapping on a category for the “I’m Flexible Search” by 30% (roughly 400ms on high-end devices), or by 65% without taking into account the server response time, or any other back end work, with minimal effort and zero manual instrumentation work.
PS Tool gave the Airbnb team the confidence needed to address their performance issues. Instead of spending hours on fruitless, premature optimization planning, they can ship features faster and quickly respond to the needs and desires of our customers.
To learn more about PS Tool, request a demo today and discover how Product Science is revolutionizing the way developers work.
