It's incredible how quickly ChatGPT has become an essential part of our lives, especially considering it was released only 7 months ago. However, OpenAI's mobile strategy took an outdated approach, even for an emerging product, as they started with the web version before launching the first mobile app, for iOS only. In contrast, more experienced competitors in both AI/ML and user attention, like Meta or Snapchat, have learned the hard way about the negative impact of delayed releases on user experience.
OpenAI's initial release was limited to iOS, leaving the majority of mobile users waiting for another 4 months before the Android version was gradually rolled out, and that too, initially in just a few countries. Now that we finally have access to both app versions, we can conduct a quick performance assessment and comparison.
As expected, we've had a similar experience with OpenAI's chatbot on mobile as we've had with ChatGPT on the web. Interacting with ChatGPT on your mobile device feels like exchanging messages with a friend on a messaging app. The process is as simple as having a one-on-one chat with a constantly available friend (so far, we haven't experienced any ANRs - Application Not Responding issues). Just type your message at the bottom of the screen, hit send, and ChatGPT will respond promptly with advice, tips, or any assistance you may need. Moreover, the mobile app seamlessly integrates with your existing ChatGPT account, ensuring you can access your past conversations across all your devices if you've enabled the "chat history & training" feature. This capability adds even more convenience to the overall experience.
Testing the performance of ChatGPT is particularly amusing, thanks to its signature feature - intentional delays in response and a human-like imitation of typing, rather than displaying a message in full immediately, like most other chat applications. With these deliberate delays as features rather than bugs, we were curious to see how the actual UI delays and latencies would feel. Well, no surprises. Just like with any other app, the delays were slightly annoying and potentially distracting, which proved to be detrimental to user engagement for as small as 50ms lags.
We put the most recent production version of the Chat GPT app to the test on an iPhone 13 Pro Max and Pixel 7, both under optimal networking conditions. Although the app boasts an extremely simple UX, our eyes spotted areas that could use further improvement. With these potential enhancements, the app has the power to deliver an even smoother user experience, taking it from good to great!
Chat history loading on iOS
When users open their chat history, they see a list of previous chat titles. However, certain chat names seem to load significantly slower than others, even though they use the iPhone 13 Pro Max with a perfect network connection. Chat GPT automatically generates these titles, and ideally, they should have been created as soon as these chats became part of the chat history. Interestingly, this delay is not observed on the Android app. With the utilization of the PS Tool Profiler, we can tackle this specific delay and achieve a reduction in loading time of up to 65%. This enhancement promises to create a smoother and more enjoyable user experience for iOS users.
“Voice-to-Text” delay on iOS
Another delay caught our attention in the voice recognition user flow. When users press the button to record their speech, complete their message, and send it, there is a noticeable lag in the app's speech conversion to text. While we don't have access to the actual code, making it challenging to pinpoint the exact reason behind this delay, there is indeed an opportunity for improvement in this aspect of the application's performance.
.gif)
Chat response delay on Android
The Android app has exceeded expectations with its exceptional performance. However, we did observe a screen blink issue in one of the main user flows. Specifically, when users send a request to chat on a Pixel 7 with a perfect network connection, they notice a momentary screen blink before the text starts appearing. Based on our experience, this could be related to an unnecessary layout update. With the help of the PS tool, we aim to optimize the loading of Chat responses and potentially reduce the blink issue by 13%, further enhancing the app's overall user experience.
About authors:
Misha Kuznetsova is a Jr. Product Scientist at Product Science and founder of Adventure Awaits App. Experienced in optimizing mobile app performance for companies of all sizes (Telegram, PicsArt, DressX, and others).
Jordan Wooten is a Product Scientist for the mobile app performance management startup, Product Science, currently living in San Francisco, California. Jordan has a degree in Mathematics with a secondary specialization in entrepreneurship, both of which fostered her interests in the intersection of STEM and startups. As a Product Scientist for Product Science, Jordan uses the PS Tool to analyze and diagnose performance issues for many popular mobile apps across various platforms and industries.
Acknowledgments: Thanks to David Liberman, Oleg Pashkovsky, Scarlett Iu, and others.
If you’re interested in tackling challenges like this, join our team! Time is humanity’s most valuable non-renewable resource. Our mission is to help all people in the world stop experiencing delays from software inefficiency.
