Learn about the key performance metrics that mobile teams should track in their mobile apps so they can deliver better user experiences. In order to build the best experiences, engineers need the best data available. Mobile makes this challenge more complex, with variables that include device types, different operating systems, and connectivities, to name just a few. With a growing number of tools that all offer varying levels of visibility into your mobile app's health, performance, and stability, it can be difficult to know exactly which metrics you should be tracking. In this post, we’ll outline the most important performance metrics you need to track and how they help you get to the root cause of issues more quickly so you can build better mobile experiences. 1. Startup time Mobile users typically check their apps on the go and are accustomed to instantaneous results. Therefore, they won’t sit around and wait for the app to load. For example, if the Uber app takes too long to load, the user will probably switch to the Lyft app instead. In addition, users who had a poor experience in the Uber app and a decent experience in the Lyft app are much more likely to become a loyal Lyft user. Now, not only have you lost revenue from that user’s session, but your cost per acquisition and churn increased, not to mention the LTV that particular customer could have brought the company. Therefore, is essential to its success. ensuring your app’s startup time meets user expectations However, if the mobile team only has access to the , they probably miss key changes and respond reactively rather than proactively. average startup time For example, let’s say a company is launching its app into a new market, and the overall feedback is negative. The average startup time shows that the app is a few milliseconds longer than it was before the launch, but it doesn’t indicate that it became dramatically slower during the launch. So there must be another problem, right? Unfortunately, because the percentage of users in the new market is only a fraction of the overall user base, it makes sense that even an abysmal startup time in the new market would only minimally impact the total user base’s startup time. Instead, the team needs to be able to segment the data to better understand how startup time impacts the business. For example, how are high-value users suffering from slow startups? Are users in a new market experiencing worse performance? Are certain devices experiencing slower startups? This data puts your company back in control of the situation and omits the guesswork in time-sensitive scenarios that impact your revenue. To learn more, check out our eBook on . how to improve mobile app startup time 2. Crash rate are a surefire way to anger customers and for a good reason! It’s essentially the equivalent of a customer walking into a brick-and-mortar store where the staff kicks them out the door halfway through their shopping. Crashes This is a major issue for the company’s brand for two reasons: It as customers feel their time is disrespected. damages the reputation of your brand It hurts your revenue as customers can’t complete their immediate transaction, and you lose the lifetime value of that customer if they decide to switch to a competitor. Here are just a few examples from various industries where a crash directly translates to lost revenue: If an e-commerce app crashes during checkout, the customer won’t be able to make their purchase and likely won’t be back. E-commerce Apps: : If a POS system crashes during a live event, none of those live customers will be able to make a purchase or enter the venue. POS Systems : If a smart device, such as a toothbrush, crashes during the setup process, it’s very likely the customer will return the product. Smart Device Apps However, while tracking the average crash rate is a good start, it’s . still not enough to understand how crashes impact your business For example, if the current crash rate is just 0.5%, you might not see the need to drill deeper. However, what if all of the crashes that occur happen at the checkout screen? That tiny percentage could be cheating the business out of significant revenue. So, in addition to looking at the major metric numbers, it’s also important to have data that shows patterns in the crash rate. Specifically, how are various high-value areas in your app performing? Which devices tend to experience the most crashes? How is the app performing for high-value users? Are there any regions that experience particularly poor crash rates? And if so, should the app be fixed or removed from those regions? By segmenting crash details, your team can better prioritize fixing issues. 3. ANR rate are typically described as freezes or glitches. Application Not Responding (ANR) errors Essentially, if the main thread is blocked, the applications cannot run effectively. Therefore, the user cannot proceed, which can have a major impact on your business. For example, a retailer we worked with had an issue with ANRs. This issue was causing startup times to increase by almost 60%, translating to an estimated revenue loss of $6.5 million per year. , the engineering team was able to quickly address the issue and reclaim that lost revenue. With the right data Beyond revenue, ANRs can also in the Google Play Store and make it less visible to new customers. negatively impact an app’s rankings To effectively track ANRs, you can look at the stack trace and see how users responded to the issue. From there, the mobile team can prioritize what to fix first based on how many people leave at various thresholds, where high-value users are most affected, and which device types and screens are most affected by ANRs. To learn more, check out . this post on investigating Android crashes like ANRs 4. Region metrics Tracking region metrics is also very important as users in different regions have different devices and different connectivities. This can have a major impact on the business for several reasons. First, regions differ in their importance to a business’s bottom line. For example, perhaps only a portion of your users are in Singapore, but they might make up a large portion of your total annual revenue. Therefore, checking region metrics on a granular level will spotlight opportunities to improve the app, particularly for high-value users. Tracking segmented region metrics is also essential when launching into a new region. For example, suppose you expand into Australia, yet that geographic area only makes up 5% of all users on launch. In that case, it won’t impact median/average metrics enough to allow the team to effectively track performance. It’s also important to take into account cultural differences, and with a tool that provides region-specific metrics, the team can test specific aspects of the app just for that region. For example, an e-commerce checkout screen that converts well in the United States may not convert as well in Dubai. In addition, granular region metrics make it easy to roll out new features slowly. For example, the team can roll out a new feature in a smaller region and see how it performs. If it performs well, roll it out to more and more regions with higher-value users. This data can guide your team in answering critical questions like: Do we need to build a new app for this region? How is this new launch performing compared to launches in other regions? Which regions are the most problematic? And should we stop serving them altogether? How are various critical areas of the app performing in one region versus another? 5. Session duration Another key metric to keep an eye on is session duration, as it signals how long users are using your app. If the average session duration changes dramatically in a week, it’s a pretty good hint that there is an issue with the app. For example, if you have a gaming app and notice the average session duration dropped from 15 minutes to 5 minutes, there’s a good chance that users had a poor experience. With this clue, you can ask questions like: Did we ship a bad release? Are users less engaged with the app? Is there a corresponding change in another metric that can help explain the change in session duration? Tracking session duration also allows the team to investigate patterns among the affected individual sessions and discover the root cause of issues. By analyzing session duration in parallel with other metrics, mobile engineers can draw a clearer picture of which issues are most intrusive to users. For example, an e-commerce store might have an OOM issue on the main scrolling feed of products that correlates strongly with a shorter session duration, whereas slow network calls on another screen have little to no correlation with the session duration. Therefore, tracking session duration is a great way to prioritize which issues should be fixed first as it directly reveals reduced user engagement. 6. Churn/retention rate It costs a lot more to acquire a new customer than it does to keep a current customer happy, and a leading cause of churn in mobile app users is a poor user experience. Downloading a new app only takes a few seconds, so if an app’s experience interrupts the user in a way that takes more than a few seconds of their time, don’t expect them to stick around. Therefore, track not only the overall churn and retention rate but also the churn and retention rates of segments of the user base (by devices, connectivities, regions, etc.). This will illuminate various opportunities to improve and prioritize retention. For example, you may find that while the churn rate is very high in one particular region, that region has few high-value users. Therefore, you may decide to invest engineering resources somewhere else where they will translate to larger business outcomes. Tracking churn and retention is also a great way to understand how users respond to new releases and various experiments. If there is a correlation between high churn and a new feature update, that’s a strong indicator that the team needs to roll back that feature update. Detailed data will allow the team to track the churn of various segments (such as specific regions or devices) to which the update is rolled out. Without segmented data, it will be difficult to see the impact that various feature updates have on small testing groups. Therefore, only once the feature is rolled out to a large group of users (and presumably caused many monthly active users to leave) will it become apparent that the feature rollout was premature. 7. User termination rate While some user terminations are nothing more than a user decluttering their phone, many terminations occur because the app has frozen, and the user has no choice but to swipe up and terminate the session. Of course, a user who is forced to terminate their session will likely be disgruntled and choose a competitor’s app instead. To prevent this, monitoring the average user termination rate is an excellent indicator of potential issues that could lead to churn, including: Failed ad loads are causing freezes Slow or broken user flows Failing purchases Server outages causing login errors Excessive media loading that causes slowdowns In addition to providing an overview of the average user termination rates, mobile teams need to know the source of every poor user experience. Therefore, one of the key features we built into Embrace is the ability to see . which screens cause frustrated users to abandon your app Therefore, rather than wasting valuable time and losing sales while engineers guess where the terminations are occurring, the mobile team is immediately directed to the issue so that they can fix the problem as efficiently as possible. 8. Timing of key user actions There are likely a few user actions within your app that absolutely must work 100% of the time. For example, if an office offers keyless entry, that feature must work every time. Otherwise, people may not be able to enter the office without calling additional support. Therefore, select a few key user actions and add them to the list of performance metrics the mobile team is tracking. In many cases, customer reviews don’t say where they experienced an issue in the app, so tracking specific user actions is a great way to catch issues that otherwise aren’t immediately apparent. This also helps you answer questions like: How many users experience issues in these critical areas of the app? Is there a direct correlation between issues in specific areas of the app and churn? How long do users try before giving up and abandoning the app? For example, one of our customers noticed that about 1% of all purchase attempts resulted in purchase failures. However, both of the associated network calls were resolving successfully, so there were no obvious errors to inspect. Therefore, they began tracking the exact moment a customer would make a purchase and found that the two network calls were happening out of order in 1% of purchase attempts, which resulted in a failed purchase. Though customers had been complaining about this issue, the mobile team couldn’t pinpoint the root cause without knowing the timing, outcome, and order of all events within the affected sessions. High-fidelity user experience data was vital in helping them reclaim 1% of their total sales. For a company doing $10 million in annual sales, that’s $100,000 that would otherwise be lost each year! 9. Memory usage It’s important to track your app’s memory consumption to identify any potential memory leaks. The efficiency of your app’s memory usage directly relates to how responsive your user experience is. This is why monitoring and optimizing memory usage plays a crucial role in delivering a seamless user experience. You can avoid issues with your app’s memory usage by: Optimizing your codebase and pinpointing areas where memory is unintentionally retained causes an increase in memory usage. Tracking your app’s memory consumption on a regular basis, noting any changes after new releases. Carefully monitoring memory-intensive operations like loading large images or processing large files and creating for abnormal spikes or sustained high memory usage. alerts A fine-tuned strategy regarding your app’s memory consumption creates responsive, stable, and effective apps that your users will love. 10. Connectivity Mobile apps operate in diverse environments with varying , including 3G, 4G, 5G, Wi-Fi, and sometimes limited or unstable connectivity. network conditions This is why it’s important to recognize the challenges posed by these varying conditions to create great user experiences. Indicators of poor network performance include: Latency and slow response times. Bandwidth usage. Inefficient API calls. Device-specific offline capabilities. For example, is an agricultural app that allows farmers and agronomists to document their findings while they’re in the field with their peers and colleagues. Farm Dog The app would crash often when network response times were exceptionally slow and when devices couldn’t determine whether they were connected. They noticed that Google Maps response times between 18-22 seconds when they should be only a few seconds. Without the proper tooling, they would need to use convoluted workarounds to solve issues using a proxy to simulate a bad network connection. However, with higher-fidelity data, they can see the exact conditions users have in the field, including: Network calls across device types, app versions, Wi-Fi, and cellular. Insight into 4xx and 5xx error trends across common domains to identify problematic pathways. The broken endpoints that prevent your users from starting up the app, loading key content, or completing crucial transactions. The duration of every network call from the client side reveals hidden points of latency. Armed with this information, the Farm Dog team simulated the issues their users faced, easily identified the problematic conditions, and fixed them. Set your mobile team up for success If you care about mobile experiences, then you need the data that enables your team to see all of the metrics mentioned above. Embrace helps you build better mobile experiences by giving you just that data, making your engineers more efficient and less bogged down by tedious work. Learn more about and download report to learn about key app frustrations according to users. Embrace The State of Mobile Experience Author: Colin Contreary Also published . here
