End-user experience monitoring is key to SaaS provider success

As customers expect the SaaS model to deliver equal performance to local apps, end-user experience monitoring becomes a core capability for providers.

Editor's note: This is the first part of a two-part series on application performance strategies for Software as a Service providers.

Software as a Service (SaaS) providers face one critical objective: They must deliver an optimal end-user experience or risk languishing.

"SaaS providers face serious performance challenges," warns SaaS industry expert Jeffrey M. Kaplan, managing director of Wellesley, Mass.-based consultancy THINKstrategies and founder of the Cloud Computing Showplace, an online directory of cloud providers. "SaaS vendors must match the response times and availability of on-premises software applications offered by traditional ISVs in order to continue to satisfy end users and displace legacy applications."

But is it reasonable for customers to expect SaaS applications to offer equal performance of on-premises software? "Absolutely," says John Essex, marketing director at New Relic, a San Francisco-based application performance monitoring vendor that delivers its core product as SaaS. "If a SaaS vendor continuously monitors the end-user experience from pre-production on, the information gathered can be used to improve performance to meet the same performance standards as on-premises software."

Many more factors affect the performance of SaaS applications than their locally accessed cousins. These factors include the performance of the application itself, the performance of the data center infrastructure on which the application runs, the performance of all the stops along the network path between application and user and, finally, performance of the browser or client software on the user's computer or mobile device. Some users may access an application from a smartphone using a cellular data network, others from desktop computers with broadband connections and others from tablets using a public Wi-Fi hotspot.

Not only do these factors individually contribute to how an application performs, but interactions among them can also affect overall performance. For example, if an application is chatty and it is located far from users, lots of network activity back and forth over long distances will slow application response times. Ensuring performance in the face of all of these variables is a tall order for SaaS providers.

Take a pulse with end-user experience monitoring tools

Are customers really satisfied? Many SaaS vendors cannot answer that question.

SaaS vendors must match the response times and availability of on-premise software applications.
Jeffrey M. KaplanTHINKstrategies

There is a common misconception that if the data center infrastructure is working, then customers are happy. But there isn't always a direct correlation between functional infrastructure and the end-user experience. Yes, the data center infrastructure must be properly sized, must scale easily with demand and must be load-balanced to avoid bottlenecks. Hosting providers offer continuous CPU, memory and disk I/O performance monitoring to determine whether servers are delivering what's expected of them. Data center infrastructure is an important piece of the performance puzzle, but its status at any given moment does not portray the full picture.

All too often, SaaS vendors learn about customer dissatisfaction from complaints to the help desk, a cascade of snarky tweets or disparaging blog comments. Customers that experience poor performance are also likely to leave negative online reviews that can be difficult to live down. It takes a number of five-star reviews to make up for a single one-star review.

"You don't want your customers to be your alerting mechanism," Essex says. "We had one mobile app vendor tell us that they found out their users were unhappy by reading user reviews. They said it was like buying a new car, having a wreck and not knowing about it for two weeks. The key is not to deploy blind, or you're asking for trouble."

But it's not always easy for SaaS providers to know what customers are experiencing. One way to find out is to conduct continual end-user experience monitoring to determine if users are happy. That monitoring should begin during pre-production so a SaaS provider can baseline an application's performance and gather information to improve performance at every step. The monitoring can occur by simulating user behavior using synthetic transactions -- a capability available from vendors such as Compuware, Keynote Systems and CA, through its Nimsoft line -- or by passively monitoring the actual user experience, as is possible with products from vendors such as New Relic and AppDynamics.

Continue reading part two: Are customers happy? How SaaS providers can use Apdex to find out

About the author:
Rebecca Wetzel is a principal with NetForecast, a network technology consulting firm based in Charlottesville, Va. She provides data communications industry insight and helps vendors and service providers develop successful marketing strategies.

Dig Deeper on SaaS implementation