A successful web analytics practice helps a business manage its performance toward goals.  If you believe the statement, then you understand that in order to manage for site performance, a web site must exist to support one or more goals.  Pretty logical, right?

Then why is that web analytics practitioners tell me they often encounter web site owners who have no clear, measurable goals?  It’s really strange.  In fact, it’s vexing and frustrating because without known goals you OBVIOUSLY can’t manage site performance.  Without goals you can’t really optimize anything and are left to simply tracking trends related to basic metrics and/or derivatives thereof.

Thus, one of the responsibilities of a web analyst is identifying and confirming site goals.  Once you have site goals, you can create KPI’s that you monitor on a regular basis investigating variances, anomalies, and outliers affecting site performance. 

Undoubtedly one of the goals you will identify for your site is some type of conversion.  A conversion is a value-generating transition on your site.  For example, the successful completion of a form that enables a visitor to download a content asset or completing the purchase of a product may be a type of conversion you measure. 

While overall site conversion rates for value-generating events are important to know, real insights come from segmenting visitors or other dimensions by conversion rates.  The best analytic tools enable you to identify the conversion event and slice the conversion by any dimension: page, geography, referrer, keywords and so on. Thus measures of conversion will be instrumental KPI’s when using analytics to manage performance. 

The notion of a “funnel” in web analytics is a sequence of one or more pages that a visitor clicks-through until reaching a final destination, the conversion event.  The “funnel” assists the analyst in understanding the discrete clickstream that led to conversion.  For example,  a pre-defined path through the pages in the subscription or download process could be considered a funnel.  Funnels can be linear and non-linear and are affected by all sorts of things like detours, fall out, abandonment… That’s a post for another topic…  Yet, the metaphor of the funnels is applicable across all sites…  This notion becomes problematic when we consider multichannel attribution of the conversion process (again another blog post when I have time).

So what’s an analyst do when they want to begin to use web analytics to manage performance against goals? Here’s are some tips:

1. Investigate the business’ revenue model. Advertising-based sites generate revenue from selling various types of ad units (rpm/cpm), contextual advertising (rpm/cpc), lead generation programs (rpl/cpl), revenue sharing, and via affiliate syndication and content sharing deals.  Ecommerce sites generate revenue by selling products or brokering services or transactions.  How does your site generate revenue?  If the goal of the site isn’t to generate revenue, then skip this step.
2. Ask key managers to identify business goals.  Top-level managers have a better grasp of the vast ecosystem of suppliers, buyers, and other priorities that you, the analyst, may not be privy to.  Your manager should be helping you put your analytics work in context of the business goals.  So ask your boss what are the site goals?  Don’t accept the answer “to drive revenue.”  Ask how the identified goals support value creation and revenue generation.  The measurement of events supporting business goals should be a focus for performance management and optimization. 
3. Identify the conversion events that support businss goals and the revenue model, including any necessary steps in the funnel.   The actions that satisfy goals are the conversion events. The transition involved when the visitor clicks and makes the site money is your discrete conversion event. The page immediately preceding the conversion event is the last step in the funnel. 
4. Determine the actors on the site.  Actors can be categorized into internal/external actors.  For this exercise, concentrate on identifying the roles and responsibilities of the internal actors who DIRECTLY influence site production.  In other words, who are people modifying the site and what do they do?  The indirect actors, like your boss, also affect the site, so make sure you consider their role and responsibility in advanced site goals and fulfilling the objectives of the business model. 
5. Determine the goals of the actors.  Like site goals based on revenue, all indirect and direct site actors will have goals specific to their jobs.  Actor goals support site goals.  Thus, actor goals can be translated to tactical KPI’s.  For example, the editorial actor may want to ensure that X number of newsletter-referred visitors subscribe to the print magazine, so you create a KPI’s “subscription conversion rate by newsletter” and “number of online subscriptions generated per newsletter.”  Based on the site goals for conversion and the number of subscriptions generated from the online channel, you can start managing editorial performance.
6. Document site goals, actor goals, conversion events, and funnels, including a diagram a hub-and-spoke model of actor roles and responsibilities and flow diagrams of funnels.  In order to establish a process for performance management via web analytics, all the actors must generally agree on roles and responsibilities.  By documenting your investigation, you confirm correctness, identify gaps in business process, and create alignment among actors and management.  You may notice breakpoints in site production processes too.  The end result is a fully-documented operational model of how your site is created, monetized, and deployed.  In the same way that you can’t manage what you don’t measure, you shouldn’t be measuring things you can’t manage.   
7. Have key managers who direct site actors sign-off approval on the documentation.  The holy sign-off confirms you correctly identified the revenue model, site and actor goals, site navigational flows that lead to conversion.  When questions arise you can reference this process artifact to backup the conversion events you defined, the KPI’s you’ve created, and subsequently the performance recommendations you will make and manage.

Then:

1. Configure your web analytics tool to report conversion rates for revenue-generating site transitions and events and to report on funnels.  All tools can do this. 
2. Build KPI’s based on specific functional goals performed by actor’s on the site. Base KPI’s around activities that support the core revenue model and the activities performed by site actors.
3. Review KPI’s with actors.  Bring your documentation and identify how the KPI’s will be used to identify performance, contextualize optimization recommendations, and help each actor be more successful at their job. 
4. Report conversion rates and KPI’s to key managers and site actors.  Optimally these performance metrics should be available for actors and other stakeholders whenever they want them, preferably in the form of dashboards elicitable from your web analytics tool.  The goal should be identified, the target value for the goal, the KPI measurement,  and any deviation from the goal should be noted along with written performance recommendations.
5. Research site performance by segmenting conversion rates and KPI’s and investigating drivers.  KPI’s provide context for understanding the actions that influence site performance.  Overall conversion rate will only tell you so much.  For example, to the SEO conversion by organic search referrer is more informative.  Other actors will require reports on segmented conversion specific to their function. 
6. Make optimization recommendations.  Whether you deliver recommendations via reporting or manage the multivariate testing function at your company, you’ll need identify the events or actions to optimize.  Then you need to get buy-in from various gatekeeping actors.
7. Test and implement optimizations.   Use a multivariate testing vendor to test combinations of content and creative that drive KPI’s and that provide lift in conversion.  Work with site actors to ensure optimization testing and controlled experimentation occurs.
8. Rollout optimization that increase conversion, improve goals, and drive revenue.  Once you are reasonably certain the optimizations you are suggesting will improve performance work with the web or product development team to realize these changes.

Thus web analytics for performance management involves:

• Goal Clarification.  Why does this site exists in the first place?  Don’t be surprised to learn different actors have different goals, and no one is aligned!  From what I hear on the street that’s a common issue!
• Stakeholder Alignment.  Do all stakeholder and actors agree on the reasons why the site exists?  If not, be prepared to mediate.
• Experience Optimization.  How is the visitor interacting with my site, and do those interactions channel visitors to conversion funnels?  Do relevant calls to action and points of resolution work for persuading visitors to convert.  What’s working?  What’s not?  Figure it out.
• Controlled experimentation.  Based on potential optimizations available, what do you test?  Multivariate testing software can help, as can VOC research.  Talk to your research team.  Use the AB testing feature of your web analytics tool…  Whatever you do, you should establish a repeatable process for doing so. 
• Outcomes measurement.  If you set up a KPI dashboard with goals, actual performance, and variances you will be able to answer that question “so I did all this stuff, so what effect did it have?”

Easy right? Now get to managing performance using web analytics!

Forrester’s Megan Burns is seeking ALL web analytics consultants to complete this 10-minute survey:

http://globaltestmarket.com/survey/s.phtml?sn=86482&lang=E&secid=0fd357

Megan will be publishing a list of every web analytics consulting firm later this year.

In addition, if you would like to meet Megan in person, listen to her speak, ask her questions, and receive a FREE copy of Forrester’s Web Analytics Vendor Evaluation (the $995 Wave), you should attend the Boston Web Analytics Wednesday I host every month.  Sign up to attend the event in Cambridge MA on 10/24:

http://www.webanalyticsdemystified.com/wednesday/?event_id=2102

Web analytics data is the raw material from which companies will realize new online products and deliver differentiated services that generate future value.

Right now as I type I can get web analytics data from so many sources.  Google Analytics and Open Web Analytics provide the data for free (once I spend the overhead to set it up).  So does Compete and Quantcast.  Many other companies are willing to broker this potential commodity to me at various price points - from the low four to seven figures. 

The price from web analytics firms for what is essentially the same data is all over the map!  Why? Perhaps because enterprise vendors know that you the customer will no longer pay for just data (thanks Google!). 

Features and services provided on top of the core data that’s valuable to a practitioner.  To the company employing the practitioner, it’s the insights generated from the data that’s valuable.

We’re seeing a lot of web analytics data operationalization via features for:

• Extension into business intelligence (and in the future leading to business analytics).  The best web analytics firms are providing open relational databases and creating methods for joining data from other systems to “extend the data model” or feed the enterprise data warehouse.  
• Automated testing and optimization.  The notion that “you aren’t doing web analytics, if you aren’t testing” provides evidence that siloed, lonely data won’t do much for your business.  In that light, automated testing is only as useful for prediction as the people setting up the tests.
• Targeting.  Using analytics data during the session or after it to automatically target content based on key visitor attributes will increase conversion.  While targeting technologies use analytics data, the value derived isn’t from the data, but from the potential conversion lift of the activity we call “targeting.”
• Proxy scoring.  Assigning a value to an event, interaction, page view, or visit can identify high-value segments and customers.  Scoring abstractions operate on the data to indicate value. 
• Profiling.  Building a picture of your online audience by aggregating data from various sources including web behavior, customer transactions, and demographic data enables one-to-one marketing.  Web data is part of the profile.  The value is in the profile.
• Integration.  Joining analytics data with data from other systems in a unified data model, or enabling machine-to-machine communication of analytics data will yield value.  Again the data is important, but the value is in the outcome.
• Alerting.  Indicating when data exceeds pre-defined upper and lower bounds and where those thresholds have been exceeded is valuable.  Once again, the data is crucial, but the alert is catalyst for value creation.

Data SEPARATED from the application, from the presentation, is extremely valuable.  When “unsiloed” and described or made available using open standards, it can be reused by other applications.  Insights realized from moving/sharing/synching data that drives enterprise value 

Yet functionality on top of the data layer doesn’t make web analytics easy and instantly drive value.  Few corporations have the slightest clue about how to take advantage of all this functionality. 

To creation value the modern web analytics practice requires:

• Dedicated in-house professionals. No duh, here.  You need people who understand the data, use ”features” to help analyze it, and who can then test hypotheses to optimize and measure outcomes.   
• Vendor and third-party professional services.  Consultants must go beyond “repeating back what you say, then claiming they can solve the problem” and deliver quantifiable, measurable value that improves business process.
• Web services. Web analytics tools need to use web services.  The “web analytics” tool of the future will take advantage of technologies that provide platform-independent protocols and standards used for exchanging data between applications. 

We’re going to continue to see highly-specialized web analytic’s “experts” at companies work with services firms (or professional services staff from vendors) to combine the ”off the shelf” web analytics products with web services technologies to create the automated marketing architectures of tomorrow. 

These “marketectures” will:

• Bridge quantative analytics data with qualitative data to identify the 360-degree view of the customer experience.  Ahh buzzwords…
• Use web services to operate on analytics data to solve highly-specific and specialized technology and business problems.  For example, using a WS to pass custom interaction events in Adobe AIR/FLEX to understand behavior in your RIA. 
• Enable HUMANS to realize new *predictive* insights from data.  Yeah, automated testing will help, but think about it… who sets up the tests? 

In wild world of online metrics, it’s a well known fact that metrics from web analytics tools and ad servers never match. Variances can be substantial. 

What I mean is that, given no “refresh rate,” the total impressions for a single ad unit, which should be served on every page request, never matches the number of total page views on the site during the same period of time.  Sigh.

Reasons why identically-named metrics from these two tools (like page views and unique visitors) don’t add up are numerous:

• Different data collection methods.  Ad servers use page tags.  Many web analytics tools use page tags, but it’s not uncommon in web analytics to use additional methods, such as logs or packet sniffers.  The methods have no shared standards for collection or storage of the same data (like visit-level data).  Thus you get apples to strawberries comparisons when attempting to correlate the dimensions from different systems.
• Unique data models.  Ad servers aren’t focused on counting page views and the other dimension of web analytics (visits, time, and so on).  Rather ad servers focus on serving and counting impressions served (and loads of related derivative calculations, like CTR, CPC, and the coolness of view–thru).   Metrics are based on an ad request and an ad code.  Ads aren’t targeted to a page (though that’s possible), but rather to a “zone” or “keyword.” What that means is that “page” dimension may not even exist in your ad server’s schema.  In other words, you aren’t looking at impressions measured on a page, but rather at the number of impressions served in a different conceptual construct.  That’s one of the reasons why people say metrics and ad-serving systems “don’t measure the same thing.” 
• Untagged pages.  Just like analytics implementations suffer from challenges related to complete code coverage of page tags, so do ad serving implementations.  Companies need to determine how to centrally manage the deployment and orchestration of page tags *of all types* and verify all the pages have tags!  Don’t just expect it to work because tagging sounds so easy!  Suspect it won’t work, and determine what you’re going to do *before* you deploy.  Too late?  Time to reengineer. 
• Non-JS executing clients.  Ad servers use page tags.  Not everyone and not all user agents execute javascript.  Everyone needs to realize that page tagging misses traffic as efficiently as it excludes it.  Period.  What percentage of the traffic you miss, you’ll never know… running and filtering your logs may provide an indication…
• Ad blocking software.  Firefox’s Adblock Plus software is a big problem for sites that have a big techie audience, and it affects all sites.  Check your browser reporting and realize a good majority of those Mozilla users may be blocking your ads.  Look at the attitudinal data you have about visitor’s to gauge whether that’s a big issue for your online audience. 
• Cookie issues.  Third-party cookies get blocked (often by privacy software).  Many ad servers still serve third party cookies, and many corporations have not tricked their DNS to accommodate this issue (ahem, CNAME).  We all know how cookie deletion affects unique visitor counts.
• Refresh rates. One page rendered in the browser and many banner “refreshes” makes it really hard to correlate page views and impressions served.
• No rich media installed, and no fallback.  If the client doesn’t have certain plug-ins, and you have no fallback, you miss ad revenue.  Meanwhile the tag executes and you count the traffic.
• Robots, spiders, and crawlers, oh my.  The web is so robotic.  The problem is amazingly understated, especially by companies who want to bill you on page views.  Different data collection methods allow some level of bots to dirty the data.  Logs are harder to efficiently filter.  When the ad server uses tags, and the analytics tool uses logs, you may get some wildly different numbers. 
• Mobile, Mobile, Mobile, Mobile.  Not all Internet-connected mobile devices will display ads, but web analytics tools will track the behavior of mobile visitors.
• Latency.  Visitors who move through the site too quickly may not execute the tag, thus no data is sent back to the server(s).  Ever wonder why vendors tell you to put the tag “high” on the page?

The influence these issues have on your site varies depending on audience.  Investigate factors causing variance and deviation between metrics systems, and educate your audience on why the numbers differ.

The concept of web analytics data sources isn’t discussed nearly as often as web analytics “data collection.”  With that in mind, I’m often asked by people just beginning to explore the wild and wonderful world of web analytics  “where does this metric come from?” 

When people ask me that question I often think back to someone who I may have seen walking in the local village a few miles away if I had lived about 170 years ago.  Ralph Waldo Emerson.  The father of American Transcendentalism once asked “whence is the flower?”  From where did it come from?  Ralphie-boy was questioning the accuracy of long-standing 19th century beliefs.  Keep in mind Mr Emerson parted ways with Harvard for questioning the Trinity (not Avinash’s ;)!

In that spirit, people are really asking me “whence is the online metric?”—from where does the metric come from?  Advertisers and your colleagues want to know because they may be questioning the origin and accuracy of our 21st century metrics! 

While folks who have been “doing web analytics” for some time know how to answer that question, I thought it would useful to blogivate on data origination for folks who are new or aren’t dealing with web analytics in full-time, day-to-day role. 

The answer is that online metrics originate from one of three data sources:  internal data sources, external data sources, and hybrid data sources.  Each source has its own particular challenges to “accuracy.”

• Internal data sources.  Web analytics technology that collects data from websites, mobile phones, or other Internet-connected devices via javascript page tagging, log file processing, or packet sniffing falls into this category.  Major vendors include Unica, Visual Sciences, Omniture, CoreMetrics, Webtrends, Google Analytics, and others.  The issue with Web analytics tools is that two tools will yield different numbers for the same data source.  That’s because each tool has its own “secret sauce” for “sessionization.”  That is, each tool counts traffic in slightly unique ways.  For example, tools may be configured to include or exclude certain filetypes or server responses.  Robotic traffic may or may not be filtered.  Web Analytics tools also depend on cookies for attributing “uniqueness” to visitors; thus, cookie deletion can overinflate unique audience numbers. 
• External data sources.  Data collected from panels, toolbars, and ISP’s–not from the actual site– are examples of external data.  Companies like Comscore, Neilsen Netratings, Compete, and Hitwise provide metrics generated from external data.  These companies provide some sort of incentive (i.e. gifts) or perceived value to entice people to participate in panels and download software that observes their Internet usage.  Self-selecting panelists are monitored, and metrics related to their behavior are projected to the entire online universe using statistical methods.  External metrics are never identical with each other because of differences in consistencies of their panels.  As you’ve probably noticed, Comscore data does not match Nielsen.  Significant divergences across panel-based measurement systems when compared to each other and web analytics tools has led to an audit by the IAB and MRC.  The hope is that auditing will vet methods and identify any potential coverage error or selection bias inherent in sources of external data.  Personally, I’m really curious to see what all the hubba bubba about auditing without guiding standards will really accomplish.
• Hybrid data sources.  Sources of metrics that use some type of internal data collection and some form of external data collection are considered hybrid data sources.  Microsoft Gatineau and Quantcast offer free services that fall into this category.  Reports in the highly anticipated web analytics offering from Microsoft include data collected from Javascript page tags combined with anonymous demographic data from Microsoft Live profiles.  Quantcast’s panel-based measurement system (i.e. external) may be augmented by adding a javascript page tag (i.e. internal) to every web site page.  Behavioral data is collected via javascript and combined with demographic data from the Quantcast panel, then reported to end-users.

In real world practice, companies use many of these different, overlapping data sources to understand their online presence.  Given the free nature of companies like Quantcast and Compete, and the pervasiveness of firms like Neilsen and ComScore in sales and agency cultures, expect that different sources regardless of type will never ever absolutely match site-specific Web analytics tools.  And that’s okay because data from all these data sources have different utility:

• Metrics from internal data sources derived from web analytics tools are very useful to site owners for identifying the behavior of their online audience.  The data can be used for site optimization, to understand what sites are referring traffic to the site, to identify conversion rates for particular marketing campaigns, to understand the broad content themes and particular search keywords driving traffic on your site, to segment and give context for other metrics and attitudinal data and more.  Metrics from site-centric sources should be provided to advertisers for comparison with external data.  Be prepared to discuss why the numbers differ!
• Data generated from external sources are useful to advertisers and agencies.  These metrics can be used for comparing the performance of a site to its competitors and for understanding audience behavior across one or more sites by demographics.  Media buyers and web strategists desiring to understand generalized Internet traffic trends and measures of site popularity use external sources.  Metrics from “audience measurement” sources should be used for comparison with internal data.  But the data won’t match, and that’s okay because you should be looking at the site performance of your competitors, not using that data to optimize your site.  Use the external data for insights into demographic makeup of your audience.  Then compare that information to data from your own internal research teams (who don’t report to web analytics). 
• Numbers from hybrid data sources blend both external and internal metrics together for both site owners and advertisers.  No duh, ay?  New insights about the online audience can be realized from segmenting visitors based on demographic and behavioral data within the same source.  We’re just entering the “early adopter” phase of this market, so I’m curious to see how it all plays out.  How will Microsoft Gatineau differentiate its hybrid analytics service and communicate the value proposition?  Will publishers adopt Quantcast’s hybrid service (and how will they make money)?  One barrier to adoption is that some companies have already combined web analytics behavioral data with audience demographic data using business intelligence and data warehousing technologies (or the more flexible and open web analytics tools that suppor open software standards).   Companies bridging data internally hope to enable a grand orchestration of automatic site optimization and content targeting (with an eye toward behavioral targeting).  Microsoft already seems to be doing this targeting to some degree on based on what I learned about “personamous” content targeting on MSN.com (at Emetrics).  So I’m curious what other online products (for example advertising offerings and site optimization technologies) the gentlemen at MSFT have up their strategic sleeves based on this hybrid source data. 

As you immerse yourself in the wondrous world of overlapping, hardly-standardized, online metrics, it’s critical to consider the source of the metric, bias from the source, and the audience for the metric.   Most critically, understand how a metric, regardless of source, relates to business goals and site objectives before using it as measure for identifying your online performance to internal and external stakeholders and taking action from your insights. 

GO RED SOX!