Web analytics standards are few and far between, which is why I’m glad to blogivate about the Web Analytics Association’s recently released standard definitions for 26 web analytics metrics.  I’m curious to see how the world will respond to these basic definitions.   Standard vocabulary and definitions educate new practitioners, enable consistency in discussions, and lead to shared understandings that foster and promote innovation.  IMHO, the web analytics industry can only benefit from standards.  I certainly think they help to:

• Clarify misunderstanding and prevent confusion.  As the Internet continues to “go mainstream” and more money is invested in the “online channel,” the capital markets will continue to scrutinize and demand consistency in measurement.  The WAA standards set a new baseline for discussing internet measurement. 
• Align other companies and bodies and people expressing standards and using non-standard vocabulary.  If the WAA definitions reach a tipping point through broad industry adoption, other standards-setting bodies and industry organization will adopt and follow suit.  However
• Create a shared vocabulary.  It is not uncommon to hear references to objects in web analytics that are archaic (pages served), industry-specific (page impressions), or conceptually obsolete for certain goals (the number of “hits” as an indicator of site success).  The “names of things” are different across competing technologies.  I hope this document furthers discussion and leads to a common, shared global web analytics vocabulary.

So what are these new standards, you ask?  Here is the standard vocabulary (thanks to my friend Avinash Kaushik whose digitization of the document I have cut and pasted here :

• Building Block Terms: Page, Page Views, Visits, Unique Visitors, New Visitor, Repeat Visitor, Repeat Visitor & Returning Visitor
• Visit Characterization: Entry Page, Landing Page, Exit Page, Visit Duration, Referrer, Internal Referrer, External Referrer, Search Referrer, Visit Referrer, Original Referrer, Click-through, Click-through Rate/Ratio, Page Views per Visit
• Content Characterization: Page Exit Ratio, Single-Page Visits, Single Page View Visits (Bounces), Bounce Rate
• Conversion Metrics: Event, Conversion

Brief definitions for all these web metrics are listed below.  Make sure you download and read the full document.  There’s a lot more to it than listed below:

• Page: A page is an analyst definable unit of content.
• Page Views: The number of times a page (an analyst-definable unit of content) was viewed.
• Visits/Sessions: A visit is an interaction, by an individual, with a website consisting of one or more requests for an analyst-definable unit of content (i.e. “page view”). If an individual has not taken another action (typically additional page views) on the site within a specified time period, the visit session will terminate.
• Unique Visitors: The number of inferred individual people (filtered for spiders and robots), within a designated reporting timeframe, with activity consisting of one or more visits to a site. Each individual is counted only once in the unique visitor measure for the reporting period.
• New Visitor: The number of Unique Visitors with activity including a first-ever Visit to a site during a reporting period.
• Repeat Visitor: The number of Unique Visitors with activity consisting of two or more Visits to a site during a reporting period.
• Return Visitor: The number of Unique Visitors with activity consisting of a Visit to a site during a reporting period and where the Unique Visitor also Visited the site prior to the reporting period.
• Entry Page: The first page of a visit.
• Landing Page: A page intended to identify the beginning of the user experience resulting from a defined marketing effort.
• Exit Page: The last page on a site accessed during a visit, signifying the end of a visit/session.
• Visit Duration: The length of time in a session. Calculation is typically the timestamp of the last activity in the session minus the timestamp of the first activity of the session.
• Referrer: The referrer is the page URL that originally generated the request for the current page view or object.
• Internal Referrer: The internal referrer is a page URL that is internal to the website or a web-property within the website as defined by the user.
• External Referrer: The external referrer is a page URL where the traffic is external or outside of the website or a web-property defined by the user.
• Search Referrer: The search referrer is an internal or external referrer for which the URL has been generated by a search function.
• Visit Referrer: The visit referrer is the first referrer in a session, whether internal, external or null. 
• Original Referrer: The original referrer is the first referrer in a visitor’s first session, whether internal, external or null.
• Click-through: Number of times a link was clicked by a visitor.
• Click-through Rate/Ratio: The number of click-throughs for a specific link divided by the number of times that link was viewed.
• Page Views per Visit: The number of page views in a reporting period divided by number of visits in the same reporting period.
• Page Exit Ratio: Number of exits from a page divided by total number of page views of that page.
• Single-Page Visits: Visits that consist of one page regardless of the number of times the page was viewed.
• Single Page View Visits (Bounces): Visits that consist of one page-view.
• Bounce Rate: Single page view visits divided by entry pages.
• Event: Any logged or recorded action that has a specific date and time assigned to it by either the browser or server.
• Conversion: A visitor completing a target action.

In order for broad-based adoption and continued relevancy of these standards, I encourage the Web Analytics Association to: 

• Create broad consensus and agreement.  I was surprised the Web Analytics Association didn’t release these standards for comment to the larger membership and the public before releasing these standard definitions.  While I support the standards, I fear the perception of “dropping” standards on practitioners and vendors without providing a period for public commentary may slow adoption as people grumble about the nuances of the language.  After all, not all vendor’s tools or reporting comply exactly to the subtleties in these standards.
• Necessitate adoption by vendors and practitioners.  The old American expression says you say “po-tay-toe” I say “po-tah-toe;” I say “to-may-toe” you say “to-mah-toe.”   For broad adoption and usage of these standards, vendors need to integrate this vocabulary into graphical interfaces, reporting, documentation, training programs, and marketing messaging.  Consultants and practitioners need to “talk the talk.”  The Web Analytics Association should think about creating a “standards certification” program to verify adherence by certain companies and consultants.
• Identify compliance by vendors.  Current vendor vocabulary doesn’t conform to the standards, and there is currently no persuasive argument for vendors to adopt the definitions and modify their offerings.  The WAA needs to let the public know which vendors comply and which don’t and to what degree!
• Go beyond definitions to focus on interoperability.  Systems integration requires more than just definitions.  I’m looking forward to when these standards are described in XML.

Excellent job, Web Analytics Association!  If you haven’t joined, you should! 

Is web analytics data normally distributed?  That question calls for another question: what web analytics variables are you measuring?  That matters.  Numeric random variables (let’s call them data) are classified into the following types:

• Discrete.  That means you count it.  The data arrives from a counting process.  In web analytics discrete random variables are counts of things like page views, visits, and unique visitors
• Continuous.  That means you measure it.  The data arrives from a measurement process.  In web analytics continuous random variables are time-based metrics.

We do both in web analytics, don’t we?  We count some things.  We measure some stuff.  And if we’re smart and have the autonomy and positional power to do so, we apply process to counting and measuring web analytics data. 

We often talk about “counting” and “measuring” like they are the same activities.  In general day-to-day online business, that’s no big deal for conceptual conversations.  But in statistics, “counting” is different than “measuring.” 

Both discrete and continuous variables may be represented by probability distributions to assess the liklihood of an outcome.  To identify probability for discrete variables, use a “binomial distribution.”  Binomial distributions take into account the probability that an outcome will occur, so you may see some skewing when plotting the data that may make it look a bit “long tail.” 

For continous random variables use the “normal distribution.” Realize your data won’t always look exactly like a bell curve.  If it looks really different and ”long tail” you may be looking at a discrete variable better suited for a binomial distribution.    

So is web analytics data “normally distributed?”  The answer is that it depends on the type of data.  Even then, the answer is “probably not.” In fact, most business data doesn’t follow a perfectly normal distribution.  Yet every day in halls of academia, very book smart people teach statistics and tell you to apply it to business data.  Are they wrong?  Insane?  Misguided?

No they aren’t (well maybe you have to be slightly insane to teach stats).  Academics realize that most distributions are not normal and do not have equal measures of central tendency (i.e. mode, median, mean).  Skewness abounds!  The normal distribution, however, can be used to approximate “real-world” distributions that have different measures of central tendency. 

A theory called the “central limit theorem” states that “if the sum of the variables has a finite variance, then it will be approximately normally distributed ( i.e., following a normal or Gaussian distribution).” In other words as the sample gets larger the distribution of the mean can be approximated by the normal distribution.  And if I remember correctly statisticians have determined that with a sample size of at least thirty, the sample distribution of the mean will be approximately normal.   Fortunately, we web analysts often have millions of data points to use…

Some time ago I actually took average visit duration for one site for which I have real data for thousands of visits and did a Lilliefors Test of Normality.  The test found no evidence that the data wasn’t normally distributed even though it looked a bit odd and the skewness was 0.741426 and the kurtosis was 4.1525665. 

If you’re thinking about applying statistics to web data, make sure you identify whether the data you are looking at is discrete or continuous.  Don’t abandon the normal distribution for certain types of web analytics data just because it doesn’t exactly look like the Liberty Bell.  Test it for normality before applying the Gaussian statistics.  If the data is highly skewed, determine whether the level of error is in acceptable limits.  Look at using other distributions for discrete variables.  

Image from http://www.weibull.com/

Web analytics and keyword metrics came up in a conversation I had last evening with a friend of mine from my days in “information retrieval“ - when Googol was a really, really large number, and we called keywords ”queries…”  Over a Belgian beer (a Cantillion), I was asked to “name the top couple of metrics I’d want to know about a set of existing keywords if I were selecting a few to continue to optimize or buy?”  

I told him that any keyword-related metric should be analyzed within the context of campaign objectives, which in order to be measured and reported need to be defined before the campaign begins.   Macro level campaign goals should be identified before performing micro-level keyword analysis.  Once campaign goals are known, analysis can focus on achieving the optimal keyword mix to fulfill them.  A single, keyword-related metric should rarely be taken as a stand-alone indicator of performance. 

Here’s a synopsis of what metrics I told him I think are useful to examine when performing keyword analysis:

• Referrers.  At a basic level, identifying the sites that sent keyword traffic is common sense (like not excluding the Googlebot ;).  You may uncover keywords for which your site’s content “accidently ranks” on a particular engine.  These rankings may not be immediately obvious from a straight list of top-performing keywords.  By digging deeper into keyword referrers, you may find sites like these: forex-cash-fast.info, gambling1×2.com, nhadep.net, nghenhac.com, and xn--q2yr34f.com.  Clickfraud?  Poor targeting by an engine?  Lost money?   So many questions can be asked from keyword referrers!
• Geography. Show me my keywords segmented by dimensions like Continent, Country, City, Zip Code to assist in planning geo-targeted campaigns and identifying the broad content themes that appeal to the geographic long tail.
• Number of Visits and Percentage of Total Site Visits.  Raw visit and percentage totals indicate the “reach” of the keyword- the degree to which a keyword has penetrated a target audience.  I could compare the number of visits to the number of searches for that keyword using Overture’s Keyword Selector Tool to assess reach and correlate whether the cost to buy or the effort to optimize the keyword is acheiving the desired effect.
• Average Visit Duration.  It’s not an engagement metric, but average visit duration does tell you whether or not the visitor remained on your site and if so for how long.  It can be useful when taken into context with the page-view to visit ratio and segmented by other dimensions, such as conversion rate.  
• Page View to Visit Ratio.  One of my favorite metrics on a per keyword basis is the view:visit ratio.  This ratio identifies the average number of pages viewed per visit for that keyword.  If your keyword should convert the visitor from the landing page, and you are seeing a page view to visit ratio greater than one, what’s up?  If your trying to persuade visitors to enter some sort of non-linear or linear, multistep funnel leading to a conversion, and your page-view to visit ratio is one, what’s up?
• Bounce rate.  A key metric that identifies what percentage of visitors enter the site on the keyword’s landing page and immediately leave.   If your bounce rate for a keyword is over 35% and you are targeting that keyword, you should think about landing page optimization.
• Conversion rate. Conversion rate is the percentage of visitors referred by the keyword who succeeded in completing a pre-identified, value generating event on the site, such as a purchase or registration.  Conversion rates measure how well the keyword acted as a trigger for driving on-site revenue.  By segmenting your keywords based on conversion rate or other dimensions, you may notice broad content themes that drive on-site success events.  These themes could be used in persuasive messaging that includes hyperlinked points of resolution moving visitors into the non-linear conversion funnel.

Then I told him to “segment, segment, segment.”

Many metrics and dimensions can be applied to the analysis of keywords beyond the few I listed above.  What metrics do you look at on a per keyword basis when planning search engine optimization efforts or when planning paid search campaigning?

In Part 1, I began a long list of reasons why your web analytics data quality may stink.  I’m continuing the list below (make sure you read Part 1 for context and to view the entire list): 

• Storing only visit level data.  May tools don’t have schemas that store raw data at the visitor level.  Instead they provide access to only visit level data.  For example, you may not be able to see all the page views during a single visit per ip address or cookied visitor.   Assess the impact of the vendor’s schema on your goals.  Companies that use analytics data to feed other systems or that want to use visitor attributes for content targeting, segmentation, optimization, or analysis may not be well-served by some vendor schemas.
• Little to no decodes or lookups.  If you use numeric codes and non-human readable naming conventions in your data, they can pass through to your reporting and prevent your colleagues from understanding the reporting.  Strange codes look like hieroglyphics!  Decoding and looking up data can eliminate the problem of non-readability and strange numerical names in your reporting… While some would say this is a reporting issue, not a data issue,  I chose to include it because it’s at the surface… it’s the data your customers see.  Not all tools decode or lookup.  Some tools allow rewriting of data in the database.
• Failure of key services supporting the application.  If you are dependent on page tags, synchronization software, web servers, databases, or any of the wondrous technology that makes it all work, failures are a real bummer.  Make sure you have monitoring and recovery processes in place so you don’t miss data!  When page tag collection fails (perhaps the page tag server went down ay?), the data is gone forever.  If the web server fails, then no logs are written, but no pages are served either - so is traffic missed?  But if the processes supporting log file analysis fail (i.e. data synch), watch out! 
• Inadequate or incorrect implementation.  If you can’t cross dimensions (like finding out what keywords referred traffic to a page), filter all of your data (for example, filtering pages to see only those viewed by the iPhone), easily create new metrics, or if the numbers aren’t adding up, you may have not adequately or correctly implemented your software or communicated your requirements to your vendor’s professional services team. 
• Limited, hard-to-extend data model. Powerful, actionable insights from web analytics are enabled by extending a data model to incorporate business specific dimensions.  For example, if every page has a category and an author, you may want to see a list of all the page views in that category or ranking of pages by most popular author.  To do that you may need to join data at the database level or take advantage of variables you pass in a page tag.  Various tools have different limits on if, how, and to what extent you can extend the data model.

So what do you do when you know your data quality is less than stellar?  Here’s some guidance:

• Don’t worry, be happy. Just by collecting the data you are collecting, you are doing better than a great majority of companies that do business on the Internet.  By asking questions about data and investigating the issues, you have a leg up on your competition.  Work on optimizing the data, expose flaws in site design or architecture that impede data collection, work with your vendor and seek help in the web analytics community if you run into real problems.  The Web Analytics Association’s Forum on Yahoo is a useful place for posting questions.  But whatever you do, stay positive and focused on solving your problems and making your web analytics practice more optimized.  Don’t get frustrated.
• Recognize the limitations in the data and do not go gently into the night.  Ask the hard questions about sampling, schemas, data retention, processing, querying and reporting to understand where the holes and noise could be in your data.  Demand answers from your vendors and quick response times to your questions about data quality.  If you vendor is frustrating you by not being responsive, talk to the boss and the vendor’s bosses, escalate, escalate, escalate until you get resolution.
• Understand the underlying elements of data collection and what can go wrong.  Learn about sessionization and why different tools and data collection methods have limitations.  Explore the more technical components of the backend, like the database and your web analytics schema - all your data is in one (or more)! Talk to your engineers.  Have them explain the technology in terms you understand.
• Evaluate your tools.  Some tools are just better suited for particular business problems than other tools.  Log files tools enable you to constantly change assumptions and reprocess data.  Page tags provide a standard data collection and transport mechanism.

With hard work on your part, you can make you web analytics data smell like roses!  I know you can!

Web analytic’s data quality and accuracy of ”the numbers” are always questioned.  With so many sources of data from different systems and vendors - both free and paid- you must be able to reconcile deviations in data from different sources, and speak intelligently about data quality and accuracy to promote adoption of web analytics at your organization. 

There are so so so many reasons why web analytics data quality can stink.   I thought it would be fun to list some of the major reasons (over two posts):

• Spiders and Bots.  If you haven’t read my series on spiders and bots, check it out.  Non-human traffic can inflate your metrics and diminish the predictive power of your analysis.  Regularly look for bots and update your filtering!
• Untagged pages.  If the bulk of your page views are being generated by a single page, and you’ve failed to tag that page, you’ve lost data.  That’s always a bummer.  Tagging must be endemic to the web development process. Ask your team how they know that every page is tagged.  Really how do they know?  Verify and reverify.  Use a tool like WASP.   Obviously this isn’t a problem with log files.  
• JS turned off.  If the browser doesn’t execute javascript, the page tag won’t fire and traffic will be missed.  If you’ve missed it, you don’t know it occurred, so you are constantly in the dark unless you compare your page tagged data to log file data, which isn’t easy at all!  How you account for missing this traffic, whether it is immaterial or not, is a business decision. Again, not a problem with log files.
• Latency.  If the page tag doesn’t fire because it failed to load, the traffic is missed.  Vendors provide recommendations about the best place for a page tag.  Your development team may not believe them, or the “global include” that may or may not exist may or may not insert the code in the suggested spot.  It’s a good idea to listen to vendors when they provide configuration recommendations.  Once again, how you account for this data discrepancy is a business decision. Not a problem with log files.
• Differences in sessionization.  Data divergence gets hairy when you are running two tools on the same site, or are replacing one tool with another.  That’s seems to be very common these days with Google Analytics.  Eric Enge over at Stone Temple Consulting in cooperation with my pal Jim Sterne has some data from the 2007 Web Analytics Shootout that you should read to help you understand how different vendor’s sessionize.
• Sampling, sampling, sampling.  My friend Avinash Kaushikdoes a good job covering issues around sampling in web analytics.  I recommend reading his post!  I’ll add that statistical methods applied to web analytics data are completely valid; however, sampling at the site, page, or database level opens the possibility that you miss key data.  Sites looking for the “long tail” of visitors and using data sampling or other data trimming methods may not find it. 
• No referrer passed.  Referrer analysis tells you what site people came from before they visited your site.  Bookmarks, typed pages, email campaigns, and bots don’t pass referrers.  Sometimes referrer information just isn’t passed by the browser.  If you overuse redirects on your sites, you may lose referrer data.  Not having this important information impacts SEO/SEM, linking campaigns, and affiliate partnering.  
• Cookie configuration and deletion.  We all know about cookie deletion thanks to Eric’s research while at Jupiter.  In addition, if your server isn’t set-up right, you may not be setting the cookie on the first request. Talk to the web server guys and gals at your company to make sure your cookie handling is optimized for web analytics.
• Proxies.  Proxies can filter out your referrers and make it look like everyone’s coming from the same ip address.  Most tools enable cookie-based visitor identification to work around the proxy issue, but if cookies aren’t set up right on your web server or configured correctly in your web analytics tool, assessing uniqueness when the bulk of traffic comes from the same IP will be problematic.     
• Time spent metrics have severe limitations.  Single-page visits and the time spent on the last page in the visit aren’t measured in “total time online.”  Time-based metrics are schoolboy metrics, so don’t overemphasize them as stand-alone indicators of “engagement.”  Assess the impact of time spent in the context of goals.  Use time as a variable for segmentation and as input into a larger engagement metric. Use them in context, especially if you are selling advertising (and who isn’t)?  My friend Jim Novo has one of the best takes on time spent metrics I’ve read.
• Failure to maintain exclusions and filters.  Data can be polluted if you are introducing new filetypes  and not excluding them from your top content reporting or if you are not maintaining your bot filters.  New bots are crawling your site right now.  What are those web developers doing now?   Do you know?  Are you checking?  What’s the process for doing so?
• Little to no historic data.  If you’ve just tagged your pages, you may have no basis for historical comparisons for quite some time (like a year!).  Data has realized value from understanding current behavior, and potential value from predicting future behavior.  And you need data to do it.    
• You’re not storing all your dynamic url’s.  Your URL’s may have many parameters in the query string (i.e. the name/value pairs after the “?”).  Some tools cut out this data and don’t make it available for querying or reporting in their applications.  If your site is database-driven and dynamic, not having access to the every URL request will limit your ability to do ad-hoc analysis, filtering, and segmentation.

Let’s continue this long list in Part 2!

Let’s continue this long list in Part 2!

Why do we do web analytics?  What value does it generate?  What is the ROI from web analytics?”  Companies trying to justify or in the process of allocating capital to “do web analytics” are wondering…  Recent research has shown that companies serious about web analytics need to invest money in people and technology.  But what do companies get in return for doing so? 

I believe that web analytics helps online business identify potential opportunities for taking action to:

• Increase revenue.  Web analytics helps you make more money.
• Decrease cost.  Web analytics helps you spend less money to make more money.
• Improve operations.  Web analytics helps you work smarter and more efficiently.

Let’s explore some of the ways web analytics can help online business across these three core actions:

Increased revenue through:                                                                                    

• More targeted advertising sales.  Content is monetized in a many ways, from cpm, cpa, cpl, ppc, and more.  Web analytics can tell you which of these methods for generating revenue and which advertising campaigns using those channels are performing most effectively.  External campaign effectiveness may be tracked using referrer data and related dimensional reporting.  Metrics related to internal campaigning, like microsites or special advertorial offerings, can be easily provided to advertisers and agencies to identify audience consistency and quality.
• Better insights into audience segments to realize incremental revenue.  Segmentation refers to dividing a total population into groups based on one or more characteristics.  A good web analytics tool easily enables you to segment on dimensions and attributes relevant to your business.  Segmenting web data enables you to answer questions about which visitors visit when and with what frequency, depth, and duration, and more, which provides otherwise unknowable insights.  New incremental revenue streams may be realized by mapping newly discovered behavioral or demographic characteristics to existing advertiser or agency demand.
• Creating effective online-marketing and editorial offerings.  Reports showing visit frequency, depth, recency, and the time periods when the online audience visits the site assist product managers, editors, and producers in optimizing, crafting, and targeting content and advertising, increasing reach and exposure time of advertiser messaging to key audience segments.
• Ensuring pages effectively lead to conversion funnels. Metrics like bounce rate, conversion rate, clickstream pathing, and conversion metrics provide indications about how to modify or tailor pages to generate value.  Funnels can provide insights about which calls to action, content, pages, sections, and campaigns yield the best conversions.

Reduced costs by:

• Increasing the effectiveness of online work products. By identifying, monitoring, and evaluating important KPI’s (key performance indicators and KKPI’s!), the business learns what works and what doesn’t work online.  Web performance data has amazing utility when evaluating, planning, and monitoring current and future trends when assessing how to reduce cost in an portfolio of online products.
• Maximizing site operations, content, and opportunities for organic and paid search.  The performance of pay-per-click, paid inclusion, and contextual advertising and linking campaigns may be audited to eliminate projects that fail to meet goals based on conversion, revenue, or KPI’s.  By tying conversion to capital budgeting, online projects that fail to meet site hurdle rates may be tailored or eliminated.  The business can then better focus on the driving profitable revenue without misallocating resources.
• Optimizing user experience and information architecture.  Overhead reducing tools like Google Site Optimizer and offerings from other companies providing site optimization services use web analytics data to programmatically alter a site to increase conversion and lift.  CMS automation can be driven off of web analytics data.
• Pinpointing the performance of online marketing campaigns.  By creating custom KPI’s, metrics, and segmented conversion rate and slicing data via custom filters and business relevant dimensions, deep insights into online performance can be attained.  Misappropriated resources and efforts can be easily recognized and eliminated.

Improved operations via:

• Deep understanding of site traffic, visitor activity, conversions, and online value-generation.  You can’t manage it, if you don’t know about. Companies most successful with web analytics dedicate a full-time staff to analyzing and contextualizing data and performance metrics from channels like organic and paid search, affiliate partnerships, and offline.  The best staff understands the impact of the web channel across the value chain.
• Contextualizing strategic decisions with accurate data.  The ability for a corporation to gain insight and intelligence into its online activities provides management with transparency into performance.  Performance must be monitored to be improved, and there’s no other way to gain true insight into online performance to than using web analytics to guide web strategy.
• Identifying site operational effectiveness in a timely manner.  When using log files, server errors and other impediments to online customer satisfaction can be quickly discovered and remediated, which reduces negative impact and minimizes risk. 
• Predicting the impact of business decisions on performance.  By applying statistical methods to web analytics data, businesses increase their abililty to predict the impact of site changes on performance. 

Every Internet business can benefit from technology that positively impacts these three important business actions.  I recommend that you consider how your projects are framed across three actions whether you’re just thinking about getting involved with web analytics, if you’re growing your web analytics practice, or if you’ve already established web analytics at your company.