Web analytics tools can be outgrown by companies, just like pants can be outgrown by people.  Over time, an analytics tool may no longer fit organizational needs or be well suited to deliver on complex organizational requirements for site optimization and multichannel integration (among other things). 

This topic led me Silicon Valley this week thanks to an invitation from Unica I headed over to Webex headquarters to record a preso in their new LiveStream studios.  A few other folks also participated in the production.  In attendance was Fireclick founder and all around cool guy, Steve O’Brien (also VP Internet Marketing at Unica), my pal and fellow blogger, Avinash Kaushik (of ZQ Insights and MarketMotive), the genial Elana Anderson (founder of NxtERA marketing and former VP Marketing Research at Forrester), the excellent bloke and all around nice guy from across the pond, Dr Alan Hall (Director of Analytics at SCL Analytics), forthcoming author and savvy multichannel marketer, Akin Arikan (Analytics Evangelist at Unica), and the jet-setting smarty Karen Hudgins (marketer at Unica).   We all had a blast getting down to business at Webex studios.  And eating at places like Burk’s and Parcel 104 - both excellent restaurants.

The title of my preso was “Symptoms you’ve Outgrown your Web Analytics Tool,” such as:

• Inadequate segmentation
• Poor visualization
• No custom reporting
• Limited Integration (covered in Part 2)
• Cost (covered in Part 2)

While I did manuscript the speech (hey I was being recorded!), it’s way too long to post, so I figured you all might enjoy me paraphrasing my own content.  So as I sit here on a Jetblue redeye, here it goes:

• Inadequate Segmentation.  Segmentation in web analytics describes the activity of categorizing and dividing your online audience and customers by their various attributes.  For example, you might choose to segment your audience based on their demographic location information to determine if a visitors from a certain geography have a higher conversion rate or behave differently on your site than visitors from another geography.  Or you may choose to use your web analytics tool to define a segment that you want to track such as visitors who clicked on a paid search term and did not convert, but came back to the site within one week.  Sounds easy, right? 

But not all web analytics tools can segment data.  The proprietary tool you run in-house may not be able to segment data.  Your expensive vendor solution may not be able to segment data easily.  Many tools only provide simple reports.  Yet basic reporting is insufficient for web analytics.  In order to understand new data relationships and the effectiveness of marketing campaigns to your massive online audience, you need to a web analytics tool that can segment data.

The idea being to do what I describe in this post on web analytics segmentation:

• Define a segment
• Identify expected segment behavior. 
• Measure current segment behavior. 
• Create “optimization hypotheses.” 
• Optimize content, offerings, user experience, and other site elements. 

How does your web analytics tool fit into the process of segementation that I described?  Does it?  Can your tool assist you in this process?  If not, you may have a nice IT tool that reports web metrics, not a marketing tool that enables you to optimize your site and landing pages to offer the best possible messages to known online segments.

• Poor Visualization.  Pictures are worth a thousand words.  Your stakeholders are already overwhelmed with data before ever presenting reports with a whole bunch of numbers.  Not everyone is quantitative.  Some stakeholders just want to be able to quickly digest data, and they prefer an aesthetically pleasing visualization instead of a spreadsheet. 

Data visualization helps stakeholders interpret important data at a glance.  Visualization helps reporting comprehension.  Good visualizations are important when you want to:

• Highlight key trends in the data
• Compare counts of things
• Identify multidimensional relationships using cube visualizations

If your tool can’t visualize your web analytics data, and you need that visualized data to assist comprehension, act as sales tool in a presentation, or as marketing collateral in a report, you have outgrown your web analytics tool.

• No Custom Reporting.  An acute inability to deliver customized reporting that meets the needs a diverse group of stakeholders is one of the signs you’ve outgrown your current web analytics tool.  The problem manifests itself in sheer frustration because people can’t get the data they need. Over time this will cause people to lose faith in web analytics because the data isn’t relevant to their jobs.  Some of the symptoms include:
  • Problems creating KPI’s.  To manage online performance, you need to be able to define Key Performance Indicators.  For example, you may want to define a view:visit ratio, that is the number of page views generated per visit.  You need to define this equation in your web analytics tool.  If you can’t define such simple KPI’s, you are limiting your success at web analytics.
  • You may not have reporting that identifies conversion rates and allows you to define custom metrics for channels like RSS, Newsletters, and Internal and External Search.  A powerful web analytics tool will be able to build custom reporting for conversion rates and other KPI’s by online channel.
  • You may have a limited ability to build reports with filtered data, such as viewing reporting of top pages on a particular day or combinations of days, or filtering data by referrer, geography, or time.
  • No ability to add core web analytics dimensions to your reports, such as creating and saving a report that shows all referrers, their keywords, conversion and bounce rates for each city in the United States.
  • Quite simply, you may only have one set of reports and you can’t build new ones at all.

In the real world practice of web analytics, you need a web analytics tool that has the ability to build as many custom reports as you want when doing analysis, to filter, add metrics, to use dimensions, do AB tests, and save all that stuff until your heart is content so you can meet business goals.  You can’t be restrained by the inability of your current tool to create custom reports. If you are you may have outgrown your web analytics tool.

I’ll post part 2 this weekend as I recover from the jet lag…     Part 2 is up, click here to view it! 

Targeting refers to the process of identifying characteristics of a segment so that relevant content may be matched to it and delivered at a time when the segment is most open to the message. The idea is the right content to the right visitor at the right time (optimally in real time). 

For example, you may visit a site, and see some type of ad unit calling out at you to “meet singles in <insert_your_city>.” When browsing real estate you may see ad units for realtors and mortgage companies.  After entering a keyword such as “car prices” and clickingthrough the SERP, you may see an ad for a local car dealer.   That’s targeting in a nutshell.  It’s simple: 

1. Visitor X has these attributes. 
2. We have content that we think will appeal to Vistor X’s attributes. 
3. Let’s show that content. 

While targeting has helped to increase ad clickthrough rates, it’s far from an ideal science.  Current methods for targeting have inefficiencies.  What if Visitor X just bought a new car after his recent marriage?  Unless the targeting engine is made aware of the visitor’s current state, the targeting may be off and not yield desired results. 

Even with limitations around “current awareness” targeting is perceived in the Internet industry as a crucial activity for maximizing the effectiveness of advertising and content.  Targeting is the next stage after A/B and multivariate testing.  Once you determine the preference of segments based on testing, you identify content to target. 

In new media, targeting is something associated with paid search campaigning, ad serving, and content optimization.  It’s not uncommon for targeting activities to be based on:

• Category and sub-category.  Conceptual constructs like “categories” of topics on a media web site or products on an ecommerce site can be targeted to include certain types of ads or messages.  The notion of a “zone” fits in here as well.  The idea is that if visitors are browsing in your category for “hardware floors” you could offer them an ad or content specific to “flooring installation services.” 
• Geography.  Country, region, city, state, DMA are all targetable constructs.  You may choose target people surfing from 02141 (Cambridge, MA) an ad for pre-sale Red Sox tix or content about Mike Lowell’s recent contract.
• Browsing environment such as the connection speed, type of browser, operating system, user software, domain, and ISP.  An ad network serves an ad for Verizon DSL to a modem-based surfer by detecting the visitor’s browsing environment.
• Time.  The idea of only showing content during specific periods of time is called “parting.”  Common types include day-parting and season-parting.  For example, a B2B site only choosing to show ads for a particular manufacturers product during business hours – the site’s busiest time of day – would be an example of day parting.
• Keyword.  There are many different types of keyword targeting.  Google does fantastic things with targeting ads based on the keywords in queries.  Content Management Systems can target content based on on-site search keywords or referring keywords.  “Keywords” may be associated as metadata with site sections or pages, similar to a zone or a category targeting on an ad server.  Once a page is associated with “keyword” metadata, you can tell your server to target that keyword (and all pages where it exists as metadata).  If two categories each with different content share a targetable keyword, I can target ads across both categories to pages tagged with that specific keyword.
• Language.  When a language is set, you can target ads to visitors with that setting. Think Google.  Keep in mind that when you target by language, the creative copy is not translated. 
• Demographics. If the ad server is aware of a segment’s demographics, such as age, gender, income, title, purchasing power, and so on, an ad can be targeted on that basis.  Sometimes this is called “profile targeting.”
• Context.  Think of Google AdSense and how it matches ads based on the semantics in site content.  Now you understand content targeting based on context.
• Profile.  Targeting is possible based on conclusions drawn and rules created from the known attributes (such as purchasing propensity) about and individual or segment.

Enter one of the holy grails of online advertising and new media: “behavioral targeting” – an advanced form of targeting. Behavioral targeting refers to the process in which content is shown to a visitor based on the web sites they visit (or have visited) and the actions they take on those sites.  

Behavioral targeting involves:

1. Knowing where a visitor “comes from” and what they’ve done in the past. 
2. Determining the context of the visitor on the site. 
3. Detecting the visitor’s current behavior.
4. Serving relevant content and/or ads matched to the behavior.

By understanding the visitor’s past history, current state, and most recent behavior the marketer can target content in order to influence some point in the customer buying cycle- often at the stages of awareness and consideration.

So where does web analytics come in?  You would think web analytics data from “web analytics” technology would provide the seed data for enabling “targeting.”  It can be but in most cases, targeting is a function provided by the ad server or network or another technology called the “behavioral targeting platform,” not the analytics tool… the data does not come directly from the web analytics tool.  I’d love to hear how well (or if at all) Omniture TouchClarity is integrated with Omniture Discover or other offerings. 

In order to make web analytics data useful for targeting (if you can at all), you will need to use your web analytics data to:

1. Define segments to target (hard to export from web analytics tools)
2. Feed those segments and associated behavioral data to another tool (achievable if you own your data and run a tool in-house.  Harder and more costly if not).
3. Report on segment performance after targeting (that requires employing the right people and enabling them with the right tools)..
4. Analyze segment performance after targeting (again employ the right people and enable them with the appropriate tools and resources).

While I’ve only covered a very little bit about “targeting” and even less about “behavioral targeting” in the context of web analytics, I hope that my simple description of current methods for targeting and some thinking about “what is BT” will help you understand the emerging ecosystem in which analytics tool are interoperating now and will interoperate in the future.

Market segmentation existed long before web analytics.  It’s a method for dividing a population into specific groups (segments) that share one or more characteristics.  The goal of segmentation is to maximize future value of that segment by optimizing your marketing mix.

Segment analysis will tell you different things about your audience than you will realize from studying overall population metrics.  In traditional market research, segments are created from demographics (such as age), psychographics (such as attitude), geography (such as zip code), behavior (such as usage patterns), and value (revenue earned and cost).

Using a web analytics tool to segment your online audience requires a bit of upfront thinking and requirements gathering before getting down to business.  Like all things web analytics, segmentation requires process.  Here are some tips that may help you create a process for web analytics segmentation:

• Determine your business objectives.  Like everything in web analytics, you can’t optimize what you haven’t defined as a goal.  A business objective driving segmentation might be to “increase conversion rate (over historical numbers)” or “to improve frequency” by offering something valuable to that segment.
• Define segments. Basic dimensions for segmentation in web analytics include: new visitors, repeat visitors, geography, time, referrer, keyword, and campaign type.  Many more dimensions and attributes can be used for segmentation too.
• Identify expected segment behavior.  By reconciling goals, historic performance data, and VOC research, you should be able to identify the expected behavior of the segment.  If your business objective is to “increase conversion rate,” your expected segment behavior might be to “complete the form” or “click on a link.”
• Measure current segment behavior. Sounds easy, right, but it will take system configuration and the right tool.  Pages may need to be (re)instrumented, tracking codes may need to be applied, query string parameters may need to be parsed, and in the worse case dimensions you want to segment or the metrics you may want to measure against may not be available in your web analytics tool.  For example, how would you use your tool identify the “conversion rate” for a segment of repeat visitors from newsletter X who came from Tokyo and previously downloaded a whitepaper?
• Create “optimization hypotheses.”  Once you’ve measured current behavior, create a hypothesis or hypotheses to test in order to optimize the behavior.  You may want to perform controlled experimentation whether a simple AB test (i.e. champion/challenger), multivariate test, or experience test.  For example, I may have detected that repeat visitors from Newsletter X responded better to Y offer after being exposed to a certain element than those visitors in the same segment who did were not exposed.  That element could have been a content theme, offer, call to action, creative, and so on.  Thus, I might create a hypothesis to test that combines elements of the user experience that I feel are key to persuading the behavior and thus fulfilling the business objective.
• Optimize content, offerings, user experience, and other site elements.  Based on your hypothesis, make subsequent changes to the elements that you think will drive the desired segment behavior.  For example, you may split traffic to two landing pages each with a completely different offer, creative, and call to action.  Or you may choose to switch out specific elements on one landing page (such as an image or call to action) using multivariate methods just to get Visitor X to “complete that form” or “click that link” to improve your “conversion rate.”
• Analyze segment behavior against hypothesis.  How did the segment perform against expected behavior and business objectives based on testing your hypotheses?  Tools that provide drill-down/drill-up and cross-dimensional capability allow to analyze segments and answer such questions. The tools I’m talking about are advanced and powerful, such as Unica NetInsight, Visual Sciences Visual Site, Omniture Discover, and WebTrends Marketing Warehouse.  Capabilities for segmentation analytics vary by tool, so make sure to dig deep into the offerings because not all tools with let you correlate metrics like “conversion rate” with dimensions like “keyword,” let alone build complex multi-dimensional segments.  In fact, some free web analytics don’t allow you to segment data at all (just filter it)!
• Go with what works.  Web analytics segmentation analysis will let you know what appeals to and works for a segment.  Success with web analytics segmentation means that you met your business goals and improved key performance of that segment.  Rinse, lather, and repeat the segmentation analysis and optimization process so your campaign outperforms and margins soar!

As a result of well-executed web analytics segment analysis and hypothesis testing you can realize new value in your customers and new opportunities in your campaigns. 

Eric Peterson of Web Analytics Demystified invites all practitioners, vendors, and consultants to participate in the Fall 2007 Web Analytics Demystified Survey.  This season’s survey focuses on web analytics tools and examines distribution of deployment and overall satisfaction.

It takes about 15 minutes to complete.  As an incentive Eric Peterson is offering a 50% discount on his Big Book of KPI’s. 

And the big bonus is that all the resulting research will be made available on the WAD site for free (the Spring 2007 survey research is here).  Please lend your voice at:

Take the Web Analytics Demystified Fall 2007 Survey Right Now!

A successful web analytics practice helps a business manage its performance toward goals.  If you believe the statement, then you understand that in order to manage for site performance, a web site must exist to support one or more goals.  Pretty logical, right?

Then why is that web analytics practitioners tell me they often encounter web site owners who have no clear, measurable goals?  It’s really strange.  In fact, it’s vexing and frustrating because without known goals you OBVIOUSLY can’t manage site performance.  Without goals you can’t really optimize anything and are left to simply tracking trends related to basic metrics and/or derivatives thereof.

Thus, one of the responsibilities of a web analyst is identifying and confirming site goals.  Once you have site goals, you can create KPI’s that you monitor on a regular basis investigating variances, anomalies, and outliers affecting site performance. 

Undoubtedly one of the goals you will identify for your site is some type of conversion.  A conversion is a value-generating transition on your site.  For example, the successful completion of a form that enables a visitor to download a content asset or completing the purchase of a product may be a type of conversion you measure. 

While overall site conversion rates for value-generating events are important to know, real insights come from segmenting visitors or other dimensions by conversion rates.  The best analytic tools enable you to identify the conversion event and slice the conversion by any dimension: page, geography, referrer, keywords and so on. Thus measures of conversion will be instrumental KPI’s when using analytics to manage performance. 

The notion of a “funnel” in web analytics is a sequence of one or more pages that a visitor clicks-through until reaching a final destination, the conversion event.  The “funnel” assists the analyst in understanding the discrete clickstream that led to conversion.  For example,  a pre-defined path through the pages in the subscription or download process could be considered a funnel.  Funnels can be linear and non-linear and are affected by all sorts of things like detours, fall out, abandonment… That’s a post for another topic…  Yet, the metaphor of the funnels is applicable across all sites…  This notion becomes problematic when we consider multichannel attribution of the conversion process (again another blog post when I have time).

So what’s an analyst do when they want to begin to use web analytics to manage performance against goals? Here’s are some tips:

1. Investigate the business’ revenue model. Advertising-based sites generate revenue from selling various types of ad units (rpm/cpm), contextual advertising (rpm/cpc), lead generation programs (rpl/cpl), revenue sharing, and via affiliate syndication and content sharing deals.  Ecommerce sites generate revenue by selling products or brokering services or transactions.  How does your site generate revenue?  If the goal of the site isn’t to generate revenue, then skip this step.
2. Ask key managers to identify business goals.  Top-level managers have a better grasp of the vast ecosystem of suppliers, buyers, and other priorities that you, the analyst, may not be privy to.  Your manager should be helping you put your analytics work in context of the business goals.  So ask your boss what are the site goals?  Don’t accept the answer “to drive revenue.”  Ask how the identified goals support value creation and revenue generation.  The measurement of events supporting business goals should be a focus for performance management and optimization. 
3. Identify the conversion events that support businss goals and the revenue model, including any necessary steps in the funnel.   The actions that satisfy goals are the conversion events. The transition involved when the visitor clicks and makes the site money is your discrete conversion event. The page immediately preceding the conversion event is the last step in the funnel. 
4. Determine the actors on the site.  Actors can be categorized into internal/external actors.  For this exercise, concentrate on identifying the roles and responsibilities of the internal actors who DIRECTLY influence site production.  In other words, who are people modifying the site and what do they do?  The indirect actors, like your boss, also affect the site, so make sure you consider their role and responsibility in advanced site goals and fulfilling the objectives of the business model. 
5. Determine the goals of the actors.  Like site goals based on revenue, all indirect and direct site actors will have goals specific to their jobs.  Actor goals support site goals.  Thus, actor goals can be translated to tactical KPI’s.  For example, the editorial actor may want to ensure that X number of newsletter-referred visitors subscribe to the print magazine, so you create a KPI’s “subscription conversion rate by newsletter” and “number of online subscriptions generated per newsletter.”  Based on the site goals for conversion and the number of subscriptions generated from the online channel, you can start managing editorial performance.
6. Document site goals, actor goals, conversion events, and funnels, including a diagram a hub-and-spoke model of actor roles and responsibilities and flow diagrams of funnels.  In order to establish a process for performance management via web analytics, all the actors must generally agree on roles and responsibilities.  By documenting your investigation, you confirm correctness, identify gaps in business process, and create alignment among actors and management.  You may notice breakpoints in site production processes too.  The end result is a fully-documented operational model of how your site is created, monetized, and deployed.  In the same way that you can’t manage what you don’t measure, you shouldn’t be measuring things you can’t manage.   
7. Have key managers who direct site actors sign-off approval on the documentation.  The holy sign-off confirms you correctly identified the revenue model, site and actor goals, site navigational flows that lead to conversion.  When questions arise you can reference this process artifact to backup the conversion events you defined, the KPI’s you’ve created, and subsequently the performance recommendations you will make and manage.

Then:

1. Configure your web analytics tool to report conversion rates for revenue-generating site transitions and events and to report on funnels.  All tools can do this. 
2. Build KPI’s based on specific functional goals performed by actor’s on the site. Base KPI’s around activities that support the core revenue model and the activities performed by site actors.
3. Review KPI’s with actors.  Bring your documentation and identify how the KPI’s will be used to identify performance, contextualize optimization recommendations, and help each actor be more successful at their job. 
4. Report conversion rates and KPI’s to key managers and site actors.  Optimally these performance metrics should be available for actors and other stakeholders whenever they want them, preferably in the form of dashboards elicitable from your web analytics tool.  The goal should be identified, the target value for the goal, the KPI measurement,  and any deviation from the goal should be noted along with written performance recommendations.
5. Research site performance by segmenting conversion rates and KPI’s and investigating drivers.  KPI’s provide context for understanding the actions that influence site performance.  Overall conversion rate will only tell you so much.  For example, to the SEO conversion by organic search referrer is more informative.  Other actors will require reports on segmented conversion specific to their function. 
6. Make optimization recommendations.  Whether you deliver recommendations via reporting or manage the multivariate testing function at your company, you’ll need identify the events or actions to optimize.  Then you need to get buy-in from various gatekeeping actors.
7. Test and implement optimizations.   Use a multivariate testing vendor to test combinations of content and creative that drive KPI’s and that provide lift in conversion.  Work with site actors to ensure optimization testing and controlled experimentation occurs.
8. Rollout optimization that increase conversion, improve goals, and drive revenue.  Once you are reasonably certain the optimizations you are suggesting will improve performance work with the web or product development team to realize these changes.

Thus web analytics for performance management involves:

• Goal Clarification.  Why does this site exists in the first place?  Don’t be surprised to learn different actors have different goals, and no one is aligned!  From what I hear on the street that’s a common issue!
• Stakeholder Alignment.  Do all stakeholder and actors agree on the reasons why the site exists?  If not, be prepared to mediate.
• Experience Optimization.  How is the visitor interacting with my site, and do those interactions channel visitors to conversion funnels?  Do relevant calls to action and points of resolution work for persuading visitors to convert.  What’s working?  What’s not?  Figure it out.
• Controlled experimentation.  Based on potential optimizations available, what do you test?  Multivariate testing software can help, as can VOC research.  Talk to your research team.  Use the AB testing feature of your web analytics tool…  Whatever you do, you should establish a repeatable process for doing so. 
• Outcomes measurement.  If you set up a KPI dashboard with goals, actual performance, and variances you will be able to answer that question “so I did all this stuff, so what effect did it have?”

Easy right? Now get to managing performance using web analytics!

Web analytics data is the raw material from which companies will realize new online products and deliver differentiated services that generate future value.

Right now as I type I can get web analytics data from so many sources.  Google Analytics and Open Web Analytics provide the data for free (once I spend the overhead to set it up).  So does Compete and Quantcast.  Many other companies are willing to broker this potential commodity to me at various price points - from the low four to seven figures. 

The price from web analytics firms for what is essentially the same data is all over the map!  Why? Perhaps because enterprise vendors know that you the customer will no longer pay for just data (thanks Google!). 

Features and services provided on top of the core data that’s valuable to a practitioner.  To the company employing the practitioner, it’s the insights generated from the data that’s valuable.

We’re seeing a lot of web analytics data operationalization via features for:

• Extension into business intelligence (and in the future leading to business analytics).  The best web analytics firms are providing open relational databases and creating methods for joining data from other systems to “extend the data model” or feed the enterprise data warehouse.  
• Automated testing and optimization.  The notion that “you aren’t doing web analytics, if you aren’t testing” provides evidence that siloed, lonely data won’t do much for your business.  In that light, automated testing is only as useful for prediction as the people setting up the tests.
• Targeting.  Using analytics data during the session or after it to automatically target content based on key visitor attributes will increase conversion.  While targeting technologies use analytics data, the value derived isn’t from the data, but from the potential conversion lift of the activity we call “targeting.”
• Proxy scoring.  Assigning a value to an event, interaction, page view, or visit can identify high-value segments and customers.  Scoring abstractions operate on the data to indicate value. 
• Profiling.  Building a picture of your online audience by aggregating data from various sources including web behavior, customer transactions, and demographic data enables one-to-one marketing.  Web data is part of the profile.  The value is in the profile.
• Integration.  Joining analytics data with data from other systems in a unified data model, or enabling machine-to-machine communication of analytics data will yield value.  Again the data is important, but the value is in the outcome.
• Alerting.  Indicating when data exceeds pre-defined upper and lower bounds and where those thresholds have been exceeded is valuable.  Once again, the data is crucial, but the alert is catalyst for value creation.

Data SEPARATED from the application, from the presentation, is extremely valuable.  When “unsiloed” and described or made available using open standards, it can be reused by other applications.  Insights realized from moving/sharing/synching data that drives enterprise value 

Yet functionality on top of the data layer doesn’t make web analytics easy and instantly drive value.  Few corporations have the slightest clue about how to take advantage of all this functionality. 

To creation value the modern web analytics practice requires:

• Dedicated in-house professionals. No duh, here.  You need people who understand the data, use ”features” to help analyze it, and who can then test hypotheses to optimize and measure outcomes.   
• Vendor and third-party professional services.  Consultants must go beyond “repeating back what you say, then claiming they can solve the problem” and deliver quantifiable, measurable value that improves business process.
• Web services. Web analytics tools need to use web services.  The “web analytics” tool of the future will take advantage of technologies that provide platform-independent protocols and standards used for exchanging data between applications. 

We’re going to continue to see highly-specialized web analytic’s “experts” at companies work with services firms (or professional services staff from vendors) to combine the ”off the shelf” web analytics products with web services technologies to create the automated marketing architectures of tomorrow. 

These “marketectures” will:

• Bridge quantative analytics data with qualitative data to identify the 360-degree view of the customer experience.  Ahh buzzwords…
• Use web services to operate on analytics data to solve highly-specific and specialized technology and business problems.  For example, using a WS to pass custom interaction events in Adobe AIR/FLEX to understand behavior in your RIA. 
• Enable HUMANS to realize new *predictive* insights from data.  Yeah, automated testing will help, but think about it… who sets up the tests? 

I’m contributing a monthly article to MediaPost’s Metrics Insider Column.  My first contribution was published last week, and I’m reposting it here to get your thoughts.  Soon I hope to describe what I mean in more detail… The article was called “The Most Measurable Medium needs and XML Standard.” In case you missed it here it is:

OVER THE LAST TWO WEEKS, my fellow Metrics Insider columnists have correctly pointed out that online metrics are neither standardized nor easily integrated across systems. Vocabulary is muddled. Numbers do not match. Data exists in silos and is isolated from related data. Systems do not adequately or easily talk to each other. Research services, ad servers, and Web analytics tools report similarly named, overlapping and often conflicting metrics. Unfortunately, these problems will not disappear anytime soon, even with emerging “standards” and continued attention paid by the industry to these important issues.

Current industry standards for Web metrics are limited, basic, and come from independent entities. Most recently, the Web Analytics Association released a set of “standards.” The WAA’s standards are elementary definitions of concepts from various periods of Internet measurement. Web 2.0 concepts like “events” are mingled with dated measurements like “hits.” Regardless, these definitions provide a very useful starting point for framing a discussion about metrics. Recently, I’ve learned that the IAB and MRC are developing a set of IAB Reach Measurement Guidelines. Let’s hope the IAB and WAA align their work efforts.

The IAB and MRC are also currently auditing “audience measurement” firms, like Comscore and Nielsen. It’s rather unclear to practitioners what standards the IAB/MRC are applying to the audit. But the hope is that auditing will expose issues of coverage error and selection bias in the black box methodologies used to create the panels and generate the audience measurement data.

It is important to note that the IAB’s audit has two parts. The first is certification, which indicates the company being audited is applying the “standards,” and the second is accreditation, which demonstrates adherence to the IAB standards.

Only time will tell if companies like Hitwise, Compete, and Quantcast will be asked to submit to auditing. It’s worth mentioning that legacy metrics “standards” (and audits) from historic organizations like ABCe still occur and carry weight with publishers and advertisers (especially outside of the United States). It’s entirely possible that newly formed organizations, like the Association for Downloadable Media will offer their perspective on “standards” for online metrics.

The idea of “standards for the standards”–however absurd it sounds on the surface — starts to seem like a good idea when considering that all these parallel efforts aren’t intersecting. Honestly though, I question whether “standards” that are purely “definitional,” even if agreed upon, will solve many of the measurement challenges companies have when trying to understand Web data and take action from it.

Standard definitions are helpful for promoting understanding and creating a controlled vocabulary for discussing online metrics, but they don’t help with what I see as a huge challenge in today ’s metrics technologies. The problem is this: currently available online metrics systems do not adequately separate data from presentation . That’s a huge limitation preventing Web data from being easily integrated with other systems.

Detailed-level Web data (the raw data) is often costly to extract, if available at all. It is nearly impossible to deliver detailed data in real time from Web analytics, ad serving, and research-based technologies in order to feed other systems. The majority of hosted (ASP) metrics systems are closed and do not allow access to key interfaces using open software standards. For the most part, today’s metrics technologies are black boxes where data goes in, but can only be extracted in various file formats after creating a report. Common export formats include csv, pdf, and doc. While XML exports are often available from many vendors, there is no standard XML schema for describing the same type of Web data across different sources!

The industry must begin collaborating and creating a standard XML schema for describing Web data. Creating a widely used, consensus-based, published, and maintained XML standard for online metrics would make it possible to more easily share, transform, and use Web data in other systems.

I firmly believe that current metrics standards must go beyond simple definitions and tackle issues pertaining to data portability and system interoperability. Then we’ll all be in a better position to reuse Web data across the enterprise value chain. Once we all agree on “standard” definitions, I encourage us to start working together to develop a standard Online Metrics Markup Language.

Last week I attended the 1st Annual SEMPhonic XChange and led a collaborative discussion on “deploying measurement systems in distributed companies.” In case you hadn’t heard about SEMPhonic, the company is a boutique consulting firm in Novato, CA (near one of my favorite towns on Earth: San Francisco).  They do some very unique work with web analytics (Functionalism) and other facets of new media technology.  A fine gentleman named Gary Angel leads the group. He’s recently hired smart folks, like long-time web analytics expert, June Dershewitz, and notable blogger and author of the Web Analytics Report, Phil Kemelor.

A few months ago the idea came up for me to lead a “huddle” at their Xchange conference.  I was intrigued with this huddle idea.  It was different. New.  I’d facilitate a discussion on a topic of my choice in a small, Socratic, group setting.  No PPT!

Fast forward to last week, and I found myself sitting in Napa, CA at COPIA: The American Center for Wine, Food, and the Arts. COPIA was a brilliant place for an industry colloquy.  Being a guy who likes culture (and conferences), this venue offered the best of both worlds.  Instead of a hotel, I found myself surrounded by gardens, vineyards, art, food, wine, web analytics, and the brightest in the industry.  Cool!

Thinking back on the event, SEMPhonic Xchange, in my opinion, is a “must-attend conference.”  Not only is the “huddle” format unique and fun in which to participate, it’s also a format that promotes deep discussion that leads to truly actionable insights. It’s a conference based on dialog and collaborative discussions between participants.  The huddle format provides 10 hours of free collaborative consulting (5 huddles) with employed people you can’t hire (and some consultants you can)!  Compare a daily consulting rate to the conference cost, and it’s a no brainer.

My discussion on best practices for the successful deployment of measurement systems lasted a little over two hours.  My group collaborated nicely (so I thought).  It included very intelligent folks like Jared from Intuit, Scott and Christel from Xilinx, Sami and Fred from Adobe, Renata and Matthew from American Express, Aaron from Webtrends, Rupa from Cisco, Amy from JPMorganChase, Jeff from New England Journal of Medicine, and Kevin from Charles Schwab.  Thanks to anyone reading this blog who attended.  Your valuable insights and knowledge sharing contributed to the success of the huddle.

At the ends of the huddle I went over a tips for a successful deployment.  Here are a few:

• Identify business goals, match those goals to business strategy, and align metrics and KPI’s to support those goals.  I always say “metrics and kpi’s have significance in position and relation to a goal.”  You can’t measure against performance unless you’ve identified your business goals.  Goals are supported by strategy.  You create KPI’s for measuring against goals and for guiding work toward objectives.  Every business unit could have different goals.  Management must align and support the standardized goals that you bake into your KPI’s (and any specific derivatives you create to support the goals of differentiated business units).
• Verify the technology implementation and the metrics collected.  A web analytics or other measurement system must be verified to ensure conformance with “standard” best practices based on the company’s goals.  You need to make sure the numbers can be trusted.  That means, validating data and reconciling differences with historic reporting systems or whatever the business demands, understands (and possibly pays for in real dollars, time, or resources allocated), such as ABCe, WAA, IAB, MRC, XYZ, 123, FBCINSA whatever it takes…
• Provide education and training.  While some analysts want to keep all the data within their analytics group, I don’t think that practice scales across a large enterprise unless you are lucky enough to have a large analytics team.   I know I can’t individually and alone satisfy the metrics needs for thousands of stakeholders or hundreds of brands.  Successful companies deploying on a large scale adopt a “train the trainer” approach.  The trainers guide their business units, become local experts, and help foment a data-driven culture.   Let the data be free and the people educated, I say. 
• Consider the corporate culture.  A metrics tool changes organizational culture (for the better or worse).  Suddenly, everyone is being measured and perhaps evaluated on goals implicit in the measurements.  Some will greet the tool with open arms while others will see it as a threat.  Solid management needs to foster buy-in and support for the tool.  That way, organizational resistance is overcome and clarity of mission is realized.
• Help business units ”use the metrics” to improve performance.  My friend Eric Peterson likes to say “web analytics is hard.”  Yes it is!  That means the expert needs to work with global business teams to mentor and teach how to separate signal from noise.   As the tool is used, business units will identify “pain points” that will need to be addressed.  The analytics team should work with business units heal the pain and improve performance.
• Plan to ”get granular” and the “get integrated” with the measurement system.   Additional requirements will come out of the woodwork due to organizational learning after you golive (even if you think you’ve elicited all reqs prior to deploying your tool and building reports).  New requirements are an early sign of success (people are “getting it” after all).   As the business learns, it will become necessary to extend the system.  Consider brainstorming about the possible ways in which the system should be extended prior to rollout, and perhaps create a basic plan for extending the system. When the system is launched, modify that system enhancement plan and execute on it to support business goals.  Employ a project manager to plan and help execute!
• Manage and guide stakeholder expectations to minimize risk.  Be realistic when for guiding and setting expectations so that you can minimize risk (to your overall mission, your job, the company ;).  Risk comes from incorrect metrics, organizatonal issues with adoption, changes in business goals, shifting managerial priorities, technology problems, and issues resulting from resource allocation.  Make sure you manage around these issues or find ways to directly deal with them. 
• Generously explain why the measurement system is being deployed and more.  You must create a well-thought out communication plan for promoting adoption and use of a metrics tool.  The communication plan should focus on answering:
  • Why the tool is being deployed.
  • What people are supposed to do with the tool.
  • How people should use the tool.
  • What key metric/KPI’s people and business units should be looking at to manage their performance.
  • How to go from “insight to action” based on analysis of the metrics.
• Share best practices and lessons learned across the enterprise.  As new insights are realized and the company starts taking action from the metrics, you should provide a way for the enterprise to communicate the “highest and best use” of the tool.  By promoting collaboration and knowledge sharing, the company is more likely to succeed quickly with the measurement tool and realize a demonstrable ROI from it.
• Realize that “premature optimization is the root of all bugs.”  When deploying a measurement system, you need to establish a baseline system before extending the system to get more “granular.”  An implementation must be granular before integrating data from other systems.  While these concepts will overlap in deployment and may occur in close proximity or in a waterfall (i.e granularity may be enabled via integration) you need to ensure you don’t put the cart before the horse.  Make sure you correctly measuring and understanding the basics (like recency, frequency, clickstream, referrers, bounced visits, and depth) then moving forward with more advanced and necessary measurements and reporting (like bounce rate, conversions, view thrus, voice of the customer, and ”engagement.”)   Set clear expectations and guidelines with the consultants.  Don’t move too fast with development.  QA, then segment away, I say… ;-) 

A lot of more was discussed and shared over in Napa and throughout the SEMphonic XChange conference.  Please share your thoughts, if you feel like it!  Thanks for reading!

Running more than one web analytics tool on a site or across a portfolio of sites is an increasingly common practice these days.  The majority of the companies that run multiple tools probably run one “for pay” tool and at least one “for free” tool.  Based on my experience, the cost of running two “for pay” solutions would be prohibitive for companies still trying to realize the “ROI” from web analytics (but it’s not unheard of in large, solvent, multinational companies).  Not surprisingly, the most common “free tool” ran next to “enterprise-level” :-) tools like Omniture, Visual Sciences, and Unica NetInsight is Google Analytics.  Data from my pal Eric Peterson’s Vendor Discovery Tool shows that GA and Visual Sciences code were found on 6% of tracked URL’s, GA and Omniture code on 4% of tracked URL’s, GA and WebTrends Hosted code on 4% of tracked URL’s.  That’s great for Google and quite an edification of their excellent product!

I’m sure that there are companies running multiple “for free” tools, and/or running multiple big ticket tools (like HBX Analytics and Visual Sciences), and/or multiple homegrown tools built from Business Intelligence technologies and databases (Oracle/Cognos).  Yet running multiple tools has risks and rewards.

Some of the risks of running more than one web analytics tools include:

• Lack of control over data. If you trying to foment a data driven culture, nothing could be more frustrating than someone outside of the web analytics team downloading a tag, linking it to their personal account, then questioning why X number in Y tool doesn’t match X number in Z tool.  To promote adoption of new technology, running a competing tool has the potential to compromise data believability.
• Numbers not matching across tools.  Different vendors “sessionize” differently so numbers will never be identical between tools.  Dynamic sites, different underlying site technologies, and unique tool configurations mean numbers won’t match.  Never ever.  Check out Eric Enge’s highly-recommended 2007 Vendor Shootout. Run two tools and be prepared to answer questions about data discrepancies from those who consume reports on the same site from both tools.
• Conflicting vocabulary.  Different tools use different terminology.  One tool may use “sessions,” while another may use “visits.”  Some tools talk about “views,” while others reference “page views.”  Some tools use the term “unique visitors,” and other tools just talk about “visitors.”  When you are rolling out “web analytics” to people who need to speak the same language, having multiple vocabularies for expressing the same or similar concepts confuses discussion and muddles actionability.
• Apples and Mangoes Comparisons.  Some tools provide only snapshots of aggregated data, while other tools let you drill down, drill up, and slice and dice on detailed level data.   Some tools enable you to add metrics on the fly to any report, and then filter and cross dimensions until your heart is content.  Two people looking at two tools on the same data may conclude different actions are warranted based on the depth of their analysis.  While a good manager can sort that out, it’s a bother.
• Potential to misallocate resources leading to needless redundancy.  Companies have limited resources. If I need to apply tags to all my sites so that I can get to the real business of analysis, then why spend valuable time applying multiple sets of tags to enable tools that serve the same purpose.
• Licensing issues.  Google Analytics or Quantcast account on a corporate site associated with a personal account whose owner would prefer not to give up the password.   
• Training issues.  When rolling out systems, training is necessary.  Tools take time to learn.  Why have resources learn multiple tools that do more or less than same thing when you got real business to take care of?

Some of the rewards from running more than one web analytics tool include:

• Comparative data.  If you’re being charged by page views, it’s nice to have an alternate reference point to validate the charges.
• Differentiated reporting.  Some tools are just better at custom and ad hoc reporting than others.  If you have an inflexible tool not fully loaded with features then maybe it makes sense to get a tool that can do all that stuff a lot cheaper than paying for additional incremental features.  Hello GA!
• Potential to enable a different level of integration.  Lots of people tell me they download Google Analytics so that they can track their AdWords campaigns. 
• Ability to leverage different features.  Several major tools are technically and functionally challenged when it comes to simple things like showing the keywords used to drive traffic to the particular page, the number of unique visitors per page, or a bounce rate.  Instead of dealing with complexities, sometimes it’s just easier to download and install a free solution like GA that does all of these things at no cost.
• Ability to leverage different data collection methods.  Time-based metrics and file downloads are inordinately easier to measure and count using log file tools than using page tag tools, imho.   Why fiddle with some esotericisms in tagging when you can just run the logs?  Or better, yet, use a hybrid approach in one tool and get the best of both data collection methods.

Is it a good idea, as a site owner or manager of a web analytics team, to run more than one tool?  The answer is it depends on your organization’s capability maturity for web analytics and how you balance risks and rewards.

The most mature companies have a centralized web analytics function.  That means the company has one “master user” and “strategic owner” for web analytics and related technologies.  The centralized web analytics function has its own resources dedicated to “doing web analytics.”  Resources may come from other groups within a company, but, regardless, the company executives have identified and placed positional power around a “web analytics champion.”  Since you’re reading my blog, you may be this person!  Cool!

When you have centralization, you control key elements of doing web analytics:

1. Measuring
2. Reporting
3. Analyzing
4. Testing
5. Evaluating outcomes

If you’ve centralized your web analytics team, you should select ONE web analytics tool as your Primary Web Analytics Tool. 

Then I think it is then I think it’s safe to use more than one web analytics tool along these guidelines:

• Standardize on one tool as your primary tool.  This tool should become the ”bible” for web analytics data at your company. 
• Give people outside of the web analytics department access to the primary tool and ONLY the primary tool.
• Use the secondary tool within your web analytics team as a supplemental tool for comparing measurements, data reconcilation/verification, or analysis that you can’t accomplish with your primary tool (such as detailed roll-up reporting). 
• Keep the overall enterprise standardized on the interface and numbers from your primary web analytics tool.  That way you prevent confusion when reporting to stakeholders outside of the web analytics team. 
• Do not provide access to the secondary tool to people outside of the web analytics team (unless the numbers match 100%! ;- ) ).

If you have a decentralized web analytics organization, I recommend that you:

• Standardize on a primary tool (whether free or paid).  Remember Google Analytics is an awesome place to start!!  And so it seems is Microsoft Gatineau!
• Work toward centralization before introducing another tool that has the potential to undermine current measures, reports, analysis, tests, and outcome evaluations.

On a final note, you may have multiple tools for reporting web analytics data (perhaps from companies like Business Objects, Cognos, Microsoft and so on).  As long as the data is synchronized with the metrics from your primary web analytics tool, that’s fantastic! 

Am I off-base?  Absolutely right?  Do you run two tools and love it, hate it, don’t care?  What do you think?