I write a monthly column for Mediapost’s Metrics Insider.  This month I wanted to talk about the different schools of thought in online metrics because at the end of the day we are all in Internet measurement together. Hope you enjoy the read:

Audience measurement and Web analytics systems are like the yin and yang of online metrics. Yin and yang are different, opposing forces, but they also complement each other. Think of Web analytics and audience measurement data in the same way: different, sometimes in opposition, but complementary.

The major difference between these systems is data collection:

• Audience measurement companies don’t collect data directly from the sites being measured. They all rely on proprietary methods. Hitwise gets data from ISPs. Compete uses a toolbar that you can download as well as ISP and panel information. Nielsen and comScore use data collected from panels to create online metrics that they believe accurately represent overall Internet usage. Due to all these different data collection methods and no shared standards across companies, metrics from audience measurement firms are never identical with each other.
• In Web analytics, data is collected directly from actual site activity. Methods include client-side data collection via javascript page tagging, server-side data collection via log file processing, or network data collection via packet sniffing. Sometimes methods such as page tagging and log file processing are combined in what’s called “hybrid data collection.” Vendors include Coremetrics, Webtrends, Unica, Visual Sciences, Omniture, Google, and others. The challenge with Web analytics tools is that each tool will calculate different numbers from the same source for identical metrics. In other words, Omniture numbers won’t match Google’s. That’s because each tool has its own “secret sauce” for “sessionization” — the fancy term for the way metrics are counted and measured by analytics technology. For example, certain tools may be configured to include or exclude certain filetypes or server responses. Robotic traffic may or may not be filtered.

It’s worth noting that a company named Quantcast uses panel data and also enables a site to add page tags to collect actual site data, which are then merged together in a completely different type of “hybrid” model.

All these different approaches to data collection lead to opposition when these systems are used for the same purpose. For example, conflict arises between the yin and yang when identifying reach using unique visitor metrics. Audience measurement firms may cry “cookie deletion” when analytics tools are used to count unique visitors, and Web analytics firms may shout back “coverage error” and “selection bias” at the unique visitor numbers from panel-based firms. Another area of opposition is demographics. I’ve been told that only audience measurement firms provide demographic data, and that you can’t get demographic data from Web analytics systems. That’s not true at all.

All enterprise-level Web analytics systems provide demographic location information at the country, city, state, and MSA levels. This information will be different than that provided by audience measurement companies.

Demographics that are harder to elicit from a Web analytics system, but are easily provided by audience measurement, include attributes like a visitor’s age, gender, occupation, income, and education.

But it is possible to integrate very detailed demographic attributes per visitor into a Web analytics system! Once demographic information is captured in a registration database, it can be joined with behavioral data in the Web analytics system and reported on. For a real-world example of analytics/demographic integration, take a look at what Microsoft is doing with Gatineau, the company’s free Web analytics offering currently in beta. Microsoft is joining Web site behavioral data with rich demographic data from MS Live profiles.

Even with differences and oppositions between these online metrics systems, companies find ways to use the data in complementary ways:

• Audience measurement data is useful for competitive intelligence. All the paid and free services provide data for comparing the performance of a site to other sites, for understanding audience behavior across one or more sites by demographics, and for understanding generalized Internet traffic trends and search terms.
• Web analytics data is useful for understanding site effectiveness, for defining key performance indicators, for determining conversion rates for marketing campaigns by channel (such as search, email, rss), for understanding what sites and keywords are driving traffic to your site, and for segmenting and reporting online metrics.

You can even use both data sources as part of the same site optimization activity. For example, you could use audience measurement data to determine that a competitor is gaining ground on a particular product or search term. Then you could look at your Web analytics tool to see how you’re doing for the same term and how visitors who searched for that keyword behave on your site. You may find a high bounce rate and low conversion rate for the keyword, so you segment that data perhaps by demographics! Next you suggest a hypothesis to minimize bounce and maximize conversion for each segment. Then you test your hypothesis, and reexamine the data. Based on the results, you then continuously improve your online performance through controlled experimentation. At the end of the day, you will drive more online revenue by understanding how the yin of audience measurement and the yang of Web analytics complement each other, than by worrying about how they differ and oppose.

Targeting refers to the process of identifying characteristics of a segment so that relevant content may be matched to it and delivered at a time when the segment is most open to the message. The idea is the right content to the right visitor at the right time (optimally in real time). 

For example, you may visit a site, and see some type of ad unit calling out at you to “meet singles in <insert_your_city>.” When browsing real estate you may see ad units for realtors and mortgage companies.  After entering a keyword such as “car prices” and clickingthrough the SERP, you may see an ad for a local car dealer.   That’s targeting in a nutshell.  It’s simple: 

1. Visitor X has these attributes. 
2. We have content that we think will appeal to Vistor X’s attributes. 
3. Let’s show that content. 

While targeting has helped to increase ad clickthrough rates, it’s far from an ideal science.  Current methods for targeting have inefficiencies.  What if Visitor X just bought a new car after his recent marriage?  Unless the targeting engine is made aware of the visitor’s current state, the targeting may be off and not yield desired results. 

Even with limitations around “current awareness” targeting is perceived in the Internet industry as a crucial activity for maximizing the effectiveness of advertising and content.  Targeting is the next stage after A/B and multivariate testing.  Once you determine the preference of segments based on testing, you identify content to target. 

In new media, targeting is something associated with paid search campaigning, ad serving, and content optimization.  It’s not uncommon for targeting activities to be based on:

• Category and sub-category.  Conceptual constructs like “categories” of topics on a media web site or products on an ecommerce site can be targeted to include certain types of ads or messages.  The notion of a “zone” fits in here as well.  The idea is that if visitors are browsing in your category for “hardware floors” you could offer them an ad or content specific to “flooring installation services.” 
• Geography.  Country, region, city, state, DMA are all targetable constructs.  You may choose target people surfing from 02141 (Cambridge, MA) an ad for pre-sale Red Sox tix or content about Mike Lowell’s recent contract.
• Browsing environment such as the connection speed, type of browser, operating system, user software, domain, and ISP.  An ad network serves an ad for Verizon DSL to a modem-based surfer by detecting the visitor’s browsing environment.
• Time.  The idea of only showing content during specific periods of time is called “parting.”  Common types include day-parting and season-parting.  For example, a B2B site only choosing to show ads for a particular manufacturers product during business hours – the site’s busiest time of day – would be an example of day parting.
• Keyword.  There are many different types of keyword targeting.  Google does fantastic things with targeting ads based on the keywords in queries.  Content Management Systems can target content based on on-site search keywords or referring keywords.  “Keywords” may be associated as metadata with site sections or pages, similar to a zone or a category targeting on an ad server.  Once a page is associated with “keyword” metadata, you can tell your server to target that keyword (and all pages where it exists as metadata).  If two categories each with different content share a targetable keyword, I can target ads across both categories to pages tagged with that specific keyword.
• Language.  When a language is set, you can target ads to visitors with that setting. Think Google.  Keep in mind that when you target by language, the creative copy is not translated. 
• Demographics. If the ad server is aware of a segment’s demographics, such as age, gender, income, title, purchasing power, and so on, an ad can be targeted on that basis.  Sometimes this is called “profile targeting.”
• Context.  Think of Google AdSense and how it matches ads based on the semantics in site content.  Now you understand content targeting based on context.
• Profile.  Targeting is possible based on conclusions drawn and rules created from the known attributes (such as purchasing propensity) about and individual or segment.

Enter one of the holy grails of online advertising and new media: “behavioral targeting” – an advanced form of targeting. Behavioral targeting refers to the process in which content is shown to a visitor based on the web sites they visit (or have visited) and the actions they take on those sites.  

Behavioral targeting involves:

1. Knowing where a visitor “comes from” and what they’ve done in the past. 
2. Determining the context of the visitor on the site. 
3. Detecting the visitor’s current behavior.
4. Serving relevant content and/or ads matched to the behavior.

By understanding the visitor’s past history, current state, and most recent behavior the marketer can target content in order to influence some point in the customer buying cycle- often at the stages of awareness and consideration.

So where does web analytics come in?  You would think web analytics data from “web analytics” technology would provide the seed data for enabling “targeting.”  It can be but in most cases, targeting is a function provided by the ad server or network or another technology called the “behavioral targeting platform,” not the analytics tool… the data does not come directly from the web analytics tool.  I’d love to hear how well (or if at all) Omniture TouchClarity is integrated with Omniture Discover or other offerings. 

In order to make web analytics data useful for targeting (if you can at all), you will need to use your web analytics data to:

1. Define segments to target (hard to export from web analytics tools)
2. Feed those segments and associated behavioral data to another tool (achievable if you own your data and run a tool in-house.  Harder and more costly if not).
3. Report on segment performance after targeting (that requires employing the right people and enabling them with the right tools)..
4. Analyze segment performance after targeting (again employ the right people and enable them with the appropriate tools and resources).

While I’ve only covered a very little bit about “targeting” and even less about “behavioral targeting” in the context of web analytics, I hope that my simple description of current methods for targeting and some thinking about “what is BT” will help you understand the emerging ecosystem in which analytics tool are interoperating now and will interoperate in the future.

Market segmentation existed long before web analytics.  It’s a method for dividing a population into specific groups (segments) that share one or more characteristics.  The goal of segmentation is to maximize future value of that segment by optimizing your marketing mix.

Segment analysis will tell you different things about your audience than you will realize from studying overall population metrics.  In traditional market research, segments are created from demographics (such as age), psychographics (such as attitude), geography (such as zip code), behavior (such as usage patterns), and value (revenue earned and cost).

Using a web analytics tool to segment your online audience requires a bit of upfront thinking and requirements gathering before getting down to business.  Like all things web analytics, segmentation requires process.  Here are some tips that may help you create a process for web analytics segmentation:

• Determine your business objectives.  Like everything in web analytics, you can’t optimize what you haven’t defined as a goal.  A business objective driving segmentation might be to “increase conversion rate (over historical numbers)” or “to improve frequency” by offering something valuable to that segment.
• Define segments. Basic dimensions for segmentation in web analytics include: new visitors, repeat visitors, geography, time, referrer, keyword, and campaign type.  Many more dimensions and attributes can be used for segmentation too.
• Identify expected segment behavior.  By reconciling goals, historic performance data, and VOC research, you should be able to identify the expected behavior of the segment.  If your business objective is to “increase conversion rate,” your expected segment behavior might be to “complete the form” or “click on a link.”
• Measure current segment behavior. Sounds easy, right, but it will take system configuration and the right tool.  Pages may need to be (re)instrumented, tracking codes may need to be applied, query string parameters may need to be parsed, and in the worse case dimensions you want to segment or the metrics you may want to measure against may not be available in your web analytics tool.  For example, how would you use your tool identify the “conversion rate” for a segment of repeat visitors from newsletter X who came from Tokyo and previously downloaded a whitepaper?
• Create “optimization hypotheses.”  Once you’ve measured current behavior, create a hypothesis or hypotheses to test in order to optimize the behavior.  You may want to perform controlled experimentation whether a simple AB test (i.e. champion/challenger), multivariate test, or experience test.  For example, I may have detected that repeat visitors from Newsletter X responded better to Y offer after being exposed to a certain element than those visitors in the same segment who did were not exposed.  That element could have been a content theme, offer, call to action, creative, and so on.  Thus, I might create a hypothesis to test that combines elements of the user experience that I feel are key to persuading the behavior and thus fulfilling the business objective.
• Optimize content, offerings, user experience, and other site elements.  Based on your hypothesis, make subsequent changes to the elements that you think will drive the desired segment behavior.  For example, you may split traffic to two landing pages each with a completely different offer, creative, and call to action.  Or you may choose to switch out specific elements on one landing page (such as an image or call to action) using multivariate methods just to get Visitor X to “complete that form” or “click that link” to improve your “conversion rate.”
• Analyze segment behavior against hypothesis.  How did the segment perform against expected behavior and business objectives based on testing your hypotheses?  Tools that provide drill-down/drill-up and cross-dimensional capability allow to analyze segments and answer such questions. The tools I’m talking about are advanced and powerful, such as Unica NetInsight, Visual Sciences Visual Site, Omniture Discover, and WebTrends Marketing Warehouse.  Capabilities for segmentation analytics vary by tool, so make sure to dig deep into the offerings because not all tools with let you correlate metrics like “conversion rate” with dimensions like “keyword,” let alone build complex multi-dimensional segments.  In fact, some free web analytics don’t allow you to segment data at all (just filter it)!
• Go with what works.  Web analytics segmentation analysis will let you know what appeals to and works for a segment.  Success with web analytics segmentation means that you met your business goals and improved key performance of that segment.  Rinse, lather, and repeat the segmentation analysis and optimization process so your campaign outperforms and margins soar!

As a result of well-executed web analytics segment analysis and hypothesis testing you can realize new value in your customers and new opportunities in your campaigns. 

Eric Peterson of Web Analytics Demystified invites all practitioners, vendors, and consultants to participate in the Fall 2007 Web Analytics Demystified Survey.  This season’s survey focuses on web analytics tools and examines distribution of deployment and overall satisfaction.

It takes about 15 minutes to complete.  As an incentive Eric Peterson is offering a 50% discount on his Big Book of KPI’s. 

And the big bonus is that all the resulting research will be made available on the WAD site for free (the Spring 2007 survey research is here).  Please lend your voice at:

Take the Web Analytics Demystified Fall 2007 Survey Right Now!

Measuring video content with web analytics isn’t super difficult, but it has its nuances and challenges.  I’ve been thinking a bit about it lately, and have had some good conversations with a few people.  Folks I know are playing around with the likes of Joost, Vuze, and Hulu, TVUNetworks, as well as using BrightCove and Videoegg.  And, man, the popularity of BitTorrent and other swarm structure 4th gen P2P networks is larger than ever.

Simply speaking video measurement can be divided into the following types:

• Instream measurement.  Refers to measuring the video itself and the various abstract elements of the video experience, such as duration metrics (average viewing time) and interaction metrics (number of stops, plays, pauses, rewinds, fast forwards, and clicks on video content).
• Outstream measurement.  Refers to measuring the content environment and user experience surrounding the video, such as the conversion metrics (percentage of visits downloading or viewing a video), behavioral metrics (referrers to the video page, players used), and content metrics (percentage videos per channel, percentage videos viewed by topic, percent videos viewed by file type). 

By categorizing the web video analytics into these two buckets, you are better able to answer meaningfully the following questions, which must be considered prior to any rollout:

1. What are the business objectives for rolling out video features on the site?
2. What format are the videos in?
3. Are the videos downloads or streams?
4. Am I using a content distribution network or streaming video network?
5. Does my web analytics tool have the features necessary for video measurement? Or should I look for a third party, niche vendor?
6. What data collection method should I use?
7. Do I understand event models?
8. What KPI’s are relevant and important based on my business goals?

To help you formulate answers to those questions, here’s some thinking:

• Business objectives.  You, the analyst, must understand why your company is rolling out video.  In other words, what’s the goal and what strategy underpins the goal?  While video is “the rage” right now, simply rolling out video because “everyone is doing it” is no strategy (though doing so may yield a strategy ;).  A goal for video deployment could be “to generate leads,” thus you measure the scenario conversion rate for the funnel resulting in the lead generation and video download (outstream video analysis).  The objective might be “to keep visitors on the site longer,” then you would measure duration and interaction (instream video analysis).  As you all know, I firmly believe that it the business goal that allows you to contextualize what you’re measuring so that you may build KPI’s.
• Video format. Lots of different video file types exist: mpegs, qt, mov, swf, flv, avi, wma, ra, wmf, mp4 and more.  You’ll need to identify the video types you want to track so you can configure your web analytics tool to measure them.  Removing or adding filters or changing your tag’s javascript might be necessary. 
• Download or streams.  Videos can be downloaded (by right clicking) or spawned in a media player.  They can also exist embedded on the page or in another object for on-page streaming.  Thus, the way you instrument your pages will differ based on the way you present the video content. For example, if you are streaming videos, you may want to use javascript (or a vendor provided scripting language) to instrument your pages to track the video.  If you are just hosting downloads, you may simply want to run your logs to detect the number of times videos were downloaded.
• Content distribution network or video network. If your video content is distributed by a CDN or a video network, you will have to apply page tags on all the pages rendered by combining your server’s content with the content served by the CDN. Some video networks provide basic reporting that you can extend with a client-side page tagging solution.  Alternatively, you can process the logs provided by a CDN. The challenge with CDN log file processing is that you will most likely not be able to merge the data with your log files for the same site, resulting in two “profiles” of analytics data related to one site: one profile with the site analytics data and one with the CDN analytics data.
• Data collection method.  If you’ve read this far in my blogivation, you probably picked up that the data collection method you have at your disposal will constrain or enable the way you measure video.  Page tags will enable you to instrument your pages with onclick functions that pass values to the javascript and in turn to the analytics server.  Packet sniffers and log files enable you to measure downloads without modifying code.   If you need modify your web analytics tool or tag configuration to track video filetypes, you can reprocess logs to access the data.  With tags any data related to downloads or interactions with the video object prior to the config change will be lost.
• Web analytics tool features. Many web analytics tools will allow you track a video play or download in your page view reports, but only two tools support true event models: Unica NetInsight and Google Analytics.  At Emetrics San Fran in May 2007, Ian Houston and I gave a preso on “from page views to events.”  It looks like the vendors agreed, ay? ;)
• Third party tools.  With the convergence of internet and television, we’re not many years away from having a single-screen for viewing the internet, tv, and movies.  Many of us already connect our TV’s to our computers (Windows Media Server), use Slingbox, have had Tivo for years, use BitTorrent and perhaps even consume content from the sites I listed at the beginning of this post.  Companies like Visible Measures, Zango, VidMetrix, and Maven Networks already provide some flavor of a video measurement solution too.
• Event models provide the conceptual and logical framework for measuring interactions that are subordinate, equal, or a replacements for the page view.  Without getting into much detail, “events” are interactions such as the play, stop, pause in a video stream, or the pan, zoom events in a online mapping experience.  In order to articulate the instream video experience, you should understand what an event model is and how it applies in Web Analytics 2.0.
• KPI’s.Based on business goals resulting from site strategy, you can build KPI’s related to instream and outstream video measurement.  For example:

Instream:

• Percentage high duration streams
• Percentage medium duration streams
• Percentage low duration streams
• Average viewing time per stream/overall across all streams
• Percentage visits who complete stream
• Percentage visits that stop stream within 10 seconds
• Percentage visits when this stream was the last video viewed
• Percentage visits when this stream was the first video viewed

Outstream:

• Conversion rates by video filetype, video topic, channel, taxonomy node, referrer, geography, keyword, and so on
• Average streams per visit
• Percent visits/views from different channels (such as email, organic search, paid search, direct, offline)
• Average time since last stream/video downloads
• Average time between stream/video downloads
• Repeat visit rate for visits involving a stream/video download

The Internet has come a long way since I saw my first streaming video over 9 years ago (VIVO for those old timers out there).  The options for consuming video content over the web are growing everyday (and not at all limited to YouTube, ay?).  I firmly believe video on the Internet is still in its infancy, and video measurement technologies both inside and outside of “web analytics” are quite embryonic.  What a huge space for growth! 

As the internet-originated video becomes even more pervasive for home entertainment and for business communication, companies will need to employ analysts who know how to create frameworks measuring video content.  Do you?