What’s the future hold for Web Analytics in 2008?  Here are a few predictions:

• Google Analytics releases a real API for getting (and perhaps setting) data.  As you know, I think GA is a fine tool for web analytics, but has severe limitations when you want to control over your data or to feed data into other systems.  Thus, I predict Google Analytics will go beyond the “Tracking API” and release a real API that allows you to at least get data out of the tool (if not set data as well).  Think of what Feedburner does with their REST-based Awareness API.  Wouldn’t that be nice to have with GA?!
• HBX Analytics goes away.  I’d be more than a bit nervous if I were an HBX customer because Omniture is going to sunset HBX and migrate everyone to SiteCatalyst, then try to aggressively sell them the rest of the suite. 
• Long live Visual Sciences.  VS is a powerful tool quite superior in some regards and very different than anything else Omniture offers.  It’s also real in-house software, not some blackbox.  VS’ extensible schema, flexibility in reporting, scalability, and performance is quite unparalleled in the industry.  I can’t envision Omniture killing it (unless they peel it apart in order to create Discover 3), like they will HBX. 
• WebTrends rebrands.  I’m not sure if you agree, but imho WebTrends Marketing Lab was an attempt to rebrand WebTrends.  I expect that interim management will continue attempting to differentiate WebTrends by rebranding products and perhaps the entire company.
• New and updated standards are released.  As a member of the IAB’s Measurement Council I can tell you that the IAB is getting ready to release the IAB Audience Measurement Reach Guidelines, which attempt to clarify and take a stand on various aspects of server/client-side analytics and audience measurement.  I also envision the WAA increasing the number of terms they define.  But standards are just dandy and quite meaningless unless they are adopted… thus…
• Standards enforcement is attempted in order to propel adoption. Existing and forthcoming standards will be enforced in 2008.  Enforcement from the WAA will probably come in the form of a publication of a matrix or documentation citing which vendors adhere to the standards and to what degree, what’s missing, what’s different, and so on.  If decision-makers who control budgets believe in standards, this type of document will cause the question ”do you adhere?” to be asked.  If vendors start losing deals because the answer is “no, not at all,” vendors will adopt the standards. 
• Internal data integration becomes more important for companies and problematic for ASP’s.  When we talk about “integration” I often think people can be a bit shortsighted.  They want to integrate data from other third-party services and tools (like Salesforce.com and their ad server).  While there is certainly real value in integrating external data with web analytics data, significant value comes from integrating web analytics with internal data, such as data residing in internally-hosted CRM systems, finance, subscription, and lead generation databases. Most vendors have barely figured out how to deal with detail-level external data integration in 2007, even though many customers are demanding it.  I expect that in 2008, internal data integration will be more commonly demanded and even more problematic for ASP’s. 
• BI tools provide better support for and integration with Web Analytics tools.  The current allotment of “enterprise” level web analytics tools are inferior to the capabilities provided by business intelligence tools from companies like Business Objects or Cognos.  Expect these BI vendors to create features for dealing with web analytics data in 2008.  Either that, or these web analytics tools need to grow up and learn a few things from BI. 
• Web Analytics as performance management.  KPI-based site optimization means using data to guide the modification of user experience to deliver on goals.   Since goals are measurable and can be plotted against performance, it’s totally logical to use web analytics as a performance management tool.  Expect to see that gestalt in tool usage come into vogue and be discussed more in 2008. 
• Web Analytics as part of business process automation.  Having the marketing department fielding page tags with campaign codes may work for some (small) companies, but when you work for an enterprise with thousands of clients and simultaneous campaigns across multiple channels, endemic tagging and subsequent tool configuration becomes challeging.  As part of the web analytics process, I expect to see tools support some level of business process automation enabling web analytics.
• Features for measuring the Mobile Web.  Right now, with a log file based tool, I can segment out Mobile traffic based on user agent.  If I want to use a page tag, I have to consider js limitations.  The mobile web is the next frontier, and I only know of one web analytics vendor who is doing a decent job measuring it right now, so I expect to see more features released this year for measuring Mobile.  

So that’s that.  Like a band named PIL once said in the song called Rise “I could be wrong, could be right!”  Am I off-base, misguided, accurate, do you disagree, agree, then let me know… I’d love to hear your thoughts and your predictions for Web Analytics 2008…

Every month I write column for MediaPost.  This month I wrote  a short summary piece I thought I’d share with you in case you missed it.  Here it is:

As 2007 ends, I thought it worth looking back, from the practitioner perspective, at just a few of the issues that have shaped Internet measurement and thus online metrics over the last year:

• The Page View is Dead, Long Live the Page View.  During 2007, technologies like AJAX and Flash continued to erode the construct of the page view.  These technologies render content in a browser but do not always make requests to the server to do so.  If the technology you are using to measure behavior requires the page request and you do not have a page request, what do you measure?  The major vendors of online metrics tried to answer that question. 

Various audience measurement companies claimed “total minutes” and other time-based derivatives were better alternatives to measuring the page view.  Web Analytics companies rolled out features for measuring “events” subordinate or equal to the page view (and highlighted already existing time-based metrics).  Ad serving companies made inroads in reconciling view-through to assist advertisers in understanding the latent effect of ad exposure on the purchasing lifecycle.  Yet all these technologies still count and report page views.

• Engagement, Engagement, Engagement.  One of the hot topics in 2007 was a carryover from 2006.  Definitions for “engagement” emerged from the worlds of advertising, social media, online metrics, and more.  Engagement has been described as “turning on a prospect to a brand idea enhanced by the surrounding context” to “repeated, satisfied interactions that strengthen the emotional connection a customer has with the brand” to “apparent interest” to the more metrical “estimate of the degree and depth of visitor interaction against a clearly defined set of goals.” 

“Engagement” is very specific to the site being measured and full of nuance.  This fact has led agencies, consultants, and various companies to create complex engagement indices consisting of measures of key behaviors.  Behaviors are tallied and segmented in order to calculate an engagement metric, which is then used as the basis for site optimization.  These indices go far beyond often-cited simple time-based measures of engagement.  For a well-thought-of example, see Eric Peterson’s Engagement Metric.

• Cookie Deletion, Again!  Jupiter Research, in 2005, first uncovered and quantified how cookie deletion can affect unique visitor numbers in web analytics systems.  The effect of cookie deletion is not quantifiable in the basic way audience measurement companies want to quantify it in 2007 – by only examining cookie deletion rates from self-selecting panelists who visited one portal site and an ad server. 

Cookie deletion behavior varies greatly by audience segment and by site.  It may be as much of an accuracy problem in web analytics as selection bias and coverage errors are in panel measurement.  It is worth noting that some audience measurement firms use cookies to collect panel data. 

• Black Box Audience Measurement.  Many questions were asked about whether audience measurement companies adequately measure “unique visitors” or “unique users” or just the frame of self-selecting “unique panelists.”  In audience measurement, counts of “unique visitors” are generated using complex, black-box mathematics that project observed metrics to the entire online universe.  The projections are always unequal to the same data provided by other audience measurement companies or web analytics tools.  Panel inconsistencies (across at-home, at-work, at-university, or specific to the geography being measured) may cause some level of bias and error. 

Accounting for the difference between actual, observed panel metrics and projected metrics is perhaps even more challenging to clarify and resolve than the measurable effect of cookie deletion. 

• The Continuing Need for Standards Enforcement.  2007 was the year two significant industry bodies continued working on standards related to online metrics: the Internet Advertising Bureau and the Web Analytics Association.  While each organization serves the needs of different constituencies, they both share the inability to enforce standards.  Both bodies can say what you should do, but not what you “must” do. 

Throughout 2007, these issues (and others) brought increased attention and scrutiny to online metrics.  Corporations are inextricably linking online metrics to site and channel strategy and performance, and thus to overall corporate profitability.  The “numbers” are now more important than ever for managing an online business and maximizing online revenue.  Nevertheless, questions are still being asked about accuracy, precision, usage, and sources of online metrics.  We have a ton of collaborative work to do in 2008 to provide the best answers and numbers we can. 

Happy New Year!

Web analytics tools can be outgrown, like houses, clothes, shoes, music, books, and ways of thinking about the world…  But how do you know when you’ve outgrown your web analytics tool?  In part 1, I began the list of five symptoms of an outgrown web analytics solution, which was spawned out a preso I recently gave.  The five symptoms include:

• Inadequate segmentation (covered in Part 1)
• Poor visualization (covered in Part 1)
• No custom reporting (covered in Part 1)
• Limited Integration 
• Cost 

So without further adieu, here’s the rest of the list and some thoughts regarding these symptoms (click here for Part 1):

• Limited Integration.  Soon after deploying a web analytics solution, you will become intimately familiar with clickstream data and simple counts of things (like page views) and measurements (like time).  You will hopefully have deployed Key Performance Indicators for understanding how effective your web site is at converting visits and meeting defined business goals.  Depending on the web analytics tool you use you may even have insight into the behavior and KPI’s of visitors from online channels like newsletters, search, and rss because you have applied “tracking codes.”

Soon will come a time when you may ask yourself how do I integrate data from other data sources?  You may want to bring data from an email service provider, CRM system, and registration databases, so that you can see delivery rates next to conversion rates from newsletters, or so that you may pass behavioral information about a visitor who registered on your web site.

You may want to move data out of your web analytics system.  Perhaps you want to feed your data warehouse?  Or you may want to feed web analytics data into a targeting system.   Simple XML-based feeds from a hosted solution may not suffice.  You will need access to your data in a open database. You may even want to stop non-human readable text and character strings from appearing in your reports. To do so you may need to lookup data using various methods in order to make reporting comprehensible.  All of these goals require some level of integration.

If your current web analytics tool can’t:

• Provide insight into all online channels
• Enable you to bring data from other systems into your web analytics platform
• Pass Web Analytics data to other systems.
• Manipulate data by looking up values, resolving urls, and decoding parameters

And do all of this at a reasonable cost in a maintainable way using in-house resources, then you may have outgrown your web analytics solution.

• Cost.  Web analytics done right isn’t cheap.  It costs money to maintain and extend whether you run an in-house solution or external solution.

When running an in-house web analytics costs are spread out across hardware and software and resources:

• Software license.
• Recurring maintenance costs.
• Servers (one or more).  Perhaps you virtualize (it cost money for the virtualization software).
• Database license(s).
• Storage.
• IT resources - people like project managers, application engineers, and dba’s.

As you expand your web analytics operation, all of these resources and technologies will need to scale.  Time will need to be devoted to maintaining it all, and time costs salary dollars. 

Your company will, hopefully, grow.  Then you will have more sites.  These will need to be tracked.  New reports will need to be created, tested, and rolled out into production.  New data and systems integration requirements will spring up.  All this has cost.

On the other hand, if you are using a hosted solution, you will need to extend the page tag and tool configuration when you want use more features or integrate systems and data.  That means spending money to use vendor professional services or consultants, unless you want to dedicate internal resources in IT who may already be overburdened.

At some point you say enough is enough. The COST is too much!  You then decide to invest in well-negotiated vendor solution that provides a lower TCO over some horizon.  When you start to run up against the barrier of cost, you may have outgrown your web analytics solution.

So that’s the list.  I can think of many more reasons why companies outgrow their web analytics tools…  What have I omitted?  What do you think?

Web analytics tools can be outgrown by companies, just like pants can be outgrown by people.  Over time, an analytics tool may no longer fit organizational needs or be well suited to deliver on complex organizational requirements for site optimization and multichannel integration (among other things). 

This topic led me Silicon Valley this week thanks to an invitation from Unica I headed over to Webex headquarters to record a preso in their new LiveStream studios.  A few other folks also participated in the production.  In attendance was Fireclick founder and all around cool guy, Steve O’Brien (also VP Internet Marketing at Unica), my pal and fellow blogger, Avinash Kaushik (of ZQ Insights and MarketMotive), the genial Elana Anderson (founder of NxtERA marketing and former VP Marketing Research at Forrester), the excellent bloke and all around nice guy from across the pond, Dr Alan Hall (Director of Analytics at SCL Analytics), forthcoming author and savvy multichannel marketer, Akin Arikan (Analytics Evangelist at Unica), and the jet-setting smarty Karen Hudgins (marketer at Unica).   We all had a blast getting down to business at Webex studios.  And eating at places like Burk’s and Parcel 104 - both excellent restaurants.

The title of my preso was “Symptoms you’ve Outgrown your Web Analytics Tool,” such as:

• Inadequate segmentation
• Poor visualization
• No custom reporting
• Limited Integration (covered in Part 2)
• Cost (covered in Part 2)

While I did manuscript the speech (hey I was being recorded!), it’s way too long to post, so I figured you all might enjoy me paraphrasing my own content.  So as I sit here on a Jetblue redeye, here it goes:

• Inadequate Segmentation.  Segmentation in web analytics describes the activity of categorizing and dividing your online audience and customers by their various attributes.  For example, you might choose to segment your audience based on their demographic location information to determine if a visitors from a certain geography have a higher conversion rate or behave differently on your site than visitors from another geography.  Or you may choose to use your web analytics tool to define a segment that you want to track such as visitors who clicked on a paid search term and did not convert, but came back to the site within one week.  Sounds easy, right? 

But not all web analytics tools can segment data.  The proprietary tool you run in-house may not be able to segment data.  Your expensive vendor solution may not be able to segment data easily.  Many tools only provide simple reports.  Yet basic reporting is insufficient for web analytics.  In order to understand new data relationships and the effectiveness of marketing campaigns to your massive online audience, you need to a web analytics tool that can segment data.

The idea being to do what I describe in this post on web analytics segmentation:

• Define a segment
• Identify expected segment behavior. 
• Measure current segment behavior. 
• Create “optimization hypotheses.” 
• Optimize content, offerings, user experience, and other site elements. 

How does your web analytics tool fit into the process of segementation that I described?  Does it?  Can your tool assist you in this process?  If not, you may have a nice IT tool that reports web metrics, not a marketing tool that enables you to optimize your site and landing pages to offer the best possible messages to known online segments.

• Poor Visualization.  Pictures are worth a thousand words.  Your stakeholders are already overwhelmed with data before ever presenting reports with a whole bunch of numbers.  Not everyone is quantitative.  Some stakeholders just want to be able to quickly digest data, and they prefer an aesthetically pleasing visualization instead of a spreadsheet. 

Data visualization helps stakeholders interpret important data at a glance.  Visualization helps reporting comprehension.  Good visualizations are important when you want to:

• Highlight key trends in the data
• Compare counts of things
• Identify multidimensional relationships using cube visualizations

If your tool can’t visualize your web analytics data, and you need that visualized data to assist comprehension, act as sales tool in a presentation, or as marketing collateral in a report, you have outgrown your web analytics tool.

• No Custom Reporting.  An acute inability to deliver customized reporting that meets the needs a diverse group of stakeholders is one of the signs you’ve outgrown your current web analytics tool.  The problem manifests itself in sheer frustration because people can’t get the data they need. Over time this will cause people to lose faith in web analytics because the data isn’t relevant to their jobs.  Some of the symptoms include:
  • Problems creating KPI’s.  To manage online performance, you need to be able to define Key Performance Indicators.  For example, you may want to define a view:visit ratio, that is the number of page views generated per visit.  You need to define this equation in your web analytics tool.  If you can’t define such simple KPI’s, you are limiting your success at web analytics.
  • You may not have reporting that identifies conversion rates and allows you to define custom metrics for channels like RSS, Newsletters, and Internal and External Search.  A powerful web analytics tool will be able to build custom reporting for conversion rates and other KPI’s by online channel.
  • You may have a limited ability to build reports with filtered data, such as viewing reporting of top pages on a particular day or combinations of days, or filtering data by referrer, geography, or time.
  • No ability to add core web analytics dimensions to your reports, such as creating and saving a report that shows all referrers, their keywords, conversion and bounce rates for each city in the United States.
  • Quite simply, you may only have one set of reports and you can’t build new ones at all.

In the real world practice of web analytics, you need a web analytics tool that has the ability to build as many custom reports as you want when doing analysis, to filter, add metrics, to use dimensions, do AB tests, and save all that stuff until your heart is content so you can meet business goals.  You can’t be restrained by the inability of your current tool to create custom reports. If you are you may have outgrown your web analytics tool.

I’ll post part 2 this weekend as I recover from the jet lag…     Part 2 is up, click here to view it! 

I write a monthly column for Mediapost’s Metrics Insider.  This month I wanted to talk about the different schools of thought in online metrics because at the end of the day we are all in Internet measurement together. Hope you enjoy the read:

Audience measurement and Web analytics systems are like the yin and yang of online metrics. Yin and yang are different, opposing forces, but they also complement each other. Think of Web analytics and audience measurement data in the same way: different, sometimes in opposition, but complementary.

The major difference between these systems is data collection:

• Audience measurement companies don’t collect data directly from the sites being measured. They all rely on proprietary methods. Hitwise gets data from ISPs. Compete uses a toolbar that you can download as well as ISP and panel information. Nielsen and comScore use data collected from panels to create online metrics that they believe accurately represent overall Internet usage. Due to all these different data collection methods and no shared standards across companies, metrics from audience measurement firms are never identical with each other.
• In Web analytics, data is collected directly from actual site activity. Methods include client-side data collection via javascript page tagging, server-side data collection via log file processing, or network data collection via packet sniffing. Sometimes methods such as page tagging and log file processing are combined in what’s called “hybrid data collection.” Vendors include Coremetrics, Webtrends, Unica, Visual Sciences, Omniture, Google, and others. The challenge with Web analytics tools is that each tool will calculate different numbers from the same source for identical metrics. In other words, Omniture numbers won’t match Google’s. That’s because each tool has its own “secret sauce” for “sessionization” — the fancy term for the way metrics are counted and measured by analytics technology. For example, certain tools may be configured to include or exclude certain filetypes or server responses. Robotic traffic may or may not be filtered.

It’s worth noting that a company named Quantcast uses panel data and also enables a site to add page tags to collect actual site data, which are then merged together in a completely different type of “hybrid” model.

All these different approaches to data collection lead to opposition when these systems are used for the same purpose. For example, conflict arises between the yin and yang when identifying reach using unique visitor metrics. Audience measurement firms may cry “cookie deletion” when analytics tools are used to count unique visitors, and Web analytics firms may shout back “coverage error” and “selection bias” at the unique visitor numbers from panel-based firms. Another area of opposition is demographics. I’ve been told that only audience measurement firms provide demographic data, and that you can’t get demographic data from Web analytics systems. That’s not true at all.

All enterprise-level Web analytics systems provide demographic location information at the country, city, state, and MSA levels. This information will be different than that provided by audience measurement companies.

Demographics that are harder to elicit from a Web analytics system, but are easily provided by audience measurement, include attributes like a visitor’s age, gender, occupation, income, and education.

But it is possible to integrate very detailed demographic attributes per visitor into a Web analytics system! Once demographic information is captured in a registration database, it can be joined with behavioral data in the Web analytics system and reported on. For a real-world example of analytics/demographic integration, take a look at what Microsoft is doing with Gatineau, the company’s free Web analytics offering currently in beta. Microsoft is joining Web site behavioral data with rich demographic data from MS Live profiles.

Even with differences and oppositions between these online metrics systems, companies find ways to use the data in complementary ways:

• Audience measurement data is useful for competitive intelligence. All the paid and free services provide data for comparing the performance of a site to other sites, for understanding audience behavior across one or more sites by demographics, and for understanding generalized Internet traffic trends and search terms.
• Web analytics data is useful for understanding site effectiveness, for defining key performance indicators, for determining conversion rates for marketing campaigns by channel (such as search, email, rss), for understanding what sites and keywords are driving traffic to your site, and for segmenting and reporting online metrics.

You can even use both data sources as part of the same site optimization activity. For example, you could use audience measurement data to determine that a competitor is gaining ground on a particular product or search term. Then you could look at your Web analytics tool to see how you’re doing for the same term and how visitors who searched for that keyword behave on your site. You may find a high bounce rate and low conversion rate for the keyword, so you segment that data perhaps by demographics! Next you suggest a hypothesis to minimize bounce and maximize conversion for each segment. Then you test your hypothesis, and reexamine the data. Based on the results, you then continuously improve your online performance through controlled experimentation. At the end of the day, you will drive more online revenue by understanding how the yin of audience measurement and the yang of Web analytics complement each other, than by worrying about how they differ and oppose.

Market segmentation existed long before web analytics.  It’s a method for dividing a population into specific groups (segments) that share one or more characteristics.  The goal of segmentation is to maximize future value of that segment by optimizing your marketing mix.

Segment analysis will tell you different things about your audience than you will realize from studying overall population metrics.  In traditional market research, segments are created from demographics (such as age), psychographics (such as attitude), geography (such as zip code), behavior (such as usage patterns), and value (revenue earned and cost).

Using a web analytics tool to segment your online audience requires a bit of upfront thinking and requirements gathering before getting down to business.  Like all things web analytics, segmentation requires process.  Here are some tips that may help you create a process for web analytics segmentation:

• Determine your business objectives.  Like everything in web analytics, you can’t optimize what you haven’t defined as a goal.  A business objective driving segmentation might be to “increase conversion rate (over historical numbers)” or “to improve frequency” by offering something valuable to that segment.
• Define segments. Basic dimensions for segmentation in web analytics include: new visitors, repeat visitors, geography, time, referrer, keyword, and campaign type.  Many more dimensions and attributes can be used for segmentation too.
• Identify expected segment behavior.  By reconciling goals, historic performance data, and VOC research, you should be able to identify the expected behavior of the segment.  If your business objective is to “increase conversion rate,” your expected segment behavior might be to “complete the form” or “click on a link.”
• Measure current segment behavior. Sounds easy, right, but it will take system configuration and the right tool.  Pages may need to be (re)instrumented, tracking codes may need to be applied, query string parameters may need to be parsed, and in the worse case dimensions you want to segment or the metrics you may want to measure against may not be available in your web analytics tool.  For example, how would you use your tool identify the “conversion rate” for a segment of repeat visitors from newsletter X who came from Tokyo and previously downloaded a whitepaper?
• Create “optimization hypotheses.”  Once you’ve measured current behavior, create a hypothesis or hypotheses to test in order to optimize the behavior.  You may want to perform controlled experimentation whether a simple AB test (i.e. champion/challenger), multivariate test, or experience test.  For example, I may have detected that repeat visitors from Newsletter X responded better to Y offer after being exposed to a certain element than those visitors in the same segment who did were not exposed.  That element could have been a content theme, offer, call to action, creative, and so on.  Thus, I might create a hypothesis to test that combines elements of the user experience that I feel are key to persuading the behavior and thus fulfilling the business objective.
• Optimize content, offerings, user experience, and other site elements.  Based on your hypothesis, make subsequent changes to the elements that you think will drive the desired segment behavior.  For example, you may split traffic to two landing pages each with a completely different offer, creative, and call to action.  Or you may choose to switch out specific elements on one landing page (such as an image or call to action) using multivariate methods just to get Visitor X to “complete that form” or “click that link” to improve your “conversion rate.”
• Analyze segment behavior against hypothesis.  How did the segment perform against expected behavior and business objectives based on testing your hypotheses?  Tools that provide drill-down/drill-up and cross-dimensional capability allow to analyze segments and answer such questions. The tools I’m talking about are advanced and powerful, such as Unica NetInsight, Visual Sciences Visual Site, Omniture Discover, and WebTrends Marketing Warehouse.  Capabilities for segmentation analytics vary by tool, so make sure to dig deep into the offerings because not all tools with let you correlate metrics like “conversion rate” with dimensions like “keyword,” let alone build complex multi-dimensional segments.  In fact, some free web analytics don’t allow you to segment data at all (just filter it)!
• Go with what works.  Web analytics segmentation analysis will let you know what appeals to and works for a segment.  Success with web analytics segmentation means that you met your business goals and improved key performance of that segment.  Rinse, lather, and repeat the segmentation analysis and optimization process so your campaign outperforms and margins soar!

As a result of well-executed web analytics segment analysis and hypothesis testing you can realize new value in your customers and new opportunities in your campaigns. 

Measuring video content with web analytics isn’t super difficult, but it has its nuances and challenges.  I’ve been thinking a bit about it lately, and have had some good conversations with a few people.  Folks I know are playing around with the likes of Joost, Vuze, and Hulu, TVUNetworks, as well as using BrightCove and Videoegg.  And, man, the popularity of BitTorrent and other swarm structure 4th gen P2P networks is larger than ever.

Simply speaking video measurement can be divided into the following types:

• Instream measurement.  Refers to measuring the video itself and the various abstract elements of the video experience, such as duration metrics (average viewing time) and interaction metrics (number of stops, plays, pauses, rewinds, fast forwards, and clicks on video content).
• Outstream measurement.  Refers to measuring the content environment and user experience surrounding the video, such as the conversion metrics (percentage of visits downloading or viewing a video), behavioral metrics (referrers to the video page, players used), and content metrics (percentage videos per channel, percentage videos viewed by topic, percent videos viewed by file type). 

By categorizing the web video analytics into these two buckets, you are better able to answer meaningfully the following questions, which must be considered prior to any rollout:

1. What are the business objectives for rolling out video features on the site?
2. What format are the videos in?
3. Are the videos downloads or streams?
4. Am I using a content distribution network or streaming video network?
5. Does my web analytics tool have the features necessary for video measurement? Or should I look for a third party, niche vendor?
6. What data collection method should I use?
7. Do I understand event models?
8. What KPI’s are relevant and important based on my business goals?

To help you formulate answers to those questions, here’s some thinking:

• Business objectives.  You, the analyst, must understand why your company is rolling out video.  In other words, what’s the goal and what strategy underpins the goal?  While video is “the rage” right now, simply rolling out video because “everyone is doing it” is no strategy (though doing so may yield a strategy ;).  A goal for video deployment could be “to generate leads,” thus you measure the scenario conversion rate for the funnel resulting in the lead generation and video download (outstream video analysis).  The objective might be “to keep visitors on the site longer,” then you would measure duration and interaction (instream video analysis).  As you all know, I firmly believe that it the business goal that allows you to contextualize what you’re measuring so that you may build KPI’s.
• Video format. Lots of different video file types exist: mpegs, qt, mov, swf, flv, avi, wma, ra, wmf, mp4 and more.  You’ll need to identify the video types you want to track so you can configure your web analytics tool to measure them.  Removing or adding filters or changing your tag’s javascript might be necessary. 
• Download or streams.  Videos can be downloaded (by right clicking) or spawned in a media player.  They can also exist embedded on the page or in another object for on-page streaming.  Thus, the way you instrument your pages will differ based on the way you present the video content. For example, if you are streaming videos, you may want to use javascript (or a vendor provided scripting language) to instrument your pages to track the video.  If you are just hosting downloads, you may simply want to run your logs to detect the number of times videos were downloaded.
• Content distribution network or video network. If your video content is distributed by a CDN or a video network, you will have to apply page tags on all the pages rendered by combining your server’s content with the content served by the CDN. Some video networks provide basic reporting that you can extend with a client-side page tagging solution.  Alternatively, you can process the logs provided by a CDN. The challenge with CDN log file processing is that you will most likely not be able to merge the data with your log files for the same site, resulting in two “profiles” of analytics data related to one site: one profile with the site analytics data and one with the CDN analytics data.
• Data collection method.  If you’ve read this far in my blogivation, you probably picked up that the data collection method you have at your disposal will constrain or enable the way you measure video.  Page tags will enable you to instrument your pages with onclick functions that pass values to the javascript and in turn to the analytics server.  Packet sniffers and log files enable you to measure downloads without modifying code.   If you need modify your web analytics tool or tag configuration to track video filetypes, you can reprocess logs to access the data.  With tags any data related to downloads or interactions with the video object prior to the config change will be lost.
• Web analytics tool features. Many web analytics tools will allow you track a video play or download in your page view reports, but only two tools support true event models: Unica NetInsight and Google Analytics.  At Emetrics San Fran in May 2007, Ian Houston and I gave a preso on “from page views to events.”  It looks like the vendors agreed, ay? ;)
• Third party tools.  With the convergence of internet and television, we’re not many years away from having a single-screen for viewing the internet, tv, and movies.  Many of us already connect our TV’s to our computers (Windows Media Server), use Slingbox, have had Tivo for years, use BitTorrent and perhaps even consume content from the sites I listed at the beginning of this post.  Companies like Visible Measures, Zango, VidMetrix, and Maven Networks already provide some flavor of a video measurement solution too.
• Event models provide the conceptual and logical framework for measuring interactions that are subordinate, equal, or a replacements for the page view.  Without getting into much detail, “events” are interactions such as the play, stop, pause in a video stream, or the pan, zoom events in a online mapping experience.  In order to articulate the instream video experience, you should understand what an event model is and how it applies in Web Analytics 2.0.
• KPI’s.Based on business goals resulting from site strategy, you can build KPI’s related to instream and outstream video measurement.  For example:

Instream:

• Percentage high duration streams
• Percentage medium duration streams
• Percentage low duration streams
• Average viewing time per stream/overall across all streams
• Percentage visits who complete stream
• Percentage visits that stop stream within 10 seconds
• Percentage visits when this stream was the last video viewed
• Percentage visits when this stream was the first video viewed

Outstream:

• Conversion rates by video filetype, video topic, channel, taxonomy node, referrer, geography, keyword, and so on
• Average streams per visit
• Percent visits/views from different channels (such as email, organic search, paid search, direct, offline)
• Average time since last stream/video downloads
• Average time between stream/video downloads
• Repeat visit rate for visits involving a stream/video download

The Internet has come a long way since I saw my first streaming video over 9 years ago (VIVO for those old timers out there).  The options for consuming video content over the web are growing everyday (and not at all limited to YouTube, ay?).  I firmly believe video on the Internet is still in its infancy, and video measurement technologies both inside and outside of “web analytics” are quite embryonic.  What a huge space for growth! 

As the internet-originated video becomes even more pervasive for home entertainment and for business communication, companies will need to employ analysts who know how to create frameworks measuring video content.  Do you? 

A successful web analytics practice helps a business manage its performance toward goals.  If you believe the statement, then you understand that in order to manage for site performance, a web site must exist to support one or more goals.  Pretty logical, right?

Then why is that web analytics practitioners tell me they often encounter web site owners who have no clear, measurable goals?  It’s really strange.  In fact, it’s vexing and frustrating because without known goals you OBVIOUSLY can’t manage site performance.  Without goals you can’t really optimize anything and are left to simply tracking trends related to basic metrics and/or derivatives thereof.

Thus, one of the responsibilities of a web analyst is identifying and confirming site goals.  Once you have site goals, you can create KPI’s that you monitor on a regular basis investigating variances, anomalies, and outliers affecting site performance. 

Undoubtedly one of the goals you will identify for your site is some type of conversion.  A conversion is a value-generating transition on your site.  For example, the successful completion of a form that enables a visitor to download a content asset or completing the purchase of a product may be a type of conversion you measure. 

While overall site conversion rates for value-generating events are important to know, real insights come from segmenting visitors or other dimensions by conversion rates.  The best analytic tools enable you to identify the conversion event and slice the conversion by any dimension: page, geography, referrer, keywords and so on. Thus measures of conversion will be instrumental KPI’s when using analytics to manage performance. 

The notion of a “funnel” in web analytics is a sequence of one or more pages that a visitor clicks-through until reaching a final destination, the conversion event.  The “funnel” assists the analyst in understanding the discrete clickstream that led to conversion.  For example,  a pre-defined path through the pages in the subscription or download process could be considered a funnel.  Funnels can be linear and non-linear and are affected by all sorts of things like detours, fall out, abandonment… That’s a post for another topic…  Yet, the metaphor of the funnels is applicable across all sites…  This notion becomes problematic when we consider multichannel attribution of the conversion process (again another blog post when I have time).

So what’s an analyst do when they want to begin to use web analytics to manage performance against goals? Here’s are some tips:

1. Investigate the business’ revenue model. Advertising-based sites generate revenue from selling various types of ad units (rpm/cpm), contextual advertising (rpm/cpc), lead generation programs (rpl/cpl), revenue sharing, and via affiliate syndication and content sharing deals.  Ecommerce sites generate revenue by selling products or brokering services or transactions.  How does your site generate revenue?  If the goal of the site isn’t to generate revenue, then skip this step.
2. Ask key managers to identify business goals.  Top-level managers have a better grasp of the vast ecosystem of suppliers, buyers, and other priorities that you, the analyst, may not be privy to.  Your manager should be helping you put your analytics work in context of the business goals.  So ask your boss what are the site goals?  Don’t accept the answer “to drive revenue.”  Ask how the identified goals support value creation and revenue generation.  The measurement of events supporting business goals should be a focus for performance management and optimization. 
3. Identify the conversion events that support businss goals and the revenue model, including any necessary steps in the funnel.   The actions that satisfy goals are the conversion events. The transition involved when the visitor clicks and makes the site money is your discrete conversion event. The page immediately preceding the conversion event is the last step in the funnel. 
4. Determine the actors on the site.  Actors can be categorized into internal/external actors.  For this exercise, concentrate on identifying the roles and responsibilities of the internal actors who DIRECTLY influence site production.  In other words, who are people modifying the site and what do they do?  The indirect actors, like your boss, also affect the site, so make sure you consider their role and responsibility in advanced site goals and fulfilling the objectives of the business model. 
5. Determine the goals of the actors.  Like site goals based on revenue, all indirect and direct site actors will have goals specific to their jobs.  Actor goals support site goals.  Thus, actor goals can be translated to tactical KPI’s.  For example, the editorial actor may want to ensure that X number of newsletter-referred visitors subscribe to the print magazine, so you create a KPI’s “subscription conversion rate by newsletter” and “number of online subscriptions generated per newsletter.”  Based on the site goals for conversion and the number of subscriptions generated from the online channel, you can start managing editorial performance.
6. Document site goals, actor goals, conversion events, and funnels, including a diagram a hub-and-spoke model of actor roles and responsibilities and flow diagrams of funnels.  In order to establish a process for performance management via web analytics, all the actors must generally agree on roles and responsibilities.  By documenting your investigation, you confirm correctness, identify gaps in business process, and create alignment among actors and management.  You may notice breakpoints in site production processes too.  The end result is a fully-documented operational model of how your site is created, monetized, and deployed.  In the same way that you can’t manage what you don’t measure, you shouldn’t be measuring things you can’t manage.   
7. Have key managers who direct site actors sign-off approval on the documentation.  The holy sign-off confirms you correctly identified the revenue model, site and actor goals, site navigational flows that lead to conversion.  When questions arise you can reference this process artifact to backup the conversion events you defined, the KPI’s you’ve created, and subsequently the performance recommendations you will make and manage.

Then:

1. Configure your web analytics tool to report conversion rates for revenue-generating site transitions and events and to report on funnels.  All tools can do this. 
2. Build KPI’s based on specific functional goals performed by actor’s on the site. Base KPI’s around activities that support the core revenue model and the activities performed by site actors.
3. Review KPI’s with actors.  Bring your documentation and identify how the KPI’s will be used to identify performance, contextualize optimization recommendations, and help each actor be more successful at their job. 
4. Report conversion rates and KPI’s to key managers and site actors.  Optimally these performance metrics should be available for actors and other stakeholders whenever they want them, preferably in the form of dashboards elicitable from your web analytics tool.  The goal should be identified, the target value for the goal, the KPI measurement,  and any deviation from the goal should be noted along with written performance recommendations.
5. Research site performance by segmenting conversion rates and KPI’s and investigating drivers.  KPI’s provide context for understanding the actions that influence site performance.  Overall conversion rate will only tell you so much.  For example, to the SEO conversion by organic search referrer is more informative.  Other actors will require reports on segmented conversion specific to their function. 
6. Make optimization recommendations.  Whether you deliver recommendations via reporting or manage the multivariate testing function at your company, you’ll need identify the events or actions to optimize.  Then you need to get buy-in from various gatekeeping actors.
7. Test and implement optimizations.   Use a multivariate testing vendor to test combinations of content and creative that drive KPI’s and that provide lift in conversion.  Work with site actors to ensure optimization testing and controlled experimentation occurs.
8. Rollout optimization that increase conversion, improve goals, and drive revenue.  Once you are reasonably certain the optimizations you are suggesting will improve performance work with the web or product development team to realize these changes.

Thus web analytics for performance management involves:

• Goal Clarification.  Why does this site exists in the first place?  Don’t be surprised to learn different actors have different goals, and no one is aligned!  From what I hear on the street that’s a common issue!
• Stakeholder Alignment.  Do all stakeholder and actors agree on the reasons why the site exists?  If not, be prepared to mediate.
• Experience Optimization.  How is the visitor interacting with my site, and do those interactions channel visitors to conversion funnels?  Do relevant calls to action and points of resolution work for persuading visitors to convert.  What’s working?  What’s not?  Figure it out.
• Controlled experimentation.  Based on potential optimizations available, what do you test?  Multivariate testing software can help, as can VOC research.  Talk to your research team.  Use the AB testing feature of your web analytics tool…  Whatever you do, you should establish a repeatable process for doing so. 
• Outcomes measurement.  If you set up a KPI dashboard with goals, actual performance, and variances you will be able to answer that question “so I did all this stuff, so what effect did it have?”

Easy right? Now get to managing performance using web analytics!

Last week I attended the 1st Annual SEMPhonic XChange and led a collaborative discussion on “deploying measurement systems in distributed companies.” In case you hadn’t heard about SEMPhonic, the company is a boutique consulting firm in Novato, CA (near one of my favorite towns on Earth: San Francisco).  They do some very unique work with web analytics (Functionalism) and other facets of new media technology.  A fine gentleman named Gary Angel leads the group. He’s recently hired smart folks, like long-time web analytics expert, June Dershewitz, and notable blogger and author of the Web Analytics Report, Phil Kemelor.

A few months ago the idea came up for me to lead a “huddle” at their Xchange conference.  I was intrigued with this huddle idea.  It was different. New.  I’d facilitate a discussion on a topic of my choice in a small, Socratic, group setting.  No PPT!

Fast forward to last week, and I found myself sitting in Napa, CA at COPIA: The American Center for Wine, Food, and the Arts. COPIA was a brilliant place for an industry colloquy.  Being a guy who likes culture (and conferences), this venue offered the best of both worlds.  Instead of a hotel, I found myself surrounded by gardens, vineyards, art, food, wine, web analytics, and the brightest in the industry.  Cool!

Thinking back on the event, SEMPhonic Xchange, in my opinion, is a “must-attend conference.”  Not only is the “huddle” format unique and fun in which to participate, it’s also a format that promotes deep discussion that leads to truly actionable insights. It’s a conference based on dialog and collaborative discussions between participants.  The huddle format provides 10 hours of free collaborative consulting (5 huddles) with employed people you can’t hire (and some consultants you can)!  Compare a daily consulting rate to the conference cost, and it’s a no brainer.

My discussion on best practices for the successful deployment of measurement systems lasted a little over two hours.  My group collaborated nicely (so I thought).  It included very intelligent folks like Jared from Intuit, Scott and Christel from Xilinx, Sami and Fred from Adobe, Renata and Matthew from American Express, Aaron from Webtrends, Rupa from Cisco, Amy from JPMorganChase, Jeff from New England Journal of Medicine, and Kevin from Charles Schwab.  Thanks to anyone reading this blog who attended.  Your valuable insights and knowledge sharing contributed to the success of the huddle.

At the ends of the huddle I went over a tips for a successful deployment.  Here are a few:

• Identify business goals, match those goals to business strategy, and align metrics and KPI’s to support those goals.  I always say “metrics and kpi’s have significance in position and relation to a goal.”  You can’t measure against performance unless you’ve identified your business goals.  Goals are supported by strategy.  You create KPI’s for measuring against goals and for guiding work toward objectives.  Every business unit could have different goals.  Management must align and support the standardized goals that you bake into your KPI’s (and any specific derivatives you create to support the goals of differentiated business units).
• Verify the technology implementation and the metrics collected.  A web analytics or other measurement system must be verified to ensure conformance with “standard” best practices based on the company’s goals.  You need to make sure the numbers can be trusted.  That means, validating data and reconciling differences with historic reporting systems or whatever the business demands, understands (and possibly pays for in real dollars, time, or resources allocated), such as ABCe, WAA, IAB, MRC, XYZ, 123, FBCINSA whatever it takes…
• Provide education and training.  While some analysts want to keep all the data within their analytics group, I don’t think that practice scales across a large enterprise unless you are lucky enough to have a large analytics team.   I know I can’t individually and alone satisfy the metrics needs for thousands of stakeholders or hundreds of brands.  Successful companies deploying on a large scale adopt a “train the trainer” approach.  The trainers guide their business units, become local experts, and help foment a data-driven culture.   Let the data be free and the people educated, I say. 
• Consider the corporate culture.  A metrics tool changes organizational culture (for the better or worse).  Suddenly, everyone is being measured and perhaps evaluated on goals implicit in the measurements.  Some will greet the tool with open arms while others will see it as a threat.  Solid management needs to foster buy-in and support for the tool.  That way, organizational resistance is overcome and clarity of mission is realized.
• Help business units ”use the metrics” to improve performance.  My friend Eric Peterson likes to say “web analytics is hard.”  Yes it is!  That means the expert needs to work with global business teams to mentor and teach how to separate signal from noise.   As the tool is used, business units will identify “pain points” that will need to be addressed.  The analytics team should work with business units heal the pain and improve performance.
• Plan to ”get granular” and the “get integrated” with the measurement system.   Additional requirements will come out of the woodwork due to organizational learning after you golive (even if you think you’ve elicited all reqs prior to deploying your tool and building reports).  New requirements are an early sign of success (people are “getting it” after all).   As the business learns, it will become necessary to extend the system.  Consider brainstorming about the possible ways in which the system should be extended prior to rollout, and perhaps create a basic plan for extending the system. When the system is launched, modify that system enhancement plan and execute on it to support business goals.  Employ a project manager to plan and help execute!
• Manage and guide stakeholder expectations to minimize risk.  Be realistic when for guiding and setting expectations so that you can minimize risk (to your overall mission, your job, the company ;).  Risk comes from incorrect metrics, organizatonal issues with adoption, changes in business goals, shifting managerial priorities, technolog