From an article I wrote for MediaPost a few weeks ago:

Reach and frequency and the core concepts of traditional media planning and advertising.  For a given site, program, channel, radio station, billboard, newspaper section, a target audience (the reach) is exposed to a certain number of occurrences of the media (the frequency).  On the web, these concepts manifest themselves in metrics collected and reported from a number of recognizable services.  Audience measurement firms, like comScore and Nielsen, web analytics firms, like Omniture and Unica, to companies somewhere in between, like Quantcast and Google, all have reach and frequency data.  Many new media metrics can be used to proxy frequency- from time-based measures, espoused by audience measurement firms, to concepts like visitor retention or the repeat visitor rate cited by web analytics firms.  On the reach side, companies refer to concepts like “unique visitors.”

These data, of course, available in free tools or in for pay tools are certainly helpful for planning campaigns.  But reach measures can be dirty (cookies, unduplicated unique users, estimates from panels, coverage error).  Frequency measures can be just as dirty (problems recording time in single page visits or visits on the last page, do page views really matter with AJAX and rich media, cookies again, and so on).  We all are aware of the challenges.

Thus using basic reach and frequency measures for planning or evaluating a campaign does not suffice.   So advertisers and agencies target demographics, like gender, age, income, education, and job title.  It’s a given that advertising in the Robb Report reaches a different audience segment than advertising in Popular Mechanics. 

These brave new days we have “behavioral” tracking too.  By taking into account visitor activity across sessions, such as past actions taken on a site or a roster of previous purchases, we can attempt to deduce what a person or segment responds to or is interested in based on their behavior.

Even with reach, frequency, demographics, and behavioral data to help guide advertising and media buying, we are missing an important attribute for maximizing the potential success of our campaigns.  We do not have an available tool, whether free or paid, for advertising or buying media on or across sites according to measures of past performance.  Such measures include ad clickthrough rates, conversion rates, goal completion rates, delivered impressions, and perhaps even harder to quantify financial measures such as ROI, ROAS, and ROMI.

Sure, historic, tacit knowledge of campaign performance exists and is used by agencies or publishers.  However, there is no shared industry source that can help us answer “how has a site for display advertisement historically performed toward goals based on the reach, frequency, demographic and behavior of its audience segments?”  Interestingly, a company minting money right now, named Google, can masterfully demonstrate performance in paid search campaigning and help advertisers unify it with segmented reach, frequency, and demographics.

Outcomes based performance measurement unified with reach, frequency, demographics, and behavior is what is missing in audience measurement tools, not frequently reported externally by web analytics tools or ad serving tools, and not available in ad planning tools.  When advertisers can target display ads, or even video ads, to desired audience segments by reach, frequency, demographics, behavior in the context of known performance, media planning will be more effective.  

What’s the future hold for Web Analytics in 2008?  Here are a few predictions:

• Google Analytics releases a real API for getting (and perhaps setting) data.  As you know, I think GA is a fine tool for web analytics, but has severe limitations when you want to control over your data or to feed data into other systems.  Thus, I predict Google Analytics will go beyond the “Tracking API” and release a real API that allows you to at least get data out of the tool (if not set data as well).  Think of what Feedburner does with their REST-based Awareness API.  Wouldn’t that be nice to have with GA?!
• HBX Analytics goes away.  I’d be more than a bit nervous if I were an HBX customer because Omniture is going to sunset HBX and migrate everyone to SiteCatalyst, then try to aggressively sell them the rest of the suite. 
• Long live Visual Sciences.  VS is a powerful tool quite superior in some regards and very different than anything else Omniture offers.  It’s also real in-house software, not some blackbox.  VS’ extensible schema, flexibility in reporting, scalability, and performance is quite unparalleled in the industry.  I can’t envision Omniture killing it (unless they peel it apart in order to create Discover 3), like they will HBX. 
• WebTrends rebrands.  I’m not sure if you agree, but imho WebTrends Marketing Lab was an attempt to rebrand WebTrends.  I expect that interim management will continue attempting to differentiate WebTrends by rebranding products and perhaps the entire company.
• New and updated standards are released.  As a member of the IAB’s Measurement Council I can tell you that the IAB is getting ready to release the IAB Audience Measurement Reach Guidelines, which attempt to clarify and take a stand on various aspects of server/client-side analytics and audience measurement.  I also envision the WAA increasing the number of terms they define.  But standards are just dandy and quite meaningless unless they are adopted… thus…
• Standards enforcement is attempted in order to propel adoption. Existing and forthcoming standards will be enforced in 2008.  Enforcement from the WAA will probably come in the form of a publication of a matrix or documentation citing which vendors adhere to the standards and to what degree, what’s missing, what’s different, and so on.  If decision-makers who control budgets believe in standards, this type of document will cause the question ”do you adhere?” to be asked.  If vendors start losing deals because the answer is “no, not at all,” vendors will adopt the standards. 
• Internal data integration becomes more important for companies and problematic for ASP’s.  When we talk about “integration” I often think people can be a bit shortsighted.  They want to integrate data from other third-party services and tools (like Salesforce.com and their ad server).  While there is certainly real value in integrating external data with web analytics data, significant value comes from integrating web analytics with internal data, such as data residing in internally-hosted CRM systems, finance, subscription, and lead generation databases. Most vendors have barely figured out how to deal with detail-level external data integration in 2007, even though many customers are demanding it.  I expect that in 2008, internal data integration will be more commonly demanded and even more problematic for ASP’s. 
• BI tools provide better support for and integration with Web Analytics tools.  The current allotment of “enterprise” level web analytics tools are inferior to the capabilities provided by business intelligence tools from companies like Business Objects or Cognos.  Expect these BI vendors to create features for dealing with web analytics data in 2008.  Either that, or these web analytics tools need to grow up and learn a few things from BI. 
• Web Analytics as performance management.  KPI-based site optimization means using data to guide the modification of user experience to deliver on goals.   Since goals are measurable and can be plotted against performance, it’s totally logical to use web analytics as a performance management tool.  Expect to see that gestalt in tool usage come into vogue and be discussed more in 2008. 
• Web Analytics as part of business process automation.  Having the marketing department fielding page tags with campaign codes may work for some (small) companies, but when you work for an enterprise with thousands of clients and simultaneous campaigns across multiple channels, endemic tagging and subsequent tool configuration becomes challeging.  As part of the web analytics process, I expect to see tools support some level of business process automation enabling web analytics.
• Features for measuring the Mobile Web.  Right now, with a log file based tool, I can segment out Mobile traffic based on user agent.  If I want to use a page tag, I have to consider js limitations.  The mobile web is the next frontier, and I only know of one web analytics vendor who is doing a decent job measuring it right now, so I expect to see more features released this year for measuring Mobile.  

So that’s that.  Like a band named PIL once said in the song called Rise “I could be wrong, could be right!”  Am I off-base, misguided, accurate, do you disagree, agree, then let me know… I’d love to hear your thoughts and your predictions for Web Analytics 2008…

Every month I write column for MediaPost.  This month I wrote  a short summary piece I thought I’d share with you in case you missed it.  Here it is:

As 2007 ends, I thought it worth looking back, from the practitioner perspective, at just a few of the issues that have shaped Internet measurement and thus online metrics over the last year:

• The Page View is Dead, Long Live the Page View.  During 2007, technologies like AJAX and Flash continued to erode the construct of the page view.  These technologies render content in a browser but do not always make requests to the server to do so.  If the technology you are using to measure behavior requires the page request and you do not have a page request, what do you measure?  The major vendors of online metrics tried to answer that question. 

Various audience measurement companies claimed “total minutes” and other time-based derivatives were better alternatives to measuring the page view.  Web Analytics companies rolled out features for measuring “events” subordinate or equal to the page view (and highlighted already existing time-based metrics).  Ad serving companies made inroads in reconciling view-through to assist advertisers in understanding the latent effect of ad exposure on the purchasing lifecycle.  Yet all these technologies still count and report page views.

• Engagement, Engagement, Engagement.  One of the hot topics in 2007 was a carryover from 2006.  Definitions for “engagement” emerged from the worlds of advertising, social media, online metrics, and more.  Engagement has been described as “turning on a prospect to a brand idea enhanced by the surrounding context” to “repeated, satisfied interactions that strengthen the emotional connection a customer has with the brand” to “apparent interest” to the more metrical “estimate of the degree and depth of visitor interaction against a clearly defined set of goals.” 

“Engagement” is very specific to the site being measured and full of nuance.  This fact has led agencies, consultants, and various companies to create complex engagement indices consisting of measures of key behaviors.  Behaviors are tallied and segmented in order to calculate an engagement metric, which is then used as the basis for site optimization.  These indices go far beyond often-cited simple time-based measures of engagement.  For a well-thought-of example, see Eric Peterson’s Engagement Metric.

• Cookie Deletion, Again!  Jupiter Research, in 2005, first uncovered and quantified how cookie deletion can affect unique visitor numbers in web analytics systems.  The effect of cookie deletion is not quantifiable in the basic way audience measurement companies want to quantify it in 2007 – by only examining cookie deletion rates from self-selecting panelists who visited one portal site and an ad server. 

Cookie deletion behavior varies greatly by audience segment and by site.  It may be as much of an accuracy problem in web analytics as selection bias and coverage errors are in panel measurement.  It is worth noting that some audience measurement firms use cookies to collect panel data. 

• Black Box Audience Measurement.  Many questions were asked about whether audience measurement companies adequately measure “unique visitors” or “unique users” or just the frame of self-selecting “unique panelists.”  In audience measurement, counts of “unique visitors” are generated using complex, black-box mathematics that project observed metrics to the entire online universe.  The projections are always unequal to the same data provided by other audience measurement companies or web analytics tools.  Panel inconsistencies (across at-home, at-work, at-university, or specific to the geography being measured) may cause some level of bias and error. 

Accounting for the difference between actual, observed panel metrics and projected metrics is perhaps even more challenging to clarify and resolve than the measurable effect of cookie deletion. 

• The Continuing Need for Standards Enforcement.  2007 was the year two significant industry bodies continued working on standards related to online metrics: the Internet Advertising Bureau and the Web Analytics Association.  While each organization serves the needs of different constituencies, they both share the inability to enforce standards.  Both bodies can say what you should do, but not what you “must” do. 

Throughout 2007, these issues (and others) brought increased attention and scrutiny to online metrics.  Corporations are inextricably linking online metrics to site and channel strategy and performance, and thus to overall corporate profitability.  The “numbers” are now more important than ever for managing an online business and maximizing online revenue.  Nevertheless, questions are still being asked about accuracy, precision, usage, and sources of online metrics.  We have a ton of collaborative work to do in 2008 to provide the best answers and numbers we can. 

Happy New Year!

In wild world of online metrics, it’s a well known fact that metrics from web analytics tools and ad servers never match. Variances can be substantial. 

What I mean is that, given no “refresh rate,” the total impressions for a single ad unit, which should be served on every page request, never matches the number of total page views on the site during the same period of time.  Sigh.

Reasons why identically-named metrics from these two tools (like page views and unique visitors) don’t add up are numerous:

• Different data collection methods.  Ad servers use page tags.  Many web analytics tools use page tags, but it’s not uncommon in web analytics to use additional methods, such as logs or packet sniffers.  The methods have no shared standards for collection or storage of the same data (like visit-level data).  Thus you get apples to strawberries comparisons when attempting to correlate the dimensions from different systems.
• Unique data models.  Ad servers aren’t focused on counting page views and the other dimension of web analytics (visits, time, and so on).  Rather ad servers focus on serving and counting impressions served (and loads of related derivative calculations, like CTR, CPC, and the coolness of view–thru).   Metrics are based on an ad request and an ad code.  Ads aren’t targeted to a page (though that’s possible), but rather to a “zone” or “keyword.” What that means is that “page” dimension may not even exist in your ad server’s schema.  In other words, you aren’t looking at impressions measured on a page, but rather at the number of impressions served in a different conceptual construct.  That’s one of the reasons why people say metrics and ad-serving systems “don’t measure the same thing.” 
• Untagged pages.  Just like analytics implementations suffer from challenges related to complete code coverage of page tags, so do ad serving implementations.  Companies need to determine how to centrally manage the deployment and orchestration of page tags *of all types* and verify all the pages have tags!  Don’t just expect it to work because tagging sounds so easy!  Suspect it won’t work, and determine what you’re going to do *before* you deploy.  Too late?  Time to reengineer. 
• Non-JS executing clients.  Ad servers use page tags.  Not everyone and not all user agents execute javascript.  Everyone needs to realize that page tagging misses traffic as efficiently as it excludes it.  Period.  What percentage of the traffic you miss, you’ll never know… running and filtering your logs may provide an indication…
• Ad blocking software.  Firefox’s Adblock Plus software is a big problem for sites that have a big techie audience, and it affects all sites.  Check your browser reporting and realize a good majority of those Mozilla users may be blocking your ads.  Look at the attitudinal data you have about visitor’s to gauge whether that’s a big issue for your online audience. 
• Cookie issues.  Third-party cookies get blocked (often by privacy software).  Many ad servers still serve third party cookies, and many corporations have not tricked their DNS to accommodate this issue (ahem, CNAME).  We all know how cookie deletion affects unique visitor counts.
• Refresh rates. One page rendered in the browser and many banner “refreshes” makes it really hard to correlate page views and impressions served.
• No rich media installed, and no fallback.  If the client doesn’t have certain plug-ins, and you have no fallback, you miss ad revenue.  Meanwhile the tag executes and you count the traffic.
• Robots, spiders, and crawlers, oh my.  The web is so robotic.  The problem is amazingly understated, especially by companies who want to bill you on page views.  Different data collection methods allow some level of bots to dirty the data.  Logs are harder to efficiently filter.  When the ad server uses tags, and the analytics tool uses logs, you may get some wildly different numbers. 
• Mobile, Mobile, Mobile, Mobile.  Not all Internet-connected mobile devices will display ads, but web analytics tools will track the behavior of mobile visitors.
• Latency.  Visitors who move through the site too quickly may not execute the tag, thus no data is sent back to the server(s).  Ever wonder why vendors tell you to put the tag “high” on the page?

The influence these issues have on your site varies depending on audience.  Investigate factors causing variance and deviation between metrics systems, and educate your audience on why the numbers differ.