I’ve been thinking a bit about targeting, and how we in the web analytics industry have just a ton of visitor or segment-level data that can be used for targeting ads or content, but most tools don’t let you use the data or easily feed it to other systems to do any targeting.  It’s rather odd, don’t you think?   Even Omniture Test and Target isn’t using, as far as I’ve learned, a single data model or the data collected from their behavioral tools, like HBX or SiteCatalyst, for targeting.  All their data models and thus, their data, are unique to the products in their platform.   So I decided to resussitate/revise a blogviation and offer it as food for thought on MediaPost.  When I reread this post, it’s more of an informational post for product managers on how I’d begin thinking about targeting with analytics data and what types of targeting are possible, so here it goes.   

Targeting refers to the process of delivering content or ads to segments or visitors based on their known attributes.  The goal of targeting is simple to understand: maximizing the performance of content or an ad by serving it to visitors at a time when they are most open to the receiving the message. 

For example, you may visit a site, and see some type of ad unit calling out at you to “meet singles in <insert_your_city>.”  When browsing a real estate site, you may see ad units for realtors and mortgage companies.  After entering a keyword such as “car insurance” and clicking through the search results, you may land on a site and see an ad for a car insurance company or land on a page that persuades you to begin the process for creating an insurance price quote.  That’s targeting in a nutshell.  It’s simple for a site owner to understand:

1. Visitor X has these attributes.  
2. We have content or an ad that we think will appeal to Visitor X’s attributes. 
3. Let’s show the relevant content or ad. 

In online media, targeting is associated with paid search campaigning, ad serving, and content optimization based on recognizing and responding to the following attributes:

• Category and sub-category.  Conceptual constructs like “categories” of topics on a media web site or products on an ecommerce site can be targeted to include certain types of ads or messages.   The idea is that if visitors are browsing your category for “hardware floors,” you could offer them an ad or content specific to “flooring installation services.” 
• Geography.  Country, region, city, state, DMA are all targetable constructs.  You may run a sports site and choose to target people surfing in from 02116 (Boston) an ad for Red Sox tickets or content about Manny Ramirez’s recent trade to the Dodgers.
• Browsing environment such as the connection speed, type of browser, operating system, user software, domain, and ISP.  An ad network could serve an ad for DSL to a modem-based surfer by detecting the visitor’s browsing environment.
• Time.  The idea of only showing content during specific periods of time is called “parting.”  Common types include day-parting and season-parting.  For example, a B2B site only choosing to show ads for a particular manufacturer’s product during business hours — the site’s busiest time of day — would be an example of day-parting.
• Keyword.  There are many different types of keyword targeting.  Search engines target ads based on keywords in queries.  Content Management Systems target content based on site search keywords or referring keywords.  “Keywords” may be associated as metadata with site sections or pages, similar to zone or category targeting on an ad server.  Once a page is associated with keyword metadata in an ad tag, you can tell your ad server to target ads to that keyword on whatever page or pages the tag was placed. 
• Language.  When a language can be detected or known in advance, you can target ads to visitors in their language.
• Demographics. If the ad server is aware of a segment’s demographics, such as age, gender, income, title, purchasing power, and so on, an ad can be targeted on that basis. 
• Context.  Think of AdSense and how it matches text ads based on the semantics in site content.  Or when, after adding a product to your cart, a site offers you “free shipping” if your total purchase exceeds a certain price.  This is content targeting based on context.
• Profile.  Targeting is possible based on conclusions drawn and rules created from attributes about an individual or segment (such as purchasing propensity or job title).
• Rules.  Serve an interstitial ad only to visitors who don’t have a cookie set for the site.
• Events.  Someone deposits a large sum of money into his bank account, so the online banking site offers him a CD product on his next login.

We’ve all heard, of course, about a very specific type of often-discussed targeting in online advertising: “behavioral targeting.”  Behavioral targeting refers to the technology and process in which an ad or content is shown to a visitor based on their past actions and reactions.

Behavioral targeting involves:

1. Collecting behavioral data about visitors.
2. Identifying when those visitors visit a site.
3. Determining the current context of visitors on the site.  
4. Detecting the visitor’s current behavior.
5. Serving relevant ads (or content) matched to the behavior.

The goal being to use past behavioral data to influence the customer buying cycle or marketing lifecycle, in order to more effectively and more quickly deliver on advertiser and site goals.

So where does Web analytics come in?  You would think Web analytics data from “Web analytics” technology would be used to enabling “targeting.”  After all the best Web analytics systems store detailed visitor level data about past behavior.  Web analytics data certainly can be used, but in most cases, targeting is a function provided by the ad server or network, perhaps the ISP, or another technology called the “behavioral targeting platform,” not from data collected by the Web analytics tool.

In order to make Web analytics data useful for targeting, you will need to use your data to:

1. Define segments to target or identify visitors to target.
2. Feed past behavioral data about segments or visitors to the targeting technology.
3. Analyze segment and visitor performance against site or advertiser goals after targeting.

Targeting has a proven ability and amazing potential to generate tremendous returns, especially when combined with the rich, detailed behavioral data available in Web analytics.  As a method for optimizing site content and advertising, targeting technologies that integrate with Web analytics data will only become more important and a necessary “must have” for innovative companies that want to maximize business opportunities on the Internet. 

From an article I wrote for MediaPost a few weeks ago:

Reach and frequency and the core concepts of traditional media planning and advertising.  For a given site, program, channel, radio station, billboard, newspaper section, a target audience (the reach) is exposed to a certain number of occurrences of the media (the frequency).  On the web, these concepts manifest themselves in metrics collected and reported from a number of recognizable services.  Audience measurement firms, like comScore and Nielsen, web analytics firms, like Omniture and Unica, to companies somewhere in between, like Quantcast and Google, all have reach and frequency data.  Many new media metrics can be used to proxy frequency- from time-based measures, espoused by audience measurement firms, to concepts like visitor retention or the repeat visitor rate cited by web analytics firms.  On the reach side, companies refer to concepts like “unique visitors.”

These data, of course, available in free tools or in for pay tools are certainly helpful for planning campaigns.  But reach measures can be dirty (cookies, unduplicated unique users, estimates from panels, coverage error).  Frequency measures can be just as dirty (problems recording time in single page visits or visits on the last page, do page views really matter with AJAX and rich media, cookies again, and so on).  We all are aware of the challenges.

Thus using basic reach and frequency measures for planning or evaluating a campaign does not suffice.   So advertisers and agencies target demographics, like gender, age, income, education, and job title.  It’s a given that advertising in the Robb Report reaches a different audience segment than advertising in Popular Mechanics. 

These brave new days we have “behavioral” tracking too.  By taking into account visitor activity across sessions, such as past actions taken on a site or a roster of previous purchases, we can attempt to deduce what a person or segment responds to or is interested in based on their behavior.

Even with reach, frequency, demographics, and behavioral data to help guide advertising and media buying, we are missing an important attribute for maximizing the potential success of our campaigns.  We do not have an available tool, whether free or paid, for advertising or buying media on or across sites according to measures of past performance.  Such measures include ad clickthrough rates, conversion rates, goal completion rates, delivered impressions, and perhaps even harder to quantify financial measures such as ROI, ROAS, and ROMI.

Sure, historic, tacit knowledge of campaign performance exists and is used by agencies or publishers.  However, there is no shared industry source that can help us answer “how has a site for display advertisement historically performed toward goals based on the reach, frequency, demographic and behavior of its audience segments?”  Interestingly, a company minting money right now, named Google, can masterfully demonstrate performance in paid search campaigning and help advertisers unify it with segmented reach, frequency, and demographics.

Outcomes based performance measurement unified with reach, frequency, demographics, and behavior is what is missing in audience measurement tools, not frequently reported externally by web analytics tools or ad serving tools, and not available in ad planning tools.  When advertisers can target display ads, or even video ads, to desired audience segments by reach, frequency, demographics, behavior in the context of known performance, media planning will be more effective.  

AVG has released an updated version that corrects the LinkScanner bot issue (build 138, July 4), which we’ve all noticed slamming our servers and analytics data over the last several weeks:

We have modified the Search-Shield component of the product to
only notify users of malicious sites.Search-Shield no longer
scans each search result online for new exploits, which was
causing the spikes that web masters addressed with us. However,
it is important to note that AVG still offers full protection
against potential exploits through the Active Surf-Shield
component of our product, which checks every page for malicious
content as it is visited, but before it is opened.

As you’ve just read in the quote above, AVG has stopped scanning each page that returns in a SERP for users of their free tool.  Instead pages will be scanned by proxy after a user clicks on the link. 

For paid users, it’s a little different.  SERP’s will still be scanned but via a pure database approach (not the DDOS approach :), which means the sites listed in SERP’s will be compared to a black list of known “bad” sites.  The blacklist is based on internal AVG research and from the real-time results reported by users who have opted-into AVG’s “prevalence reporting system” (a feature of AVG 8).  This means AVG is still scanning sites, but on a very limited basis, thus the detrimental effects on analytics should be very minimal and only caused by users who participate in prevelance reporting.  Still some data pollution will occur…  

AVG hasn’t confirmed that they’ve released a fix to the “noscript” issue I mentioned.  I do know they are working on it and have fixed the problem in internal builds.  Regardless, if the LinkScanner is working in the way they say it is, the problem will be negligible (but some data pollution will still occur ;).

Kudos to AVG Corporate, Roger Thompson, Pat Bitton, Greg Mosher, and all the other engineers who listened to the community on the web and worked quickly to fix the problem.  Now let’s hope the the build 138 update works as described. Time will tell.

AVG has obfuscated their user agent.  One of the current agents for customers of their free and paid tool now cloaks itself as IE6:

Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)

In addition to the easily detectable user agents:

Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1;1813)

User Agent:Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1;1813)  

User Agent:Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)

This news is not good.  If you filter SV1 agent, you risk filtering legitimate traffic from the IE6 browser.  A few folks have commented to me that one should filter the user agent anyway, because 1) IE6 is in decline and 2) most IE6 users have .NET installed, which will show in the user agent.  Still filtering it makes me a little uneasy.

Is this the death toll for log file analysis and services provided by ABCe (since they can’t filter this user agent either)?  Maybe it is.  AVG is touting that agent lacks HTTP Accept-Encoding, which is just dandy, but that information isn’t normally captured in logs.

So the current situation is this:

1. AVG has two user agents.  Both are filterable, but the SV1 agent is problematic to filter because you risk filtering legitimate traffic.
2. Both agents in the current version request gifs in noscript tags, inflating counts in page tag implementations with noscript configurations.  AVG claims they will fix this issue.
3. The bot uses”mad” bandwidth.  I’ve heard stories of bandwidth increasing 100x normal levels.  Some webmasters are serving dummy files to the recognizable user agents, some aren’t serving content to IE 6 browsers (crazy), and some are redirecting the bot back to AVG (thus inflating AVG’s bandwidth, LOL!).
4. Evidence points to this bot NOT inflating clicks from paid search (i.e. PPC) and thus NOT committing click fraud.   But it doesn’t remain out of the realm of possibility that the scanner may be accessing an ad vendor click redirector and causing a click.  Not trying to spread FUD here, just making a point. 
5. AVG is looking at option of checking either an external db (hosted by AVG) or a local cache to verify sites in SERP’s have been “scanned by AVG,” instead of repeatedly scanning sites every time they are listed in SERP, to reduce the bandwidth issue and minimize fraudulent entries in log files.
6. AVG is thinking about enabling white listing of sites, so they are skipped by the scanner.
7. AVG is thinking about exposing a meta-tag that instructs the scanner to ignore the site.

Good luck with this nasty bot!  Interestingly, here’s how you smurf a site with the AVG LinkScanner. 

Here’s the deal.  AVG LinkScanner doesn’t execute javascript nor take cookies.  I had that confirmed by the Chief Research Officer at AVG, Roger Thompson. 

So why is the AVG user agent showing up in that data collected from certain page tag configurations?  The AVG LinkScanner currently requests gifs in noscript tags!

A best practice in web analytic’s page tag configuration is to use the noscript tag to serve the gif to non-javascript executing browsers.  Here’s some commonly seen (obscured) code for doing that:

<noscript>
<div><img alt=”foo” id=”bar” width=”1″ height=”1″ src=”http://
foo.bar.com/xyzab57yw10000s1s8g0boozt_9t1x/foo.gif?baruri=/
nojavascript&xy.js=No&xy.tv=1.2.3″ mce_src=”http://
foo.bar.com/xyzab57yw10000s1s8g0boozt_9t1x/foo.gif?baruri=/
nojavascript&xy.js=No&xy.tv=1.2.3″div>
</noscript>

<NOSCRIPT>
<IMG
src=”//foo.bar.com/xyz.gif?Log=1&URL=/javascript_disabled” mce_src=”//foo.bar.com/xyz.gif?Log=1&URL=/javascript_disabled”
BORDER=”0″ WIDTH=”1″ HEIGHT=”1″ />
</NOSCRIPT>

<noscript>
<img src=http://pt.foobar.com/images/xyz.gif?js=0” height=”1″
width=”1″
border=”0″ hspace=”0″ vspace=”0″ alt=”"> 

Thus, if you are using noscript tags in your page tag *and* someone with the AVG Linkscanner views a SERP (search engine results page)  from Google/Yahoo/MSN that lists your site, the traffic from the LinkScanner will be counted. 

Of course the simple solution to fix this problem is to exclude the user agent: 

Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1;1813)

If don’t have full control over your page tag based web analytics implementation (i.e. hosted), you need to verify that your vendor has excluded this agent.   And you should have them audit your data going back to April, and refund/credit you any money.  Good luck with that though!

How big is the problem?  Well, it depends!

The amount of AVG traffic will vary dramatically by site.  Your site must show up in the SERP’s on computers of visitors that have AVG LinkScanner installed, and you must be using noscript tags to serve the gif.

I’ve made AVG aware of this issue.  And frankly, they’ve been a fantastic company to work with, so I’m sticking with them (for now ;).  First they allowed me to join a private Google group to discuss my findings, both the Head of Global Communications and Chief Research Officer quickly responded to all my emails (good social media response), and their engineers are looking into this issue so that they can fix it…  That’s pretty impressive and quick response.  So cheers to them!

It’s worth mentioning that the LinkScanner isn’t _supposed_ to request images, so I do think this issue will get fixed.

Only time will tell whether or not AVG obfuscates the user agent so it looks just like a “normal” browser.  Let’s hope not! 

What I do find interesting is that I’m already hearing that an agent exists with the string (Mozillia/4.0 (compatible; MSIE 6.0; Windows NT 5.1;1813). Note the “ia” mispelling of Mozilla as incorrectly documented here.  And it accepts cookies.  So AVG’s agent is already being spoofed.  Not good, not good.

The  well-researched answer is “no.”  The AVG LinkScanner Bot appears to prefetch the js and the gif (and pretty much everything else on the page), which for certain tools and their tag configurations generates false page views and visits (and the derivatives thereof), just like it’s “legitimate” traffic. 

If your tag configuration is set up with noscript tags, AVG will fetch the content in the tags, including the gif, which means that:

• The bot may be infesting the data of customers of web analytics vendor who configure page tag-based data collection in this way. 
• The bot may be inflating the data in such products/services offered by various web analytics companies.
• Customers may be paying for server calls generated by this bot.

Vendors, of course, could easily filter the user agent to protect their customers:

Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1;1813) 

But I haven’t heard a peep from any SaaS vendors about excluding the user agent, filtering already collected data, or refunding customers the cost of robotically generated server calls (regardless of AVG). Have you?

Think about this: many SaaS page tag vendors don’t provide detailed visitor-level data and user agent reporting.  That means that their customers have no ability to investigate this bot or detect it by filtering their reported data by the the true user agent.

I’ve been talking about JS executing bots screwing with web data for about a year now.  SEOMoz and the folks at SlickSurface confirmed it quite recently (quoting me no less in their fantastic analysis).  So they do exist…

Now let me tell you a little story.  Once upon a time I was at a conference called eMetrics when the CEO of a company came up to me and said “hey I read your blog about bot detection, and I looked in my web metrics tool for traffic with high page view to visit ratios.”  Then he narrated a story to me about how he found a bunch of traffic that had page view to visit ratios of 5,000 to 1.”  I said “do you use page tags” He said “that’s all my vendor provides, so yeah.”  And I said “you’ve found a javascript executing bot in your data.”  “I know” he said. “Well did you call your vendor and let them know?”  I said.  Now for the punch line:  he told me that the vendor (who shall remain nameless) told him “well, the traffic executed server calls”  And they wouldn’t give him a refund!

It’s worth mentioning that this bot definitely affects log file tools and packet sniffer tools.  Both must be configured to filter the AVG LinkScanner user agent.

Now here’s the rub for me.  I use AVG!!!  But I now find it increasingly difficult to support the company or continue using their products.  Why?  Because they are wearing a “bad hat” here:

• First, they are fully aware of the affect of this bot on web analytics systems. They just don’t seem to care (yet).  UPDATE:  They have set up a Google Group to discuss this issue.  They must understand how companies of all types in all sectors use web analytics data to optimize their sites, set their marketing budgets, determine expected server load, and much more.  What do their Internet Marketers think? 
• Second, the Link Scanner tool may have a short shelf life and may offer limited protection.  Malware creators will easily adjust. Check out what my friend Steve McInerney, a very smart security expert, said on the Web Analytics Association’s Yahoo Forum:

What strikes me about this particular solution by AVG is how
incredibly … stupid it is on several fronts.

1. Noticeably impacting a users bandwidth is, technically, a security
breach in the first place, aka Denial of Service Attack.

2. Some of us live in countries that have rather severe bandwidth
charges/limits and the like, whom shall I send my excess bandwidth
bill to?

…(this) method is fundamentally
flawed. ie malware ignores any first request and only infects on a
second request - alternate cloaking. Whatever. This type of “solution”
only provides weak protection for a strictly limited period of time.

…not just “no security” but bad
security. Because folk feel they are being protected when they are
not, and hence will take greater risks and hence inflict greater harm
on themselves.  

Ignoring the balance of positive to harm that this problem inflicts on
the users who use this product.

• Third, AVG just doesn’t seem to “get it” yet.  They are potentially messing with the ability to drive commerce via data driven decision making, e-commerce analytics, site optimization, and online media measurement!  To quote The Register “chief of research Roger Thompson - who designed the AVG LinkScanner - indicated he may do away with that unique user agent. His chief concern is security, and he doesn’t want webmasters or malware writers gaming his scanner. “In order to detect the really tricky - and by association, the most important - malicious content, we need to look just like a browser driven by a human being,” he argues.

WebMasterWorld has some good stuff about to say here.  Read the Register’s first article here.  And check out the dude’s blog who broke the news first and responses from AVG here and here.

Interesting stuff. So what do you all think? Have you seen evidence of this bot in user agent data from your page tag solutions that use the noscript tag for the image? 

A question any practitioner of Internet-based analytics will be asked by many different stakeholders is “why don’t the numbers match?”  Counts of the identically named metrics from ad servers don’t match the web analytics tool, which don’t match the for-pay third party audience measurement tools, which don’t match the free audience measurement tools, which never match any of the homegrown internal measurement tools.  And none of them ever match each other.

So it’s a good question certainly valid to ask.  The answers are even fairly easy to understand, but the root causes are often difficult to pinpoint and even harder, if possible at all, to remedy.  The fact of the matter is that data discrepancies in analytics result for a multitude of reasons, such as:

• Different data collection methods.  We have a bunch of tools and services that collect web data using various, non-standardized, proprietary data collection methods.  Ad servers use javascript page tags.  Many web analytics tools use page tags too, but it’s not uncommon in web analytics to use additional methods, such as log files or packet sniffers.  Or perhaps a combination of these methods, called hybrid data collection.  And all the tools have different algorithms for processing the data collected.

On the audience measurement side, data is collected from self-selecting panels who install proprietary software (i.e. toolbars and so on) on their computers, perhaps at work or at their university, but most likely at home.  Then, the collected data from different panels is rolled-up and combined, and the limited subset of the Internet population that chooses to be monitored, in exchange for some incentive, is inflated and projected to the entire Internet audience using proprietary statistical methods.  We also have data collected from a limited set of geographically specific ISP’s.  And regardless of whether we’re talking about audience measurement or web analytics, the different data collection methods often, but not always, involve cookies and all their inherent issues of cookie deletion.  

• Unique data models.  Ad servers aren’t focused on counting page views and the other dimension of web analytics (visits, time, and so on).  Rather ad servers focus on serving and counting impressions served (and loads of related derivative calculations, like CTR, CPC, and view–thru).  Metrics are based on an ad request and an ad code.  Ads may or may not be targeted to a page, and instead to various constructs, like a “zone” or “keyword.”  What that means is that the “page” dimension may not even exist in your ad server’s data model.  In other words, you aren’t looking at impressions measured on a page, but rather at the number of impressions served in a different conceptual construct.  That’s one of the reasons why people say metrics and ad-serving systems “don’t measure the same thing.”
• Untagged pages.  Specific to technologies that collect data or serve ads using javascript page tags, there are challenges to ensuring and verifying complete coverage of page tags across every page on a site.  When the pages aren’t all tagged with the different tags for the assorted technologies, guess what?  The numbers won’t come close to falling within tolerable variances.  And questions and skepticism will ensue.
• Non-JS executing clients and ad blocking software.  Let’s imagine for the moment, your site is perfectly tagged for all technologies, so the numbers between your ad server will be close to your web analytics system, right?  Nope, regardless of data model issues, not all browsers execute javascript and many Firefox users have installed Ad Block Plus. 
• Cookie issues.  When you’re counting based on cookies, third-party cookies get blocked (often by privacy software).  Many ad servers and web analytics tools still serve third party cookies, and many corporations have not tricked out their DNS to accommodate this issue.  And we all know how cookie deletion affects unique visitor counts, even if you use first-party cookies.
• Many other issues.  Latency from visitors moving off the page prior to the tag executing to latency in the call to pick up an ad from a third party while your ad server counts the traffic (so your ad count differs from the agency’s count), to refresh rates making it hard to correlate page views and impressions, to no rich media installed and no fallback, to robotic traffic not being filtered from logs or tags, to certain types of user agents (such as mobile devices) not executing javascript… there’s a whole host of other factors that cause data discrepancies.

And of course, there’s always the nebulous issue around the complete lack of consensus-based, enforceable standards for online measurement.  No industry organization can say what vendors or companies “must” do, only what they “should” do… And no industry body is going to get successful companies to change their secret sauce just because they said so…

So what’s a practitioner to do?  Understand the potential sources of discrepancies.  Work with your team (from IT to vendors) to prevent and minimize the root causes when possible.  Educate your team when discrepancies are not remediable.  Ensure you use the different sources of metrics judiciously in the context of your business goals.  Finally, realize that none of the tools are more “correct” than any other.  All of our analytics tools serve different, and sometimes overlapping, business purposes - from counting ads, to influencing media buying, to sizing audiences, to measuring business performance, and to optimizing the site.

Web Analytics Key Performance Indicators (KPI’s) are critical for breaking through the dataglut spewing forth from your web analytics tool.   I mean there’s a just a ton of data in web analytics, and the majority of it tends not to be very useful or applicable for improving your business performance.  While it’s wonderful to have a tool that lets you cut, cross, and slice loads of data every which way but loose, its can be a real challenge to frame the data or put it in context in a way that helps your business optimize the web site.   That’s why I like KPI’s - they identify meaningful, business-focused relationships in your analytics data.  By understanding KPI drivers, setting expectations for KPI performance, and analyzing your KPI’s toward defined goals for those KPI’s, you increase understanding of data, alleviate data confusion, and provide focus for the usage of your web analytics tool.

For those of you who don’t have a KPI strategy or who are just getting into analytics, an easy way to understand a KPI is to consider the example of when you are driving somewhere and trying to get there within a certain period of time.  If your goals is drive 60 miles (kilometers, my European friends) in exactly 60 minutes, you know that you need to drive 60 miles per hour (or KPH).  If you go faster, you will arrive early, if you go slower you won’t meet your goal and will arrive past your deadline.   So as you travel along the road, you measure the KPI of your speed. That’s what is important to measure on your trip.  Of course you may measure other KPI’s like the amount of fuel left or the miles you’ve traveled… those certainly may be KPI’s you measure.  But you definitely don’t need to measure you compression ratio or oil pressure even though it’s available data from your car.  In the same way, when you are looking at web analytics data, you don’t want to track everything, only those things that are important to your business performance toward goals. 

Several activities can assist the creation of KPI’s.  Here are a few of them:

• Determine the Business Strategy.  Why is the company funding and developing an online mission?  What is the strategy?  KPI’s can help you figure out if it’s working.  To find the KPI’s that will help, the web analyst should be asking the question how can web analytics be used to formulate, implement and evaluate cross-functional decisions that will enable an organization to achieve objectives? How will web analytics be used in the process of specifying the organization’s objectives, developing policies and plans to achieve these objectives, and allocating resources to implement the policies and plans to achieve the organization’s objectives?
• Define the Site’s Goals and why the Site Exists.  I covered this in a post a few months ago.  A understanding of why your site exists enables you to effectively use online metrics.  You need to define the purpose of your site in order to create effective KPI’s.  Once you’ve defined your site’s purpose, you are positioned to examine Web data in way that helps you determine whether your site delivers on its purpose — does it exist effectively?   Create your KPI’s, identify goals for your KPI’s, and track your performance against those goals.
• Recognize Value Drivers.  How does the business make money on the site? Monetization, in cases where profitability is important, influences what you should be measuring.  If you run a media site, you probably make money from content consumption (the recency and frequency of content consumption), conversation (social media, such as contributions or comments), and conversion (the rate at which people complete certain value driving actions, like signing up for newsletters, rss feed, webcasts, print subscriptions, or downloading certain content types, like white papers).  So you create goals for and measure KPI performance around those value drivers.
• Map Organizational Roles.  Classify your organization into audiences for your KPI’s based what they do on your web site.  You may create KPI’s around function or action of the actors who receive your KPI reports.  Function defines the group that KPI’s are focused for, such as product development or editorial.  Action defines what those people do on the site to make it successful.  By understanding function and action of key actors on your sites, you gain insight into the type of data needed in KPI’s and the number of different KPI reports you may need to roll out.
• Understand the Customer.  KPI’s purely focused on internal function and actions are important, they need to be customer focused.   If you think measuring conversion is important, while your customers tend to come to your site for informational or non-transactional purposes and then go elsewhere to convert, you may be disconnected from the reality of why your site exists.   Learn customer goals from VOC (voice of customer) data and by examining historic behavioral data of key segments.  Make sure you don’t create KPI’s that are vain or inane.  Instead create KPI’s that help you guide action internally so that your business meets the needs of your customers.

Framing your KPI development around the five bullet points I listed above will help you create KPI’s that assist your team in guiding business performance toward goals - while not forgetting to consider some of the core elements of online business: business strategy, site performance goals, value drivers, the human organization, and the customer. 

Now segment, segment, segment your KPI’s!

The infoglut in web analytics is enormous.  So much data.  Companies report that 69% of all people who consume the data don’t understand it.  How does a business go about making sense of it all?  Formulating a comprehensive KPI (Key Performance Indicator) strategy is a big part of differentiating signal from noise and directing appropriate tool usage.  We’ve all heard about KPI’s before.  They are ratios or derivatives of metrics that pinpoint critical, business relevant web performance.   My good friend, Eric, even wrote a book (a BIG one) about it. 

The process of moving an organization through KPI Change Management starts with a well formulated plan for doing so.  Here are some tips for formulating your KPI plan: 

• Educate senior management and get managerial buy-in.  Education and buy-in can take shape via a number of methods.  Maybe you publish and circulate an internal-only white paper about the importance of KPI’s measurement.  Maybe you leave Eric’s book on the chair of your C-level executives.  Perhaps you hold a meeting and present the web site optimization process and how measurement via KPI’s provides the foundational informational on which to make site optimization decisions.  Perhaps you take your boss out to lunch and explain that basic reporting and tool access is helpful, but “Web analytics is hard” and that KPI’s give context to the data to staff that’s otherwise somewhat confused about what they pull for the tool.  You explain that KPI’s provide a focal point for centering analysis around business goals.  Whatever the method, the goal is managerial approval that “yes, you can do KPI’s.”
• Determine the audience for the KPI’s and train them.The importance of KPI’s will vary by stakeholder, and your KPI strategy needs to take that into account. Different segments of stakeholders will be interested in specific KPI’s, and you must accommodate that need.  As an analyst, you should identify the functional roles and job responsibilities of the people who are going to receive KPI reports.   Everyone may not be the right choice (though it could be), and it may make sense to concentrate a KPI rollout on the needs of the few or it may make sense to “go broad.”  Follow up with comprehensive training about your KPI project and how KPI’s can most effectively be used.
• Start with simple, well-qualified, highly relevant KPI’s.  While some folks with want to throw a “kitchen sink” strategy at KPI’s.  That’s a mistake.  If you report more than 5 to 10 KPI’s (imho) per stakeholding group you may end up with a set of unworkable, confusing, and neglected reports.  It’s better to report just a few, well qualified, highly relevant KPI’s.  How do you qualify them? By mapping KPI’s to important business objectives.  How do you know they are highly-relevant? Because you’ve compelled management to buy-in and to agree that they are critical indicators of site success. 
• Elicit the business goals for the KPI’s, compare KPI’s to goals, and report associated variances (i.e. deviations). Make sure you have determined business performance goals for KPI’s.  Goals give context for performance. It’s that simple.  Without goals, you have no context for determining what’s good and what’s bad.  If your conversion rate KPI is 5%.  Great!  So what though?  If you know your goal is 3%.  Awesome job.  If you know your goal is 10%.  Stop reading now, and get back to work - you have much work cut out for you. 
• Identify the frequency and format for reporting.  You need to determine a frequency that is timely and sustainable, and the format in which you present KPI reports needs to common enough that people can easily examine the data. Perhaps you deliver the reporting in Excel, make it available directly in your tool, use Xcellius, or create reports using a BI tool. 
• Automate the delivery of the reporting.Without automation, you may put on the Report Monkey suit and enter Excel hell.  Critical to the successful rollout of any KPI reporting is an automation plan.  Do you email reports, put them in a shared directory, create a set of reports in the tool and provide access, or deliver them in weekly presentations?  The best choice is the option that gets people to use them, listen, and understand what you are trying to do with KPI’s.
• Following the reporting up with analysis and guidance.  Depending on the size and scale or your organization and the resources you have to work with, it may not be possible to provide every stakeholder with detailed analysis.  But you need to do your best to follow up KPI reporting with true analysis and guidance.  Why are KPI’s going up or down?  What are the drivers of the changes? 
• Segment, segment, segment. Site level KPI’s are helpful in understanding overall audience and customer behavior, but they hide important details.  When you slice a KPI by a specific segment, you will realize insights that help you conclude what action to take next.  Overall site repeat visit rate is 37%, but the repeat visit rate for customers who use your “product lookup tool” is 96%.  What does that data indicate about how you market the site, or about why people are coming to the site? 
• Test, test, test.  As you measure > report > analyze > guide based on KPI’s you will undoubtedly determine actions to take on the site.  You should be testing the hypothesis behind these actions via controlled experimentation.   

There’s obviously a lot more to talk about here - from what constitutes a good KPI, to what types of KPI’s different stakeholders should examine, to what are the best KPI’s for particular site types and more.  I guess there’s more blog posts for that, but in the meantime I hope you’ve found this blogviation useful.  Let me know if you have any thoughts to share.

At some point in the career of a web analyst, you will be asked to investigate, assess, and possibly judge the current state of how a company “does” web analytics.  What are some of the areas you should ask about?  Here are some thoughts and a few questions to ask to help inform your analysis (and grease your mental gears):

• Business strategy.  Why does the organization do web analytics?  What’s the goal of having a web analytics team?  Who defines the strategy?  What is the strategy?
• Analytics organization and team structure.  Who is the chief owner of web analytics?  What does the analytics team look like?  How has the team structure been formalized in the organization?  Is the web analytics team effectively staffed and have enough control over resources to do the job?
• Process.  What analytics processes have been defined?  How does a site or site feature progress from not being measured to being effectively measured?
• Data collection. What methods for data collection are being used?  How much data is being collected, and for how long is it stored, and at what level (i.e. detail, aggregate)?
• Reporting.  What data is reported?  What do the reports look like?  Who creates them?  How are they distributed, and in what format?  To whom?  When?  How?
• Analysis.  What’s the difference in this company between reporting and analysis?  How is analysis communicated to stakeholders?  When?  How?
• KPI’s.  What Key Performance Indicators are you measuring?  How are they relevant to the business?  What actions have people taken from KPI analysis that improved business performance?
• Segmentation.  What audience and customer segments exist?  What audience and customer dimensions and attributes are segmented?  Why are they meaningful to the business?  What has the business learned and what action has been taken from the current segmentation analysis strategy?
• Technology.  What analytics technologies are being used?  What does the schema for web analytics look like?  What homegrown technologies are used?  What external technologies have you bought or deployed for analytics?
• Integration.  How is web analytics data integrated with other internal and external data?  Is it integrated with other systems, how? 
• Site Optimization.  Does the company test landing pages, and/or use AB or Multivariate testing software?  If so, whose software, and what business gains have been realized?
• Advertising/Advertisers. How is analytics used to inform or enable advertisers and advertising?
• Privacy.  What safeguards does the company take in protecting analytics data? 
• Qualitative Data.  Is qualitative data contextualized with web analytics data? Do you capture voice-of-customer data?  Use Net Promoter Scores?  Have a research department?  Does web analytics collaborate with research? 

Those are just a few questions to ask.  Many others can be asked.  What would you want to know, and what would you ask?  Please leave a comment.  I’d love to hear your thoughts.

Now for some random thoughts:

• News from Orem.  API / Fusion / Video Tracking… cool.  I’m pretty psyched that Omniture announced a web services API.  That’s fantastic, and confirms how truly important integration is now and will be in the future for analytics data (as I’ve been saying for years… Google will be next). 

Omniture has announced a new methodology, Fusion, and improved capabilities for tracking video.  All sounds very exciting.  But, like Eric, I’m wondering what revolutionary new methodology Fusion really is?  Or is just what Eric’s been saying for the last 4 yearsbranded by Omniture and delivered by the Great Belkin? 

Regarding the video capabilities, I haven’t seen a real demo yet, but I wasn’t immediately impressed with what I saw on my friend Marshall’s blog.  Instead of quartile tracking, it seems like you track the playhead (the part of the video playing) across audience aggregates in increments of one-twelfth, and you get some bubbly visualization (what would that look like with 10,000 videos on your site?), and better access to forums.

I’m hoping I haven’t seen the whole ball of wax, and I look forward to Omniture giving me the grand tour. 

But for a playhead visualization, I was much more impressed with what I saw from Visible Measures and their engagement curve.  And what the heck are those folks at Divinity Metrics up to for measuring video? 

• News from Novato.  One of my favorite gangs of web analytics folks reside in Northern California.  My colleagues at Semphonic have just released a rather impressive “Omniture Implementation Toolkit.” 

I was able to procure a copy, and I’m totally impressed.  It’s full of hard-learned and hard-earned real world practitioner knowledge.  If you are trying to implement Omniture, it is well worth the money. 

Now I’m not sure if this document competes with or acts as a companion to Fusion.  All I can say is that I know the folks at Semphonic are smart, savvy, and very experienced, and there are thousands of Omniture customers out there who could benefit from this document.

• X Change Conference.  I am totally excited for X Change brought to us this year by Semphonic and Web Analytics Demystified.  The last X Change in Napa at COPIA was one of the most intimate, educational, stimulating, and enjoyable conferences that I’ve been too (and did I mention the wine?).  It was pure “class” all the way (in both the sense of style and learning, and did I mention the wine? ;-). 

This year attendance is limited to 100 folks (99 if you count me ;).  Last year, I huddled on “Deploying Measurement Systems in Globally Distributed Enterprises.”  

If you aren’t familiar with X Change or Semphonic  check them out, and make sure to read a few of my favorite bloggers - the prolific deep thinker and expert Gary Angel, the always impressive (and fun) June D(ershewitz), and bright author and web analytics veteran, Phil Kemelor.