A question any practitioner of Internet-based analytics will be asked by many different stakeholders is “why don’t the numbers match?”  Counts of the identically named metrics from ad servers don’t match the web analytics tool, which don’t match the for-pay third party audience measurement tools, which don’t match the free audience measurement tools, which never match any of the homegrown internal measurement tools.  And none of them ever match each other.

So it’s a good question certainly valid to ask.  The answers are even fairly easy to understand, but the root causes are often difficult to pinpoint and even harder, if possible at all, to remedy.  The fact of the matter is that data discrepancies in analytics result for a multitude of reasons, such as:

• Different data collection methods.  We have a bunch of tools and services that collect web data using various, non-standardized, proprietary data collection methods.  Ad servers use javascript page tags.  Many web analytics tools use page tags too, but it’s not uncommon in web analytics to use additional methods, such as log files or packet sniffers.  Or perhaps a combination of these methods, called hybrid data collection.  And all the tools have different algorithms for processing the data collected.

On the audience measurement side, data is collected from self-selecting panels who install proprietary software (i.e. toolbars and so on) on their computers, perhaps at work or at their university, but most likely at home.  Then, the collected data from different panels is rolled-up and combined, and the limited subset of the Internet population that chooses to be monitored, in exchange for some incentive, is inflated and projected to the entire Internet audience using proprietary statistical methods.  We also have data collected from a limited set of geographically specific ISP’s.  And regardless of whether we’re talking about audience measurement or web analytics, the different data collection methods often, but not always, involve cookies and all their inherent issues of cookie deletion.  

• Unique data models.  Ad servers aren’t focused on counting page views and the other dimension of web analytics (visits, time, and so on).  Rather ad servers focus on serving and counting impressions served (and loads of related derivative calculations, like CTR, CPC, and view–thru).  Metrics are based on an ad request and an ad code.  Ads may or may not be targeted to a page, and instead to various constructs, like a “zone” or “keyword.”  What that means is that the “page” dimension may not even exist in your ad server’s data model.  In other words, you aren’t looking at impressions measured on a page, but rather at the number of impressions served in a different conceptual construct.  That’s one of the reasons why people say metrics and ad-serving systems “don’t measure the same thing.”
• Untagged pages.  Specific to technologies that collect data or serve ads using javascript page tags, there are challenges to ensuring and verifying complete coverage of page tags across every page on a site.  When the pages aren’t all tagged with the different tags for the assorted technologies, guess what?  The numbers won’t come close to falling within tolerable variances.  And questions and skepticism will ensue.
• Non-JS executing clients and ad blocking software.  Let’s imagine for the moment, your site is perfectly tagged for all technologies, so the numbers between your ad server will be close to your web analytics system, right?  Nope, regardless of data model issues, not all browsers execute javascript and many Firefox users have installed Ad Block Plus. 
• Cookie issues.  When you’re counting based on cookies, third-party cookies get blocked (often by privacy software).  Many ad servers and web analytics tools still serve third party cookies, and many corporations have not tricked out their DNS to accommodate this issue.  And we all know how cookie deletion affects unique visitor counts, even if you use first-party cookies.
• Many other issues.  Latency from visitors moving off the page prior to the tag executing to latency in the call to pick up an ad from a third party while your ad server counts the traffic (so your ad count differs from the agency’s count), to refresh rates making it hard to correlate page views and impressions, to no rich media installed and no fallback, to robotic traffic not being filtered from logs or tags, to certain types of user agents (such as mobile devices) not executing javascript… there’s a whole host of other factors that cause data discrepancies.

And of course, there’s always the nebulous issue around the complete lack of consensus-based, enforceable standards for online measurement.  No industry organization can say what vendors or companies “must” do, only what they “should” do… And no industry body is going to get successful companies to change their secret sauce just because they said so…

So what’s a practitioner to do?  Understand the potential sources of discrepancies.  Work with your team (from IT to vendors) to prevent and minimize the root causes when possible.  Educate your team when discrepancies are not remediable.  Ensure you use the different sources of metrics judiciously in the context of your business goals.  Finally, realize that none of the tools are more “correct” than any other.  All of our analytics tools serve different, and sometimes overlapping, business purposes - from counting ads, to influencing media buying, to sizing audiences, to measuring business performance, and to optimizing the site.

Back from another excellent eMetrics.  I’m a very big fan of the eMetrics Marketing Optimization Summit…  Props go to Jim Sterne for growing this event from a little seed into an incredible, blogworthy blossom.  How involved is Jim in eMetrics?  I’d say he’s completely immersed in every little piece - he even came up to me at the SF WAW (way to go June D!) to find out about the renegade AV work I did in one of the sessions, and to get my take on how it could have been avoided.  He’s that intimately connected to what’s going on.  Macro and micro, micro and macro.  And when you have one of the best Internet Marketers in the world, keeping a tight rein on the Clydesdale of conferences, you know you’re in for one heck of fun ride. 

And so it was for about 500+ of the top web analytics in the beautiful Palace hotel.  Props to consummate conference organizers Matt Finlay and his crew at Rising Media for keeping the road smooth as we all trotted on it as well.  Fanny, you are one helpful polyglot of a marketing manager!  I never knew German keyboards were so wild… Thanks.

The eMetrics sessions were informative and actionable.  The lobby bar and after-hours parties fun and enlightening.  You really can’t ask for more out of a conference.  As I flew home thinking back on it all, there was a lot to blog about, including:

• It’s all about attitude, dude – as in attitudinal data.  Like my father says “it’s all about your attitude.”  And so it is on the Internet in 2008.  From ForeSeeResults, to iPerceptions, to OpinionLab, to CRMMetrix, the often missing link in customer analytics is attitudinal data.  I’m talking here about Voice of Customer (VOC) technology that allows you to ask a question set to site visitors and then apply some sort of algorithm or model to express the meaningfulness of the data in quantifiable terms.  From the American Customer Satisfaction Index to 4Q.  VOC technology enables you to participate in a continuous, automated dialog with your customers in order to identify problem points on your web site and enable you to measure purpose and success of your most valuable segments.  Expect to see some of the big players gobble up these smaller companies.  Omniture, Unica, WebTrends, and CoreMetrics should be thinking about acquisition in this space to round out their offerings.
• Testing, 123… as in multivariate, MVT.  The rage is site optimization technologies beyond the simple A/B, champion challenger, test.  In this category you find folks like SiteSpect (the only non-intrusive multivariate testing solution!).  I’m a big fan of these guys (and was in 2006 long before they ever sponsored a WAW, thanks to a nice demo from Larry at my old job).  Eric Hansen and his crew have specialized software that you install in your data center.  No futzing with damned tags.  Swap out your variations, create different recipes, determine what’s statistically significant in giving you a lift to your macro or micro conversion goal, and you’re off to the races.  The good folks at Google are doing it and doing it well with Google Site Optimizer (thanks for the t-shirts!).  Interwoven is baking in Optimost to the CMS, and Omniture has their Test and Target integrated with the Business Optimization Suite.  Accenture has Memetrics.  Kefta too. And what ever happened to Verster?

In a nutshell, these technologies enable you to test variations of content themes, colors, creative, calls to action, points of resolution, buttons, navigational elements, –whatever you want to call the stuff on the screen—to determine what combination performs best against your goals.  But of course, this is all just software, so don’t get too excited.  The tests are about as good as the people creating them…  And complex tests that take a long time to execute may not finish.  Imagine 1-800-Flowers starting a test in January and not finishing until March, missing Valentine’s Day.  Or Intuit running a test beyond April 15th for a tax product.  Go humbly and carefully into this space, my friends, or you may end up optimizing for everyone and appealing to none.

• Tying it all back to the dollar for profit-generating sites and to the mission of non-profit generating sites…  It seems like a “no, duh” moment but metrics for the sake of metrics can be a big waste of time.  If you can’t tie metrics or visitor actions back to value on a revenue-producing site or to the betterment of a non-profit site’s core mission, then what’s really the point of the measurement…  That’s why I’m a big fan of the stuff ZaaZ does.  They totally get the fact of how actionable metrics turn the wheel of Internet commerce and ad-based models, and they can model it all to prove it out the ROI.  Folks like newly elected WAA Director Alex Langshur’s company Public InSite do similar stuff for content driven sites.  That is they know how to use metrics to optimize the channel to goals, not to just puke confusing data, like most web analytics tools do.  Again, it’s all about the people you hire, not the tools you use… My good friend Avinash, right again!
• The emergence and rise of deeply psychological and neuro-behavioral methods for automating persuasion and conversion.   Anyone who knows my good friend Joseph Carrabis, over at NextStage Evolution, knows that besides being one heck of giant kite flying, music master, he’s also got the models and the patents to help target and respond to human behavior across programmable devices.  We’re already seeing some companies, like Seven Billion Joe’s, er People, taking what he’s been saying for years and going to market with it.  The idea here being that if you can identify the affective, behavior, and motivational drivers of site visitors, you can maximize cognition in elements on the site (like pictures, text, informational flow) to appeal to target segments and persuade/provoke desired behavior.  It’s like a higher rung on the optimization ladder.  It’s not test what they see, it’s figure out how they think, then make the site better because of it.  Cool stuff.  Blows my mind.
• Integrated, multichannel marketing.  Just ask my good friend Akin Arikan, author of the newly released Multichannel Marketing.  (Disclaimer: I was a technical editor on the book.  It’s easy to do when you edit brilliance).  Make sure to check it out!  Marketing in general will become more Internet-centric, but will continue to clutch the roots of broadcast and print.  You will have the database marketer and statistical modelers working with a union of web channel and offline data.  What’s preventing it now?  A unified marketing database.  You see companies like Salford Systems circulating in this space.  And take a look at Unica’s blend of Enterprise Marketing Management…  I’d stay tuned to see what Unica has up their sleeve for bringing together online and offline.  When you can segment and target across online and offline campaigns, if I were pure web channel player only, like Omniture or CoreMetrics, I’d be a bit concerned that people are waking up to open systems, not closed black boxes.  WebTrends is already moving in this direction…  But they all remain far behind Unica when it comes to multichannel marketing.

And that’s just a few of the things the phenomenal eMetrics got me thinking about…  I hope to see you in Washington DC in October! 

Wow!  It’s been a few weeks since I’ve had any time to blogviate. 

What other things do web analysts do?  Besides blog and do WAA stuff… And ensure tool configuration/administration, date collection, data verification/validation, reporting, KPI generation, conversion optimization, deep site analysis, stakeholder guidance, outcomes evaluation and so on… Well the fun answer is “it depends” on a things like your boss, the organization you work and the holy org chart, your recognized skill set, and what you want to do.   But as I talk to my colleagues in the industry, I’ve noticed some web analysts do a lot of different things.  Here’s a few beyond the norms (or in some case maybe part of the norm, but not often discussed):

• Write business requirements.  You may be writing biz reqs for the extension and maintenance of your own tool, or you may be asked to participate in the definition of the metrics strategy for product or site features.  The analyst may define the attributes, capability, and characteristics that are necessary to accomplish given business objectives.  Generally these biz reqs will be functional (the system must do this in this way and look like this) and not technical (but every so often you may need to justify why you keep saying “ah, page tags, not logs” or vice-versa or packet sniffers or hybrid).  Fun!  And time consuming! 

• Participate in product development and usability discussions.  A rich topic here for sure.  As web analysis sort of fractures into those who study how the site routes visitors, navigational elements, information architecture, and into those who prepare AB and MV tests and report the results, it’s not uncommon for analysts to be called into to determine what should go where and what functionality should or should not exist on the site in order to drive business or conversion goals.

• Contribute to the keyword set.  As I explained in my last post, web analytics is morphing into multichannel analytics.  Analysts are increasing leveraged to participate in and analyze the outcomes of SEO and SEM.  Based on keyword data, I have a few friends who spend a ton of time selecting and managing the keyword portfolio and even the bids! 

• Have a say in “strategy”.  Analysis informs tactical decision making, which is guided by strategy (and analysis and decision making and strategy again).  When fully leveraged, a web analyst has much to offer the strategic decision making process.  Think about something as simple as using referrers to establish content syndication and affiliate partnerships…  Cool.

• Guide the content agenda.  For those who work in what my buddy, Alex Langshur (who runs a boutique consultancy in the public sector), calls “content-rich” and “mission driven” sites, the web analytics tool has utility as an editorial or content research tool.  From understanding what keywords/phrases are driving traffic to determining whether the editorial plan is actually mapped to the information demands of site visitors, web analysts can have a lot to say, if asked.  But be weary, the last thing an editor wants is some hot shot web jockey telling them what to write. That’s not what I’m saying to do, rather, some analysts work with content and editorial teams to ensure frequently demanded content topics are rounded out on the site, expanded on/developed, put on the content plan, or simply just known about, so the content folks can do what they do… 

• Code. Yeah, some of us know how to do it, and many of us just don’t tell anybody.  Because “that’s not what I want to do anymore” as my friend who works at a local agency told me the other night.  My personal opinion is that code is better left to the coders, but any web analyst who can throw down with web development and talk about things like X-Forwarded From headers will only make themselves more valuable to the organization.  Then again, some analysts would rather analyze data than futz around with overly esoteric tags and variables and the plumbing of web pages.  Then again some of us love that.

• Direct IT.  Those of us fortunate enough to have control over our web analytics technology already know they’ll be spending perhaps inordinate amounts of time with our good buddies in IT.  They may be the audience for your business requirements, or you just may need to connect with them to ensure your technology is factored into the larger plan for next generation integrated, service oriented architectures.

• Due diligence on acquisitions.   A fun one for you MBA’ers is when you get drafted into the acquisition or merger process, having to examine the target’s web traffic.  You gain real insight into the core of their web business, and may even find things, I’ve heard, like page view inflation from not filtering bots on including things like favicon.ico to inflate page views.  Heh!

And more!  So yeah, it’s not all about spending all day just thinking about who comes to the site, why, what do they do, and do they complete their purpose according to specific goals.  While that is all a big and important part of it, the role of web analyst can go far beyond tradition, if you are capable and you work for the right business that lets you excel!

Hello good readers!  Every month I write a column for MediaPost’s Metrics Insider.  Here’s my most recent one:

Companies that derive revenue from multiple channels often have two analyst teams: the “database marketing team” and the “Web analysis team.”  These groups tend not to communicate.  In some companies, however, these teams are merging to form the “multichannel analytics team.”  This specialized team analyzes, reports, and evaluates both Web data and offline data — often in coordination with the “business intelligence team.”  The emergence of this new team structure makes sense for companies that are shifting their offline business models to become more online-centric, and thus want to understand value-generating connections among channels. 

Several macro-level catalysts are necessitating the shift to a multichannel approach to data collection and analysis.  The ongoing mainstreaming of the Internet channel for enabling commerce, conversation, and relationship marketing is certainly pushing this movement.  The burgeoning set of analytics tools that integrate with other technologies to enable event detection and trigger a customer-specific response is also promoting change in the way companies think about connecting offline and online data to improve overall business performance.

If database marketers and Web analysts are evolving into a new type of team, then what roles are necessary on this new multichannel team?  Here are a few:

• Web Analyst.  The overall Web analytics professional has a deep understanding of the Web channel.  This person uses a Web analytics tool to understand the performance of site traffic, online marketing campaigns, and to segment Web data in order to understand how visitors referred from certain channels navigate (or don’t) through the site.  They understand, measure, and report whether the site is fulfilling its purpose for conversion, task completion, and other KPIs when compared to business goals.  
• Site Optimizer.  A niche type of Web analytics professional, the site optimizer is in charge of determining the right approach for configuring and reporting the results for AB (champion/challenger) and multivariate tests.  This person is all about testing components of site and page design to yield the best combination of elements that provides a lift in a particular metric against a goal, such as conversion rate.  Content targeting may also fall under this person.
• Social Metrician.  Another niche type of Web analytics professional, the social media measurer is concerned about the performance of customer touchpoints outside of the main Web site.  He or she collects, monitors, and analyzes data related to things that happen “out there, on the Internet,” such as syndicated video, mobile, widgets, blogs, social networks, and other social media.
• Database Marketer.  The traditional offline analyst and database miner.  This role analyzes data from channels that are not online but may reference and promote online interaction, such as television, radio, print, catalogs, and direct mail.  Of course, these analytics skills can be applied to online data as well!
• Search Analyst.  The analytics professional in charge of keyword identification/selection, keyword management, bidding, and analyzing the outcomes of search.  He or she may be in charge of analyzing site performance against known SEO goals too, not just SEM.
• Market Researcher.  The traditional market researcher gathers, analyzes, and reports data about the overall market, key competitors, and customers. 
• Qualitative Analyst.  Part market researcher and part analyst, this individual is in charge of online customer and visitor surveying, relating customer feedback and visitor opinions to the context of on-site behavior to help deduce “why” people did something on your site.
• Ad Analyst.  Solely dedicated to assessing the performance of advertising campaigns, the ad analyst assesses and educates clients on ad campaign performance both online and offline.
• Audience Measurer.  The wielder of an audience measurement tool informs competitive decisions, influences media plans, and provides benchmarking and competitive data to give context to other data analysis activities, such as keyword bidding or media buying. 

How would these professionals all work together?  The market researcher’s data is used to help craft a customer-focused and competitively differentiated campaign strategy.  The audience measurer provides data that focuses the strategy on the right online demographics and sites, while the database marketer mines historic data to figure out the best-performing offline tactics for the identified demographics. 

Let’s say a mix of search, social media, and online and offline display ads are selected as part of the campaign.  The search analyst concentrates on SEO/SEM, while the ad analyst tracks the performance of display ads.  The social metrician examines the social media ecosystem’s response to the campaign.  The Web analyst analyzes how campaign-referred visitors behave and navigate through the site, taking into account the context of the qualitative analyst’s voice-of-customer data.  Meanwhile, the site optimizer tests landing pages and funnels to ensure they effectively convert visitors and fulfill business goals. 

For many companies, it would be unrealistic and perhaps impossible to find and hire people to fill each of the roles I’ve presented above.  In fact, in most companies these roles and activities are completed by only a few people, if at all.  An option for companies that seek to expand or combine teams is to look at consultants, contract workers, and full-time equivalents allocated across multiple people.

That said, companies that are unable to bridge together online and offline analytics teams will miss important data points.  In the digital future, we’ll see different types of analytics professionals working together across channels to yield profitable insights that support campaign and business goals.

At some point in the career of a web analyst, you will be asked to investigate, assess, and possibly judge the current state of how a company “does” web analytics.  What are some of the areas you should ask about?  Here are some thoughts and a few questions to ask to help inform your analysis (and grease your mental gears):

• Business strategy.  Why does the organization do web analytics?  What’s the goal of having a web analytics team?  Who defines the strategy?  What is the strategy?
• Analytics organization and team structure.  Who is the chief owner of web analytics?  What does the analytics team look like?  How has the team structure been formalized in the organization?  Is the web analytics team effectively staffed and have enough control over resources to do the job?
• Process.  What analytics processes have been defined?  How does a site or site feature progress from not being measured to being effectively measured?
• Data collection. What methods for data collection are being used?  How much data is being collected, and for how long is it stored, and at what level (i.e. detail, aggregate)?
• Reporting.  What data is reported?  What do the reports look like?  Who creates them?  How are they distributed, and in what format?  To whom?  When?  How?
• Analysis.  What’s the difference in this company between reporting and analysis?  How is analysis communicated to stakeholders?  When?  How?
• KPI’s.  What Key Performance Indicators are you measuring?  How are they relevant to the business?  What actions have people taken from KPI analysis that improved business performance?
• Segmentation.  What audience and customer segments exist?  What audience and customer dimensions and attributes are segmented?  Why are they meaningful to the business?  What has the business learned and what action has been taken from the current segmentation analysis strategy?
• Technology.  What analytics technologies are being used?  What does the schema for web analytics look like?  What homegrown technologies are used?  What external technologies have you bought or deployed for analytics?
• Integration.  How is web analytics data integrated with other internal and external data?  Is it integrated with other systems, how? 
• Site Optimization.  Does the company test landing pages, and/or use AB or Multivariate testing software?  If so, whose software, and what business gains have been realized?
• Advertising/Advertisers. How is analytics used to inform or enable advertisers and advertising?
• Privacy.  What safeguards does the company take in protecting analytics data? 
• Qualitative Data.  Is qualitative data contextualized with web analytics data? Do you capture voice-of-customer data?  Use Net Promoter Scores?  Have a research department?  Does web analytics collaborate with research? 

Those are just a few questions to ask.  Many others can be asked.  What would you want to know, and what would you ask?  Please leave a comment.  I’d love to hear your thoughts.

Now for some random thoughts:

• News from Orem.  API / Fusion / Video Tracking… cool.  I’m pretty psyched that Omniture announced a web services API.  That’s fantastic, and confirms how truly important integration is now and will be in the future for analytics data (as I’ve been saying for years… Google will be next). 

Omniture has announced a new methodology, Fusion, and improved capabilities for tracking video.  All sounds very exciting.  But, like Eric, I’m wondering what revolutionary new methodology Fusion really is?  Or is just what Eric’s been saying for the last 4 yearsbranded by Omniture and delivered by the Great Belkin? 

Regarding the video capabilities, I haven’t seen a real demo yet, but I wasn’t immediately impressed with what I saw on my friend Marshall’s blog.  Instead of quartile tracking, it seems like you track the playhead (the part of the video playing) across audience aggregates in increments of one-twelfth, and you get some bubbly visualization (what would that look like with 10,000 videos on your site?), and better access to forums.

I’m hoping I haven’t seen the whole ball of wax, and I look forward to Omniture giving me the grand tour. 

But for a playhead visualization, I was much more impressed with what I saw from Visible Measures and their engagement curve.  And what the heck are those folks at Divinity Metrics up to for measuring video? 

• News from Novato.  One of my favorite gangs of web analytics folks reside in Northern California.  My colleagues at Semphonic have just released a rather impressive “Omniture Implementation Toolkit.” 

I was able to procure a copy, and I’m totally impressed.  It’s full of hard-learned and hard-earned real world practitioner knowledge.  If you are trying to implement Omniture, it is well worth the money. 

Now I’m not sure if this document competes with or acts as a companion to Fusion.  All I can say is that I know the folks at Semphonic are smart, savvy, and very experienced, and there are thousands of Omniture customers out there who could benefit from this document.

• X Change Conference.  I am totally excited for X Change brought to us this year by Semphonic and Web Analytics Demystified.  The last X Change in Napa at COPIA was one of the most intimate, educational, stimulating, and enjoyable conferences that I’ve been too (and did I mention the wine?).  It was pure “class” all the way (in both the sense of style and learning, and did I mention the wine? ;-). 

This year attendance is limited to 100 folks (99 if you count me ;).  Last year, I huddled on “Deploying Measurement Systems in Globally Distributed Enterprises.”  

If you aren’t familiar with X Change or Semphonic  check them out, and make sure to read a few of my favorite bloggers - the prolific deep thinker and expert Gary Angel, the always impressive (and fun) June D(ershewitz), and bright author and web analytics veteran, Phil Kemelor.

In Part 1, I mentioned that the Web Analytics team will look very different depending on company and business goals.  I identified three elemental constituents (business strategy, analytics, and technology) necessary to select a web analytics tool, and I divided them up into three different folks who fill those roles when you’re selecting an analytics technology.

Once the tool is selected, companies will want to create a structured team framework with defined roles and responsibilities in order to successfully deploy the tool.  What I’m describing is a suitable team-structure that enables you to successfully deploy a tool in your organization that finally gets you to a point where you are able to do web analysis. The team structure I describe below lets you get to the hub-and-spoke model that my good friend, Eric Peterson, described in these Part 1 and Part 2 of “what’s your web analytics communication strategy?”   What Eric excellently describes takes the team to the next level of actually doing Web Analytics.  It’s excellent stuff that I encourage you to read.

A formalized team structure for rolling out a web analytics tool may have the following constituents: 

• Executive Advisory Board.  Beyond the Executive Sponsor mentioned in Part 1, these board members are the ones who really control the budget and strategy at the highest level.  They may be your boss, your bosses’ boss, or board members at your company. Regardless, they are the analytics project champions at the highest level in your organization – often C-level executives.  They support the project structure and analytics strategy, confirm the scope of the project, and approve any budget allocation.

• Steering Committee.  You may be on the steering committee, Mr Web Analyst, or it may consist of very senior representatives of all the internal teams that the project touches.  These people work to define the strategic direction of the project, decide on how to resolve critical issues that come up during the rollout, and generally handle any escalations.

• Web Analytics Expert.  That’s probably you, fine reader.  You will provide analytics-based strategy and informed decision making across all aspects of the project. You’re obviously critical to the success of this project, and will ensure technical, tactical, procedural, functional, and financial adherence across the entire analytics program.   You are the chief evangelist, and will define the overall reporting and KPI structure.  In addition, you will be responsible for the overseeing the partnership with your vendor. Other things you may do will include managing costs, coordinating schedules, risks and resources, and reporting overall project status and important communications (often with the help of a project manager) to the steering committee and advisory board.

• Web Analytics Team.  If you are lucky enough to have a team, these folks will gather and document project and technology requirements, liason with business stakeholders, lead training, build awareness of and evangelize web analytics, and in general work with those who receive reporting and leverage the tool.  In many companies the solo web analytics expert will do all this stuff (and drink a lot of coffee or green tea too!).

• Project Manager.  A web analytics rollout can be complicated. While the solo web analytics team member may be expected to project manage, it may make sense to give that role to a formal project manager (y’know a PMP) who works with the Web Analytics Expert to manage the schedule, risks, resources, communications, change, and quality management plans.

• Business Partners.  Since web analytics will touch many different groups, you will need to ensure your analytics team communicates with them.  Business partner are critical stakeholders.  They can’t be neglected.  They will provide business requirements, test the technology, and work with analytics team to ensure the technology, reporting, KPI’s, and analysis you rollout helps drive business performance.

• Subject Matter Experts (SME).  Similar to business partners, these folks will probably be more technical in nature.  The Technology Expert you worked with when selecting the project will transition into a roll as a SME.  You may have one SME who oversees the overall technology architecture, another who coordinates BI resources, another who QA’s the system, another who creates interfaces to your data warehouse, and perhaps another who acts an IT contact covering issues across the operating system, database, security, and networks (especially if you are running an in-house tool).

• Vendor Professional Services Team Members.  Last, but certainly not least, are the folks sent from your vendor to do what you want them to do.  From installing the application (in a in-house environment), functional training, to advanced configuration, these people are critical to ensuring that you don’t make simple, avoidable mistakes that can thwart your efforts and delay the successful rollout, golive, and extension of the project.

In reality, you may not be able to effectively isolate all of these groups to support your analytics rollout.  To some degree I’ve presented big company structure above.  In smaller companies, one or only a few people may do all of the interlaced activities necessary to rollout a web analytics tool.  Regardless, I think the groupings I’ve presented above define the primary roles and responsibilities necessary for success when rolling out a web analytics tool (in fact I presented things in a general way to apply to other rollouts as well).  The next challenge comes once your up and running (make sure to read Eric’s posts)… You need to use the data to improve business performance and guide strategy, decision making, and online tactics that reduce expense and yield profitable revenue.


Image by Jim Sterne, from Emetrics 07 San Fran.

The best answer to that question is that “it depends.” The members of the Web Analytics team vary widely by company based on a number of factors, such as the company size, where you are in your rollout, capability maturity for analytics, established corporate processes, the number of sites to implement and maintain, the granularity of the implementation, the technology used, the number of people to which you give access, support requirements, and many more company-specific factors.

For many companies, the number of web analysts can be counted on one finger of one hand.  The lone cowboy is expected to champion the effort, and pretty much do everything under the sun - from orchestrating the tagging to reading the data to being a project manager.  Sure, that can work.  It just means empowering one individual to get the entire job done and giving them the budget, resources, authority, and clearance to make all the decisions - and communicate up the chain.  In reality though, few companies can find the right person who can do it all. Does it take a village to do web analytics?  We’ll not quite, but it does take many different people to select, implement, extend, and maintain a web analytics platform.

Over the next two (or maybe more) posts I’m going to cover my take on what skill sets, roles, and responsibilities are necessary on for doing web analytics - from when you start thinking (and believing) that you need a web analytics tool, to when you implement, to the ongoing day-to-day operations of the web analytics department and maintenance of the tool.

When you are just beginning you web analytics selection, prior to implementation, you want a small, focused web analytics team (watch out for too many cooks!):

• An Executive Sponsor.  This person is usually the HIPPO (highest paid person in the room) - until their boss gets involved ;).  For some companies this could be a C-level executive, VP, or Director.  The Executive Sponsor is in charge of setting the broad-based strategic vision for the analytics roll-out.  They may have hired you!  They help to set the overall scope of the rollout, remove obstacles, and set and control the budget. They are who you go to “escalate.”
• A Web Analytics Expert.  This person is most likely you. You may be an MBA, a techie, a marketer, an IT person, or someone who was promoted into the position.  Lucky you!  You will be in charge in identifying a vendor consideration set, writing an RFP (if you do one), identifying business requirements, collaborating with internal stakeholders, doing the due diligence with the vendor, determining the features and components needed in the web analytics product, figuring out the appropriate financial model, championing for the budget, communicating with internal stakeholders, debating the merits of the technology with your internal team, and generally supervising and stewarding the whole selection process along so that the job gets done (and your executive sponsor looks good).
• A Technology Expert.  This person could be you too, Ms. Web Analyst. Or it could be a systems architect, a data warehousing expert, a dba, an application engineer, or another tech-savvy colleague with a computer science degree (or maybe not - a degree from the school of hard knocks). This person will vet the underlying technology provided by the vendor.  You want this person to ask deep, hard questions about the innards of the technology offering to ensure the technology will match and scale to your internal technical requirements.  Say you want to integrate internal data with your web analytics tool.  This person should know all about your corporate systems, what data your company has, where/how it’s stored, other technology projects, and so on.  They’ll help you ensure technology you are leaning toward fits into the technology ecosystem at your company at a very deep level.

After short-listing vendors, doing the due diligence, pilot/proof of concept(s), you’ll finally make a decision about what tool to buy (or perhaps you’ll determine a free tool meets your requirements now (but will it in the future is the question you should be asking… LOL!).

At the “buy” decision is made, the Web Analytics team will grow to include a more people with different skill sets, roles, and responsibilities.  I’ll cover that in my next blog post.

One of my favorite personalities in Web Analytics is Phil Kemelor - author of the Web Analytics Report.  Phil is also Vice President of Strategic Consulting Services at SEMphonic and an analyst at CMS Watch. 

I first encountered Phil in late 2006 when he called me up to discuss my take on the vendor landscape at that time.  We instantly hit it off because, like him, I seek truth, and have a lot of trouble believing vendor spin and hype.   We found each other’s insights “refreshing,” and we’ve kept in touch ever since, grabbing food together/swinging back a few beers/hanging out at conferences, chatting via email, and just generally staying in touch.

Fast forward to late 2007, early 2008, when I find out that Phil was revising his awesomely comprehensive Web Analytics Report and releasing the 2.0 version.  We’ll the time has come, and WAR 2.0 is out.  If you can swing the cost (it’s over $1000 US), I highly recommend purchasing it, especially if you are new to the industry and trying to make a vendor selection, or if you are old to the industry and want to get a solid sense of current vendor capabilities.

The 343-page report goes over all sort of juicy stuff - from beginner information about “what is web analytics” to concepts around the “web analytics business case” to deeper dives into “web analytics technology.”  It covers an abundance of useful information about data collection, data sampling, data exporting, dashboarding/reporting, segmentation, licensing, organizational requirements, and how to select a web analytics vendor.  He’s also done a good job, imho, discussing 15 different vendor tools, which shows you that the vendor landscape is a lot larger than just Google Analytics and Omniture - that’s for sure.

So if you have the $$$ to spend, check it out.  It’s an excellent addition to my analytics library.  Good work, Phil!

What’s the future hold for Web Analytics in 2008?  Here are a few predictions:

• Google Analytics releases a real API for getting (and perhaps setting) data.  As you know, I think GA is a fine tool for web analytics, but has severe limitations when you want to control over your data or to feed data into other systems.  Thus, I predict Google Analytics will go beyond the “Tracking API” and release a real API that allows you to at least get data out of the tool (if not set data as well).  Think of what Feedburner does with their REST-based Awareness API.  Wouldn’t that be nice to have with GA?!
• HBX Analytics goes away.  I’d be more than a bit nervous if I were an HBX customer because Omniture is going to sunset HBX and migrate everyone to SiteCatalyst, then try to aggressively sell them the rest of the suite. 
• Long live Visual Sciences.  VS is a powerful tool quite superior in some regards and very different than anything else Omniture offers.  It’s also real in-house software, not some blackbox.  VS’ extensible schema, flexibility in reporting, scalability, and performance is quite unparalleled in the industry.  I can’t envision Omniture killing it (unless they peel it apart in order to create Discover 3), like they will HBX. 
• WebTrends rebrands.  I’m not sure if you agree, but imho WebTrends Marketing Lab was an attempt to rebrand WebTrends.  I expect that interim management will continue attempting to differentiate WebTrends by rebranding products and perhaps the entire company.
• New and updated standards are released.  As a member of the IAB’s Measurement Council I can tell you that the IAB is getting ready to release the IAB Audience Measurement Reach Guidelines, which attempt to clarify and take a stand on various aspects of server/client-side analytics and audience measurement.  I also envision the WAA increasing the number of terms they define.  But standards are just dandy and quite meaningless unless they are adopted… thus…
• Standards enforcement is attempted in order to propel adoption. Existing and forthcoming standards will be enforced in 2008.  Enforcement from the WAA will probably come in the form of a publication of a matrix or documentation citing which vendors adhere to the standards and to what degree, what’s missing, what’s different, and so on.  If decision-makers who control budgets believe in standards, this type of document will cause the question ”do you adhere?” to be asked.  If vendors start losing deals because the answer is “no, not at all,” vendors will adopt the standards. 
• Internal data integration becomes more important for companies and problematic for ASP’s.  When we talk about “integration” I often think people can be a bit shortsighted.  They want to integrate data from other third-party services and tools (like Salesforce.com and their ad server).  While there is certainly real value in integrating external data with web analytics data, significant value comes from integrating web analytics with internal data, such as data residing in internally-hosted CRM systems, finance, subscription, and lead generation databases. Most vendors have barely figured out how to deal with detail-level external data integration in 2007, even though many customers are demanding it.  I expect that in 2008, internal data integration will be more commonly demanded and even more problematic for ASP’s. 
• BI tools provide better support for and integration with Web Analytics tools.  The current allotment of “enterprise” level web analytics tools are inferior to the capabilities provided by business intelligence tools from companies like Business Objects or Cognos.  Expect these BI vendors to create features for dealing with web analytics data in 2008.  Either that, or these web analytics tools need to grow up and learn a few things from BI. 
• Web Analytics as performance management.  KPI-based site optimization means using data to guide the modification of user experience to deliver on goals.   Since goals are measurable and can be plotted against performance, it’s totally logical to use web analytics as a performance management tool.  Expect to see that gestalt in tool usage come into vogue and be discussed more in 2008. 
• Web Analytics as part of business process automation.  Having the marketing department fielding page tags with campaign codes may work for some (small) companies, but when you work for an enterprise with thousands of clients and simultaneous campaigns across multiple channels, endemic tagging and subsequent tool configuration becomes challeging.  As part of the web analytics process, I expect to see tools support some level of business process automation enabling web analytics.
• Features for measuring the Mobile Web.  Right now, with a log file based tool, I can segment out Mobile traffic based on user agent.  If I want to use a page tag, I have to consider js limitations.  The mobile web is the next frontier, and I only know of one web analytics vendor who is doing a decent job measuring it right now, so I expect to see more features released this year for measuring Mobile.  

So that’s that.  Like a band named PIL once said in the song called Rise “I could be wrong, could be right!”  Am I off-base, misguided, accurate, do you disagree, agree, then let me know… I’d love to hear your thoughts and your predictions for Web Analytics 2008…

Every month I write column for MediaPost.  This month I wrote  a short summary piece I thought I’d share with you in case you missed it.  Here it is:

As 2007 ends, I thought it worth looking back, from the practitioner perspective, at just a few of the issues that have shaped Internet measurement and thus online metrics over the last year:

• The Page View is Dead, Long Live the Page View.  During 2007, technologies like AJAX and Flash continued to erode the construct of the page view.  These technologies render content in a browser but do not always make requests to the server to do so.  If the technology you are using to measure behavior requires the page request and you do not have a page request, what do you measure?  The major vendors of online metrics tried to answer that question. 

Various audience measurement companies claimed “total minutes” and other time-based derivatives were better alternatives to measuring the page view.  Web Analytics companies rolled out features for measuring “events” subordinate or equal to the page view (and highlighted already existing time-based metrics).  Ad serving companies made inroads in reconciling view-through to assist advertisers in understanding the latent effect of ad exposure on the purchasing lifecycle.  Yet all these technologies still count and report page views.

• Engagement, Engagement, Engagement.  One of the hot topics in 2007 was a carryover from 2006.  Definitions for “engagement” emerged from the worlds of advertising, social media, online metrics, and more.  Engagement has been described as “turning on a prospect to a brand idea enhanced by the surrounding context” to “repeated, satisfied interactions that strengthen the emotional connection a customer has with the brand” to “apparent interest” to the more metrical “estimate of the degree and depth of visitor interaction against a clearly defined set of goals.” 

“Engagement” is very specific to the site being measured and full of nuance.  This fact has led agencies, consultants, and various companies to create complex engagement indices consisting of measures of key behaviors.  Behaviors are tallied and segmented in order to calculate an engagement metric, which is then used as the basis for site optimization.  These indices go far beyond often-cited simple time-based measures of engagement.  For a well-thought-of example, see Eric Peterson’s Engagement Metric.

• Cookie Deletion, Again!  Jupiter Research, in 2005, first uncovered and quantified how cookie deletion can affect unique visitor numbers in web analytics systems.  The effect of cookie deletion is not quantifiable in the basic way audience measurement companies want to quantify it in 2007 – by only examining cookie deletion rates from self-selecting panelists who visited one portal site and an ad server. 

Cookie deletion behavior varies greatly by audience segment and by site.  It may be as much of an accuracy problem in web analytics as selection bias and coverage errors are in panel measurement.  It is worth noting that some audience measurement firms use cookies to collect panel data. 

• Black Box Audience Measurement.  Many questions were asked about whether audience measurement companies adequately measure “unique visitors” or “unique users” or just the frame of self-selecting “unique panelists.”  In audience measurement, counts of “unique visitors” are generated using complex, black-box mathematics that project observed metrics to the entire online universe.  The projections are always unequal to the same data provided by other audience measurement companies or web analytics tools.  Panel inconsistencies (across at-home, at-work, at-university, or specific to the geography being measured) may cause some level of bias and error. 

Accounting for the difference between actual, observed panel metrics and projected metrics is perhaps even more challenging to clarify and resolve than the measurable effect of cookie deletion. 

• The Continuing Need for Standards Enforcement.  2007 was the year two significant industry bodies continued working on standards related to online metrics: the Internet Advertising Bureau and the Web Analytics Association.  While each organization serves the needs of different constituencies, they both share the inability to enforce standards.  Both bodies can say what you should do, but not what you “must” do. 

Throughout 2007, these issues (and others) brought increased attention and scrutiny to online metrics.  Corporations are inextricably linking online metrics to site and channel strategy and performance, and thus to overall corporate profitability.  The “numbers” are now more important than ever for managing an online business and maximizing online revenue.  Nevertheless, questions are still being asked about accuracy, precision, usage, and sources of online metrics.  We have a ton of collaborative work to do in 2008 to provide the best answers and numbers we can. 

Happy New Year!