A question any practitioner of Internet-based analytics will be asked by many different stakeholders is “why don’t the numbers match?”  Counts of the identically named metrics from ad servers don’t match the web analytics tool, which don’t match the for-pay third party audience measurement tools, which don’t match the free audience measurement tools, which never match any of the homegrown internal measurement tools.  And none of them ever match each other.

So it’s a good question certainly valid to ask.  The answers are even fairly easy to understand, but the root causes are often difficult to pinpoint and even harder, if possible at all, to remedy.  The fact of the matter is that data discrepancies in analytics result for a multitude of reasons, such as:

• Different data collection methods.  We have a bunch of tools and services that collect web data using various, non-standardized, proprietary data collection methods.  Ad servers use javascript page tags.  Many web analytics tools use page tags too, but it’s not uncommon in web analytics to use additional methods, such as log files or packet sniffers.  Or perhaps a combination of these methods, called hybrid data collection.  And all the tools have different algorithms for processing the data collected.

On the audience measurement side, data is collected from self-selecting panels who install proprietary software (i.e. toolbars and so on) on their computers, perhaps at work or at their university, but most likely at home.  Then, the collected data from different panels is rolled-up and combined, and the limited subset of the Internet population that chooses to be monitored, in exchange for some incentive, is inflated and projected to the entire Internet audience using proprietary statistical methods.  We also have data collected from a limited set of geographically specific ISP’s.  And regardless of whether we’re talking about audience measurement or web analytics, the different data collection methods often, but not always, involve cookies and all their inherent issues of cookie deletion.  

• Unique data models.  Ad servers aren’t focused on counting page views and the other dimension of web analytics (visits, time, and so on).  Rather ad servers focus on serving and counting impressions served (and loads of related derivative calculations, like CTR, CPC, and view–thru).  Metrics are based on an ad request and an ad code.  Ads may or may not be targeted to a page, and instead to various constructs, like a “zone” or “keyword.”  What that means is that the “page” dimension may not even exist in your ad server’s data model.  In other words, you aren’t looking at impressions measured on a page, but rather at the number of impressions served in a different conceptual construct.  That’s one of the reasons why people say metrics and ad-serving systems “don’t measure the same thing.”
• Untagged pages.  Specific to technologies that collect data or serve ads using javascript page tags, there are challenges to ensuring and verifying complete coverage of page tags across every page on a site.  When the pages aren’t all tagged with the different tags for the assorted technologies, guess what?  The numbers won’t come close to falling within tolerable variances.  And questions and skepticism will ensue.
• Non-JS executing clients and ad blocking software.  Let’s imagine for the moment, your site is perfectly tagged for all technologies, so the numbers between your ad server will be close to your web analytics system, right?  Nope, regardless of data model issues, not all browsers execute javascript and many Firefox users have installed Ad Block Plus. 
• Cookie issues.  When you’re counting based on cookies, third-party cookies get blocked (often by privacy software).  Many ad servers and web analytics tools still serve third party cookies, and many corporations have not tricked out their DNS to accommodate this issue.  And we all know how cookie deletion affects unique visitor counts, even if you use first-party cookies.
• Many other issues.  Latency from visitors moving off the page prior to the tag executing to latency in the call to pick up an ad from a third party while your ad server counts the traffic (so your ad count differs from the agency’s count), to refresh rates making it hard to correlate page views and impressions, to no rich media installed and no fallback, to robotic traffic not being filtered from logs or tags, to certain types of user agents (such as mobile devices) not executing javascript… there’s a whole host of other factors that cause data discrepancies.

And of course, there’s always the nebulous issue around the complete lack of consensus-based, enforceable standards for online measurement.  No industry organization can say what vendors or companies “must” do, only what they “should” do… And no industry body is going to get successful companies to change their secret sauce just because they said so…

So what’s a practitioner to do?  Understand the potential sources of discrepancies.  Work with your team (from IT to vendors) to prevent and minimize the root causes when possible.  Educate your team when discrepancies are not remediable.  Ensure you use the different sources of metrics judiciously in the context of your business goals.  Finally, realize that none of the tools are more “correct” than any other.  All of our analytics tools serve different, and sometimes overlapping, business purposes - from counting ads, to influencing media buying, to sizing audiences, to measuring business performance, and to optimizing the site.

One of the many things on my mind in the online world these days is “deep packet inspection.” 

First, let me digress, packet sniffing isn’t new to web analytics.  From Accrue to Omniture (Visual Discover Sensor?) to AuriQ to Metronome Labs.  Packet sniffers are used to “do web analytics.”  It’s an uncommon method when compared to javascript page tags.

Web analytics packet sniffers are used to write logs for sessionization (and thus measure) the traffic on behalf of site owners (who don’t want to use tags or logs).  Once you’ve logged and sessionized you know what content people have looked at or downloaded on your site. 

“Deep packet inspection,” like WA sniffers looks at the entire payloadof packets in real-time across a huge number of simultaneous sessions.  Deep packet inspection, like regular packet sniffing, examines the files downloaded and the content of the pages viewed - the whole ball of wax. 

Deep packet inspection is being offered as a hardware/software technology by companies like FrontPorch and Sandvine (in the US) and Phorm(in the UK).  These companies are selling the technology to ISP’s (like Charter, Comcast, and Virgin Media) so that they can monitor the sites visited and the keywords used by customers, and then use the data collected for behavioral targeting.  The ISP’s want a slice of the juicy, lucrative online ad business.

What’s the difference?  Site owners collect data about what you do on ONE site (or a portfolio of their sites).  ISP’s collect data about what you do on EVERY site you visit.  As I understand it, some of these companies create an anonymous profile of your surfing activity by assigning a unique key to your browser.  Then they monitor the site’s visited by your browser, and use that data so that the ISP, or the companies to which they sell your data, can serve you what they conclude to be relevant, behaviorally targeted ads. 

Get it?  Packet sniffing by site owners = knowing about one site you visit.  Deep packet inspection by ISP’s = knowing about every site you visit.

Now to digress… In web analytics, we know that web analytics data is collected anonymously.  Unless there’s a login, you don’t know exactly who is coming from that IP address.  And in many cases, most companies data warehouses only contain purchase information, not the entire clickstream.  Once the data is collected, if you have the right architectures you can decode cookie values to people, and make that data non-anonymous (i.e PII).  Not difficult to do with some smart BI folks on your side.  

An ISP already knows who you are and can already identify the sites you visit.  Probably not that easily though on individual level.  They can dig through the logs, etc… 

So what’s the big deal and all the hoo-hah about  the “deep packet inspection” Phorm and FrontPorch are doing?   It’s the data they are collecting and the repository they are building containing data about every site you visit and all the content you view and download… Of course, these companies say that it’s all done anonymously and that your “privacy” is preserved “to the greatest extent possible.” 

Now let me quote Sir Tim Berners-Lee about the data collected from Phorm’s ISP tracking: “It’s mine - you can’t have it. If you want to use it for something, then you have to negotiate with me. I have to agree, I have to understand what I’m getting in return.”

And that’s the point of the blogviation, Tim is correct.  In web analytics, we do this - we try to operate within Tim’s constraints.  We enable opt-in with P3P statements and disclosures when you register/login.  Privacy policies disclose what we are doing with the data.  It’s just ethical and smart business practice to do so.

Thus, I think FrontPorch and Phorm and all the ISP’s who want a piece of online advertising should adhere to the following five rules for their services.

1. Move to an obvious “opt-in” model with full disclosure.  Tracking via “deep packet inspection” should be an all opt-in model.  If you want anonymous data from your browser collected so that you can be behaviorally targeted, then you should opt-in to be.  Right now, it’s seems to be all opt-out.  You probably don’t know if it’s being done to you.  It’s buried in fine print you’ve probably never read.  Is that your fault you didn’t read the fine print? Yeah, but the point is it shouldn’t be buried in the fine print…
2. Provide me with access to the data collected.  If I opt-in, I should be able to see the data collected from my browser.  It’s very simple.  I demand to see what you are collecting about my browser.  If you are building a profile, then I demand to see the data collected in the profile.  If it’s all anonymous, then explain how it is in detail, and then follow rule #1.
3. Enable me to edit or prevent the data from being collected.  If I opt-in, I want to be able to edit or prevent certain types of data from being collected.  If you’re tracking my browser, alert me before the data is transmitted, so I can decide if I want to share it.  If a profile is built, I want to be able to edit it!
4. Let me opt-out at any time EASILY. If I’ve opted in, and I’m unhappy with the service, allow me to opt-out simply.  Having to set an opt-out cookie on my browser is absolutely and completely absurd.  I want to be able to fully opt-out at the ISP level, just once forever, not at the browser level every time cookies are deleted.  Make it easy and permanent, not easily deletable.
5. Disclose who you sell my data too.  Like online list rentals, the next step in all this ISP profiling is selling the data to third-parties.  Let me know what you’re doing with my data-before you do it- so I can opt out or prevent it from being sold to parties to which I don’t want it being sold.

Consumers must be given a choice for preserving their privacy.  Anonymity to the “greatest extent possible” is not enough and neither are short-sighted opt-out cookies.  Companies like Phorm and Front Porch would be wise to apply these rules to regulate themselves.  Otherwise freedom-loving governments will almost certainly regulate them. 

And I haven’t even mentioned the issues with net neutrality and deep packet inspection (i.e. traffic shaping and access restrictions (called “throttling” as Clint points out in the comment), have I?

Back from another excellent eMetrics.  I’m a very big fan of the eMetrics Marketing Optimization Summit…  Props go to Jim Sterne for growing this event from a little seed into an incredible, blogworthy blossom.  How involved is Jim in eMetrics?  I’d say he’s completely immersed in every little piece - he even came up to me at the SF WAW (way to go June D!) to find out about the renegade AV work I did in one of the sessions, and to get my take on how it could have been avoided.  He’s that intimately connected to what’s going on.  Macro and micro, micro and macro.  And when you have one of the best Internet Marketers in the world, keeping a tight rein on the Clydesdale of conferences, you know you’re in for one heck of fun ride. 

And so it was for about 500+ of the top web analytics in the beautiful Palace hotel.  Props to consummate conference organizers Matt Finlay and his crew at Rising Media for keeping the road smooth as we all trotted on it as well.  Fanny, you are one helpful polyglot of a marketing manager!  I never knew German keyboards were so wild… Thanks.

The eMetrics sessions were informative and actionable.  The lobby bar and after-hours parties fun and enlightening.  You really can’t ask for more out of a conference.  As I flew home thinking back on it all, there was a lot to blog about, including:

• It’s all about attitude, dude – as in attitudinal data.  Like my father says “it’s all about your attitude.”  And so it is on the Internet in 2008.  From ForeSeeResults, to iPerceptions, to OpinionLab, to CRMMetrix, the often missing link in customer analytics is attitudinal data.  I’m talking here about Voice of Customer (VOC) technology that allows you to ask a question set to site visitors and then apply some sort of algorithm or model to express the meaningfulness of the data in quantifiable terms.  From the American Customer Satisfaction Index to 4Q.  VOC technology enables you to participate in a continuous, automated dialog with your customers in order to identify problem points on your web site and enable you to measure purpose and success of your most valuable segments.  Expect to see some of the big players gobble up these smaller companies.  Omniture, Unica, WebTrends, and CoreMetrics should be thinking about acquisition in this space to round out their offerings.
• Testing, 123… as in multivariate, MVT.  The rage is site optimization technologies beyond the simple A/B, champion challenger, test.  In this category you find folks like SiteSpect (the only non-intrusive multivariate testing solution!).  I’m a big fan of these guys (and was in 2006 long before they ever sponsored a WAW, thanks to a nice demo from Larry at my old job).  Eric Hansen and his crew have specialized software that you install in your data center.  No futzing with damned tags.  Swap out your variations, create different recipes, determine what’s statistically significant in giving you a lift to your macro or micro conversion goal, and you’re off to the races.  The good folks at Google are doing it and doing it well with Google Site Optimizer (thanks for the t-shirts!).  Interwoven is baking in Optimost to the CMS, and Omniture has their Test and Target integrated with the Business Optimization Suite.  Accenture has Memetrics.  Kefta too. And what ever happened to Verster?

In a nutshell, these technologies enable you to test variations of content themes, colors, creative, calls to action, points of resolution, buttons, navigational elements, –whatever you want to call the stuff on the screen—to determine what combination performs best against your goals.  But of course, this is all just software, so don’t get too excited.  The tests are about as good as the people creating them…  And complex tests that take a long time to execute may not finish.  Imagine 1-800-Flowers starting a test in January and not finishing until March, missing Valentine’s Day.  Or Intuit running a test beyond April 15th for a tax product.  Go humbly and carefully into this space, my friends, or you may end up optimizing for everyone and appealing to none.

• Tying it all back to the dollar for profit-generating sites and to the mission of non-profit generating sites…  It seems like a “no, duh” moment but metrics for the sake of metrics can be a big waste of time.  If you can’t tie metrics or visitor actions back to value on a revenue-producing site or to the betterment of a non-profit site’s core mission, then what’s really the point of the measurement…  That’s why I’m a big fan of the stuff ZaaZ does.  They totally get the fact of how actionable metrics turn the wheel of Internet commerce and ad-based models, and they can model it all to prove it out the ROI.  Folks like newly elected WAA Director Alex Langshur’s company Public InSite do similar stuff for content driven sites.  That is they know how to use metrics to optimize the channel to goals, not to just puke confusing data, like most web analytics tools do.  Again, it’s all about the people you hire, not the tools you use… My good friend Avinash, right again!
• The emergence and rise of deeply psychological and neuro-behavioral methods for automating persuasion and conversion.   Anyone who knows my good friend Joseph Carrabis, over at NextStage Evolution, knows that besides being one heck of giant kite flying, music master, he’s also got the models and the patents to help target and respond to human behavior across programmable devices.  We’re already seeing some companies, like Seven Billion Joe’s, er People, taking what he’s been saying for years and going to market with it.  The idea here being that if you can identify the affective, behavior, and motivational drivers of site visitors, you can maximize cognition in elements on the site (like pictures, text, informational flow) to appeal to target segments and persuade/provoke desired behavior.  It’s like a higher rung on the optimization ladder.  It’s not test what they see, it’s figure out how they think, then make the site better because of it.  Cool stuff.  Blows my mind.
• Integrated, multichannel marketing.  Just ask my good friend Akin Arikan, author of the newly released Multichannel Marketing.  (Disclaimer: I was a technical editor on the book.  It’s easy to do when you edit brilliance).  Make sure to check it out!  Marketing in general will become more Internet-centric, but will continue to clutch the roots of broadcast and print.  You will have the database marketer and statistical modelers working with a union of web channel and offline data.  What’s preventing it now?  A unified marketing database.  You see companies like Salford Systems circulating in this space.  And take a look at Unica’s blend of Enterprise Marketing Management…  I’d stay tuned to see what Unica has up their sleeve for bringing together online and offline.  When you can segment and target across online and offline campaigns, if I were pure web channel player only, like Omniture or CoreMetrics, I’d be a bit concerned that people are waking up to open systems, not closed black boxes.  WebTrends is already moving in this direction…  But they all remain far behind Unica when it comes to multichannel marketing.

And that’s just a few of the things the phenomenal eMetrics got me thinking about…  I hope to see you in Washington DC in October! 

Web Analytics Key Performance Indicators (KPI’s) are critical for breaking through the dataglut spewing forth from your web analytics tool.   I mean there’s a just a ton of data in web analytics, and the majority of it tends not to be very useful or applicable for improving your business performance.  While it’s wonderful to have a tool that lets you cut, cross, and slice loads of data every which way but loose, its can be a real challenge to frame the data or put it in context in a way that helps your business optimize the web site.   That’s why I like KPI’s - they identify meaningful, business-focused relationships in your analytics data.  By understanding KPI drivers, setting expectations for KPI performance, and analyzing your KPI’s toward defined goals for those KPI’s, you increase understanding of data, alleviate data confusion, and provide focus for the usage of your web analytics tool.

For those of you who don’t have a KPI strategy or who are just getting into analytics, an easy way to understand a KPI is to consider the example of when you are driving somewhere and trying to get there within a certain period of time.  If your goals is drive 60 miles (kilometers, my European friends) in exactly 60 minutes, you know that you need to drive 60 miles per hour (or KPH).  If you go faster, you will arrive early, if you go slower you won’t meet your goal and will arrive past your deadline.   So as you travel along the road, you measure the KPI of your speed. That’s what is important to measure on your trip.  Of course you may measure other KPI’s like the amount of fuel left or the miles you’ve traveled… those certainly may be KPI’s you measure.  But you definitely don’t need to measure you compression ratio or oil pressure even though it’s available data from your car.  In the same way, when you are looking at web analytics data, you don’t want to track everything, only those things that are important to your business performance toward goals. 

Several activities can assist the creation of KPI’s.  Here are a few of them:

• Determine the Business Strategy.  Why is the company funding and developing an online mission?  What is the strategy?  KPI’s can help you figure out if it’s working.  To find the KPI’s that will help, the web analyst should be asking the question how can web analytics be used to formulate, implement and evaluate cross-functional decisions that will enable an organization to achieve objectives? How will web analytics be used in the process of specifying the organization’s objectives, developing policies and plans to achieve these objectives, and allocating resources to implement the policies and plans to achieve the organization’s objectives?
• Define the Site’s Goals and why the Site Exists.  I covered this in a post a few months ago.  A understanding of why your site exists enables you to effectively use online metrics.  You need to define the purpose of your site in order to create effective KPI’s.  Once you’ve defined your site’s purpose, you are positioned to examine Web data in way that helps you determine whether your site delivers on its purpose — does it exist effectively?   Create your KPI’s, identify goals for your KPI’s, and track your performance against those goals.
• Recognize Value Drivers.  How does the business make money on the site? Monetization, in cases where profitability is important, influences what you should be measuring.  If you run a media site, you probably make money from content consumption (the recency and frequency of content consumption), conversation (social media, such as contributions or comments), and conversion (the rate at which people complete certain value driving actions, like signing up for newsletters, rss feed, webcasts, print subscriptions, or downloading certain content types, like white papers).  So you create goals for and measure KPI performance around those value drivers.
• Map Organizational Roles.  Classify your organization into audiences for your KPI’s based what they do on your web site.  You may create KPI’s around function or action of the actors who receive your KPI reports.  Function defines the group that KPI’s are focused for, such as product development or editorial.  Action defines what those people do on the site to make it successful.  By understanding function and action of key actors on your sites, you gain insight into the type of data needed in KPI’s and the number of different KPI reports you may need to roll out.
• Understand the Customer.  KPI’s purely focused on internal function and actions are important, they need to be customer focused.   If you think measuring conversion is important, while your customers tend to come to your site for informational or non-transactional purposes and then go elsewhere to convert, you may be disconnected from the reality of why your site exists.   Learn customer goals from VOC (voice of customer) data and by examining historic behavioral data of key segments.  Make sure you don’t create KPI’s that are vain or inane.  Instead create KPI’s that help you guide action internally so that your business meets the needs of your customers.

Framing your KPI development around the five bullet points I listed above will help you create KPI’s that assist your team in guiding business performance toward goals - while not forgetting to consider some of the core elements of online business: business strategy, site performance goals, value drivers, the human organization, and the customer. 

Now segment, segment, segment your KPI’s!

Next Sunday afternoon I am moderating a panel at eMetrics San Fran.  The panel is called ”Web Analytics -vs- Audience Measurement.”  Andrea Hadley at NetSetGo was the brainchild of this panel idea (and yes that is her picture on her site :).  In fact, I was a panelist on the same panel at eMetrics Toronto, filling in for my friend Marshall Sponder.  Since he’s going to be in San Fran, I yielded my seat 0n the panel and decided to stand up at the podium.   Other panelists include Jodi McDermott, Director of Product Management, at ClearSpring, and some other surprise guests (from comScore and IAB maybe)… You’ll have to show up and find out…

The panel description is as follows:

Are you confused about the number of customers visiting your website? Are the metrics reported by your web analytics tool different from the metrics reported by your online media, or by audience measurement organizations? The WAA invites eMetrics Marketing Optimization Summit attendees and the local San Francisco business community of web marketers, publishers and agencies to attend this community meeting. A panel of experts will discuss the value of the metrics, methods and tools used by web analytics practitioners, online advertising media and audience measurement organizations. Find out how-to use these metrics and tools to better understand your customers, your website’s competitive standing and overall website value.

The goals for this panel include:

• Adding clarity around the tools and data associated with each set of technology and metrics - web analytics technologies and website data, ad servers and ad data, and audience measurement tools and data.
• Learning how each data source can be used to expand our understanding of customers, how effective our website is as a business channel, the website’s competitive standing and value, and so on.
• Providing insight into the role of the web analytics practitioner and how this role is growing in importance and influence over business, marketing, product, and strategic decisions.
• Discussing the role of the Web Analytics Association (WAA) and how the WAA serves the practitioner.  That the WAA is an unbiased organization that doesn’t serve advertisers, publishers, or technology vendors, rather that the WAA serves and exists for the benefit and betterment of the the practitioner and the web marketer/strategist.
• Articulating the announcement made at eMetrics Toronto on the important collaboration between the IAB and the WAA for standards review.

My goal as the moderator is not to critique, demean, or criticize audience measurement, Internet advertising technologies, or to embellish or hype up web analytics tools.  Rather I hope to clarify the differences between the technologies and speak about the value they hold together - like I did in my article for MediaPost called the Yin and Yang of Online Metrics.

So why am I telling you all of this on my blog???  Well it’s because I really want your help, whether you are going to eMetrics or not…  Since I’m the moderator, I get to ask the questions, and I don’t want to just ask “my” questions, I want to know what questions YOU would ask if you had the chance to ask.  Of course, those of you reading this and attending the panel will be given the microphone if you raise your hand.

Please help my crowdsource by telling me in comments or via email to judah (at) webanalyticsdemystified.com:

What questions would you ask to clarify the differences and value between web analytics and audience measurement tools?

Any questions you think worth asking from “why don’t the numbers match?” to complexly “what are the differences between audience measurement and web analytics systems in terms of data collection?” would be awesome and appreciated.  Thanks in advance for your help!  I’m eager to see if this social media experiment in blog-based crowdsourcing actually works!

Wow!  It’s been a few weeks since I’ve had any time to blogviate. 

What other things do web analysts do?  Besides blog and do WAA stuff… And ensure tool configuration/administration, date collection, data verification/validation, reporting, KPI generation, conversion optimization, deep site analysis, stakeholder guidance, outcomes evaluation and so on… Well the fun answer is “it depends” on a things like your boss, the organization you work and the holy org chart, your recognized skill set, and what you want to do.   But as I talk to my colleagues in the industry, I’ve noticed some web analysts do a lot of different things.  Here’s a few beyond the norms (or in some case maybe part of the norm, but not often discussed):

• Write business requirements.  You may be writing biz reqs for the extension and maintenance of your own tool, or you may be asked to participate in the definition of the metrics strategy for product or site features.  The analyst may define the attributes, capability, and characteristics that are necessary to accomplish given business objectives.  Generally these biz reqs will be functional (the system must do this in this way and look like this) and not technical (but every so often you may need to justify why you keep saying “ah, page tags, not logs” or vice-versa or packet sniffers or hybrid).  Fun!  And time consuming! 

• Participate in product development and usability discussions.  A rich topic here for sure.  As web analysis sort of fractures into those who study how the site routes visitors, navigational elements, information architecture, and into those who prepare AB and MV tests and report the results, it’s not uncommon for analysts to be called into to determine what should go where and what functionality should or should not exist on the site in order to drive business or conversion goals.

• Contribute to the keyword set.  As I explained in my last post, web analytics is morphing into multichannel analytics.  Analysts are increasing leveraged to participate in and analyze the outcomes of SEO and SEM.  Based on keyword data, I have a few friends who spend a ton of time selecting and managing the keyword portfolio and even the bids! 

• Have a say in “strategy”.  Analysis informs tactical decision making, which is guided by strategy (and analysis and decision making and strategy again).  When fully leveraged, a web analyst has much to offer the strategic decision making process.  Think about something as simple as using referrers to establish content syndication and affiliate partnerships…  Cool.

• Guide the content agenda.  For those who work in what my buddy, Alex Langshur (who runs a boutique consultancy in the public sector), calls “content-rich” and “mission driven” sites, the web analytics tool has utility as an editorial or content research tool.  From understanding what keywords/phrases are driving traffic to determining whether the editorial plan is actually mapped to the information demands of site visitors, web analysts can have a lot to say, if asked.  But be weary, the last thing an editor wants is some hot shot web jockey telling them what to write. That’s not what I’m saying to do, rather, some analysts work with content and editorial teams to ensure frequently demanded content topics are rounded out on the site, expanded on/developed, put on the content plan, or simply just known about, so the content folks can do what they do… 

• Code. Yeah, some of us know how to do it, and many of us just don’t tell anybody.  Because “that’s not what I want to do anymore” as my friend who works at a local agency told me the other night.  My personal opinion is that code is better left to the coders, but any web analyst who can throw down with web development and talk about things like X-Forwarded From headers will only make themselves more valuable to the organization.  Then again, some analysts would rather analyze data than futz around with overly esoteric tags and variables and the plumbing of web pages.  Then again some of us love that.

• Direct IT.  Those of us fortunate enough to have control over our web analytics technology already know they’ll be spending perhaps inordinate amounts of time with our good buddies in IT.  They may be the audience for your business requirements, or you just may need to connect with them to ensure your technology is factored into the larger plan for next generation integrated, service oriented architectures.

• Due diligence on acquisitions.   A fun one for you MBA’ers is when you get drafted into the acquisition or merger process, having to examine the target’s web traffic.  You gain real insight into the core of their web business, and may even find things, I’ve heard, like page view inflation from not filtering bots on including things like favicon.ico to inflate page views.  Heh!

And more!  So yeah, it’s not all about spending all day just thinking about who comes to the site, why, what do they do, and do they complete their purpose according to specific goals.  While that is all a big and important part of it, the role of web analyst can go far beyond tradition, if you are capable and you work for the right business that lets you excel!

Hello good readers!  Every month I write a column for MediaPost’s Metrics Insider.  Here’s my most recent one:

Companies that derive revenue from multiple channels often have two analyst teams: the “database marketing team” and the “Web analysis team.”  These groups tend not to communicate.  In some companies, however, these teams are merging to form the “multichannel analytics team.”  This specialized team analyzes, reports, and evaluates both Web data and offline data — often in coordination with the “business intelligence team.”  The emergence of this new team structure makes sense for companies that are shifting their offline business models to become more online-centric, and thus want to understand value-generating connections among channels. 

Several macro-level catalysts are necessitating the shift to a multichannel approach to data collection and analysis.  The ongoing mainstreaming of the Internet channel for enabling commerce, conversation, and relationship marketing is certainly pushing this movement.  The burgeoning set of analytics tools that integrate with other technologies to enable event detection and trigger a customer-specific response is also promoting change in the way companies think about connecting offline and online data to improve overall business performance.

If database marketers and Web analysts are evolving into a new type of team, then what roles are necessary on this new multichannel team?  Here are a few:

• Web Analyst.  The overall Web analytics professional has a deep understanding of the Web channel.  This person uses a Web analytics tool to understand the performance of site traffic, online marketing campaigns, and to segment Web data in order to understand how visitors referred from certain channels navigate (or don’t) through the site.  They understand, measure, and report whether the site is fulfilling its purpose for conversion, task completion, and other KPIs when compared to business goals.  
• Site Optimizer.  A niche type of Web analytics professional, the site optimizer is in charge of determining the right approach for configuring and reporting the results for AB (champion/challenger) and multivariate tests.  This person is all about testing components of site and page design to yield the best combination of elements that provides a lift in a particular metric against a goal, such as conversion rate.  Content targeting may also fall under this person.
• Social Metrician.  Another niche type of Web analytics professional, the social media measurer is concerned about the performance of customer touchpoints outside of the main Web site.  He or she collects, monitors, and analyzes data related to things that happen “out there, on the Internet,” such as syndicated video, mobile, widgets, blogs, social networks, and other social media.
• Database Marketer.  The traditional offline analyst and database miner.  This role analyzes data from channels that are not online but may reference and promote online interaction, such as television, radio, print, catalogs, and direct mail.  Of course, these analytics skills can be applied to online data as well!
• Search Analyst.  The analytics professional in charge of keyword identification/selection, keyword management, bidding, and analyzing the outcomes of search.  He or she may be in charge of analyzing site performance against known SEO goals too, not just SEM.
• Market Researcher.  The traditional market researcher gathers, analyzes, and reports data about the overall market, key competitors, and customers. 
• Qualitative Analyst.  Part market researcher and part analyst, this individual is in charge of online customer and visitor surveying, relating customer feedback and visitor opinions to the context of on-site behavior to help deduce “why” people did something on your site.
• Ad Analyst.  Solely dedicated to assessing the performance of advertising campaigns, the ad analyst assesses and educates clients on ad campaign performance both online and offline.
• Audience Measurer.  The wielder of an audience measurement tool informs competitive decisions, influences media plans, and provides benchmarking and competitive data to give context to other data analysis activities, such as keyword bidding or media buying. 

How would these professionals all work together?  The market researcher’s data is used to help craft a customer-focused and competitively differentiated campaign strategy.  The audience measurer provides data that focuses the strategy on the right online demographics and sites, while the database marketer mines historic data to figure out the best-performing offline tactics for the identified demographics. 

Let’s say a mix of search, social media, and online and offline display ads are selected as part of the campaign.  The search analyst concentrates on SEO/SEM, while the ad analyst tracks the performance of display ads.  The social metrician examines the social media ecosystem’s response to the campaign.  The Web analyst analyzes how campaign-referred visitors behave and navigate through the site, taking into account the context of the qualitative analyst’s voice-of-customer data.  Meanwhile, the site optimizer tests landing pages and funnels to ensure they effectively convert visitors and fulfill business goals. 

For many companies, it would be unrealistic and perhaps impossible to find and hire people to fill each of the roles I’ve presented above.  In fact, in most companies these roles and activities are completed by only a few people, if at all.  An option for companies that seek to expand or combine teams is to look at consultants, contract workers, and full-time equivalents allocated across multiple people.

That said, companies that are unable to bridge together online and offline analytics teams will miss important data points.  In the digital future, we’ll see different types of analytics professionals working together across channels to yield profitable insights that support campaign and business goals.

eMetrics Toronto is about two weeks away, and Boston Web Analytics Wednesday is tomorrow night.   I’m pretty excited for both events.  

I think eMetrics is thee premier web analytics and marketing optimization conference.  I’ve been to a bunch of them, and I always walk away with new ideas with which I can hit the ground running when I get back to work.   The real world case studies, practitioner advice, vendor spiels, and the cadre of notable analytics and marketing professionals who attend and make themselves available to you, the attendee, is unmatched.  

Web analytics pals Jim Sterne (and Andrea Hadley in Toronto) in association with Rising Media ensure the content and tracks focus on the knowledge needed today to drive the real-world application and usage of analytics and marketing measurement technologies in way that enables your online competitive advantage tomorrow.  eMetrics is far beyond a beginners conference (though there’s always plenty of good stuff for beginners).  Gone are the days talking about simple definitions and whose tool can do what (though they’ll be plenty of that stuff covered in some sessions, at lunch, in the bar).   Today’s eMetrics is focused on the application of analytics in business context, especially in the context of marketing.  The conference, as I see it, is about what you should measure and how to take action from what you measure in order to improve business performance and increase top-line revenue and bottom line profits (while reducing middle-line expenses).

I’ll be there in two sessions.  The first session I’m in is on Monday afternoon and occurs in two parts.  The first part is where I’ll fill in for my buddy Marshall Sponder, Director of the WAA and Chair of the Social Media Committee, and give an update on Social Media Committee activities.  During the second part of session, I’ll participate on a panel moderated by Chris Williams, Managing Director, Media Contacts Canada, and panelists Brent Bernie, President, comScore Inc., Paula Gignac, President, IAB Canada, and Jodi McDermott, Director, Product Management Clearspring Technologies.  We’ll be talking about the importance of standards, collaboration between industry organizations, and the real-world application of all the measurement tools you can use to run an online business (and more).

In the second session, on Wednesday afternoon, I’ll be on another panel speaking about the analysis of audience behavior and engagement levels on media and content sites.  Sitting alongside me will be moderator Mitch Joel, President, Twist Image, Sibel Satiroglu, Interactive Marketing, Hewlett Packard, and Mike Sukmanowsky, Web Analytics Specialist, Rogers Media. 

Should be some good conversation.

For those people trying to gain insights from the data, to those actively working on taking action from the the data, and for those who are trying to manage it all, I highly recommend attending one of 2008’s eMetrics Summits.  If you’re in Canada or nearby or really anywhere on Earth, now’s the time to sign up for Toronto.   

And for those in the Boston area tomorrow night (3/19), please join me at the monthly Web Analytics Wednesday I host.  This month, Jonathan Mendez, Founder of RAMP Digital, will be presenting on the “Crawl, Walk, Run” approach to onsite targeting.   Omniture is sponsoring and providing free food and drinks for all attendees, so swing by and swing back a drink whilst enjoying some smart conversation.

See you at both events.

That’s the first question to answer when determining strategy for using online metrics.  You should be able to answer in 10 seconds.  If you don’t know, or if key stakeholders can’t agree on your site’s purpose, then you are unable to use online metrics efficiently.  And, worse yet, you are missing chances for improving your business performance. 

Your web site exists for a purpose, perhaps multiple purposes, such as:

• Providing information or data.  Many sites entice people to visit for access to valuable, differentiated information or data.  Traffic is then monetized primarily through site advertising.  Many internal and external analytics packages will tell you where visitors come from and what they do onsite, which, when combined with demographic information, can be used to qualify a specific audience to an advertiser.
• Generating leads.  A content asset is placed on a site and gated using a form.  People fill out the form and download the asset.  The information captured in the form is stored and used by the company that generated the leads or profitably sold to another company.
• Selling products.  The typical ecommerce model involves acquiring customers via some method or offer, providing a product catalog or landing page, and creating a strong call to action and funnel that persuades people to purchase a product.
• Connecting people.  The explosion of social networking sites where people connect to other people, interact with each other, and use widgets, apps, and data services is a modern phenomenon in which many of us participate. 

Understanding why your site exists enables you to effectively use online metrics.  Once you’ve defined your site’s purpose, you are positioned to examine web data in way that helps you determine whether your site delivers on its purpose – does it effectively exist? 

Metrics and ratios that help you assess if you site fulfills its purpose are called Key Performance Indicators (KPI’s) – see Eric Peterson’s Big Book of KPI’s for a detailed review of the topic:

• For information or data driven sites, you may want to look at KPI’s that measure goal or task completion and conversion rates.  For example, if your site’s purpose is to expose video content to an audience, then a relevant KPI would be the percentage of all visitors that streamed a video or the number of streams per visit. 
• For lead generation sites, a key KPI you will track is the lead conversion rate.  In other words, of all the visitors that came to your site, what percentage of visitors successfully filled out a form and generated a lead. 
• For ecommerce sites, a key KPI that you might track is average order value, which is how much money the average visitor who purchases a product spends on a single transaction.
• For social networking sites, you may want to measure the average time between visits (latency) and the repeat visitor rate. 

But here’s the challenge with KPI’s: they are all academic, unless you have business goals for KPI’s.  KPI’s help you track progress toward predefined business goals.  What are the business goals associated with your site’s purpose?  For your informational site, what’s the goal for video streams per visit or time spent?  For your lead generation site, what’s the goal for the lead conversion rate?  By comparing business goals for KPI’s to actual KPI’s, you can begin to answer the question: “is my site successfully existing and fulfilling its purpose?”

You will continue to answer that question by segmenting your KPI’s, investigating distributions beyond averages, and using other techniques for data analysis.  You may ask: do certain referring sites, have a lead generation conversion rate higher than other referring sites, and why?  Do certain audience segments spend more time on site?  If so, where do they go on the site and what do they do?  If my goal for average time between visits (latency) to my site is five days, and certain customer segments haven’t visited in ten days (recency), what does that indicate about current business performance?

By defining why your site exists, creating KPI’s based on your site’s purpose, establishing business goals for KPI’s, and investigating what’s driving those KPI’s, you can enhance your online business performance in a way that increases bottom-line profit – from optimizing user experience and landing pages, to more efficiently allocating your marketing budget, to improving your product mix, and much more.

In Part 1, I mentioned that the Web Analytics team will look very different depending on company and business goals.  I identified three elemental constituents (business strategy, analytics, and technology) necessary to select a web analytics tool, and I divided them up into three different folks who fill those roles when you’re selecting an analytics technology.

Once the tool is selected, companies will want to create a structured team framework with defined roles and responsibilities in order to successfully deploy the tool.  What I’m describing is a suitable team-structure that enables you to successfully deploy a tool in your organization that finally gets you to a point where you are able to do web analysis. The team structure I describe below lets you get to the hub-and-spoke model that my good friend, Eric Peterson, described in these Part 1 and Part 2 of “what’s your web analytics communication strategy?”   What Eric excellently describes takes the team to the next level of actually doing Web Analytics.  It’s excellent stuff that I encourage you to read.

A formalized team structure for rolling out a web analytics tool may have the following constituents: 

• Executive Advisory Board.  Beyond the Executive Sponsor mentioned in Part 1, these board members are the ones who really control the budget and strategy at the highest level.  They may be your boss, your bosses’ boss, or board members at your company. Regardless, they are the analytics project champions at the highest level in your organization – often C-level executives.  They support the project structure and analytics strategy, confirm the scope of the project, and approve any budget allocation.

• Steering Committee.  You may be on the steering committee, Mr Web Analyst, or it may consist of very senior representatives of all the internal teams that the project touches.  These people work to define the strategic direction of the project, decide on how to resolve critical issues that come up during the rollout, and generally handle any escalations.

• Web Analytics Expert.  That’s probably you, fine reader.  You will provide analytics-based strategy and informed decision making across all aspects of the project. You’re obviously critical to the success of this project, and will ensure technical, tactical, procedural, functional, and financial adherence across the entire analytics program.   You are the chief evangelist, and will define the overall reporting and KPI structure.  In addition, you will be responsible for the overseeing the partnership with your vendor. Other things you may do will include managing costs, coordinating schedules, risks and resources, and reporting overall project status and important communications (often with the help of a project manager) to the steering committee and advisory board.

• Web Analytics Team.  If you are lucky enough to have a team, these folks will gather and document project and technology requirements, liason with business stakeholders, lead training, build awareness of and evangelize web analytics, and in general work with those who receive reporting and leverage the tool.  In many companies the solo web analytics expert will do all this stuff (and drink a lot of coffee or green tea too!).

• Project Manager.  A web analytics rollout can be complicated. While the solo web analytics team member may be expected to project manage, it may make sense to give that role to a formal project manager (y’know a PMP) who works with the Web Analytics Expert to manage the schedule, risks, resources, communications, change, and quality management plans.

• Business Partners.  Since web analytics will touch many different groups, you will need to ensure your analytics team communicates with them.  Business partner are critical stakeholders.  They can’t be neglected.  They will provide business requirements, test the technology, and work with analytics team to ensure the technology, reporting, KPI’s, and analysis you rollout helps drive business performance.

• Subject Matter Experts (SME).  Similar to business partners, these folks will probably be more technical in nature.  The Technology Expert you worked with when selecting the project will transition into a roll as a SME.  You may have one SME who oversees the overall technology architecture, another who coordinates BI resources, another who QA’s the system, another who creates interfaces to your data warehouse, and perhaps another who acts an IT contact covering issues across the operating system, database, security, and networks (especially if you are running an in-house tool).

• Vendor Professional Services Team Members.  Last, but certainly not least, are the folks sent from your vendor to do what you want them to do.  From installing the application (in a in-house environment), functional training, to advanced configuration, these people are critical to ensuring that you don’t make simple, avoidable mistakes that can thwart your efforts and delay the successful rollout, golive, and extension of the project.

In reality, you may not be able to effectively isolate all of these groups to support your analytics rollout.  To some degree I’ve presented big company structure above.  In smaller companies, one or only a few people may do all of the interlaced activities necessary to rollout a web analytics tool.  Regardless, I think the groupings I’ve presented above define the primary roles and responsibilities necessary for success when rolling out a web analytics tool (in fact I presented things in a general way to apply to other rollouts as well).  The next challenge comes once your up and running (make sure to read Eric’s posts)… You need to use the data to improve business performance and guide strategy, decision making, and online tactics that reduce expense and yield profitable revenue.


Image by Jim Sterne, from Emetrics 07 San Fran.