Two weeks ago I had the good fortune of attending the eMetrics Marketing Optimization Summit in Washington, DC. It was my 6th eMetrics! I can’t believe I’ve been to six eMetrics. For those of you who aren’t familar with eMetrics (and who wouldn’t be if you’re reading this blog) it’s the big show for analytics and Internet marketing professionals.

One of the cool things about eMetrics is that it covers the gamut of measurement technologies and their application to real business activities. You can get informed by the best and brightest practitoners, consultants, and visionaries. In attendence were folks like Eric Peterson, Jim Novo, Bryan Eisenberg, Justin Cutroni, Bob Page, Matt Cutler, Gary Angel, Jim Sterne (of course!), Alex Langshur, Jon Lovett, Anil Batra, Phil Kemelor, June Dershewitz and many other smart, fun, and friendly folks I should mention.

The event has an awesomely grand, wide focus across several areas in more than 70 different sessions. Tracks include: acquisition, conversion, retention, social media metrics, data driven organization, industry insights, predictive analytics, introductory analytics training, advertising, and landing page testing. All in all, it’s the place to be to learn all about the big picture and the little details on analytics and optimization.

When I think back on what I learned - a lot - a few key themes kept resonating with me:

• Testing, testing, testing, and more testing. Easier said than done methinks on large complex websites with a lot of stakeholders, but the emphasis continues to be on using AB and/or a multivariate testing approach to optimize images, calls to action, points of resolution, site functionality, offers, forms, headings, and more in order to figure out what recipe gives you the best lift toward your goals.
• Different strokes for different folks when it comes to measuring social media, word of mouth, and user generated content and determining ROI. Companies can measure all sorts of stuff: traffic from social sites, tweets on twitter, trackbacks, mentions in blogs, comments, Facebook, but what really matters to measure is, for the most part, completely dependent on goals and the company.  There doesn’t seem to be one single set of social media KPI’s applicable in all situations, and that kind of makes sense to me.
• The measurement industry is fragmenting. The analytics landscape is awash with smaller players who want to do things like alert you when the data changes (cool!), measure your mobile experience (excellent!), track the engagement of your videos (rad!), help you orchestrate tagging (thank you!), configure your tool (yeehaw!), build you a custom solution (please do!), and generally fill in any gaps in functionality or services that the big vendors can’t.
• The requirement to integrate.From integrating internal and third party data with analytics systems to feeding certain analytics data and key events into data warehouses to bridging voice of customer data with clickstream and customer experience management technologies, to bringing together visitor level data about performance across online and offline campaigns. Integration will be a major concern for companies that are serious about competing on analytics.
• The problem of attribution. First click, last click, direct, indirect, multi-attribution models, and more, there are some huge issues that online marketers are dealing with when it comes to attributing on-site success to a business activity. No one really agrees on what method is better or which is correct.  Expect to see a lot more discussion in this area once the executives start asking.

And so much more too… If only I had more time to blog. My advice is to get yourself to the nearest eMetrics in 2009. Flag me down when you are there. And if you were there, let me know in the comments some of things that stuck out to you.

Anyone who has spent any time doing web analytics knows that you get a lot of requests for analysis.  The bigger the company and the more serious they take analytics, the more requests you will get.  And the requests will range all over the map - from simple data pulls to dashboarding/reporting requests to comprehensive analytics projects to statistical modeling to pretty much everything under the sun - whatever people consider “analytics.”

People have even told me that requests for analysis can even be a tactic for people outside your analytics team to pass the buck in order to make an excuse for not delivering on their own work.  The idea being that because you do analytics, you instantly have all the data at your fingertips and can deliver it in a flash, so why aren’t you helping me? 

False conclusion.  Analytics is hard and complex.  And the bigger the corporation and more integrated the analytics, the harder and more complex.

In many cases with web data, the data may not already have been collected, the data may have to be sourced from another system, extracted, transformed, or made available in a dashboard or a report - if at all even possible to report and analyze. 

The data may only exist as a figment in the imagination of the requester. 

The data may be impossible to report because you have inadequate data model or an entrenched BI team that doesn’t get how the schema affects reporting and doesn’t care enough to help. 

And real analysis of the data takes uninterrupted time, which you may or may not have time to get to today, next week, or next year because you are working at correcting the data model, ensuring the right data is collected, letting queries run, dealing with the intracacies of overly complex tools, building reports, dealing with other requests, working on already prioritized projects, and trying to answer your email and attend meetings. 

I am sure this all sounds familiar to the 1000’s of practitioners who subscribe to my blog…

That’s why when you, data and analysis consumers, ask for data or analysis, you sometimes don’t get it quickly from the web analytics team.  You are just one of the many people making many requests to what in many companies is an already overburdened, often understaffed team dealing with non-standardized or clearly-defined data coming out of many different systems and third party applications.   

Thus, when you are a web analyst, you need to carefully consider what you say yes to do and how you respond to requests for analytics work.  Because as soon as you say “yes, will do” you’ve committed to what could be the analytics equivelent of daisy pulling in a sunny meadow on a summer’s day or a Herculean cleaning of smelly crap-filled stables in freezing weather.   What follows is some advice on how to prioritize your requests for reports and analysis:

• Is Revenue at Risk?  Anyone who has worked with me knows this is one of my favorites. If revenue is at risk, then the analysis will be done!  Profitable revenue is the chi, the lifeforce of any business.  And analytics that supports revenue generation is of the highest kind of analysis.  But you gotta prove revenue is at risk.  Just because you work for a group that produces revenue doesn’t mean your request going unfulfilled puts revenue at risk.  Tell the team, exactly why revenue is at risk…  Not just that you think it is.
• Who’s asking?  Is it your boss asking, her boss, their boss’ boss?  Then the work gets done.  We’re not talking HIPPO here.  We’re just talking MOPPO (most powerful person in the organization!).  Keep your boss happy.
• How difficult is the request?  Just because something is “too hard” doesn’t mean it won’t get done, but as an analytics professional you need to set delivery expectations when requests are so difficult that they will take time.  Perhaps the schema needs to be modified, changed, or just simply remodeled in order to get that data, maybe you need to rewrite the tags, reconfigure the tool, build a bunch of new reports, figure out the data delivery tool.  Maybe 5 other groups need to work collaboratively in coordination with all their other projects just to get the data to a point where it can be reported.  Manage the expectations of the requester.    
• Can it be self serviced? Just because requesters don’t know how to use the tool (RTFM), it’s too slow, don’t know where the report is, can’t understand the report, don’t get web analytics, don’t know how to write SQL, or don’t where to look, doesn’t mean the web analytics team is going to do for you what your job requires you to do.   The analytics team should teach self-servicing as a best practice because wasting time easy fishing in shallow waters means you may miss the big analytics catch in the deep data pool! 
• When is the analysis needed?  Of course, it helps to know when the analysis is needed in order to prioritize.  Requester wants the weight of the world at microsecond N during the equinox by the end of the day tomorrow?  They’re probably out of luck unless 1) revenue is at risk or 2) they are the boss.   A week?  Well maybe, but the weight of world requires querying the Atlas database and queries don’t run like Mercury.   The analyst needs to set expectations based on a number of interplaying factors about when the work can be delivered.
• Why is the analysis needed?  Just curious about the number of X that goes to Y from Z?  Time spent on page Z of your microsite? Or do you need to make a real business decision to advance the core mission of your company?  By communicating to your analytics team the importance of the request’s “why” you can get better service.   Analysts that know why they are delivering can more effectively prioritize.

As an analyst, use these questions above to:

• Help you prioritize your work
• Figure out what’s really important
• Frame how to manage expectations
• Deliver what’s really necessary to drive the business as soon as possible
• Not get caught in the tarpit of wasted time constantly servicing low value requests

And if you made it this far into this blogviation, leave me a comment on what you think.  Am I right, wrong, on target, misguided, and how do you do it?
 

AVG has obfuscated their user agent.  One of the current agents for customers of their free and paid tool now cloaks itself as IE6:

Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)

In addition to the easily detectable user agents:

Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1;1813)

User Agent:Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1;1813)  

User Agent:Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)

This news is not good.  If you filter SV1 agent, you risk filtering legitimate traffic from the IE6 browser.  A few folks have commented to me that one should filter the user agent anyway, because 1) IE6 is in decline and 2) most IE6 users have .NET installed, which will show in the user agent.  Still filtering it makes me a little uneasy.

Is this the death toll for log file analysis and services provided by ABCe (since they can’t filter this user agent either)?  Maybe it is.  AVG is touting that agent lacks HTTP Accept-Encoding, which is just dandy, but that information isn’t normally captured in logs.

So the current situation is this:

1. AVG has two user agents.  Both are filterable, but the SV1 agent is problematic to filter because you risk filtering legitimate traffic.
2. Both agents in the current version request gifs in noscript tags, inflating counts in page tag implementations with noscript configurations.  AVG claims they will fix this issue.
3. The bot uses”mad” bandwidth.  I’ve heard stories of bandwidth increasing 100x normal levels.  Some webmasters are serving dummy files to the recognizable user agents, some aren’t serving content to IE 6 browsers (crazy), and some are redirecting the bot back to AVG (thus inflating AVG’s bandwidth, LOL!).
4. Evidence points to this bot NOT inflating clicks from paid search (i.e. PPC) and thus NOT committing click fraud.   But it doesn’t remain out of the realm of possibility that the scanner may be accessing an ad vendor click redirector and causing a click.  Not trying to spread FUD here, just making a point. 
5. AVG is looking at option of checking either an external db (hosted by AVG) or a local cache to verify sites in SERP’s have been “scanned by AVG,” instead of repeatedly scanning sites every time they are listed in SERP, to reduce the bandwidth issue and minimize fraudulent entries in log files.
6. AVG is thinking about enabling white listing of sites, so they are skipped by the scanner.
7. AVG is thinking about exposing a meta-tag that instructs the scanner to ignore the site.

Good luck with this nasty bot!  Interestingly, here’s how you smurf a site with the AVG LinkScanner. 

Here’s the deal.  AVG LinkScanner doesn’t execute javascript nor take cookies.  I had that confirmed by the Chief Research Officer at AVG, Roger Thompson. 

So why is the AVG user agent showing up in that data collected from certain page tag configurations?  The AVG LinkScanner currently requests gifs in noscript tags!

A best practice in web analytic’s page tag configuration is to use the noscript tag to serve the gif to non-javascript executing browsers.  Here’s some commonly seen (obscured) code for doing that:

<noscript>
<div><img alt=”foo” id=”bar” width=”1″ height=”1″ src=”http://
foo.bar.com/xyzab57yw10000s1s8g0boozt_9t1x/foo.gif?baruri=/
nojavascript&xy.js=No&xy.tv=1.2.3″ mce_src=”http://
foo.bar.com/xyzab57yw10000s1s8g0boozt_9t1x/foo.gif?baruri=/
nojavascript&xy.js=No&xy.tv=1.2.3″div>
</noscript>

<NOSCRIPT>
<IMG
src=”//foo.bar.com/xyz.gif?Log=1&URL=/javascript_disabled” mce_src=”//foo.bar.com/xyz.gif?Log=1&URL=/javascript_disabled”
BORDER=”0″ WIDTH=”1″ HEIGHT=”1″ />
</NOSCRIPT>

<noscript>
<img src=http://pt.foobar.com/images/xyz.gif?js=0” height=”1″
width=”1″
border=”0″ hspace=”0″ vspace=”0″ alt=”"> 

Thus, if you are using noscript tags in your page tag *and* someone with the AVG Linkscanner views a SERP (search engine results page)  from Google/Yahoo/MSN that lists your site, the traffic from the LinkScanner will be counted. 

Of course the simple solution to fix this problem is to exclude the user agent: 

Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1;1813)

If don’t have full control over your page tag based web analytics implementation (i.e. hosted), you need to verify that your vendor has excluded this agent.   And you should have them audit your data going back to April, and refund/credit you any money.  Good luck with that though!

How big is the problem?  Well, it depends!

The amount of AVG traffic will vary dramatically by site.  Your site must show up in the SERP’s on computers of visitors that have AVG LinkScanner installed, and you must be using noscript tags to serve the gif.

I’ve made AVG aware of this issue.  And frankly, they’ve been a fantastic company to work with, so I’m sticking with them (for now ;).  First they allowed me to join a private Google group to discuss my findings, both the Head of Global Communications and Chief Research Officer quickly responded to all my emails (good social media response), and their engineers are looking into this issue so that they can fix it…  That’s pretty impressive and quick response.  So cheers to them!

It’s worth mentioning that the LinkScanner isn’t _supposed_ to request images, so I do think this issue will get fixed.

Only time will tell whether or not AVG obfuscates the user agent so it looks just like a “normal” browser.  Let’s hope not! 

What I do find interesting is that I’m already hearing that an agent exists with the string (Mozillia/4.0 (compatible; MSIE 6.0; Windows NT 5.1;1813). Note the “ia” mispelling of Mozilla as incorrectly documented here.  And it accepts cookies.  So AVG’s agent is already being spoofed.  Not good, not good.

Mobile analytics for Internet-enabled wireless devices is a fairly hot topic for companies seeking to acquire customers, extend their brand, or expose content in “innovative” ways.  Obviously, the iPhone and Blackberry are pushing development in this area forward, but there really aren’t a lot of players in this space. 

Nedstat, CoreMetrics, and Omniture offer capabilities mixed into their current offerings.  Nedstat even carves out some mobile specific reporting.  You can gain some insight into mobile activity from companies that enable log file processing, like Unica and WebTrends, but be prepared to configure a bunch of filters to isolate the data.

Lesser known companies pushing mobile offerings include: Amethon, Mobilytics, Bango, TigTags, Xiti, and AdMob.  Some of these mobile players are even offering capabilities where they cross-sell analytics as an integrated part of their ad networks, content delivery  and transactional processing systems, marketing and barcoding services, and even as infrastructure or network appliances.

On the audience measurement side, we’ve seen comScore acquire M:Metrics, which was no surprise to me.

On the multivariate testing side, we see my friends at SiteSpect offering mobile MVT testing capabilities. 

And I’ll bet we see Google get into this space within the next 6 months…  I’d even wager an announcement at eMetrics DC…

From what I can gather, when we’re talking about “mobile analytics” we’re talking about “mobile browser” activity across a variety of handsets, not everything that happens on the device. 

Measurement issues in this area include:

• Data Collection.  As many of you know, not all mobile browsers will execute javascript.  They cached the imagesThus, vendors offer us choices.  Folks like Mobilytics and Bango use an image-based data collection method, while Amethon offers a packet sniffer (they call it wireline detection), and we even have Omniture and Coremetrics talking about “no tag” implementations - what my good friend Phil Kemelor mentioned on his CMS Watch blog (”To compensate, you need to stuff the image tag with query strings that will collect the data you require for reporting.”)  Then we have Unica and WebTrends with log files.  Interestingly, packet sniffing has some advantages here because some devices pass unique id’s (such as the phone number) in the HTTP header or other unique id’s.
• Unique visitor identification due to lack of cookie support and IP addresses changing.  IP addresses change, I’m told, as they switch from tower to tower.   In addition many mobile devices will take the IP address of the gateway, making all the devices look the same “person.”  I’ve certainly seen evidence of the host changing pretty quickly during a mobile session. Compounding the difficulty in assessing “uniqueness” is that not all mobile devices support cookies.  In web analytics, cookies are used to define uniqueness.  The fallback method when you can’t use a cookie is IP address/user agent.  If you can’t set cookies and the IP address and user agents are the same, how do you identify uniqueness?   However, when you can detect a unique value in the header, you can easily detect uniqueness.
• Handset capability detection.  Does the device support WAP pushing, streaming video, ringtones, downloading video clips, and so on?
• Phone and Manufacturer identification.  Database from WURFL and DeviceAtlas can be used to identify phone and manufacturer device attributes.  Larger vendors are further behind on integrating this data into their current offerings, whereas the smaller niche players are making use of it. 
• Screen resolution detection.  The Mobile Marketing Association’s (MMA) standards for the four “standard” screen sizes may carry enough weight to push this disdained piece of metrics trivia available from javascript based tagging in web analytics into a brighter spotlight.
• Traffic source detection.  Capabilities for traffic sources seem rudimentary.  I don’t just want to know about search and direct entry.  But I want detection of sources from my marketing and advertising campaigns, rss feeds, and email newsletters, if mobile visitors are coming in from those channels.   Interestingly, Bango solves the campaign tracking issue by pushing you to a Bango-specific URL.
• Geographic identification.  Where are the visitors viewing your site coming from?  And what does the mobile audience environment “look like” in each country.  From this information you can extrapolate country-specifics for site optimization.  But not all devices enable geographic detection because the gateway’s IP address is used or the IP address from the network is used, not a GPS signal.
• No standards.  There are few, if any, commonly supported mobile standards and no web data standards, so the problem is no standards for the devices and no standards for the tools.  There are no standards.  Did I mention that there are no standards. 

So I was thinking, what would I want to see in a mobile analytics solution?  Allow me to riff here.

• Dashboards for KPI and specific-metric reporting.  Views, visits, visitors, referrers, popular pages, traffic sources, resolutions, geography, time-based reporting and custom defined KPI’s….
• Support for multiple data collection methods.  Logs, no-js image tags , and packet sniffers.  Let me pick what I need for whatever application fits my goals.
• Support for mobile-specific constructs not present in historic web analytics data.  Manufacturers, operators, handsets, and device capabilities.
• Advertising-based reports.  CTR, CPM, eCPM, that stuff…
• Tracking for mobile downloads, installed applications, SMS, and MMS.  Seems like a no-brainer.
• API’s.  Closed systems are dead ends for integrated marketing, so give me an API or enable pre-built integrations with other systems, like CRM.
• Segmentation.  By country, by device, by network, by manufacturer, and so on.  It’s necessary.
• Repeat or return visitor identification.  Simple measures of recency and frequency, core to media buying and planning and to site optimization, should be a data point available in mobile analytics.
• Conversion and goal metrics.  Do visitors on mobile devices convert better, worse, the same?  Do they reach site goals?  Without tying performance data  and outcomes to mobile visitor activity, I’m left wondering…
• Value scoring for engagement or proxy scoring for revenue and ROI analysis.  I want to be able to score attributes or actions to approximate an engagement score or to identify value or indicate revenue. 
• Non-human traffic and web-browser based detection and reporting.  Mobile pages are full of links.  The ads are links.  Mobile vendors must support detecting, filtering, and reporting, non human and web-based agents from pure mobile agents - otherwise the mobile data gets muddled and skewed.
• Data Export.  Must be able to export reports to Excel or Word, and email them.

So there’s a quick blogviation on Mobile.  Am I right, wrong, what did I miss?  Let me know…

Back from another excellent eMetrics.  I’m a very big fan of the eMetrics Marketing Optimization Summit…  Props go to Jim Sterne for growing this event from a little seed into an incredible, blogworthy blossom.  How involved is Jim in eMetrics?  I’d say he’s completely immersed in every little piece - he even came up to me at the SF WAW (way to go June D!) to find out about the renegade AV work I did in one of the sessions, and to get my take on how it could have been avoided.  He’s that intimately connected to what’s going on.  Macro and micro, micro and macro.  And when you have one of the best Internet Marketers in the world, keeping a tight rein on the Clydesdale of conferences, you know you’re in for one heck of fun ride. 

And so it was for about 500+ of the top web analytics in the beautiful Palace hotel.  Props to consummate conference organizers Matt Finlay and his crew at Rising Media for keeping the road smooth as we all trotted on it as well.  Fanny, you are one helpful polyglot of a marketing manager!  I never knew German keyboards were so wild… Thanks.

The eMetrics sessions were informative and actionable.  The lobby bar and after-hours parties fun and enlightening.  You really can’t ask for more out of a conference.  As I flew home thinking back on it all, there was a lot to blog about, including:

• It’s all about attitude, dude – as in attitudinal data.  Like my father says “it’s all about your attitude.”  And so it is on the Internet in 2008.  From ForeSeeResults, to iPerceptions, to OpinionLab, to CRMMetrix, the often missing link in customer analytics is attitudinal data.  I’m talking here about Voice of Customer (VOC) technology that allows you to ask a question set to site visitors and then apply some sort of algorithm or model to express the meaningfulness of the data in quantifiable terms.  From the American Customer Satisfaction Index to 4Q.  VOC technology enables you to participate in a continuous, automated dialog with your customers in order to identify problem points on your web site and enable you to measure purpose and success of your most valuable segments.  Expect to see some of the big players gobble up these smaller companies.  Omniture, Unica, WebTrends, and CoreMetrics should be thinking about acquisition in this space to round out their offerings.
• Testing, 123… as in multivariate, MVT.  The rage is site optimization technologies beyond the simple A/B, champion challenger, test.  In this category you find folks like SiteSpect (the only non-intrusive multivariate testing solution!).  I’m a big fan of these guys (and was in 2006 long before they ever sponsored a WAW, thanks to a nice demo from Larry at my old job).  Eric Hansen and his crew have specialized software that you install in your data center.  No futzing with damned tags.  Swap out your variations, create different recipes, determine what’s statistically significant in giving you a lift to your macro or micro conversion goal, and you’re off to the races.  The good folks at Google are doing it and doing it well with Google Site Optimizer (thanks for the t-shirts!).  Interwoven is baking in Optimost to the CMS, and Omniture has their Test and Target integrated with the Business Optimization Suite.  Accenture has Memetrics.  Kefta too. And what ever happened to Verster?

In a nutshell, these technologies enable you to test variations of content themes, colors, creative, calls to action, points of resolution, buttons, navigational elements, –whatever you want to call the stuff on the screen—to determine what combination performs best against your goals.  But of course, this is all just software, so don’t get too excited.  The tests are about as good as the people creating them…  And complex tests that take a long time to execute may not finish.  Imagine 1-800-Flowers starting a test in January and not finishing until March, missing Valentine’s Day.  Or Intuit running a test beyond April 15th for a tax product.  Go humbly and carefully into this space, my friends, or you may end up optimizing for everyone and appealing to none.

• Tying it all back to the dollar for profit-generating sites and to the mission of non-profit generating sites…  It seems like a “no, duh” moment but metrics for the sake of metrics can be a big waste of time.  If you can’t tie metrics or visitor actions back to value on a revenue-producing site or to the betterment of a non-profit site’s core mission, then what’s really the point of the measurement…  That’s why I’m a big fan of the stuff ZaaZ does.  They totally get the fact of how actionable metrics turn the wheel of Internet commerce and ad-based models, and they can model it all to prove it out the ROI.  Folks like newly elected WAA Director Alex Langshur’s company Public InSite do similar stuff for content driven sites.  That is they know how to use metrics to optimize the channel to goals, not to just puke confusing data, like most web analytics tools do.  Again, it’s all about the people you hire, not the tools you use… My good friend Avinash, right again!
• The emergence and rise of deeply psychological and neuro-behavioral methods for automating persuasion and conversion.   Anyone who knows my good friend Joseph Carrabis, over at NextStage Evolution, knows that besides being one heck of giant kite flying, music master, he’s also got the models and the patents to help target and respond to human behavior across programmable devices.  We’re already seeing some companies, like Seven Billion Joe’s, er People, taking what he’s been saying for years and going to market with it.  The idea here being that if you can identify the affective, behavior, and motivational drivers of site visitors, you can maximize cognition in elements on the site (like pictures, text, informational flow) to appeal to target segments and persuade/provoke desired behavior.  It’s like a higher rung on the optimization ladder.  It’s not test what they see, it’s figure out how they think, then make the site better because of it.  Cool stuff.  Blows my mind.
• Integrated, multichannel marketing.  Just ask my good friend Akin Arikan, author of the newly released Multichannel Marketing.  (Disclaimer: I was a technical editor on the book.  It’s easy to do when you edit brilliance).  Make sure to check it out!  Marketing in general will become more Internet-centric, but will continue to clutch the roots of broadcast and print.  You will have the database marketer and statistical modelers working with a union of web channel and offline data.  What’s preventing it now?  A unified marketing database.  You see companies like Salford Systems circulating in this space.  And take a look at Unica’s blend of Enterprise Marketing Management…  I’d stay tuned to see what Unica has up their sleeve for bringing together online and offline.  When you can segment and target across online and offline campaigns, if I were pure web channel player only, like Omniture or CoreMetrics, I’d be a bit concerned that people are waking up to open systems, not closed black boxes.  WebTrends is already moving in this direction…  But they all remain far behind Unica when it comes to multichannel marketing.

And that’s just a few of the things the phenomenal eMetrics got me thinking about…  I hope to see you in Washington DC in October! 

Next Sunday afternoon I am moderating a panel at eMetrics San Fran.  The panel is called ”Web Analytics -vs- Audience Measurement.”  Andrea Hadley at NetSetGo was the brainchild of this panel idea (and yes that is her picture on her site :).  In fact, I was a panelist on the same panel at eMetrics Toronto, filling in for my friend Marshall Sponder.  Since he’s going to be in San Fran, I yielded my seat 0n the panel and decided to stand up at the podium.   Other panelists include Jodi McDermott, Director of Product Management, at ClearSpring, and some other surprise guests (from comScore and IAB maybe)… You’ll have to show up and find out…

The panel description is as follows:

Are you confused about the number of customers visiting your website? Are the metrics reported by your web analytics tool different from the metrics reported by your online media, or by audience measurement organizations? The WAA invites eMetrics Marketing Optimization Summit attendees and the local San Francisco business community of web marketers, publishers and agencies to attend this community meeting. A panel of experts will discuss the value of the metrics, methods and tools used by web analytics practitioners, online advertising media and audience measurement organizations. Find out how-to use these metrics and tools to better understand your customers, your website’s competitive standing and overall website value.

The goals for this panel include:

• Adding clarity around the tools and data associated with each set of technology and metrics - web analytics technologies and website data, ad servers and ad data, and audience measurement tools and data.
• Learning how each data source can be used to expand our understanding of customers, how effective our website is as a business channel, the website’s competitive standing and value, and so on.
• Providing insight into the role of the web analytics practitioner and how this role is growing in importance and influence over business, marketing, product, and strategic decisions.
• Discussing the role of the Web Analytics Association (WAA) and how the WAA serves the practitioner.  That the WAA is an unbiased organization that doesn’t serve advertisers, publishers, or technology vendors, rather that the WAA serves and exists for the benefit and betterment of the the practitioner and the web marketer/strategist.
• Articulating the announcement made at eMetrics Toronto on the important collaboration between the IAB and the WAA for standards review.

My goal as the moderator is not to critique, demean, or criticize audience measurement, Internet advertising technologies, or to embellish or hype up web analytics tools.  Rather I hope to clarify the differences between the technologies and speak about the value they hold together - like I did in my article for MediaPost called the Yin and Yang of Online Metrics.

So why am I telling you all of this on my blog???  Well it’s because I really want your help, whether you are going to eMetrics or not…  Since I’m the moderator, I get to ask the questions, and I don’t want to just ask “my” questions, I want to know what questions YOU would ask if you had the chance to ask.  Of course, those of you reading this and attending the panel will be given the microphone if you raise your hand.

Please help my crowdsource by telling me in comments or via email to judah (at) webanalyticsdemystified.com:

What questions would you ask to clarify the differences and value between web analytics and audience measurement tools?

Any questions you think worth asking from “why don’t the numbers match?” to complexly “what are the differences between audience measurement and web analytics systems in terms of data collection?” would be awesome and appreciated.  Thanks in advance for your help!  I’m eager to see if this social media experiment in blog-based crowdsourcing actually works!

Wow!  It’s been a few weeks since I’ve had any time to blogviate. 

What other things do web analysts do?  Besides blog and do WAA stuff… And ensure tool configuration/administration, date collection, data verification/validation, reporting, KPI generation, conversion optimization, deep site analysis, stakeholder guidance, outcomes evaluation and so on… Well the fun answer is “it depends” on a things like your boss, the organization you work and the holy org chart, your recognized skill set, and what you want to do.   But as I talk to my colleagues in the industry, I’ve noticed some web analysts do a lot of different things.  Here’s a few beyond the norms (or in some case maybe part of the norm, but not often discussed):

• Write business requirements.  You may be writing biz reqs for the extension and maintenance of your own tool, or you may be asked to participate in the definition of the metrics strategy for product or site features.  The analyst may define the attributes, capability, and characteristics that are necessary to accomplish given business objectives.  Generally these biz reqs will be functional (the system must do this in this way and look like this) and not technical (but every so often you may need to justify why you keep saying “ah, page tags, not logs” or vice-versa or packet sniffers or hybrid).  Fun!  And time consuming! 

• Participate in product development and usability discussions.  A rich topic here for sure.  As web analysis sort of fractures into those who study how the site routes visitors, navigational elements, information architecture, and into those who prepare AB and MV tests and report the results, it’s not uncommon for analysts to be called into to determine what should go where and what functionality should or should not exist on the site in order to drive business or conversion goals.

• Contribute to the keyword set.  As I explained in my last post, web analytics is morphing into multichannel analytics.  Analysts are increasing leveraged to participate in and analyze the outcomes of SEO and SEM.  Based on keyword data, I have a few friends who spend a ton of time selecting and managing the keyword portfolio and even the bids! 

• Have a say in “strategy”.  Analysis informs tactical decision making, which is guided by strategy (and analysis and decision making and strategy again).  When fully leveraged, a web analyst has much to offer the strategic decision making process.  Think about something as simple as using referrers to establish content syndication and affiliate partnerships…  Cool.

• Guide the content agenda.  For those who work in what my buddy, Alex Langshur (who runs a boutique consultancy in the public sector), calls “content-rich” and “mission driven” sites, the web analytics tool has utility as an editorial or content research tool.  From understanding what keywords/phrases are driving traffic to determining whether the editorial plan is actually mapped to the information demands of site visitors, web analysts can have a lot to say, if asked.  But be weary, the last thing an editor wants is some hot shot web jockey telling them what to write. That’s not what I’m saying to do, rather, some analysts work with content and editorial teams to ensure frequently demanded content topics are rounded out on the site, expanded on/developed, put on the content plan, or simply just known about, so the content folks can do what they do… 

• Code. Yeah, some of us know how to do it, and many of us just don’t tell anybody.  Because “that’s not what I want to do anymore” as my friend who works at a local agency told me the other night.  My personal opinion is that code is better left to the coders, but any web analyst who can throw down with web development and talk about things like X-Forwarded From headers will only make themselves more valuable to the organization.  Then again, some analysts would rather analyze data than futz around with overly esoteric tags and variables and the plumbing of web pages.  Then again some of us love that.

• Direct IT.  Those of us fortunate enough to have control over our web analytics technology already know they’ll be spending perhaps inordinate amounts of time with our good buddies in IT.  They may be the audience for your business requirements, or you just may need to connect with them to ensure your technology is factored into the larger plan for next generation integrated, service oriented architectures.

• Due diligence on acquisitions.   A fun one for you MBA’ers is when you get drafted into the acquisition or merger process, having to examine the target’s web traffic.  You gain real insight into the core of their web business, and may even find things, I’ve heard, like page view inflation from not filtering bots on including things like favicon.ico to inflate page views.  Heh!

And more!  So yeah, it’s not all about spending all day just thinking about who comes to the site, why, what do they do, and do they complete their purpose according to specific goals.  While that is all a big and important part of it, the role of web analyst can go far beyond tradition, if you are capable and you work for the right business that lets you excel!

That’s the first question to answer when determining strategy for using online metrics.  You should be able to answer in 10 seconds.  If you don’t know, or if key stakeholders can’t agree on your site’s purpose, then you are unable to use online metrics efficiently.  And, worse yet, you are missing chances for improving your business performance. 

Your web site exists for a purpose, perhaps multiple purposes, such as:

• Providing information or data.  Many sites entice people to visit for access to valuable, differentiated information or data.  Traffic is then monetized primarily through site advertising.  Many internal and external analytics packages will tell you where visitors come from and what they do onsite, which, when combined with demographic information, can be used to qualify a specific audience to an advertiser.
• Generating leads.  A content asset is placed on a site and gated using a form.  People fill out the form and download the asset.  The information captured in the form is stored and used by the company that generated the leads or profitably sold to another company.
• Selling products.  The typical ecommerce model involves acquiring customers via some method or offer, providing a product catalog or landing page, and creating a strong call to action and funnel that persuades people to purchase a product.
• Connecting people.  The explosion of social networking sites where people connect to other people, interact with each other, and use widgets, apps, and data services is a modern phenomenon in which many of us participate. 

Understanding why your site exists enables you to effectively use online metrics.  Once you’ve defined your site’s purpose, you are positioned to examine web data in way that helps you determine whether your site delivers on its purpose – does it effectively exist? 

Metrics and ratios that help you assess if you site fulfills its purpose are called Key Performance Indicators (KPI’s) – see Eric Peterson’s Big Book of KPI’s for a detailed review of the topic:

• For information or data driven sites, you may want to look at KPI’s that measure goal or task completion and conversion rates.  For example, if your site’s purpose is to expose video content to an audience, then a relevant KPI would be the percentage of all visitors that streamed a video or the number of streams per visit. 
• For lead generation sites, a key KPI you will track is the lead conversion rate.  In other words, of all the visitors that came to your site, what percentage of visitors successfully filled out a form and generated a lead. 
• For ecommerce sites, a key KPI that you might track is average order value, which is how much money the average visitor who purchases a product spends on a single transaction.
• For social networking sites, you may want to measure the average time between visits (latency) and the repeat visitor rate. 

But here’s the challenge with KPI’s: they are all academic, unless you have business goals for KPI’s.  KPI’s help you track progress toward predefined business goals.  What are the business goals associated with your site’s purpose?  For your informational site, what’s the goal for video streams per visit or time spent?  For your lead generation site, what’s the goal for the lead conversion rate?  By comparing business goals for KPI’s to actual KPI’s, you can begin to answer the question: “is my site successfully existing and fulfilling its purpose?”

You will continue to answer that question by segmenting your KPI’s, investigating distributions beyond averages, and using other techniques for data analysis.  You may ask: do certain referring sites, have a lead generation conversion rate higher than other referring sites, and why?  Do certain audience segments spend more time on site?  If so, where do they go on the site and what do they do?  If my goal for average time between visits (latency) to my site is five days, and certain customer segments haven’t visited in ten days (recency), what does that indicate about current business performance?

By defining why your site exists, creating KPI’s based on your site’s purpose, establishing business goals for KPI’s, and investigating what’s driving those KPI’s, you can enhance your online business performance in a way that increases bottom-line profit – from optimizing user experience and landing pages, to more efficiently allocating your marketing budget, to improving your product mix, and much more.

I’ll get back to talking about the web analytics team soon, but I’ve been getting a few emails from folks just starting out who are a bit confused about data collection.  So I figured I’d blog about it…

When web analysts talk about data collection, they are referring to the method by which counts and measures of things, like page views and durations, are captured by a web analytics tool.  If you’re new to web analytics, data collection can be slightly confusing.  There are three “generally-accepted” methods for data collection in the web analytics industry: 

• Page tags.  Client-side data collection involves using little snippets of HTML code that reference a JS file and communicate via a beacon to a “page tag server” - the machine that collects the data so it can be sessionized by the web analytics tool (it may not be called that by your vendor).  As a web analyst, if you are using page tags you will have lots of fun tagging every page on your web site and instrumenting the tags with custom variables and campaign codes.  Reasons why people like page tags are numerous, and include the fact that they are fairly efficient in filtering out non-human traffic (as long as the robot doesn’t execute javascript) and can count proxy cached pages (improving accuracy). Page tags are probably the most ubiquitous method for collecting web data today.
• Log files. Server-side data collection involves parsing text-based log files generated by Web servers.  The server, when instructed to do so, logs every request received by clients in a file called the “log file.”  There are many formats for log files.   Each line in a log file is called a “hit” and contains lots of different stuff - from the ip address, a request date/time stamp, the item requested, user agent, referrer, and more.  Many “hits” make up a single page view - that’s why it’s incorrect to use the term “hits” to refer to “page views.”  As a web analyst you will be defining the format of the log file within your tool and moving and synchronizing log files so that they can be processed by your tool.  Some people will claim log file analysis is dated (historic may be more appropriate), or less accurate than page tags (due to caching issues).  Other people like logs because they can reprocess their data. 
• Packet sniffers.  Network data collection involves deploying either software or hardware that intercepts and logs traffic coming over a network.  Every packet is captured and decoded according to a configuration you define.  Your web analytics tool can be configured to process the data captured and decoded by the sniffer.  Packet sniffers are a less common approach for data collection by web analytics vendors.  

Interestingly some vendors offer “hybrid” data collection, which combines multiple data collection methods.  This mode could be considered a “fourth type” of data collection.  Most commonly hybrid data collection means using logs and page tags to collect different data elements, but other combinations are possible as well. 

As you investigate the best data collection method for your implementation ensure you deeply consider the pros and cons of each method.   For example page tags capture information about the browser (like screen resolution) that logs just can’t.  But what about if you need to measure non-javascript executing clients, like some mobile devices?  Log files capture information about crawlers (i.e. robotic traffic) that page tags just can’t.  But can you adequately filter robotic traffic and maintain host exclusions?  Packet sniffers capture pretty much everything, but can be challenging to customize to your exact data needs (and you’ll need a fair amount of IT support). 

Which one is correct for your implementation?  It depends on your business goals defining what you need to measure…