Part 2: Web Analytics Tools – How Do I Know I’ve Outgrown Mine?
Web analytics tools can be outgrown, like houses, clothes, shoes, music, books, and ways of thinking about the world… But how do you know when you’ve outgrown your web analytics tool? In part 1, I began the list of five symptoms of an outgrown web analytics solution, which was spawned out a preso I recently gave. The five symptoms include:
- Inadequate segmentation (covered in Part 1)
- Poor visualization (covered in Part 1)
- No custom reporting (covered in Part 1)
- Limited Integration
- Cost
So without further adieu, here’s the rest of the list and some thoughts regarding these symptoms (click here for Part 1):
- Limited Integration. Soon after deploying a web analytics solution, you will become intimately familiar with clickstream data and simple counts of things (like page views) and measurements (like time). You will hopefully have deployed Key Performance Indicators for understanding how effective your web site is at converting visits and meeting defined business goals. Depending on the web analytics tool you use you may even have insight into the behavior and KPI’s of visitors from online channels like newsletters, search, and rss because you have applied “tracking codes.”
Soon will come a time when you may ask yourself how do I integrate data from other data sources? You may want to bring data from an email service provider, CRM system, and registration databases, so that you can see delivery rates next to conversion rates from newsletters, or so that you may pass behavioral information about a visitor who registered on your web site.
You may want to move data out of your web analytics system. Perhaps you want to feed your data warehouse? Or you may want to feed web analytics data into a targeting system. Simple XML-based feeds from a hosted solution may not suffice. You will need access to your data in a open database. You may even want to stop non-human readable text and character strings from appearing in your reports. To do so you may need to lookup data using various methods in order to make reporting comprehensible. All of these goals require some level of integration.
If your current web analytics tool can’t:
- Provide insight into all online channels
- Enable you to bring data from other systems into your web analytics platform
- Pass Web Analytics data to other systems.
- Manipulate data by looking up values, resolving urls, and decoding parameters
And do all of this at a reasonable cost in a maintainable way using in-house resources, then you may have outgrown your web analytics solution.
- Cost. Web analytics done right isn’t cheap. It costs money to maintain and extend whether you run an in-house solution or external solution.
When running an in-house web analytics costs are spread out across hardware and software and resources:
- Software license.
- Recurring maintenance costs.
- Servers (one or more). Perhaps you virtualize (it cost money for the virtualization software).
- Database license(s).
- Storage.
- IT resources - people like project managers, application engineers, and dba’s.
As you expand your web analytics operation, all of these resources and technologies will need to scale. Time will need to be devoted to maintaining it all, and time costs salary dollars.
Your company will, hopefully, grow. Then you will have more sites. These will need to be tracked. New reports will need to be created, tested, and rolled out into production. New data and systems integration requirements will spring up. All this has cost.
On the other hand, if you are using a hosted solution, you will need to extend the page tag and tool configuration when you want use more features or integrate systems and data. That means spending money to use vendor professional services or consultants, unless you want to dedicate internal resources in IT who may already be overburdened.
At some point you say enough is enough. The COST is too much! You then decide to invest in well-negotiated vendor solution that provides a lower TCO over some horizon. When you start to run up against the barrier of cost, you may have outgrown your web analytics solution.
So that’s the list. I can think of many more reasons why companies outgrow their web analytics tools… What have I omitted? What do you think?
Judah added the following ...
Hi Justin,
How can I refuse a blogging request from a fellow Red Sox fan and author of GA Shortcuts? 
GA must be causing certain companies to reconsider their web analytics expense. I’d be more than a little nervous if I were attempting to sell into the small or mid market. I still remain unconvinced that GA or any of the free tools are giving the Enterprise vendors a run for their money at large corporations with a unified analytics function. Free tools just don’t provide anywhere near comparable levels of control over data processing or tool configuration, data granularity, reporting customization, or features for integration as their “for pay” counterparts. But I’ve always said that if GA wanted to rule the analytics world, the big GOOG certainly has the resources to make it so. 
Your comment on non-clickstream tools is a good fodder for a discussion over beers sometime. If the same business owner is lucky enough to control all those technologies, your prediction may be true. On the other hand, established companies who don’t operate fully in the online channel may already have established business owners (and budgets) for testing, surveying, and competitive data.
Thanks so much for commenting and reading my blog!
Judah
Steve added the following ...
I for one, would love to sit in on that “over beer” conversation. 
I suspect you’ve kinda touched on it Judah, but I’d raise the spectre of “diminishing returns”, to somewhat loosely follow Justin’s thread. In that while your existing tool may no longer be adequate to the task desired; the next one up makes the ROI just not worth it.
Such that “Good Enough” becomes sufficient, even tho not Best.
I recognise I’m not talking just WA tools here, but all sorts of software and IT related projects that I’ve been involved in over the years.
Thoughts?
Cheers!
- Steve
Justin Cutroni added the following ...
The more the merrier. Now If I could just get that virtual beer program running…
Judah, I completely agree with your point re: organizations with unified analytics. They probably can’t use a GA. But I think they’re in the minority.
I think Steven articulated my point better than I did. How many organizations buy a web analytics tool and then never fully use it? It’s like MS Word. 90% of us use 10% of the features.
Have a great weekend and GO PATS.
Justin
Judah added the following ...
Steve: You are hereby invited to join me (and Justin) for beers anytime you are in the states. We’ll even drink Fosters or Mountain Goat.
I think gaining competitive advantage using analytics requires a substantial bet by senior management that it will all work, and that includes funding initiatives where the ROI from buying a tool is harder to attribute. If you’re doing simple landing page optimization comparing conversion rates etc, the ROI is fairly easy to identify. Did the levers you pulled, generate a measurable return that you can quantify in today’s dollars given a hurdle rate? And you know what, if all I want to look at is bounce rate or goal conversion, and have no need to integrate data other than paid search or feed the data anywhere else, GA is a godsend.
But when we talk about huge scale data integration roundtripping from the data warehouse exposing it all via a web analytics tool, the ROI from the resulting usage is harder to attribute in advance, and when you need to create automated business processes across online and offline channels with hundreds of campaigns and advertisers, do you say nahhh, let’s download the free tool and instead spackle millions of pages with hard coded tags that will inevitably become unmanageable over the long term as you realize the metadata you need to expose wasn’t hardcoded at go-live? And then get a bunch of little aggregated XML feeds you load into the DWH on some schedule? Short-term, limited ROI gain for long-term competitive advantage pain?
Google provides a free analytics tool for one primary reason: to validate their advertising model to advertisers, so folks continue to spend money buying traffic. Google isn’t trying to enable the enterprise with a web analytics tool that can serve as a layer in the corporate SOA or as the yin to BIs yang.
Overall, I think what you are saying makes a lot of sense, and what I’m saying is based on needs of Enterprises (read: mid to large cap companies). And as Justin points out below, not everyone has those needs…
And I should add there are very few people I know who could even begin to lead a large enterprise in pulling it all off.
And of course it all depends on goals. Goals. Goals. Goals.
So I hope that answers your question or at least gives you some food for thought.
And I will conclude with “aussie, aussie, aussie.”
Best,
Judah
Justin: I agree. Check your FB account. I sent you BoozeMail! The real beer will have to come when you are next in Beantown or up North, or at Emetrics…
I completely agree with what you’re saying. Not all companies have the requirements (nor do they have the visionary who can set the strategy and execute on it.)
I think using GA at an Enterprise as a tool to augment your web analytics arsenal could be beneficial. Especially when you find prebuilt integrations exist between GA and some partner you use, and to enable the same functionality in your Enterprise tool you need custom coding to script vars or whatever. You know my big issues with GA are no api, limited data model, no control over processing etc… I think it is perhaps the prettiest tool out there and has some really actionable reporting thanks to the Great Avinash working *with* Google as a consultant. And don’t get my wrong, I dig the GA, just not for complex large scale implementations across many sites with a large number of tool users, report consumers, integration needs, etc…
We all use Word, and don’t just default to WordPad because you never know when you need to generate a Table of Contents, create a new template, use macros…
GO PATS!
Best,
Judah
Steve added the following ...
So this IT Nerd, Business Marketeer and Rocket Scientist walk into a bar….
Shudder. Sounds exactly like a bad joke.
Excellent response Judah. I hadn’t thought of the complete integration perspective - that tends to be such a rarity in Government. Isolated little fiefdoms is more the norm. Sadly.
Perfectly on topic: This is where I like Avinash’s 90/10 rule. It’s nothing specific to WA per-se. The same rule should be applied to just about any software product. Pure automation is really *hard*. Still need a brain to deal with the funkies.
Literally: *Cheers*! (clink)
- Steve
Who just made the first batch of christmas puddings to ship to the rellies. The smell of brandied pudding permeates the entire house. Damn.
Justin Cutroni added the following ...
Judah,
As always a wonderful post, thanks for sharing your experience. I would love to hear your thoughts on the ‘reciprocal’ topic. Web Analytics Tools – Am I getting the most out of mine?
I think many of the low cost tools (read: GA) are forcing organizations to re-evaluate how much they spend on their tools. I also think that analytics budgets may come under pressure when other, non-clickstream tools (testing, surveys, competitive data) are added to the budget. Tighter budgets may mean less for clickstream applications.
Thanks,
Justin