Performance, Performance, Performance
From an article I wrote for MediaPost a few weeks ago:
Reach and frequency and the core concepts of traditional media planning and advertising. For a given site, program, channel, radio station, billboard, newspaper section, a target audience (the reach) is exposed to a certain number of occurrences of the media (the frequency). On the web, these concepts manifest themselves in metrics collected and reported from a number of recognizable services. Audience measurement firms, like comScore and Nielsen, web analytics firms, like Omniture and Unica, to companies somewhere in between, like Quantcast and Google, all have reach and frequency data. Many new media metrics can be used to proxy frequency- from time-based measures, espoused by audience measurement firms, to concepts like visitor retention or the repeat visitor rate cited by web analytics firms. On the reach side, companies refer to concepts like “unique visitors.”
These data, of course, available in free tools or in for pay tools are certainly helpful for planning campaigns. But reach measures can be dirty (cookies, unduplicated unique users, estimates from panels, coverage error). Frequency measures can be just as dirty (problems recording time in single page visits or visits on the last page, do page views really matter with AJAX and rich media, cookies again, and so on). We all are aware of the challenges.
Thus using basic reach and frequency measures for planning or evaluating a campaign does not suffice. So advertisers and agencies target demographics, like gender, age, income, education, and job title. It’s a given that advertising in the Robb Report reaches a different audience segment than advertising in Popular Mechanics.
These brave new days we have “behavioral” tracking too. By taking into account visitor activity across sessions, such as past actions taken on a site or a roster of previous purchases, we can attempt to deduce what a person or segment responds to or is interested in based on their behavior.
Even with reach, frequency, demographics, and behavioral data to help guide advertising and media buying, we are missing an important attribute for maximizing the potential success of our campaigns. We do not have an available tool, whether free or paid, for advertising or buying media on or across sites according to measures of past performance. Such measures include ad clickthrough rates, conversion rates, goal completion rates, delivered impressions, and perhaps even harder to quantify financial measures such as ROI, ROAS, and ROMI.
Sure, historic, tacit knowledge of campaign performance exists and is used by agencies or publishers. However, there is no shared industry source that can help us answer “how has a site for display advertisement historically performed toward goals based on the reach, frequency, demographic and behavior of its audience segments?” Interestingly, a company minting money right now, named Google, can masterfully demonstrate performance in paid search campaigning and help advertisers unify it with segmented reach, frequency, and demographics.
Outcomes based performance measurement unified with reach, frequency, demographics, and behavior is what is missing in audience measurement tools, not frequently reported externally by web analytics tools or ad serving tools, and not available in ad planning tools. When advertisers can target display ads, or even video ads, to desired audience segments by reach, frequency, demographics, behavior in the context of known performance, media planning will be more effective.
Helen Vetrano added the following ...
Great post =) Surely there is an opportunity to combined outcomes with reach, frequency, demographics, and behavior data as you have described—historical and predictive. It would be great to know if, for example, males age 18-35 of the greater Boston area, who frequent Amazon.com 3x a week and search for the latest Manga are more likely to click your ad on Amazon with a 3% conversion rate compared to another segment or have historically registered this rate.
Considering the very specific elements of this segment, Bostonian-Manga-Loving-Guys, I wonder how this data could be distributed and who would manipulate it to create the segment…Maybe I should have chosen a broader target =)
Regardless, I can see how other elements beyond the above 4 segmentation attributes can affect outcomes and produce dubious stats. Mainly, elements that the advertiser controls—messaging, calls-to-action, landing pages, design, etc. So a vendor may tell me that this segment can generate $xx ROI, but is that really the case in the context of execution?
Judah added the following ...
Toby: Thanks for your astute comment. I agree that the publisher would be made more responsible, and in most cases the agency and client, I think, would also be made more responsible. The publisher, would be more accountable, in the sense of the delivering the qualified, core audience that “converts” to the advertiser via their content/editorial offerings, while the agency would be accountable for optimizing the creative, and the client in ensuring the asset advertised fulfills audience expectation. You make a good point in dividing into “direct response” versus “brand awareness.” I agree that awareness, affinity are challenging to measure (engagement?).
Helen: Thanks for the excellent feedback! Your example makes me think BI would do this. A few WA tools could expose this data if configured to do so, but only at the site level, which would be good for understanding audience in the context of all those dimensions on the site itself, but not across sites. Perhaps Quantcast is the closest to being able to measure segment outcomes/demo/behavior/reach/frequency across sites. You have identified the issue with conversion in a nutshell. Parts of the conversion equation are outside of the site’s control and subject to the agencies creative input, which of course should be provided in a framework that allow for testing to ensure the prediction has a better chance of fulfilling itself during execution. Thanks for commenting!
Helen and Toby: Apologies it’s taken so long to respond, I’ve been on vacay! 
Toby Christophersen added the following ...
You raise a valid point with regard to the lack of a communal pool for historical ad performance data for Online, however I wonder, in a world where Online competes directly with TV, Radio, Print etc for allocated spend in campaigns whether its measurability is to an extent an Achilles heal, given that for example lead generation rates from radio spots at particular times on particular channels are not easily measured or historically archived, and that this type of uncertainty is traditionally acceptable in other media.
Further, if such a pool existed, it effectively makes the publisher, not the client or its agency, responsible for ensuring that the creative running in all ad spots on its site(s) is of sufficient quality to engender direct response at a rate which would not reflect badly on the site(s) public performance data in the long term.
Finally, this type of performance data still somewhat fails to take into account the effect of ads where brand or product exposure/awareness/perception etc rather than direct response is the campaign objective.