Beat Your Control by Testing the Big Things
Saturday, April 25, 2009 at 01:40PM When you test direct mail, for example, test the big things rather than tweaks.
“Tweak testing” pervades the risk adverse environments in many industries. This dramatically reduces their chances of competing with those firms that understand the need to compete aggressively with their testing programs.
Why? Because breakthroughs of 25% plus in costs per sale that displace controls do not happen with small changes. Big changes in results require big differences in your package tests.
The testing priorities are clear. The priority sequence is as follows. 
Typical direct mail test grid
1. The list
2. The offer
3. The format
4. The main selling proposition
So don’t try to structure the test so that you are testing paper stocks, envelope sizes, first class versus 3rd class and so forth. These all impact response to some degree, but by the time you figure it all out everything will have changed. So concentrate on the majors.
Besides, you have limited staff resources for analyzing test results as well as fixed test budgets. Spend your assets on the things that make a significant difference.
When testing offers, don’t try to make the two packages by forcing the same package contents or format. Test packages, not package elements. Every package possesses its own unique character. After all, you are looking for a breakthrough, not an analytical report that has too many variables to make sense out of anyway.
So in direct mail, test packages against each other and not package elements against each other. Your chances of beating your controls are greatly enhanced through this approach.
View Printer Friendly Version
Email Article to Friend
Reader Comments (4)
Wonderful post, very interesting and informative. Your priority list is something I'd have to agree with. First and foremost, you'd have to work on the list. No matter how great your offer is or whatever you are selling, your direct mail campaign will not be effective if you do not have a specific target audience and if your list is not accurate.
We used to have a saying as direct marketers. It said "He who controls the list controls the business."
Target marketing (i.e. the list) lies at the core of the direct marketing discipline.
Thank you for your comment.
Great article. I use a similar test grid like the one you have listed and I feel that it helps me figure out what works and what doesn't and how best to focus my monies and campaign.
melissadataarticles,
Thank you for your comment. Do you have any suggested improvements on your grid you could share?
Ted