Testing 1234 -- Strengths, limitations and opportunities in market testing

When you read through this posting, you'll see a "Test 1234" where you can help out, individually, because the only way I can figure out why our emails aren't working so well is to find if there is a real problem here. Testing can be invasive, irritating, subtle, or (perhaps i will find out) useful in involving your community in your marketing and business challenges. Thanks for helping out. You can click on the image to send the test email or email buckshon@cnrgp.com, but remember to phone or comment to validate the results.

Is your bright marketing idea really so bright? Often, I have to admit, my "best ideas" bomb in the real world, have unintended negative consequences, or simply don't work quite the way I thought they would.

Then of course, there are the ideas that didn't seem to have much going for them in the first place, but for some reason seemed worthy of pursuing. This blog, for example.

Is testing a way around this challenge? Can we keep ourselves out of trouble by trying the idea on a small enough sample and/or short enough time that we can tell if our marketing ideas are good, bad or ugly -- and catch disasters before they happen?

The answer at first may seem to be an obvious "yes", and I'm pretty confident I've saved my skin from some bad idea implementation by trying things out with a simple test first.

In one case, for example, we were exploring starting a new leads service, providing really valuable data in co-operation with a major partner, using relatively low cost print-on-demand technology and providing genuine value and service to readers.

My sixth sense told me to be careful, so I asked a non-employee sales representative (who I know was good at the type of selling required) to work with me for a couple of days to see if the idea would fly. It bombed.

But here is the rub, and challenge, of testing. Maybe the test really wasn't valid. Possibly some variable within the testing process had thrown the numbers off, and with a slight rejigging of the concept, either within framework, market or focus, it would have worked wonderfully. We'll never know, of course.

Conversely, we know of awkward tests that cause real market problems; specifically the infamous "mike tests" where politicians and celebrities are caught with their pants down (or, to use another cliche, their foots in their mouths) when they say things that really aren't meant for broadcast in an AV test.

Finally, in the construction marketing community, we have to contend with the reality that many of our most interesting and important projects arise from long-term and interconnected relationships, which run counter to the principals of simple A/B testing and other measuring methodologies. If the process has a long lead time, how can we figure out when and what to test, and where are the boundaries of good metrics and simply wasting our time counting things that aren't really important?

Here are some thoughts and guidelines that can perhaps help out on our testing methodologies.

If it can be measured, it can be tested.
You will find test results are most meaningful and easy to implement if you have some simple metrics to assess your ideas and results. Online resources provide a wealth of measurement tools, including hits, conversion rates, and the like, and you can develop simple processes to tell if something is working or not. Once you have a baseline, for example, on responses to your emails, you can try different things with A/B samples, sending one message to half the list and another message (with the change you are trying to evaluate) to the other. You'll know which works right away.

Testing works best if is real time, not obtrusive, and doesn't seem to be a test.
The "Hawthorne Effect", where the fact that by letting people know you are testing changes behaviour, is a real risk if you are obvious in your evaluations. So you need to be discreet. Market surveys are often problematic -- many people hate surveys and won't respond (your most important sources of information), or you need to be intrusive with phone calls and personal contacts (thus alienating the person you are seeking to survey, or again, you achieve invalid results.)

You need to catch subtle clues as well as objective measurements in your testing (carefully).
Sometimes the real results are not the ones you think you were going to find; so you need to probe into the observations you obtain and listen carefully to subtle clues. Of course you may have it wrong, but you may also score major insights.

So, yes, test your assumptions, your ideas and your marketing strategies, especially if you are planning to commit major resources to the process. But be wary of the testing pitfalls.

And now for a real-time test (and a request for your assistance) because we are having some strange email problems. Some emails I and my team are sending are not going through; in other cases, people are sending emails to us, which we aren't receiving. The problem may be spam blocks; I don't know.

1. Please send an email marked "Test 1234" to me at buckshon@cnrgp.com.
2. At the same time, please post a comment to this blog (which operates on a different server) or phone me at 888-432-3555 ext 224. (If you wish to leave a comment and remain anonymous, just make reference to something in your email which won't disclose your identity.)

If you complete this test, I'll send you a copy of my new publication: The Art and Science of Publicity, but most importantly, a really big "Thank You".

The more responses the better -- the key measure will be the number of communications we receive through the blog comments and phone messages which fail to correlate with the number of emails we receive. But I'll learn other things, as well, which I'll share with you tomorrow.

Again, simply email a note to me at buckshon@cnrgp.com with a subject line: "Test 1234" and communicate offline, either by a blog comment or phone call to 888-432-3555 ext 224 to tell me you've sent the email. Thanks.

Read More

Testing your ideas

Yesterday morning, I floated a trial balloon on this blog, suggesting that the Design and Construction Report could be an effective opportunity to associate your interests (as Construction Marketing Ideas blog readers) with our own marketing strategies.

I then presented the idea to our weekly general meeting, and received what amounted to a collective "sigh" from my company's sales team. They effectively told me, "no, this won't work."

Glub. "Brilliant ideas" go nowhere without execution. So, the question is, am I wrong to assume there is a valid marketing correlation between responses and connections from this blog and the marketing of publicity-focused services, or should I push forward and (even more dramatically) invest time, resources, and money in making it happen?

Most of the time, these sorts of questions are aired and responded quietly within our own businesses.

But this blog's foundation is its openness and experimentation -- you can see ideas maturing, tested, and sometimes failing, right in front of your eyes. The result hopefully show you what can work for your own business, and which ideas should be dropped like lead balloons. (That's the second balloon analogy in this posting.)

In practice, most successful business people process dozens, even hundreds, of ideas a year. These arrive through their own experiences, insight flashes, employees, external media publicity and conferences, and sometimes, the initiative of outside sales representatives.

We need to process the ideas quickly and sort them into ones worthy of further experimentation and implementation. Most ideas rightfully end up right in the trash can, but unfortunately, I'm sure, many worthy innovations are not introduced because of inertia, fear, laziness, or simply lack of knowledge.

The question is, how can you sift the ideas effectively and quickly and focus your energies where they will be most useful? I'm going to go out on a limb and suggest these tests are worthy of consideration.

Is the idea you are testing low risk, easy to implement, and not likely to cause too much disruption in existing operations?
This is an "easy to say yes" idea -- but one which probably won't have that much impact long-term.

If you are asking someone else to do something, would you do it yourself? If not, why not?
My sales representatives said "no" to the idea of marketing leads generated by this blog for the Design and Construction Report. I suppose I could force them to do some work here -- but if I'm not ready to do it myself, why should I ignore their resistance?

Can you find quick tests that will validate your assumptions, without burning bridges or draining resources?
A few years ago, a well-placed person proposed a business idea that seemed relevant and valid, but would have required commitments of resources and effort beyond normal levels. We spent months setting up the concept, even developing prototypes. But I hesitated and decided I needed to see how the market would respond. I contracted with a reliable but non-employee representative to work with me for three days to see whether he could sell the idea. It failed, miserably. Maybe the idea should have received a 'second chance' but I realized that we were entering the space of diminishing returns. I dropped it.

Will the idea produce immediate revenue or reduce costs right away, or is it an "investment for the future"?
The former two variables are easy to say "yes" and move forward, the latter requires a much higher commitment threshold (and the risk increases with the investment size).

Can you find a test that will quickly provide you with insights and results at minimum cost and effort?
The three day test described above for an idea we had worked on for months made sense (and the investment in a few days pay for one person proved to be worth every cent), but do you want to spend that much time and effort validating other ideas, at the earlier stages of their development. You need some quick tests and measuring tools to tell you whether to take things further, or drop the idea.

For the idea I posted yesterday, the negative feedback from my staff is a major warning to go slow, if at all.

I can now take things in two directions: Pick up the phone or send some emails to a sampling of potential clients to see their interest (that is, do the work myself, with sampling, one by one), or survey the readers with a quick online poll.

I decided on the latter -- it is fast, easy, and if it works, will provide immediate and actionable leads. And if it doesn't work, I'll know equally quickly, and we can move on to other ideas.

If you haven't already received the survey invitation email from me, you can see it here -- and once you complete the survey, you will also see the results!

Read More