Category: Digital Marketing

Posted on

Important Questions about Online Experimentation

To supplement our reading this week. I wanted to highlight on some important points that the article brings up, but I feel deserve a bit more time and explanation.

Why do most online experiments test a single variable at a time?

Well, this comes down to basic scientific principles. To determine causality of an outcome, we must be able to narrow down what change in the environment led to that outcome. 

Maybe you’ve heard the latin phrase “ceteris paribus” or more likely its English translation “holding all else equal”. Well that’s what we’re talking about. Holding all variables except our focal variable equal or in their current state. For example, if I want to figure out if adding a rocket ship emoji to my email subject line will result in a higher number of email opens, well I would send the exact same email to two statistically equal audiences—which we usually do through random sampling—

but the only difference would be that minor addition of the rocket ship emoji. If we then see a statically significant change in our email opens, then we can deduce that the emoji addition that we made actually caused the uptick in email opens. At this point we’d probably integrate that change into our email campaign and then send then email with the emoji addition to the remainder of our email marketing list.

If we change multiple elements of our email. For example, we added the rocket ship emoji and we also changed the font we used in the body of the email or the call-to-action label we used or heck even the time of day we sent the email then with most email management systems we have no way of knowing whether it was the emoji or the other element that we changed that caused the uptick in email opens. 

So, if you’re going to do online experimentation, only experiment with a single variable at a time. Once you’ve established the source of a positive effect, incorporate that change and now test another variable. Firms like Google and Amazon run thousands of these types of experiments every single day. Can you run experiments with multiple variables simultaneously. Yes! But systems that can properly analyze them are expensive and these experiments are more difficult to interpret. 

And there many different variables you could choose to manipulate to see if there is room to improve the outcomes of your email campaigns. We’ve talked about the subject line. You can also change the sender name. Perhaps one name might resonate better with an audience than another. Calls-to-action are of course a really important element to promotion emails. What wording will work best for your customers? What about the button size or color? Personalization is another really important element. People tend to respond more favorably when their name is used, but what about incorporating information related to purchases they’ve made in the past? Retargeting emails like abandoned cart notifications fall into this area. Day and time can make a big difference. When do your customers tend to open their emails? The design of your email is a big one. More likely than not you would be test more than one simple tweak as part of a design change, so you may not be able to know whether it was the color scheme change, typography change, and use of images, but you would know whether one design was more effective than another. Frequency, or how often you send your emails could be another way to test, though this would be a little different that a typical A/B test. Finally, target audience is another element that can be tested. Think about breaking your audience lists into groups based on how they respond to your emails and you might also find other similarities that can help you define a new audience segment.

What is A/B/C testing?

We’ve established that the correct way to test elements to determine causality is to only change one variable and hold all else equal. So then what is A/B/C testing. Wouldn’t that be testing more that one variable. 

Well, no. Recall that when we engage in A/B testing we have one variable, again let’s say email subject line. But that one variable has two states or treatments: with the rocket ship emoji and without the emoji. 

Given a sufficiently large enough sample, we could actually test three maybe even four treatments of the same variable. So I could try the standard version of the subject line, one with a rocket ship added, and maybe one with a puppy emoji added…A, B, and C. This is still only testing one variable, but with more that two test treatments.

The is a common occurrence with real-time online testing. For example, Amazon might run a simultaneous experiment where they divvy up their online customers into multiple groups. The first group to whom they show the current variation of a page design (for example showing vehicle body styles in black&white, the other groups get different variation including an alternate version where the vehicles are in full-color). With real-time online experiments like this, a firm has the ability to immediately shift course and serve up a version of the page that is more successful, or in other words, leads to more conversions.

Do online experiments need to be driven by an underlying theory?

In the scientific method, we often start by forming a hypothesis about something based on existing theories. We then devise an experiment that we can use to test it, run the experiment, and analyze the results. Theory is the lifeblood of scientific research. But an email marketing campaign or a search engine website are commercial endeavors. We can use the scientific method to test variables, and we should, but that doesn’t necessarily mean that we have to have a theory that drives our initial interest in experimentation. 

Business is a fast-moving, highly-complicated environment. As marketers, we’re continuously looking for an edge to help improve our outcomes. Sometimes we have a gut instinct about what might appeal to our customers and that is a completely valid starting point for experimentation in the business world. If you’re in the healthcare industry and your gut feeling is your customers will likely stay on your website longer if you use a color palette of blues and not yellows, then that’s an easy thing to test and you don’t need to dive headlong into color theory to find justification for an experiment. These types of experiments are inexpensive and quick and they can often reveal surprising outcomes. 

Posted on

How Email Technically Works

For this video I thought I’d get a little technical. You already know the basics of how email marketing works because by now you have likely earned your HubSpot Email Marketing Certification, but do you know how email marketing technically works, the actual mechanism behind how we know when the user opens and clicks through on an email? Let’s get into it.

When the consumer receives an email from a marketer, there’s a lot going on in the background that you’ll never see. The consumer may or may not have an idea that their email address is part of an email marketing list that’s been segmented and profiled and sliced and diced in any number of ways. But what most probably don’t know about is the tracking technologies that make all of it work.

Mailchimp, Campaign Monitor, Emma, Constant Contact—All the major email marketing platforms work on the same underlying principle: beacons. Also known as web beacons or tracking pixels. What’s a beacon? A beacon is a small 1-pixel x 1-pixel graphic embedded into the email that recipients receive. 

For example, if I receive an email marketing message from Williams-Sonoma, within that email is a beacon. The beacon will tell Walgreens whether or not I opened the email, and of course if I click anything in the email then Walgreen’s analytics system will then be able to register a clickthrough of the email. 

You can’t see it, but the beacon is there. And it’s the most important image in the entire email. Now you might ask yourself how on earth does a tiny little graphic help them know that I’ve open the email?

Here’s how it works: When an email management program is used, each email that is sent out using that system has  Code for a beacon embedded within it. 

The act of opening the email starts the process of your email program requesting all of the graphics that were embedded in the email. One of these graphics is that invisible beacon. Because the beacon has a unique filename, the server that serves up the beacon 

now knows when, and in many cases where, the email was opened and clicked and by whom because that unique filename is tied to a specific account. By the way, it’s only when you actually open on the email to view it are the graphics loaded. If it’s just sitting in your inbox, no graphics are loaded, so no tracking information is captured.

Here’s an example email from Walgreens. Looks like a fairly normal, if not a bit bland, email. There are a few embedded graphics. One dominant call-to-action, the “Get offer” button. What you can’t see in this view is the beacon. But if we view the raw source for the email, we can find it. 

Let’s open the raw source…and scroll all the way to the bottom…and here it is. The email marketing system that Walgreens uses actually is nice enough to clearly label the tracking pixel or beacon. By they way, the reason you see all of the “3D”s in the text is because email uses a specific encoding mechanism for transportation because email is based on an early text-based format. 

Of course all of this works because we as consumers prefer to see emails with embedded graphics, or HTML-based emails. HTML emails are essentially little webpages that are sent to your email inbox. What would happen if we didn’t use HTML emails and instead used the old standard of text-based emails? Well, because you can’t embed graphics into a text-based email, you would completely disable the email marketing platform’s ability to track your mail opens. It used to be the case that when you signed up for an email subscription you were given the option to select whether you wanted text-based or HTML/graphics based email. Those options have largely disappeared as marketers have become more and more reliant on email marketing and value the advantages that HTML-based emails give them.

Remember, the next time you receive a promotional email, there’s a lot going on in the background to ensure that they send can track when you open and/or clickthrough that email. And if you don’t like, there is, of course, another option for you as well.

Posted on

The Long Tail is a Power Law

So I’d like to introduce you to a friend of mine. His name is George. George Zipf. Besides having an exceedingly cool last name, George here was a pretty smart guy. George studied at Harvard and became a world renown linguist. He became an expert in what is known as quantitative linguistics, meaning the study of how we learn language, how language changes, and the structure of language, including the frequency of word usage. Now that last one is important as that’s what made George famous. He studied the frequency of word usage in language. He discovered something interesting. When mapping the frequency of word usage in any language, only a few words are used very often, and most are only rarely used. And how does one determine the frequency of words in a language? Well, you count a lot of words in a lot of books.

To put that in scientific language: given a corpus of natural language word usage, the frequency of any word is inversely proportional to its rank in the frequency table.

Well, what does that mean exactly? Let’s take a look at the most commonly used words in the English language.

You can find this on Wikipedia. Unsurprisingly, “the” in the most common word, follow by be, to, of, and, and a and so on.  Let’s go to the virtual whiteboard and write this out. Alright, let’s write out our words. We have, the, be, to, of, and, and a. If “the” is the most popular word that would be 1 or fractionally-speaking 1/1. The second most popular occurs half as often, so it is half or 1/2. The third happens 1/3 of the. The forth 1/4 and so on.

If we graph this out it looks like this: a line that quickly falls and flattens out, but never actually hits zero. An asymptote. This is what’s called a power law.

The interesting things is that George Zipf and others started to notice that a lot of other data sets from the physical and social sciences seemed to follow the exact same “Zipfian” pattern. 

And thus was born Zipf’s Law.

Let’s look at an example completely unrelated to to word frequency: City populations in the United States.

Here’s our list of the major cities in the US and here are their populations. If Zipf’s Law is actually reflected in city size than we would expect the second most populous city to be right around half as big as the most populous. So Los Angeles should be around half the size of New York City, or 4 million people. 

And that’s what we see. The predicted population using Zipf’s law (or multiplying New York City’s population by one half) gives us 4 million, which ends up only being off by 7%. 

Looking at Chicago, which would be one third the size of New York City, the results are darn close. For the rest of the top cities, you can see the results are also pretty close.

So what does this have to do with digital marketing?

Let’s look at an important example, search marketing. Let’s say you’re a logistics management software development firm looking to advertising yourself on Google Search. Because you know that Google’s Search Ads system is based on a real-time bidding auction, you know that bidding so your ad comes up for the popular keyword searches is going to give you the most number of individuals seeing your ad.

If you bid on the keyword “logistics” you are guaranteed to get exposure to the most number of eyeballs. The problem is, you’re selling logistics management software. The keyword “logistics” is overly broad. Let’s narrow our keyword bidding a bit. Now we bid on the keyword “logistics management software”. Because the keyword is narrower, fewer people will see it, but chances are those people are going to be more relevant to us because they typed in “logistics management software” or something similar into the search engine.

Now, you’ve done your marketing research and your know that the vast majority of logistic management software purchases are preceded by purchasing managers reading up on successful deployment case studies. Wouldn’t it be great if we could target those people? Well, we can. Again, just narrow the keyword we bid on to “logistics management software case studies” and now our ad will show up to fewer people, but those people are far more likely to convert to a sale.

The great thing about this is, as you increase the complexity of your keywords and more narrowly target your audience—looking for only those people typing in the search terms that will match your keywords—your competition goes down. And when your competition goes down, so does your bid cost. Thus those lowest-common denominator keywords in the so-called “fat head”, which are expensive and a fairly blunt instrument, are far less appealing than the keywords that sit here in the long tail.

How is Google addressing the Long Tail? 

Obviously, the more narrow people are with the search terms they use to find things the better result that gives the user. But does Google benefit from pushing people towards more common, “fat head” searches instead? Turns out, they do.

Only 15% of Google’s daily search engine queries are unique. Just a few years ago, that number was 20%. When unique queries are run by a search engine, it is more processor-intensive and thus more expensive. So convincing users to utilize search results that have already been indexed is more cost effective. Here are a few ways they do that.

When you start typing into the google search field you are offered suggestions. This is Google subtlety steering you toward those pre-indexed pages. In some cases, as in this Disneyland example, you don’t even have to finish typing a question before the Google dropdown provides an answer. Finally, if you’ve already made a query, you’ve established a context. And future searches operate within the search context. For example, if I do a search for the original opening date of Disneyland, but then start a new query with “Who is the…” Google will fill in the rest using artificial intelligence trying to figure out what you might want. Again, subtlety steering you toward pre-indexed, less expensive results.

To sum up, George Zipf observed a pattern in nature that’s reflected in many physical and social science phenomena. This pattern creates the same long tail distributions that we can use to our advantage as digital marketers. Oh and by the way, the long tail doesn’t just apply to search engine keywords, it’s can be used to understand a number of phenomena that are relevant to digital marketers.

Posted on

The upside of showrooming: How online information creates positive spill-over for the brick-and-mortar retailer

The ubiquitous nature of mobile internet devices (i.e., smartphones and tablet computers) has led to an increase of their use within the retail environment as a shopping assistive technology. Consumers use them for a variety of shopping-related tasks, the most significant of which is researching product information. The use of these devices has clearly impacted how consumers shop, but what is not clear is how these devices affect shopper satisfaction, trust in the retailer and subsequent shopper intentions. The purpose of this paper is to better understand these relation- ships and extend existing research on the use of mobile internet devices in the retail industry. Several hypotheses are offered, and survey data from a nationwide random sample of consumers tested the hypotheses using structural equation modeling. Results indicate that shoppers’ satisfaction and trust in an online information source creates a spill-over effect on satisfaction and trust toward the retailer. Additionally, retailer repatronage intentions increase as a result of this spill-over effect. Contributions to emerging mobile marketing literature and theory, managerial implications, and future research recommendations are discussed.

Spaid, B. I., O’Neill, B. S., & Ow, T. T. (2019). The upside of showrooming: How online information creates positive spill-over for the brick-and-mortar retailer. Journal of Organizational Computing and Electronic Commerce, 29(4), 294-315.

Spaid, O’Neill & Ow 2019Download

Posted on

Conversions

What is a conversion?

As you know, one of the fundamental aspects of our job as a marketer is to capture value in the marketplace. All the efforts we go to and all of the resources we allocate and spend— to create, communicate, and deliver value—all have capturing of value as their ultimate goal. Of course, we do this by adding customers. But an individual doesn’t necessarily have to buy something from us for some level of value capture to occur. What we’re really trying to do as marketers is convince the recipients of our marketing message to do something. We’re trying to change them from being one thing in our eyes to something else. We’re trying to convert them.

In reality, we don’t typically convert complete strangers directly into customers. There are often many steps in between; many conversion points that this individual passes through. At each conversion point the fuzzy picture we have of who this person is becomes clearer until we hopefully add them as a customer.

MarketingSherpa, an online resource for digital marketers, defines a conversion as “The point at which a recipient of a marketing message performs a desired action.” A “desired action” could be just about anything. And that’s kind of the point.

Because everyone’s customer journey is different, marketers have to provide many different ways for customers to engage with the brand, help them get the information they need, provide the best route to become a customer, and then a repeat customer. 

Conversions and the Customer Journey

Let’s look at the customer journey graphic that I introduced during the Content Marketing lecture. This graphic shows conversion taking place and leading directly to dollars (the purple section). This customer journey is a good example of what an online retailer might use. Where the key conversion is when a sale is actually made.

Now, let’s look at Hubspot’s version of the customer journey. Hubspot is a software company that provides online lead tracking and analytics tools. Notice how one of the stages in their overall process is called Convert. Where “visitors” become “leads”. This is typical of service-based firms or manufacturers with complex products where leads are the most important metric for digital marketers. Why? Because often in these businesses the leads are worked by sales people in offline processes where detailed questions and concerns must be addressed before a lead can convert to a customer.

Let’s look at some examples of all of this in action.

Online Retailer Example

That’s say I’m in the market for a new electric guitar. I might start at a place most people start, and that’s Google. So let me enter my search terms. I have my eye on a Fender Stratocaster Elite and now I’m taken to a search engine results page. You can see Sweetwater here is the top ad result. You can also see the Fender website is here and Sweetwater appears in the organic results. I’m going to go ahead and click this link here for Sweetwater. And the second I click that link I’ve now been converted from a stranger to a visitor to their website. And now Sweetwater is going to be tracking me while I’m on their website. Well, this blue color is nice but let’s say I actually like more of a sunburst look on my guitar. Yeah, that looks fantastic. I think I’ll put that one in my cart. So I’ll hit Add to Cart. 

This is where Sweetwater really excels at an online retailer. they use every opportunity they have to add additional value to the shopper. Notice that we don’t go directly to the shopping cart, but an intermediate page that allows the shopper to customize their purchase with a variety of add-ons. Ok, let’s go to the actual shopping cart now. Notice how this page also takes the additional opportunities to provide more product and service offerings. You can add a discounted t-shirt to your order (another product). You can get special financing terms (a service) and you can even donate to a good cause. After I go through the checkout process i have been officially converted to a customer. Don’t forget a good retailer isn’t done with conversion just because the sale was made. A good retailer, or any business for that matter, strives to delight its customers and convert them into loyal customers that will keep coming back. How does Sweetwater do this? They do this in all and very big ways. From including a little bad of candy with every shopped purchase to hosting one of the largest free music gear tryout events around, Sweetwater’s GearFest.

Online Services Firm Example

So that was a great example of how a retailer might handle conversion. Let’s look at how a services business does conversion.

Again, let’s start our search on Google. Something bad has happened and you need an  accident lawyer. We can see from the results that there are a number of legal firms to choose from. Huh, the law firm in the second  ad slot sounds familiar. Maybe I’ll click on that.(Don’t worry, I didn’t really click the ad. No ad budgets were harmed during the making of this video).

Now that we’re on the website, we’ve been successfully converted from a stranger to a visitor. You can see from this page that it is very clearly setup to help answer questions that someone in the market for an accident lawyer might be asking. All the different types of accidents are clearly presented with a very obvious next step button below. By the way, this “Start now” button is what we call a call-to-action or CTA. Calls-to action are attention getting buttons or text links designed to get you continue on the customer journey.

Let’s click the call-to-action.

Now law firms like this are high touch, meaning they want to make sure that they personally answer all the questions a potential customer, what they would call a client, has. It’s not surprising that they would put the firm phone number front and center. In fact, their phone number appear four different times on this page. Besides the large phone number in the center on the page, it’s in the header and twice in the footer on every page. But also notice there’s a live chat feature. And finally if we scroll down the page a little, there’s a contact form we can fill out and provide more details about our accident. If someone calls the phone number, initiates a live chat, or submits the contact form they will be successfully converted into a lead. This is the point high-touch sales will take over. And make no mistake, a big part of being a successful lawyer is being a good salesperson. You’re selling yourself, your skills, and the firm. You have to be convincing if you’re going to get new clients.

From a digital marketing standpoint, the contact form is the easiest to track. Every time that form is submitted, it will trigger a series of actions, one of which is the completion of an online goal that marks the conversion of a website visitor into a lead. This firm likely has other ways of tracking interactions with leads over the phone and with the live chat system.

Now you’ve seen how conversions can be different things to different types of businesses. And that a single customer can be converted multiple times along the customer journey. All of those conversion points are important, but it’s that key conversion, when a visitor buys or becomes a lead that is most crucial.