If we decide to collect our own data, the big issue is what method to use. Very often, the method will be dictated by the nature of our research question, as we will see in Part Three. The two main methods are surveys and experiments. Quasi-experiments are a good compromise when practical limitations, such as cost, make experiments problematic.
Surveys
In a survey, we go out and gather data from the world, more or less taking it as we find it. The best known type of surveys are polls, where we contact various individuals and ask them for the information. The best known type of polls are opinion polls, where the information we ask for is the opinion of the person contacted. But actually, there are a number of types of surveys and polls:
. An opinion pollis where we survey people to get their opinions. We will probably also get some facts (such as their age) from them at the same time. Correlating demographic facts with opinions and preferences is important to marketing strategy.
. A poll of people can be used to obtain facts (as the sample perceives them) and preferences, as well as opinions. A good example is an exit poll, at election time.
. A surveydoes not have to ask people questions at all. When a surveyor measures land, he is doing a survey of the landscape, defining measure- ments that go onto a map (a data collection tool). In business, one common type of survey is the inventory, where we either count all items, or, for large inventories, use sampling and estimate inventory size and condition.
Experiments
Experiments differ from surveys and other non-experimental studies in that an intervention, called an experimental manipulation, is performed. The experimental manipulation is usually intended to model some change that might happen in the real world. This change is directly connected to our research question. For instance, if we wanted to know whether our new ingredient improved the flavor of a soft drink, we might add the new ingredient to some bottles of the soft drink and not to others.
In an experiment useful in a business context, the goal is to determine whether or not there is a correlation between the intervention and some desirable, or undesirable results. Ordinarily, we think of the intervention as causing the changed results, if any. Within the context of a properly designed experiment, statistics provides information that may lead to a conclusion about causation. Statistical calculations alone can only demonstrate the presence or absence of correlation, not causation.
When we conduct an experiment, we are looking to see if there is a change resulting from the intervention. Therefore, it is very common in experimental studies that the intervention is only done to some subjects or units. The subjects or units that are modified are part of what is called theexperimental group, because they receive the experimental manipulation. The other subjects or units do not receive the intervention. They are part of thecontrol group.
KEY POINT
The comparison between the results for the control group and the results for the experimental group will show us a difference that may have been caused by the intervention, if there is a difference. Without the control group, it would be hard—perhaps impossible—to demonstrate that the intervention was the cause of a particular effect.
Ideally, we want to choose which subjects or units receive the intervention on a random basis. In that way, the experimental and control groups are random (or at least quasi-random) sub-samples of our overall sample for our experimental study. Just as there are many ways to sample from our population, there are even more ways to divide our sample into groups who receive different interventions (or no intervention at all for the control group). This process is called assignment to groups and is a key feature to different kinds of experimental designs.
Quasi-experiments
Between surveys and experiments are quasi-experiments. Quasi- Experimentation: Design and Analysis Issues by Cook and Campbell (1979) is the best source of information on quasi-experimental studies. Quasi- experiments are used when proper control groups cannot be used or are too expensive (or unethical), but something more capable of finding possible causes is needed than a survey can provide. Quasi-experiments include interventions, but who gets what intervention (or any intervention) is not determined in the best way possible. Group assignment is performed in a non-random fashion.
There are many, many kinds of experiments and quasi-experiments. Each of them presents its own unique statistical challenges. In Business Statistics Demystified, we will focus on the most common experimental designs used in business.
Writing a Statistical Report for Business
Statistics can be intimidating. (You probably know that already.) While clarity is vital in any business report, it is much more important and much more difficult in a statistics report. As with any report, the first rule is to keep your target audience in mind. How often do they read this type of report?
How sophisticated are they in terms of mathematics and statistics? Knowing how to present statistics with a minimum of numbers is a critical skill.
Finally, remember the old seminary school adage: ‘‘Tell ’em what you’re gonna tell ’em. Tell ’em. Tell ’em what you told ’em.’’ Both the introduc- tion and the conclusion should be expressed in straightforward ordinary language, laying out the business decision and relevant information.
Reading a Statistical Report
If you can plan and write a good statistical report, you also know most of what you need to be able to read and evaluate a statistical report.
Why is it harder to read a statistical report than to write one? Because, here in Business Statistics Demystified, we are teaching you to write a good, honest, accurate, useful statistical report. But many of the reports you will read will be poorly done, biased, vague, or useless. And those problems may be hidden under layers of pristine prose and clear charts.
KEY POINT
One of the most common uses of a basic understanding of statistics for business is the ability to read a statistical report and answer the questions: Is this report any good? Is it relevant? Will it support the decisions we need to make?
The problem of bad statistics is not a new one. In fact, Huff and Geis wrote the best manual on reading statistics and seeing through bias in 1954, How to Lie With Statistics. We highly recommend this entertaining and educational little volume. In it, on pages 122–142, they encourage readers to watch for these misleading techniques found in statistical reports, and in articles and advertisements that present themselves as being based on statistics:
. Who says so?Does the corporate or individual source of the informa- tion imply a possibility of conscious or unconscious bias?
. How does he know? Examine issues of population, sample size, and sample bias. Sample bias can arise from intentionally biased selection, from poor design, or from self-selection.
. What’s missing?Watch for an average that doesn’t specify mean, med- ian, or mode, and for conclusions not based on comparisons. If results are presented as an index, we must ask whether the index varied because of a change in the factor being discussed, or a change in the base of the index. And we have to pay attention to extraneous factors that may affect report results in such a way that the claims of correla- tion remain unproven.
. Did somebody change the subject? The link from data to statistics to conclusions may not be valid. For example, a survey may get people’s self-report of their behavior, but that may be biased, and not represent what they actually do.
. Does it make sense? Sometimes, the underlying theory is invalid. In researching the rapid growth of the cellular telephone industry, the second author found numerous projections that indicated that, if cur- rent trends continued, everyone in the world would own five or ten cell phones in another decade or so. But that just isn’t going to happen.
And, if we go back in time, we will find out that this very kind of error was cautioned against in 1954, inHow to Lie with Statistics.
Unfortunately, many of these misrepresentations and errors are hard to detect unless the survey data and research methods are available. If we have
only the results of the study in hand, all we can do is be suspicious. If we can’t acquire original data and a record of methods used, and we don’t trust the source to be making a better effort than we could to be accurate and unbiased, we should gather our own data and do our own statistics.
Otherwise, our company is making a business decision based on opinions and assumptions, disguised as a statistical report.
Quiz
1. A statistical study is. . .
(a) A project that makes statistics available to an audience (b) A project whose outputs are statistics
(c) A summary of the statistics
(d) A document containing charts and graphs
2. What is nota key to making a good statistical report for business?
(a) Focus on the business decision
(b) Save the details for the back of the report (c) Put supporting materials first
(d) Put supporting materials last
3. Which of these isnota type of information available for a business decision?
(a) Hard facts
(b) Data plus statistics (c) Assumptions and guesses
(d) All of the above aretypes of information available for business deci- sions.
4. The first step in conducting a statistical study is. . .
(a) Define the business decision to be supported, and the question to be answered
(b) Create a research plan (c) Collect the data (d) Analyze the data
5. The _______ stage can be conducted in any order; the _______ stage must be conducted in a specific order.
(a) Study; planning (b) Planning; study (c) Planning; report (d) Report; planning
6. Thebusinessreason for a well-planned statistical study is to reduce _______.
(a) Error (b) Bias (c) Bad data (d) Costs
7. The process of converting information into numbers is. . . (a) Sample size
(b) Statistical analysis (c) Encoding
(d) Data
8. Which is not amethodfor collecting data?
(a) Survey (b) Experiment (c) Quasi-experiment
(d) All of the above are methods
9. The _______ group receives the intervention; the _______ group does not receive the intervention.
(a) Control; experimental (b) Experimental; control (c) Assignment; experimental (d) Experimental; assignment
10. What should you ask yourself when reading a statistical report?
(a) Who says so?
(b) What’s missing?
(c) Does it make sense?
(d) All of the above
CHAPTER
Planning a Statistical Study
The need for planning in business is illustrated by the 1:10:100 rule for Planning : Building : Using. Missing a step in planning that would take one hour adds ten hours of work time to a project. If the work is missed during the project, it costs the customer a hundred hours of lost operations to fix the problem. For example, if an architect makes a mistake that he needs to fix, it would take him one hour of architectural work (planning) to fix it. But, if the error didn’t get caught until after construction started, it would take ten hours (and cost ten times as much) for the construction team to fix it. And if the error was missed during construction, it would take 100 hours (with four days of lost rental income) to move everyone out and fix the problem once the building was in use. In construction, this is not surprising. What is amazing, but true, is that this rule applies in every project we do, whether we are building something, or gathering and analyzing data, or making a phone call, or preparing a report. It has been measured and proven over and over.
The 1:10:100 rule is about as close to a universal law as you will get in
111
Copyright © 2004 by The McGraw-Hill Companies, Inc. Click here for terms of use.
business. You can reduce the costs of your statistical study by a factor of ten by planning it well.
BIO BITES
The first author spent a brief time as an assistant account executive at a small mid- town Manhattan advertising agency. I shared an office with three other junior executives. My boss had asked me to gather some information, via telephone, from five or ten companies. I had done such small, informal phone surveys before, so I felt confident. I stumbled through the first phone call so badly that my office mate, who was not a particularly nice guy, took pity on me. He sat me down and showed me how to plan a five-minute phone call.
I remember how shocked I was that something that small was worth planning for.
My office mate had a nice, simple system that took no more than three minutes, using a pencil and one sheet of paper. The plan was no more than a list of topics, in order, that I needed to cover in the phone call. I use it to this day.
The lesson: No matter the size of the task, so long as the planning process takes less than ten or twenty percent of the time the task will take, it is worth taking the time to plan.
For a number of especially obvious reasons, statistics benefits more from planning than do most areas of business. In statistics, because of all the calculations and the need for both precision and accuracy, tiny mistakes can have catastrophic consequences. In addition, in business statistics, we have to present to an audience that doesn’t know statistics. If we spend half our time planning, that is probably not too much, and may not be enough.
We will need a detailed research plan in order to conduct the research that will produce the contents of our statistical report. The structure of our plan will differ depending on whether we will be collecting our own data or obtaining pre-collected data from some source. In either event, these are the steps for creating a research plan:
. determine the plan objectives . state the research questions . assess the practicality of the study . plan the data collection
. plan the data analysis . plan the statistical report . prepare a budget
. writing up the plan and getting it approved
This chapter will show you how to do this process from beginning to end.
Determining Plan Objectives
A statistical study begins with a directive, perhaps from your boss, that may be written or verbal. It could be as simple as your boss saying, ‘‘Find out about our customers in Chicago.’’ Or it could be a full, clear list of questions, a sample report for how the results should look, and a date it is due. In all probability, the directive will not contain enough information for your plan.
By meeting with your boss and others, try to get clear answers to all these questions:
. When is the report due?
. How much money and time are budgeted for the statistical study?
. What decision(s) will the information support? What is the issue, and what are the specific questions? Is there a specific act of planning, opportunity under consideration, or problem to be solved?
. What questions need to be answered?
. Is there a sample of a similar report or study, done in the past, or done by another company, that you could use as a model? If not, find some, and show them to your boss and have the boss define what is needed.
. Will the report be presented orally, in writing, or both?
. Are there any rules or restrictions governing how you do the study?
For example, does your boss want you to use available data, or to do a survey or experiment, or is it left up to you?
In an ideal business, you would get all this information right up front. In reality, getting it may be like pulling teeth. If so, figure that you were asked to create the study, and be creative! Trust your judgment, come up with what you think is best, and then outline what you plan to do. Illustrate it with a similar report from another study, or a mock-up of the report you will create.
Bring it to your boss, and get an okay, or, if your boss has some ideas, there may be changes. Once you have the business issues, format, budget, and deadline defined, you are ready to turn the business questions into statistical research questions.
Defining the Research Questions
Now is the time to bring in a statistician to help with the plan. Together, you can translate the business questions into a series of statistical issues that will
define the study:
. What is our population?
. What is our sample size? How will we obtain our sample?
. Will we gather our own data, or use data from someone else?
. Will we generate our own statistics, or use statistical results from some- one else?
. What statistical measures and procedures will give us information that will support our business decision?
There is no easy way to do all this, and it definitely requires a consulting statistician. As we explain in detail in Chapter 9 ‘‘Meaningful Statistics,’’
statistics has been defined primarily to serve science, and the notion of statistical significance, in particular, was defined in ways useful to science.
As a result, it is very difficult to translate business value and business significance into statistical significance. This is exactly what a statistician can help you do.
CRITICAL CAUTION
Many statistical consultants will charge more per hour to rescue a project than to plan for one, so we save money both in terms of hours spent and on the hourly rate.
As you work with the statistician, your job is to keep all the business requirements and constraints in mind. The statistician suggests methods of getting the data and the statistics, and statistical procedures that will provide useful results. The structure of the research questions will tell that statistician what statistical procedures are most useful. The content of the questions will help you define the population, the sample size, and where to get the data.
Together, you seek a cost-effective way of creating a study that will provide statistical results that support the decision to be made.
For example, the decision to start a new advertising campaign requires a lot of information. Knowing the answer to the question: ‘‘Who will like our product?’’ might be helpful, but it is poorly phrased. A question like: ‘‘What demographic groups who are prone to purchase our product are currently unaware of it?’’ is better. A question such as: ‘‘Is this product more likely to be purchased by older or younger people?’’ is too general. Rephrase it as:
‘‘Which age groups, 8–18, 19–34, 34–50, 50 and over, are most likely to purchase our product?’’
Turning a general or vague business question into a good research question requires the following:
. Defining the population. In the above example, are we talking about potential customers in the USA, or in just one city, or worldwide?
. Defining the attributes of the population you want to measure.Age is one attribute. Gender is another. What about household income level, buying for self or others, spending power, or ethnicity? All might be valuable.
. Defining the required precision for each attribute.For example, an age range may be just as useful as an age exact to the year.
. Align all of these elements with business and statistical standards. For example, certain age ranges are used by magazines in describing their readership. If we use a different set of age ranges, we may not be able to use our study to decide which magazines we should advertise in.
Even if we get all of this right, have we really supported a business decision? It’s not clear that we have. If all we know is the demographics of people in the right market associated with likely interest in our product, we probably still don’t know how to reach these people. This study, or another study, or expert advice, will be needed to determine whether we reach the target audience by print ads, television ads, posters in a supermarket, demonstrations at health expositions, household parties with personal networking, or whatever. Why? Because the real business question is not
‘‘Who buys our product?’’ That’s not enough; we need to know, ‘‘How do we reach people who would want to buy our product?’’ and ‘‘What information, in what context, influences people who would want to buy our product to actually make the purchase?’’
As we work to define the population, the sample, the questions, and measures, we also have to think about how we will get that sample, and how we will ask the questions and take the measurements. From the business side, the key issue is cost. From the statistical side, we have to address two other concerns:
. Preventing bias. In various situations, people will report consistently inaccurate results, or tell others what they think others want to hear.
See Chapter 15 ‘‘Creating Surveys’’ for more about this. If we are using experiments or quasi-experiments, we will have to avoid bias as well.
For this, take a look at Chapter 6 ‘‘Getting the Data’’ and consult with a statistician.
. Ensuring data meet the requirements of the assumptions of the statis- tical procedures we will use. Each statistical procedure has certain