you must fit your analysis to the time and resources available to you. If your client needs extremely precise results, and those results are not available from existing research, then you must inform the client of the resources necessary to get the degree of precision she requests. If your client will be satisfied by knowing what the effects are likely to be, without needing to know the magnitudes, you are much more likely to be able to complete your work quickly and easily. You will do neither yourself nor your client any favors by promising a highly sophisticated analysis that you cannot deliver in the time frame requested. Either you will be sleepless and underpaid, or your client will be dissatisfied with your work. It is much less likely that your client will be upset if you provide specific, detailed results when general comments would have served, except that you spent time and resources she might have preferred that you use elsewhere.
1. AGREEING UPON AN ANALYSIS BUDGET
There are several issues that will need to be resolved when you first discuss the analysis with your client. Will you spend six months or six hours on the study? Will your budget enable you to hire fellow analysts, or will you (and you alone) have to do the work? Will you be able to develop new data sources, or must you rely on someone else's study of an issue that‘s sort of (but not quite) similar to your issue? If you have to submit a bid to conduct a study, as consulting firms frequently do, you will have to develop an explicit budget and time line for your work, as well as a description of your expected final product. If you work directly for your client, your time and budget constraints may be negotiated directly with your employer. Regardless, these issues must be resolved before the work begins, to ensure that your time and effort are spent as your client wants.
Often the scope of the project will be determined even before you know that you will work on it: your client may want a result immediately, or she may have already decided that a multi-year, multi-million-dollar analysis will be necessary. In other cases, you will have an opportunity to suggest the appropriate scope for the analysis. Money spent on policy analysis, from a societal perspective, is money not spent on other activities, such as implementing policies. The purpose of the analysis is to improve the decision that will ultimately be made about the policy. At a theoretical level, a study is worth conducting if the benefits of the study – the improvement in the decision – outweigh the costs -- the time and money invested in the analysis. Under some circumstances, conducting a
study is unlikely to improve a decision. Several factors should influence both whether to conduct an analysis and what the scope of the analysis should be:
1.1 Might a decision change because of your analysis?
Is the decision-maker looking for the best alternative? It is possible that the decision is already determined – perhaps because a law mandates an action or because the decision-maker has already decided. If so, then nothing will change because of the study, and you might have more influence by working on another topic. On the other hand, an analysis may nevertheless be useful if future actions might change or mitigate the effects of the policy.
1.2 How well understood at this time are the impacts of the policy?
If the effects of the policy are completely unknown, then there is value in determining whether those effects are likely to be large or negative. If the policy is well studied, then another study may not be the best use of society's money. Sometimes, though, policy-makers may not respond to a study because they may not be convinced that it is really correct. Having studies that confirm the original results could add important credibility to the original analysis.
1.3 How large are the impacts of the decision relative to the amount of money necessary to conduct the study?
If the differences in impacts across alternatives are large – if millions of dollars, acres, or lives are at stake, for instance – then spending at least a small amount (and possibly a large amount) studying impacts is likely to be worthwhile. If the differences in impacts are small, though, then spending a lot of money on a study is not socially productive.
Those factors are useful for you and your client if you are deciding the socially optimal level of analysis to conduct. Is it likely, though, that the socially optimal level of analysis will be the basis of either your or your client‘s decision on analysis scope? In reality, you and your client will each have your own objectives in this negotiation. At an individual level, she may want a highly credible and completely thorough analysis done in almost no time for almost no money, while you may want to be well paid
for a long period of time and not have to work very hard. Both of you are likely to recognize the unfeasibility of these positions. Settling the scope of the analysis will require negotiation and careful communication to make sure that you both understand the work to be done and the time and budget you will face.
Analysis can also be conducted for political reasons. A policy-maker may be reluctant to make a decision because she is uncertain or indecisive, or because she does not approve of the proposed alternative, or because she hopes that another study will overturn results of the last study. In these cases, claiming a need for further study can bring policy implementation to a halt at least until the study is completed (a phenomenon sometimes called “paralysis by analysis”). While you may not approve of the decision to postpone, keep in mind that you are not the decision-maker. It is her job to face the consequences of this decision.
The analyst does not get blamed for doing good analysis, though the decision-maker can get blamed for requiring more analysis than a policy needs.The remainder of this chapter will discuss some of the issues you will run into as you determine the scope of your analysis. This list is suggestive rather than conclusive: other issues may arise as you become more experienced in policy analysis. In all cases, we encourage you to discuss in advance with your client how to balance the tradeoffs between limited time, money, and resources, and the desire for the best possible information.