You've commissioned an economic impact analysis to give your organization the edge it needs in its fight to develop your local economy. But before you start making claims and promoting your study's results, make sure they're accurate.
I know what you’re probably asking... "Fact-check my study? Why would I commission a study from an outside expert, receive it, and then turn around and do all the work over just to validate it?" And that’s fair.
I’m certainly not suggesting that you repeat all the work you’ve commissioned. But there are a few simple preliminary checks you can perform, as the recipient, to ensure that the study you’ve been given is as valuable as possible. Remember, you’ll be the one championing the study’s results, not your consultant. You’ll be the one using the numbers to support your statements and you’ll be the one putting your name behind their validity. So, before you do, make sure they’re worthy!
There are several elements of your study that you can review and approve quickly in your efforts to validate results, but I’ll only be sharing one of those elements here: framework.
Check Your Study’s Framework
As simple as it sounds, a study's general framework is arguably the most important element to review and approve before standing behind its results. Put simply, before you promote a study’s results, check its framework to ensure that those results actually reflect what you plan to say.
A study may employ quality base data, be methodologically sound, and thus be considered (by most standards) accurate and reliable. But if that study doesn’t fundamentally analyze the location, the industry, and the change in production that is of interest to you, then it’s results are NOT ones which you should promote. In a scenario like this, the results of the study might not truly suggest anything about your scenario.
The last thing you want to do is stand firmly behind a collection of results which, accurate or not, are irrelevant to the situation you’re discussing and, consequently, destroy your own credibility.
Given that economic impact analyses examine the financial effects of changes in production on local economies, there are three fundamental components at the heart of every analysis: study area, industry, and input. So, the very first step to take during a preliminary assessment of a study’s results is to simply confirm the accuracy of these components.
a. Confirm the Study Area
The geographic location being analyzed in an economic impact analysis is usually referred to (within the study) as the study area. So, firstly, in order to assess the effects of a change in production on a local economy, the study must define the geographic economy it considers to be local.
It’s important to make sure that the economy it defines as local appropriately reflects the region which you intend to examine. For instance, if a study’s examining impacts throughout a single state in the U.S., it might define the local economy as being all economic activity which occurs within that state, but none occurring outside of it. On a larger scale, if a study is examining impacts throughout the entire nation, it might define the local economy as being all economic activity occurring within the United States as a whole.
When reviewing the study area, it’s important to understand that even though a report will likely recognize it as one region, that one region could be comprised of multiple actual geographies. While a simple study may truly analyze just one distinct region (e.g. the state of New York), many studies actually analyze multiple regions, but recognize them as one aggregate study area (e.g. the states of New York, Pennsylvania, and New Jersey amassed into one composite region).
Be SURE that the study area examined in the study you’ve commissioned, and the actual geographic location which you hope to have examined, are in line with one another.
b. Confirm the Industry
Secondly, in order to assess the effects of a change in production on a local economy, the study must define which industry is experiencing the change in production.
When confirming the industry analyzed, it’s important to check that the primary economic sector which the study analyzes best reflects the true industry in which the change in production will be observed.
It’s important to note here that, the more specificity a study can provide regarding the industry which observes a change in production, the more accuracy a study can reasonably ensure. For instance, if a study aims to analyze the impacts of an increase in production in organic farming, citing an economic sector which reflects the farming of organic food items would be a better fit than one which simply reflects farming.
As is the case with study areas, the same rule applies when confirming a study’s industry; a simple study may recognize just one industry (e.g. a construction industry in a report which models the impact of a construction project), but more commonly, studies will define changes in production throughout several separate, but interrelated, industries (i.e. construction industries, manufacturing industries, etc.).
Be SURE that the industry examined in the study you’ve commissioned, and the actual industry which you hope to have examined, are accurately aligned.
c. Confirm the Inputs
The change in production being analyzed in an economic impact analysis is usually referred to (within the study) as the input. So, thirdly, in order to assess the effects of a change in production on a local economy, the study must define the input itself.
If a study is examining the economic impacts of the construction of a new hotel on a local economy, it must clearly define how much money is expected to be spent in the production of that hotel, or how many jobs that project is expected to create or support.
As is the case with specificity regarding study areas and/or industries analyzed, it’s just as important for a study to provide as much specificity as possible with regards to inputs. For instance, the ability to itemize a project’s spending with as much detail as possible all but guarantees exponentially more accurate results. Ultimately, the more specific spending and/or labor information your study can analyze, the more accurately it can estimate which other industries throughout the economy will experience stimulated activity resulting from the inital input.
It’s also important to remember here that the number of jobs created or supported by a change in production throughout an industry, or throughout multiple industries, might be used as an input.
Regarding specificity, as is the case with using dollars of spending as an input, it’s extremely important for a study to provide as much detail as possible with regards to the number of jobs created or supported as a result of the change in production, and with regards to the industries in which those jobs are created or supported.
Be SURE that the inputs traced by the study you’ve commissioned, and those which you hope to have examined, are represented as correctly as possible.
As mentioned above, there are several elements of your study that you can quickly review and approve in your efforts to validate results; its framework is merely one of them. To read about the others, click the link below to download our newest white paper 5 Questions to Ask When Fact-Checking an Economic Impact Analysis.