Five Questions with Jeffrey Lyons
In our “Five Questions” feature we ask scholars, activists, and public officials five short questions about their work (and other things).
Professor Jeffrey Lyons is the School of Public Service Survey Director and an Assistant Professor of Political Science. His research focuses on American politics, specifically public opinion, political behavior, political psychology, and state politics.
You have the designation as the School of Public Service Director of Survey Research. What does this entail?
This means that I am in charge of both of the School of Public Service’s surveys – the Treasure Valley Survey and Idaho Public Policy Survey. But, I work a lot with Greg Hill and Vanessa Fry at the Idaho Policy Institute, as well as faculty throughout the school to make these surveys happen. We do everything from soliciting community feedback on the kinds of questions that people are interested in, to presenting the results to elected officials and groups involved in addressing policy problems.
Can just anyone write effective survey questions? Is this an art, a science, or both?
I definitely think anyone can write effective questions, but it is a lot harder than it looks. There is a science element to it in that there are some established practices for certain kinds of questions, but there is definitely an art to it as well. You have to put yourself in the shoes of the respondent and try to guess all of the possible issues, confusions, and objections that a range of different people could have. Writing a question that can be answered by both someone who is a highly knowledgeable expert on a topic as well as someone who has never heard of the issue or may not know any information about it is a challenge. It is easy to write a question for one person or the other, but it’s difficult to do both at the same time, especially while keeping the questions brief so that people don’t hang up their phones.
Tell us a little more about the survey work you have been doing with SPS and IPI.
Both of our surveys are 1,000-person samples so that we have enough respondents to keep our sampling margin of error low (at around 3%), and to give us enough people to do some subgroup analysis of specific geographies or demographic groups. We have moved to a mixed-mode survey where we are contacting folks on cell phones, land line phone, over email, and via text message. The hope is that we can minimize some of the issues associated with people not answering their phones by contacting respondents in a variety of different ways.
For each survey that we do, we spend a fair amount of time reaching out to people in the community such as elected officials and policymakers, about the issues and topics that they could use public opinion information on. Because one of the goals is to be a resource to decisionmakers, we are always trying to find out what challenges people are focused on where public opinion data might be helpful. On both surveys this means that we have a core group of questions that we have been asking consistently over time that is related to general topics of interest, as well as groups of questions that are unique to each year and might be focused on a particular topic that is being discussed. For example, on our Treasure Valley survey we have been asking some general questions about perceptions of growth in the Treasure Valley for the last four years, as well as deeper dives on topics related to growth that people in the community want information on, such as willingness to pay for a specific public transportation project.
On the questions that we have been asking consistently year after year, we have seen some interesting changes, such as large increases in the share of people in the Treasure Valley who think that growth is occurring too fast. But, we have also seen a lot of stability in attitudes across Idaho, for example people are very consistent in the belief that education is the biggest challenge the state, and we keep seeing that year after year.
What kind of impacts have you seen from the survey work you have done?
We do a number of presentations of our survey work to elected officials at both the state and local level, and we definitely hope that the information we are providing is helpful for understanding what peoples’ policy preferences are. For the most part there is very little publicly available survey data in the state of Idaho, so decisionmakers are often relying on other ways of trying to know what the public thinks such as emails and phone calls from constituents or comments at public hearings. While these are very useful and important, the members of the public who engage in these activities are probably not representative of everyone. So, there have been a number of times where we have presented our results and we see eyebrows raise.
In terms of impact, it is hard to measure that exactly, but I definitely think that our survey work has changed a number of the conversations surrounding policy here in the Treasure Valley and in Idaho. We have seen our findings cited by legislators when introducing bills at the statehouse, have been part of community conversations about growth, and have received national news coverage from The Wall Street Journal, LA Times, and Forbes, among others. We have presented our survey results to just about every government entity in the area from the Nampa City Council to Governor Brad Little. So, the information is out there, and even if policymakers don’t act on the public preferences that we see in our survey research, I would argue there is a lot of value in it being one consideration in the discussion.
What do you like to do in your free time? Write more surveys?
I try to do other things in my free time than write more surveys! But, I do spend a lot of time reading other surveys and I’m always following national and state polling on policy and politics. Other than writing and reading about surveys, I spend most of my free time chasing my toddler around the house, and I try to go fishing when I can.