Initial Evaluation of the Addiction Studies Program for State Legislators

Introduction

While there are many federal efforts designed to prevent illicit drugs from entering this country and to reduce the illegal distribution and use of these drugs, most of the issues associated with this national problem are felt and dealt with by states and cities on a local level. For example, most of the criminal justice activity associated with prosecution, sentencing and monitoring of drug-related offenders is handled by state and local authorities. Similarly, the school-based prevention efforts and the great majority of publicly funded treatment efforts designed to reduce drug use are financed, regulated and administered with state funds. In turn, the policies governing these state-level activities associated with drug use reduction are under the province of state legislators. At the state level, those who serve in legislatures are usually business people, teachers and farmers whose main job is not government and who discharge their responsibilities as part-time employees. These citizen-legislators typically learn about the important issues facing state government through personal experience, discussing the issues with their staff and peers and through on the job training.

With regard to the many issues of drug use, abuse and addiction, there is a large and growing science base of information. Impartial, factual, science-based information is available to inform those who are charged with setting policies about the most effective prevention and treatment strategies, the relative effectiveness of different sentencing, probation and parole practices with drug-related offenders, prevention of relapse and many other health and social issues affected by drug abuse. Yet this science based information is often quite technical, not in a form that is easy to understand or pertinent to the issues faced by lawmakers in states.

With support from the National Institute on Drug Abuse, David Friedman from Wake Forest School of Medicine and Sue Rusche from National Families in Action, designed and developed the Addiction Studies Program for State Legislatures, a two-day seminar designed to educate and inform state legislators and their staffs about the research that is relevant to the many health and social issues associated with drug use and abuse. The goals of these sessions were not only to inform these state lawmakers, but also to introduce scientific information and research evidence into the laws, regulations and budgets that affect state drug abuse issues. Thus, participation was restricted to state legislators or their aides and travel fellowships were provided from the NIDA support. The support and resources supplied by Allison Colker and the National Council of State Legislatures assured appropriate mailing lists to invite participants, as well as publicity for the program and early information to assure those interested that there was no political or “industry” agenda associated with the program. The Treatment Research Institute at the University of Pennsylvania is a fourth partner in sponsoring the Addiction Studies Program.

The two-day sessions consisted of one-hour presentations from some of the most respected physicians and scientists working on issues related to drug abuse. Presentations included hand-outs and visual materials that were made available to the audiences. Speakers were chosen not only for their scientific expertise and their impartiality, but also for their ability to communicate and interact with the legislators. Each presentation was made in a conversational style and substantial time was set aside during each day for the attendees and speakers to interact and discuss specific issues and cases.

With the goals of the program in mind we present the initial findings from an evaluation of all 24 legislators, still in office, and staff members who participated in the first two sessions of this program, in September of 2005. This initial evaluation was designed by one of the participants in the program (McLellan) but the data collection was conducted in April, 2006 – six months following the two-day seminars - by a trained and experienced research assistant who had not been part of the program. There were two parts to the data collection – consistent with the two purposes of the program. An initial set of questions asked all participants their opinions about the relevance, quality and utility of the presentations they heard. Did the information come across in a manner that was understandable and relevant to the attendees? The second set of questions asked the participants about the impact of the information to that point and about the likelihood that the research they heard would have a future impact on the legislation, regulation and budgets associated with drug abuse issues.

Methods

The questionnaire consisted of eleven questions in two overall areas, satisfaction and impact on your work. The first three questions asked participants to rate their satisfaction with the quality of the presentations, quality of the information, and relevance of the information to their work on a ten point scale with one being poor and ten being excellent. Each participant was directed to select only one rating. The final eight questions regarding impact on your work were split into two broad questions and asked participants to indicate if the information they heard resulted in or will lead to any new or modified: 1) legislation pertinent to alcohol and drug issues; 2) regulations; 3) reporting requirements; and 4) budget changes. Participants were asked to check yes or no for each category. An area for comments was available after each question.

The evaluation form was emailed to each of the participants based on a list of attendees provided by the Addiction Studies Program for Legislatures. Each participant was asked to take a few minutes to complete the attached evaluation form and return the completed form via email or fax. There were a total of 27 legislators or their aides on the list of attendees. All 27 were contacted and asked to complete the evaluation. If the attendee did not respond within a week to the initial email request, they were contacted via phone to request the completed evaluation. A few of the attendees received a combination of phone calls, emails, and faxes to request their completed evaluation. Out of the 27 attendees on the list, 24 returned completed evaluations. Two of the remaining attendees responded that they signed up for the seminars, but were unable to attend and were removed from the list of possible attendees. The last legislator on the attendance list was no longer in the state legislature when contacted for the evaluation. The final total of completed evaluations was 24 out of 25 attendees for both two-day seminars.

Summary of Results

Nine presentations on various scientific issues in addiction were given to each of the two sets of legislators and aides in the September 14 & 15 and the September 16 & 17 sessions. The presentation topics were the same for both sessions and seven of the nine presenters presented to both sets of legislators. When we examined the results from the attendees at the two sessions we saw no significant differences in the patterns of answers between the two sessions. For these reasons, we combined the answers from the two sets of attendees and discuss those combined results in the text that follows.

Questions 1 – 3 – Ten-point ratings of the quality and relevance of the information provided, and the quality of the presentations.

Twenty four of 25 attendees answered each question. Average ratings for the questions ranged between 8.5 and 9.1 on a ten point scale. The lowest rating given was a 7 and that was only given by three attendees to one question. These ratings suggest broad satisfaction with the content and relevance of the information provided in these sessions and with the understandability of the presenters. Please see Table 1 for the individual answers and comments on these questions.

Questions 4 – 7 – Yes/No answers to whether the information presented “has resulted in:”

A – new legislation pertinent to alcohol or drug abuse issues
B – new regulations or quality standards regarding some aspect of drug abuse
C – new reporting or monitoring requirements for the state substance abuse agency
D – new budget changes for the state substance abuse agency.

A – 24 of the 25 participants answered this question: 14 (58%) indicated that the information had already influenced or led to some new legislation.

B – 24 of the 25 participants answered; 6 (25%) indicated that the information had already led to a new or modified state regulation pertaining to substance abuse.

C - 24 of the 25 participants answered this question: 8 (33%) indicated that the information had already influenced or led to new reporting requirements for the state substance abuse agency.

D - 24 of the 25 participants answered this question: 8 (33%) indicated that the information had already influenced or led to a budget change for the state substance abuse agency.

Please see Table 2 for the individual answers and comments on these questions.

Since the evaluation interviews occurred only six months after the two legislator sessions had been presented, it was quite optimistic to think that the information provided would have led to changes of this nature in this relatively short time frame. For this reason, we also asked each attendee whether the information they had received would lead to these same tangible changes in the future.

Questions 8 – 11 – Yes/No answers to whether the information presented “would lead in the future to:”

A – new legislation pertinent to alcohol or drug abuse issues
B – new regulations or quality standards regarding some aspect of drug abuse
C – new reporting or monitoring requirements for the state substance abuse agency
D – new budget changes for the state substance abuse agency.

A – 24 of the 25 participants answered this question; 21 of those (88%) indicated that the information would influence or lead to some new legislation.

B – 24 of the 25 participants answered; 14 of those (58%) indicated that the information would influence or lead to a new or modified state regulation pertaining to substance abuse.

C - 24 of the 25 participants answered; 14 of those (58%) indicated that the information would influence or lead to new reporting requirements for the state substance abuse agency.

D - 24 of the 25 participants answered this question: 20 of those (83%) indicated that the information would influence or lead to a budget change for the state substance abuse agency.

Please see Table 2 for the individual answers and comments on these questions.

Discussion

The goals of the Addiction Studies Program for State Legislatures are quite ambitious: present attractive, understandable research information that would be understandable to and used by state legislators and staff in designing legislation, regulations and budgets to deal with addiction-related health and social issues. The evaluation results to this point are still quite preliminary (24 participants from 14 states) and likely reflect the comments of those legislators who were most involved even prior to their participation in the program. It will thus, be important to continue this program and the evaluation to determine whether it will be attractive to and used by a broader segment of the intended legislator audience.

Despite this caveat, the results were uniformly positive with respect to the perceived attractiveness and relevance of the information – suggesting that the format used to this point, the topics and the types of speakers were well appreciated. One recurring comment in this regard was the request for additional research findings with regard to economics and financing issues, as these are most pertinent to the work of legislators. This appears to be an easy accommodation. Beyond the perceived attractiveness of the information, it was heartening and a bit surprising that so many of those polled suggested that what they learned had already led to modifications of legislation, reporting and budget – and that virtually all participants suggested that the information from the program would lead to those types of changes. Again, it is possible and even likely that these attendees may be those who had already been interested in drug abuse issues. Even if this accounts for some of the rapid, positive impact the comments of the attendees suggest that what they heard at the sessions informed the nature of the actions they ultimately took. Again, it will be important to continue to hold the scheduled meetings and to repeat this evaluation to see if these early favorable trends continue.