CSAT Publications -- TAP 25, The Impact of Substance Abuse Treatment on Employment Outcomes Among AFDC Clients in Washington State

This study had two purposes. The primary purpose was to assess the effect of substance abuse treatment on employment and earnings among AFDC clients admitted to treatment in Washington State. A second purpose was to illustrate the potential utility of computerized State administrative databases for tracking client outcomes and conducting timely evaluation research.

a. The Effects of Substance Abuse Treatment on AFDC Clients' Employment Outcomes

b. The Utility of Computerized State Administrative Databases for Tracking Client Outcomes and Performing Evaluation Research

A. The Effects of Substance Abuse Treatment on AFDC Clients' Employment Outcomes

This study's finding that substance abuse treatment was associated with increased employment and earnings among AFDC clients during a 2-year post-treatment period is notable, especially in light of the fact that the primary goal of substance abuse treatment is rehabilitation, not employment. This finding was robust across the three treatment modalities examined: outpatient treatment, intensive inpatient (21- or 28-day non-hospital) treatment, and methadone maintenance. AFDC clients in these treatment groups were more likely than their counterparts in comparison groups to become employed-whether employment was measured in terms of any earnings whatsoever or quarterly earnings above $1,500. Among the subset of clients who were employed, those in treatment groups had aggregate income that exceeded by $1,700 to $3,200 (30 to 60 percent) aggregate income for clients in comparison groups.  

Time on welfare proved to have a substantial effect on AFDC clients' earnings prospects. AFDC clients who had been on welfare for 4 to 6 quarters of the 6-quarter pretreatment period had lower aggregate earnings in the 8-quarter post-treatment period than clients who had been on welfare for only 1 to 3 quarters. Regardless of the amount of time AFDC clients had spent on welfare, however, outpatient substance abuse treatment was associated with a significant increase (p < .05) in earnings. Among AFDC clients who had been on welfare for 4 to 6 quarters in the pretreatment period, those who received outpatient substance abuse treatment had aggregate earnings of $3,993 in the 8 quarters following treatment; their counterparts in the outpatient comparison group earned only $1,615 in the same period. Among AFDC clients who had been on welfare for 1 to 3 quarters in the pretreatment period, those who received outpatient substance abuse treatment had aggregate earnings of $6,192 in the 8 quarters following treatment; their counterparts in the outpatient comparison group earned only $3,814 in the same period.

Because this study did not randomly assign subjects to treatment and comparison groups, an important question is whether there is selection bias. Random assignment of subjects is alwayshighly desirable, but is not possible for studies that rely on secondary analysis of data obtained from administrative databases. Thus, appropriate caution must be used in the interpretation of the findings from this study (and similar studies).

The validity of a study such as this rests largely on the extent to which statistical analysis is able to control for factors that may affect outcomes and treatment classification. Although the study was able to control for a large number of factors-including differences in AFDC clients' pretreatment earnings-unmeasured selection effects could still have influenced the results in some unknown way.

The study's ability to control for differences in AFDC clients' pretreatment earnings, though not eliminating possible selection problems, does provide some measure of reassurance. The measure of AFDC clients' (pretreatment) baseline earnings was consistently the most important independent predictor of post-treatment earnings. Regression models indicated that for every $1 of (pretreatment) baseline income, AFDC clients in this study earned approximately $2 to $4 in post-treatment income, other things being equal. In short, the AFDC clients who worked more before receiving treatment for substance abuse tended to work more after receiving such treatment.

The study's findings regarding the positive effects of substance abuse treatment on AFDC clients' employment and earnings need to be balanced with the observation that earnings were relatively low among the AFDC clients in the sample-too low for the clients to achieve self-sufficiency. Among the 5,664 AFDC clients in the study population, 42 percent had no earned income in the 8-quarter post-treatment period, and an additional 14 percent had aggregate earned income in that period of under $1,000.

TANF has introduced strong incentives to work. Thus, AFDC clients today may be earning somewhat more than the data presented in this report suggest. But as recently as the third quarter of 1998-1 year after TANF's implementation-the level of quarterly earnings among the Washington State AFDC clients in the study population was very modest (see Figure III-15), despite the incentives for employment.

Among individuals with substance abuse problems, the goals of employment and self-sufficiency underpinning TANF may be achievable for only a small minority unless ancillary vocational services are provided in conjunction with substance abuse treatment. Earlier studies from Washington State indicate that vocational services enhance employment and earnings when delivered in conjunction with treatment.²⁰ Unless AFDC clients treated for substance abuse problems receive vocational services, there is little likelihood that many of them will be able to find jobs and become economically self-sufficient. There is a very real possibility that AFDC clients who are unable to obtain employment may be forced to rely on other local and State health and welfare systems once their time allotment for financial assistance under TANF is exhausted. If so, the costs of operating these health and welfare systems will certainly increase.

B. The Utility of Computerized State Administrative Databases for Tracking Client Outcomes and Performing Evaluation Research

State substance abuse agencies are under increasing pressure to demonstrate accountability for their budgets and must plan and administer substance abuse programs in the face of rapidly changing environments. These needs require the capability to perform timely analyses and outcome evaluations at a reasonable cost.

State computer databases contain client-level data representing important outcome indicators of substance abuse treatment-including employment, public assistance payments, Medicaid utilization and expenditures, and criminal justice information. This study in Washington State illustrates the potential utility of State administrative databases in tracking clients' outcomes and conducting timely evaluation research.

The experience of Washington State in conducting outcome studies demonstrates the important advantages of establishing a database infrastructure that can provide data in a timely and efficient manner for evaluation and research purposes. The advantages of using State administrative databases include the following:

• Clients can be tracked over extended periods of time.
• Baseline (pretreatment) data can be incorporated into the analysis, reducing problems of selection bias.
• Large numbers of clients can be studied at little extra cost.
• Problems of client recall and case attrition are reduced.

From the standpoint of performing evaluations, however, State databases do have inherent limitations. Since the information contained in computer databases is collected for administrative purposes, the desired measures may not be available in the preferred form. The types of information gathered and stored in State databases is subject to change, and changes may create problems for longitudinal analyses. Finally, information contained in State administrative databases may not be updated frequently enough to make the information as current as might be desired.

A number of useful lessons emerged from this study that merit brief discussion:

• First, it is important that a person familiar with the different State databases either extract the data or serve as a communication link with others who will perform this function. Many detailed questions arise regarding the definition of variables and the organization and completeness of the information contained in databases that an outside researcher cannot anticipate. Addressing these questions early-before data extract files are created-helps to avoid subsequent analytical problems.
• Second, to the extent possible, data should be collected that will allow the triangulation of measures. As part of this study, detailed treatment activity data were collected and used to verify discharge-date information obtained from theclient discharge records. This proved very useful in improving the reliability of the discharge information.
• Third, it is generally useful to obtain data on more cases than research design considerations (e.g., ensuring statistical power and external validity) may dictate. Inevitably some cases will have to be dropped because of incomplete or unreliable data. Further, the need for subanalyses that were not initially planned may emerge, which may require additional cases.

The increasing complexity of the policy and program environment of substance abuse treatment is placing more emphasis on timely evaluation and outcomes research. By utilizing information available in computer databases, States can greatly improve their capacity to perform timely evaluation and outcome studies.

For further information, contact the CSAT staff at 240-276-2750.