Evaluation is mandatory for many Drug and DWI Courts. Even when regular program evaluation isn’t required, in these times of severely curtailed funding, survival may depend on demonstration of good client outcomes. Moreover, evaluation is essential for continuing to improve program effectiveness.

Research also suggests that good outcomes may result when clients are continually monitored and services/supervision adapted in response to progress or setback. Major references. Shared management of diverted offenders by all stakeholders may also help achieve good outcomes.

Reliable data is essential and TRI-CEP™ is a product for court officials who need data. Whether to validly evaluate and justify continued operation - or promote team management and rapid response to client status – the data that TRI-CEP collects and arranges in secure, simple-to-read reports will help judges regardless of what they need most.

Early TRI-CEP Development: Evaluation and the Missing Data Problem
Beyond Evaluation: Continuous Client Monitoring and Rapid Response
TRI-CEP: Dual-Purpose Tool for Program Evaluation and Client Management

Early TRI-CEP Development: Evaluation and the Missing Data Problem

Development of TRI-CEP began in 2005, when Douglas Marlowe, J.D., Ph.D. and David Festinger, Ph.D. from TRI’s Section on Law & Ethics set out to design an electronic evaluation system that simplified collection and analysis of the essential data elements courts needed to conduct reliable and valid program evaluations. The elements included those identified through their own drug court research, and other elements recommended by an expert panel convened by the National Drug Court Institute (NDCI) (Dr. Marlowe serving as one of the panel experts.)

The sad truth was (and still is) that valid evaluations of Drug and DWI Courts have been continuously frustrated by lack of data – missing but essential information without which informed and defensible self-assessment – and from that, program improvement - is not possible. When this data is collected, rarely does it account for missed sessions, hearings, or drug tests, or the dates these were supposed to have occurred or entered in a format that permits accurate statistical analysis. In most cases, evaluators must extract the data from written records or spreadsheets with few available options for reconciling inconsistencies or accounting for missing entries.

With more diverted offenders causing larger case loads, courts must be efficient. They need systems that are easy to operate. Hence, TRI-CEP was designed to collect only the data elements essential for conducting rigorous, meaningful evaluations. In addition to essential demographic data, (e.g. age of client, primary drugs of choice), other essential evaluation elements include results of biological testing, attendance at treatment sessions or status hearings with the judge, and criminal infractions committed while in the Drug/DWI Court program.

Beyond Evaluation: Continuous Client Monitoring and Rapid Response

Even though their initial intent was to facilitate state-of-the-art program evaluations, Marlowe and Festinger reasoned that the data to be collected/reported in TRI-CEP were some of the same elements that could facilitate rapid adjustment by the court and other stakeholders corresponding to a change in client status. For example, by collecting and reporting information on biological testing results or treatment progress (two of the elements essential to program evaluation), TRI-CEP could also generate alerts to judges and other stakeholders when a client was progressing well or suffering setbacks – along with recommended changes to court supervision schedules – rewards or sanctions - or to treatment plans.

These principles of “continuous monitoring” and “adaptive response” are not new to Drug or DWI Courts. Since 2002, research has suggested that graduated (judicially imposed) sanctions or rewards applied rapidly and in conjunction with client behavioral successes or infractions are associated with better outcomes on drug use and criminal recidivism. Clinically, the same principles are increasingly being applied as treatment systems embrace a chronic disease model, where substance abuse patients are continually monitored and the level and intensity of treatment services adjusted based on documented client progress and/or setbacks.

With the addition of empirically derived algorithms, Marlowe and Festinger reasoned that TRI-CEP could also be a decision support tool – one that alerts judges, treatment providers, case managers, and probation/parole officials when the treatment or supervision plan should be adapted, and even what the adaptation should be.

TRI-CEP: A Dual-Purpose Tool Supporting Program Evaluation and Client Management

TRI-CEP thus became a dual-purpose system, one that could help court officials evaluate and foster rapid adaptation based on continuous monitoring. For TRI-CEP to serve in either of these roles and be maximally useful, it had to be:

• Research based: incorporate research findings as to the operative elements of the Drug/DWI Court model and principles of effective management of drug-involved offenders.
• Useful: meet the needs of judges who must 1) be capable of evaluating their own programs; 2) respond quickly and effectively when a client’s situation warrants it; and 3) share management of clients with other stakeholders.
• Versatile: be both an evaluation and decision support tool.
• User friendly: employ visual and navigational techniques proven successful in other sectors, such as the on-line retail industry where the goal is to capture essential information from all customers, even those in a hurry or technically naïve.
• Secure: incorporate industry-standard encryption with adjustable levels of access for different types of users.

Program Evaluation: TRI-CEP was developed to be a web-based evaluation tool specifically crafted to capture essential data elements recommended by the NDCI expert panel. All programmatic data related to such variables as biological testing results, counseling and hearing attendance, infractions and accomplishments, are entered directly into TRI-CEP by the court, treatment counselors, case managers, probation or parole officers, and lab technicians.

All transmissions are protected using industry-standard 128 bit SSL encryption, and levels of access can be regulated for different types of users. For example, the judge might be entitled to view all data-entry screens whereas a probation officer might be limited to viewing only the probation and treatment screens.

TRI-CEP routinely prompts users for missing data. It is also arranged just like one’s own appointment calendar and automatically date-stamps all entries. Following the principle of “less is more,” users only see those data-entry screens that are directly relevant to their jobs and do not need to wade through multiple screens or columns of information. The entry screens look just like the types of web pages encountered when purchasing items or conducting Internet searches. There are no spreadsheets to look at and no lines of data. Instead, simple-to-use windows ask for information in a straightforward and intuitively simple manner. It generally takes less than two minutes to enter all of the required weekly data elements for any one client. What is not immediately apparent to the front-end user is that the data elements are stored in an analyzable format permitting immediate and real-time statistical reports to be generated for various needs. The data can also be presented in graphic or tabular form to assist in interpretation of the results.

TRI-CEP also monitors data entry, tracking whether data elements are entered in a timely manner or shown as “pending” for unduly long periods of time. This promotes accountability by permitting the judge and/or other managerial staff to address problems of poor data-entry in real time rather than months after the fact when it is simply too late to retrace steps and figure out where the problem exists.

Client Monitoring and Management: TRI-CEP was also designed to automatically calculate when a participant is non-compliant (triggering increased supervision) or non-responsive (triggering increases or changes in clinical services), and transmit an alert to everyone granted appropriate access. The alert indicates the appropriate course of action that should be taken based upon an adaptive algorithm built into the system. The algorithms (e.g., how many missed sessions should trigger a response and what the response should be) are set based on local preference, ensuring that judges and other staff members are aware when adaptations are called for and what the appropriate response should be.

TRI-CEP also captures information on when new conditions are applied (e.g., requirements to attend more frequent court hearings or a case management program). This allows for a precise measurement of the proportion of adaptations that were implemented as intended, as well as the time delay between the determination of non-compliance or non-responsiveness and the ordering of the response.

According to TRI’s adaptive clinical model, lack of compliance – or failure to respond appropriately to circumstances typically under clients’ control - should trigger an increase in criminal justice monitoring or supervision. Failure to respond to treatment should trigger a clinical response through increased level or intensity of treatment services. In line with this model, TRI-CEP is designed to allow jurisdictions set specific criteria for both supervision compliance and treatment response. Moreover, for the alerts to be actionable by the stakeholders, as well as for security reasons, they are visible to judges and court personnel, while treatment-related alerts are also visible to counselors and case managers.

Finally, to convey fairness and rational decision-making, graphs and tables of client-level data generated by TRI-CEP are designed to be shared with and easily understood by clients if desired by the local jurisdiction. Some courts may even choose to display graphs in the courtroom during clients’ regular status hearings. This may allow for more informed interactions between the judge and client, and/or help the client, his or her significant others in attendance, and other clients awaiting their hearing, better understand why certain judicial responses were called for.