Applying a Results Framework to Social Program Evaluation in San Antonio

Posted July 15, 2019, By the Annie E. Casey Foundation

Researchers and evaluators are learning new tools to improve practices.

With today’s increas­ing acces­si­bil­i­ty to data, met­rics focused on desired results can dri­ve any pro­gram — and steer pow­er­ful com­mu­ni­ty ini­tia­tives. But many com­mu­ni­ties still rely on tra­di­tion­al meth­ods to eval­u­ate their pro­grams, often result­ing in lim­it­ed suc­cess­es in indi­vid­ual settings.

Researchers at the Uni­ver­si­ty of Texas San Anto­nio (UTSA) are using their work with the Casey Foundation’s Expand­ing the Bench in Per­for­mance Man­age­ment Ini­tia­tive to apply results-based meth­ods to eval­u­at­ing a vari­ety of efforts in San Anto­nio. Using the Results-Based Account­abil­i­ty (RBA) frame­work, the researchers have estab­lished met­rics for eval­u­a­tion tied to spe­cif­ic indi­ca­tors, engag­ing experts and orga­ni­za­tions across com­mu­ni­ty sec­tors to come up with those met­rics together.

Expand­ing the Bench in Per­for­mance Man­age­ment focus­es on bring­ing RBA to human ser­vices pro­fes­sion­als of col­or and to schools that serve larg­er num­bers of stu­dents of col­or. Unlike tra­di­tion­al frame­works, RBA starts with an end goal, then works back­ward to select the most effi­cient, effec­tive pro­gram meth­ods. In the process, it engages a wider range of com­mu­ni­ty orga­ni­za­tions to pro­duce strong solu­tions and make a greater difference.

The chal­lenge? Apply­ing the frame­work to eval­u­a­tion and train­ing eval­u­a­tors to use it.

Sofia Bahena, prin­ci­pal inves­ti­ga­tor, and Eri­ca Sosa, direc­tor of UTSA’s Cen­ter for Com­mu­ni­ty-Based and Applied Health Research, are work­ing to dis­sem­i­nate RBA through­out UTSA and the San Anto­nio com­mu­ni­ty. Bahena leads an RBA train­ing pro­gram, now in its sec­ond year, to train fac­ul­ty mem­bers from under­rep­re­sent­ed racial groups from five dif­fer­ent depart­ments and two col­leges. Bahena has also start­ed a writ­ing group among fac­ul­ty mem­bers to increase the num­ber of RBA-relat­ed arti­cles in aca­d­e­m­ic jour­nals and fur­ther dis­sem­i­nate the RBA frame­work. Over time, the fac­ul­ty mem­bers will be able to train stu­dents in RBA across mul­ti­ple disciplines.

One of the trained fac­ul­ty mem­bers, Sosa, is using her new skills to improve the lives of chil­dren and fam­i­lies in San Anto­nio through col­lab­o­ra­tive, inter­dis­ci­pli­nary health research. Sosa’s cur­rent project focus­es on improv­ing health indi­ca­tors, includ­ing rates of dia­betes and obe­si­ty, across Bexar Coun­ty in Texas. Under the RBA frame­work, Sosa has recruit­ed 15 part­ners from diverse sec­tors across the coun­ty to iden­ti­fy pri­or­i­ty health issues, cre­ate a plan to devel­op data and then set up strate­gies to address the health issues. The Cen­ter, mean­while, is putting the plans into action by work­ing on col­lab­o­ra­tive grants to fund local ini­tia­tives aimed at increas­ing health equi­ty and elim­i­nat­ing poor health outcomes.

The San Anto­nio research part­ners are show­ing that putting results first strength­ens eval­u­a­tion. With more per­spec­tives and infor­ma­tion, we believe find­ings will be more rel­e­vant and action­able in the end,” says Mar­i­an Amoa, a senior asso­ciate at the Casey Foundation.

It’s a dif­fer­ent way of work­ing, accord­ing to the researchers. Typ­i­cal­ly, eval­u­a­tors are the experts’ and guide com­mu­ni­ties in set­ting up eval­u­a­tion plans and col­lect­ing rel­e­vant data. But using RBA as a guide, Sosa is tap­ping into local exper­tise by work­ing with a local health depart­ment, school dis­tricts, a health col­lab­o­ra­tive, nine UTSA aca­d­e­m­ic depart­ments and sev­er­al agencies.

The RBA frame­work engages com­mu­ni­ty orga­ni­za­tions through­out all lev­els of plan­ning and action,” explains Sosa. It’s an inclu­sive, com­pre­hen­sive approach that brings every­one togeth­er and elim­i­nates jar­gon so that everyone…understands the lan­guage and shares equal pow­er, not just the experts.’”

It’s excit­ing work,” she adds. With RBA, we’ll go from plan­ning to action very quickly.”

Learn more about Expand­ing the Bench in Per­for­mance Management

Popular Posts

View all blog posts   |   Browse Topics