Tracking and Learning From Program Data

Track­ing data and per­for­mance is vital to improv­ing pro­gram­ming for pub­lic agen­cies and non­prof­it human ser­vice orga­ni­za­tions. A new report from the Urban Insti­tute, fund­ed in part by the Annie E. Casey Foun­da­tion, pro­vides dozens of prac­ti­cal sug­ges­tions to help these orga­ni­za­tions improve how they com­pile and use data to get bet­ter results.

The report, Dos and Don’ts: Tips for Strength­en­ing Your Per­for­mance Man­age­ment Sys­tems, iden­ti­fies 63 dos and don’ts in five facets of per­for­mance man­age­ment: col­lect­ing and ana­lyz­ing per­for­mance data, using per­for­mance find­ings and pre­sent­ing and dis­sem­i­nat­ing data.

“With­out rel­e­vant, time­ly, and rea­son­ably accu­rate per­for­mance infor­ma­tion, pro­gram man­agers will be fly­ing blind,” the report’s authors note. ​“Per­for­mance mea­sure­ment includes iden­ti­fy­ing a program’s mis­sion goals or objec­tives, select­ing per­for­mance indi­ca­tors and iden­ti­fy­ing data col­lec­tion procedures.”

Sug­ges­tions for Man­ag­ing Data in Five Key Areas

“Hav­ing the right data — and the abil­i­ty to ana­lyze and talk about those data in real time — is crit­i­cal to help­ing orga­ni­za­tions under­stand and improve what they are doing,” says Ilene Berman, a senior asso­ciate with the Foundation’s Evi­dence-Based Prac­tice Group. ​“By engag­ing in reg­u­lar per­for­mance reflec­tion and improve­ment con­ver­sa­tions, orga­ni­za­tions can posi­tion effec­tive pro­grams for greater scale to improve out­comes for more young peo­ple and families.” 

High­lights of the report’s rec­om­men­da­tions include:

Col­lect­ing Data

• Have orga­ni­za­tion­al lead­ers sig­nal to staff that per­for­mance is a top priority. 
• Seek input on which mea­sures to track from a range of stake­hold­ers, includ­ing pro­gram participants.
• Obtain peri­od­ic feed­back from par­tic­i­pants about how well the ser­vice is delivered.

Ana­lyz­ing Data

• Iden­ti­fy one staff mem­ber to lead data track­ing and analysis.
• Com­pare pro­gram out­comes bro­ken out by demo­graph­ic char­ac­ter­is­tics, includ­ing race and eth­nic­i­ty, to learn how dif­fer­ent groups of par­tic­i­pants are expe­ri­enc­ing the program.
• Com­pare per­for­mance results over time to spot trends.
• Invite pro­gram par­tic­i­pants to sug­gest ser­vice improvements.

Using Per­for­mance Findings

• Use per­for­mance data inter­nal­ly to learn and improve.
• Don’t turn per­for­mance pre­sen­ta­tions into a ​“gotcha” exer­cise, which may lead staff to fudge data to avoid penalties.
• Rec­og­nize team mem­bers — through awards or mon­e­tary rewards — respon­si­ble for strong per­for­mance results.

Pre­sen­ta­tion

• Clear­ly define per­for­mance indi­ca­tors to avoid any con­fu­sion about what is being measured.
• Don’t hide bad news; instead, iden­ti­fy plans to cor­rect the problem. 
• Include sto­ries that explain the mean­ing and val­ue of the data being shared. 

Dis­sem­i­na­tion

• Devel­op reg­u­lar score­cards on pro­gram per­for­mance and share with staff and pub­lic officials.
• Ensure pro­gram man­agers have ready access to up-to-date per­for­mance data.

This report is a com­pan­ion to a pre­vi­ous Urban Insti­tute report, Eval­u­a­tion Guide for Pub­lic Ser­vice Pro­gram Man­agers, which pro­vides strate­gies for pub­lic and non­prof­it pro­gram man­agers to strength­en their deci­sion mak­ing using eval­u­a­tion information.