Results from introduction of dashboards
Demand for technical support to introduce dashboards continues
GMS’s fourth project year saw an increase in demand for the PR Management Dashboard and the CCM Summary, with almost half of technical support assignments relating to performance improvement of Global Fund grants through introduction of a management change process using PR management dashboards.
Twice the number of countries (12) requested assistance with introduction of dashboards as did the previous year. Of these, all but two requests were for the whole-of-country approach, a standardized approach to introducing a country to both dashboards–the PR Management Dashboard and the CCM Summary.
As the culminating event in the process to introduce the PR Management Dashboard, GMS conducted a qualitative impact evaluation of the PR dashboard pilot. A report of the evaluation report is scheduled for release in 2017.
In 2016, GMS developed two additional tools–a management tool for subrecipients and a dashboard for regional grants. Success stories about these new tools provide additonal infomration about them.
Executive Summary Recaps Qualitative Evaluation of 2014 Pilot of Principal Recipient Management Dashboard
An executive summary of the report on the 2014 pilot of the Principal Recipient Management Dashboard is now available to read and download. On March 15, 2017, GMS reported to USAID on a qualitative evaluation of the PR Management Dashboard’s impact on grant performance.
After piloting the dashboard in 2014 in six countries with seven local grant implementers in Africa, Asia and Latin America, GMS assessed institutionalization of the PR Management Dashboard and management improvement among the implementers, known in Global Fund parlance as principal recipients, or PRs. The evaluation ran from November 2015 to February 2016, aiming to collect answers to three questions: What changes in management took place at the PRs? What factors fostered or inhibited use of the dashboard? Did changes in management lead to improved grant performance?
This evaluation found that PRs varied in the degree to which they adopted the dashboards, with some using them for more limited purposes than originally postulated. PRs that fully institutionalized the dashboard showed improved grant ratings. Partial adoption, such as for internal management only, while sufficient for those PRs’ immediate needs, led to more attenuated results. Different degrees of adoption were influenced in part by varying initial motivation for using the dashboard as well as certain managerial and attitudinal factors.
These results suggest that introduction of the PR Management Dashboard through the whole-of-country approach for all PRs and the CCM, rather than with individual PRs, has more potential for risk management and performance impact on grants financed by the Global Fund.
Whole-of-country approach enabling activities to expand technical support capacity for dashboard introduction
GMS has standardized the introduction of dashboards in the whole-of-country approach wherein all principal recipients (PRs) in a country obtain management dashboards (PR Management Dashboards) and the country coordinating mechanism (CCM) receives a summary dashboard (CCM Summary) to support oversight. The steps include assessing capacities of all PRs; completing indicator selection through workshops with PRs and subrecipients (SRs); producing first dashboards; and coaching PR staff on analysis and use of data. For each PR, GMS trains a dashboard focal point, who is responsible for producing future dashboards for the country. GMS then trains a CCM dashboard focal point to produce the CCM Summary, using an application that imports and displays data from PR dashboards by disease.
The number of technical-support teams and number of hours required to introduce the PR Management Dashboard and CCM Summary vary depending on number of grants, capacity of PRs and capacity of the CCM oversight committee.
Expansion of pool of consultants capable of introducing dashboards
A high priority during for GMS during its latest project year was to expand the pool of technical-support providers willing to fund training of consultants in dashboard introduction and technical-support assignments as well. Other donors joined the U.S. government in financing training of consultants in dashboard introduction.
GIZ’s BACKUP Initiative was the first technical-support provider to join the U.S. government in financing training of 23 consultants in whole-of-country introduction of dashboards, in October 2015 in Brighton, England.
GMS subsequently delivered the training twice more with cofinancing from GIZ: for 15 consultants in Cape Town, South Africa, in March 2016; and for 25 francophone consultants in Casablanca, Morocco, in September 2016, with additional support from France Expertise.
As of November 2016, France Expertise has committed to fund a dashboard assignment in Congo, Brazzaville, while BACKUP Initiative is providing complementary funding for a coordinated approach to introduction of dashboard in Liberia and Malawi, and for an independent team in Togo.
Success of PR Management Dashboard drives demand for development of additional management tools
During GMS’s latest project year (October 2015–September 2016), demand for new types of dashboards mushroomed, leading GMS to develop two additional tools for regional grants and subrecipients (SRs)—the Regional Grant Dashboard and the Subrecipient (SR) Management Tool. Development of these new tools will result in a suite of four dashboards available to PRs and CCMs by the end of the GMS project.
Success stories and news
A first regional dashboard: tracking services in 10 countries for the grant "TB in the Mining Sector" (TIMS)
During 2016, GMS consultants and staff collaborated with managers at principal recipient (PR) Wits Health Consortium (WHC) to create and deliver a first-ever regional PR management dashboard that allows the PR and its subrecipients track data across 6 tuberculosis service packages in 10 countries. In the process, WHC refined the tools and systems its program staff and partners use to report on dashboard indicators, thereby strengthening its ability to meet reporting requirements of the Global Fund to Fight AIDS, Tuberculosis and Malaria.
A unique tool and the first of its kind, the Regional Dashboard displays country and implementing entity performance. It allows regional grants and their unique, multicountry structures to communicate and report effectively to the various individual countries. A bonus: the regional coordinating mechanism that governs the TIMS grant has adopted it to oversee the grant.
Update: Evaluation of pilot of the Principal Recipient Management Dashboard
Analysis of data collected in the course of the evaluation of the pilot of the PR Management Dashboard concluded that 15 months following the end of the pilot, 3 of the 7 PRs in the pilot had fully institutionalized management changes.
Management changes included adopting regular production and analysis of dashboards, addressing grant-performance problems using data produced by the dashboard, and showing improvement or stabilization of performance ratings. The remaining 3 pilot participants had partially institutionalized the management-change process.
Evaluation data show that performance improvement is achieved by purposeful problem-solving efforts targeted through detailed issue analysis, using successive dashboards to measure reactivity in priority indicators. PRs and subrecipients (SRs) were observed to have improved data quality over successive dashboards.
Greater accountability and dialogue between and among PRs and SRs accelerates the process: active support by senior management is needed. Public-sector PRs may be constrained by limited authority to make management and structural adjustments specific to grant performance.
Burkina Faso: Dashboards for principal recipients and the country coordinating mechanism
Overcoming the unrest of 2015, Burkina Faso’s country coordinating mechanism (CCM) and principal recipients (PRs) pursued a rigorous program of management and governance strengthening, and to this end obtained GMS support to carry out several steps of the Global Fund to Fight AIDS, Tuberculosis and Malaria’s funding model. In one assignment a GMS team installed the PR Management Dashboard for the PRs and the CCM Summary, a dashboard for CCMs. The request was intended to address startup delays and implementation and performance issues. Two GMS teams trained the PRs and the CCM’s oversight committee how to install, analyze and use the 5 resulting dashboards for decision making to improve performance. By the end of September 2016, each of the PRs had produced 3 dashboards.
A review by GMS showed that the PRs are using the tool and making progress toward performance targets. The absorption of funds has steadily improved. Burkina Faso demonstrates that while country context can adversely impact grant performance, with the right tools and practices PRs can accelerate funds absorption and regain ground lost in delayed start-up.