by Ilana Cohen, London @H2Oyeah
The Benefits and Limitations of a Water Point Monitoring Tool
An estimated 30-40% of water points across sub-Saharan Africa do not function (RWSN). People falsely counted as having access to safe water from these may revert to unsafe or far away sources. Water ministries, district governments, or the NGOs that installed them do not do know which ones are non-functional, or where the greatest needs are for repairs or new investments. Sadly, this sustainability problem is an old one, but new approaches supported by new ICT tools may be the agents of change. This blogpost looks into the experiences of three different organisations using Water for People’s innovative technology “FLOW.”
The challenge of water point monitoring:
The historical focus on new water infrastructure above maintenance has meant regular monitoring is often absent from ad hoc water projects, and local authorities or water ministries have not had the resources nor capacity to monitor. Effective monitoring is also complex. A single pump test one day does not indicate functionality in the dry season or even at the end of a day’s use, and whether the system is being effectively managed so that maintenance funds will be available. Conducting paper-based water point surveys requires transfer to digital records for sharing and updating data; the process is time consuming, error-prone, and hinges on correctly pairing paper surveys of each water point with GPS data.
The NGO, Water for People, is changing the game with a new attitude and a new tool. Their “Field Level Operations Watch” (FLOW) is a remote technology system for robust monitoring needed to ensure sustainable services. FLOW uses an android application to collect monitoring indicators with GPS information for each water point. Cloud computing sends data from the phone to a dashboard system that analyses service level and sustainability instantly, plotting the results on Google maps (see Figure 1). Where mobile networks don’t exist or are down, the application can still collect data offline that can later be downloaded for analysis. FLOW’s design enables indicator data to be customised, and collected digitally for ease, accuracy and rapid transfer.
Figure 1: FLOW outputs on Water for People’s Bolivia programmes
The promise of FLOW led Water for People to start using the beta version a year and half ago, and to share it with other key partners, including the World Bank’s Water and Sanitation Programme (WSP) and the IRC International Water and Sanitation Centre. Based on conversations with staff from each, the remainder of this post describes how these organisations are using the tool and synthesizes some of the key benefits, current limitations and how it is evolving.
Water for People has integrated FLOW in eleven of their country programmes in Latin America and Africa. The tool is key to their bold commitment to ensure their programmes lead to water and sanitation services that are for “Everyone, Forever.” By making FLOW’s monitoring results of their programmes publically available, this transparency strengthens their accountability to users and donors. Keri Kugler, from Water for People, explained they embed the tool for continued monitoring at the local level by training local partner organisations, or those responsible for operations and maintenance to operate FLOW. She clarified that unlike many prototypes from the Water Hackathon, FLOW is not currently designed as an incident reporting system (e.g. Ushahidi) so they don’t give FLOW-enabled phones to all users, though as android ownership increases, it may be made available to them.
IRC is testing FLOW in a similar approach for their “sustainable services at scale” (Triple-S) initiative in three districts in Ghana. They recently completed a baseline survey of water point functionality and sustainability, and shared their overall positive experience here. Similar to Water for People, the operation has engaged local water support staff, with the intent to eventually embed regular monitoring, as possible. IRC’s Nicolas Dickinson says they are considering expanding FLOW to other districts in Ghana and possibly other countries and programmes. He notes this version of FLOW still has challenges to scaling up, as it needs significant technical support to succeed (I’ll come back to the specific issues).
The World Bank WSP is using FLOW to conduct baseline surveys of water points Sierra Leone and has done the same in Liberia. In these contexts, previous civil unrest has left water infrastructure destroyed or in disrepair, so that an updated picture of infrastructure and service levels is urgently needed to inform resource distribution. WSP’s Maximilian Hirn discusses their use of FLOW in Liberia in this video. His team trained hundreds of enumerators to use FLOW to map and record functionality of more than 10,000 water points across the country. The findings of just over 60% functionality, and nearly 1 million citizens still without any access to improved resources, are shaping a new infrastructure investment plan.
Benefits of FLOW across these contexts include customisation, centralisation of data, reduction of data errors and survey time, and ease of use on the ground.
In Ghana, IRC customised FLOW with survey questions to assess sustainability of community-managed services based on country-specific water committee criteria. For example, they used indicators of committee governance, financial management, and operations as well indicators of external support programmes to assess whether the administration can support lasting services. Yet Keri from Water for People points out that for organisations who have not previously done this level of service monitoring, it can actually be a challenge to design their own meaningful indicators and questions to make FLOW a valuable tool.
FLOW is a major improvement from paper data handling. When used online, it immediately centralises data collected from multiple locations. FLOW instantly analyses survey data, enabling detection of data quality issues (e.g. survey triangulation can highlight errors to correct them while the surveyor is still onsite).
However, these benefits of real-time data collection were non-existent in WSP’s use of FLOW in Liberia and Sierra Leone due to limited mobile networks. Still, offline digital data collection with FLOW has been more efficient and effective than previous paper-based surveys with high error rates. The android application alone is far easier and reliable than paper reports because it automatically identifies the next appropriate question based on previous inputs, eliminating enumerator errors from accidentally skipping relevant questions. It also avoids logistical issues of re-printing and distributing revised surveys.
All three informants remarked how easy the android application is to use, with training requiring just two days. Nicolas of IRC notes that enumerators and interviewees are excited about using the technology. Maximilian has found that even older enumerators, less accustomed to phones, can quickly get the hang of it.
The limitations of FLOW currently include frequent software changes, human/institutional capacity issues, with a need to test it at scale.
Keri explains that Water for People consciously decided to deploy the beta version and face the challenge of constant bug fixes in order to immediately use and refine it in real contexts. Maximilian and Nicolas also attest to experiencing these frustrations. Nicolas notes that troubleshooting needed at this early stage requires some level of technical capacity for field operators.
Nicolas and Maximilian both identify current limitations in scaling-up FLOW beyond pilot districts and baseline surveys, for regular, nationalised monitoring. Maximilian explains that the android application cannot recall data entered previously about a specific water point to update the same file, but rather must create a new file for the same point. Though the data can be manipulated (to some extent) when downloaded, Maximilian highlights that this makes continuous monitoring with the tool problematic. Water for People is looking to address this, though they want to maintain a continuous timeline of data and not overwrite the last year’s data. Additionally, with the standard GPS margin of error, it is near impossible to tell if a point added very near last year’s point is an update of the existing one or a new one- unless a water point coding system is added.
Added to these technical issues, institutional capacity may be insufficient to support integrating the technology for continued monitoring, as is the case in Liberia and Sierra Leone. Maximilian fears that once the survey project is complete, water authorities will only be able to manually edit FLOW outputs converted to Excel worksheets, which diminishes the software’s long-term benefits. This puts into question whether a one-time investment in FLOW is worthwhile if it can’t realistically be used continuously. Despite lamenting other logistical challenges (e.g. frequent replacement of mobile batteries) some of which would arise with any large-scale survey using either paper or mobile (e.g. impassable roads), Maximilian sees the benefits as outweighing the drawbacks, yet noting the tool alone is “no magic solution for sustainability.”
Nicolas notes that the next test is to see how well FLOW can be scaled up for districts and national governments to use on their own. He suggests that an important balance for FLOW (and other ICT innovations in this sector) as it evolves, is to be generic and simple enough, but also interoperable with different systems and information needs.
These limitations mean FLOW isn’t currently usable by any NGO or water ministry currently, and though the demand is there, Water for People has not yet released it free of charge. The organisation has been focused on their own water and sanitation programmes and simply hasn’t had capacity to support widespread expansion for other organisations to use FLOW- but they’re building this through a new partnership with Akvo, and also hope to make the code-open source by April 2012.
As a side note, there are alternatives to FLOW. IRC considered the benefits of WaterAid’s Water Point Mapper tool, which is designed to be used offline, but felt customisation for their surveying would be more challenging. Maximilian notes that EpiSurveyor is a general surveying application for mobiles that may have some features that FLOW lacks, but for WSP has stuck with FLOW for its simplicity, and now their familiarity using it.
On the whole, early experiences with FLOW show that it is an important piece of the sustainability picture, addressing many issues with water point mapping and monitoring. However, its full suite of benefits can’t be attained without networks or sufficient institutional and human capacity to integrate regular monitoring, as well as some technical improvements. At the same time, Water for People is using FLOW to make big step toward accountability to support commitments to sustainability. Nicolas suggests ICT tools such as FLOW can be very valuable to improving water and sanitation, yet it is not always easy for WASH professionals to see this; it’s a matter of gradually testing how they can be applied in different contexts and at different levels.
Special thanks to:
Keri Kugler, Senior Manager of Programmatic Data, Water for People
Nicolas Dickinson, Programme Officer, IRC International Water and Sanitation Centre
Maximilian Hirn, Economist, Water and Sanitation Program, World Bank