FORECASTING MODULE
FORECAST AND WARNING OUTPUTS
DATA-GATHERING MODULE
Telemetry Meteorological
forecasts Hydrometric
database Weather radar Satellite
imagery
OPERATOR INTERFACE AND INFORMATION CONTROL
Snowmelt
model Rainfall-runoff
model Flow-routing
model Hydrodynamic
model
FORECAST DATABASE AND INFORMATION-SYSTEM MODULES
Although the open architecture system has the capacity to use models of different types and from different origins (legacy systems), the major suppli- ers will provide their own range of compatible and complementary models. Typically these will include:
(a) A hydrological-cycle (rainfall–runoff) model;
(b) A river-channel hydraulic model;
(c) A river-channel and flood-plain model;
(d) An automatic alert-generating model.
As well as the comprehensive nature of the data management and range of model types, the sophis- ticated operating systems provide a wide range of display modules and graphics, which form the graphic user interface (GUI). All contain systems for graphical, tabular and map-based summaries.
5.4 OPERATION AND MAINTENANCE OF HYDROLOGICAL NETWORKS
This is a highly critical aspect of flood manage- ment and the operations of a flood forecasting and warning system. It covers a wide range of equip- ment, mobility and the capacity to maintain communications with third parties or service providers, and as such requires maintaining long- term financial arrangements for recurrent expenditure. It is this latter aspect that is often neglected in aid funding arrangements and also given inadequate consideration in the financial organization of national government budgets. Key foci to be considered are detailed in the following subsections.
5.4.1 Field-monitoring equipment
Instruments have to be maintained in good work- ing order. Where manual and mechanical instruments are still used, for example operator- read instruments and chart recorders, then regular visits are required and calibration of instruments maintained. Current meters will have to be returned periodically to a certified calibration facility. Not all countries have such facilities, in which case finan- cial, logistic and customs arrangements have to be in place.
The increasing use of electronic instruments reduces the need for maintenance visits, but it cannot be assumed that these instruments are maintenance free. A consistent, high standard of accuracy and reliability is often overstated by manufacturers and suppliers, so the onus falls upon the user to care- fully review collected data. Thus, a strict programme of data checking by knowledgeable staff is required.
It should not be assumed that because an instru- ment or data logger is in an electronic sealed unit,
it is accurate forever. Instruments that are accessed by an operator, either to exchange memory units or download onto a data transfer module or PC, need to be visited regularly to keep databases up to date. A three-month visiting interval should be the minimum target.
Power supplies are of critical importance to field instruments. Even in developed countries a mains power supply is not always available, and it is often necessary and desirable to install instruments in remote locations where there are no power supplies.
Battery, wind and solar power all provide solutions for on-site power, but all require maintenance and servicing. In order to keep instrument downtime and network disruption to a minimum, it is neces- sary to have adequate spare instruments and parts, and also plans and arrangements for servicing by manufacturers. Electronic instruments can mostly only be calibrated by manufacturers, which again may require international shipment.
5.4.2 Office equipment
As with field equipment, office items are increas- ingly becoming electronic. Standard items within a hydrological unit include telephones, fax machines, photo copiers and computer systems of varying complexity. These items quickly become obsolete as manufacturers develop products and change models. Thus the maintenance programme has to make allowance for service and replacement of equipment that manufacturers no longer support. It is important that hydrological services are linked through the World Wide Web and have e-mail facilities. These are all services that carry recurrent costs with service providers, which can be both government and private-sector organizations.
In some countries the poor reliability of telecom- munications services and power supplies requires that internal communications are supported by radio (wireless) communications. Once again, equipment items in this field are constantly evolv- ing, so the same issues regarding spares, service and replacement exists as for field instruments. It may also be the case that licence fees are required to keep dedicated radio bandwidths, especially if contact is operationally critical in emergency situations.
5.4.3 Operational licences and warranty
Hydrological services are increasingly reliant on computer models and other software, for example for flood estimation, databases and GIS. These items are most likely to be provided by interna- tional companies, rather than be developed in-house. Thus the maintenance is dependent on
CHAPTER 5. MONITORING NETWORKS 5-11
the supplier and this is provided through service agreements, software licensing, upgrade provi- sions and warranty arrangements. These are all recurrent expenditure items that have to be identi- fied at the time of acquiring the facilities, and adequate finance provided on a continuing basis.
If suppliers are not fully represented in a country, these support arrangements have to be paid for in foreign currency, so exchange facilities are required.
5.4.4 Staff issues
Hydrometric units depend on a range of skilled staff with both technical and professional back- grounds. The educational and work experience must be recognized by suitable remuneration, organizational structures and career opportunities, which can be problematical in what are often small units. Worldwide, there is a problem of staff reten- tion, as the technical and professional qualifications and skills required in hydrology are similar to broader fields of civil engineering and science.
Larger organizations both within government and the private sector can often offer better salaries and better career prospects. In the case that hydrology is a small part of a major organization, for example an irrigation or water resources service, internal staff transfers are often the only way to achieve career development. This may result in loss of experienced staff due to their seeking a managerial or executive post, or the drafting in of inadequately qualified or inexperienced staff. The latter situation frequently occurs when a larger organization is restructuring, reducing staff numbers or filling vacancies when there is an embargo on external recruitment.
As hydrological services are invariably govern- ment organizations, they can suffer as part of the broader problems of cutbacks, poor pay scales and low morale. This situation is exacerbated when government salary payments are delayed and other allowances, for example housing, mobility and subsistence allowances, are not forthcoming. Without being particularly danger- ous, hydrological duties do entail some risk, for example the tasks of flood monitoring and river gauging, and may require staff to be away on duty or working extra hours in emergency situa- tions. It is important that these factors are recognized and rewarded.
5.4.5 Mobility and transport
Field operation is very dependent on adequate trans- port. Typical major requirements are for suitable field vehicles, boats and outboard motors, and these require high-quality maintenance for reliability and safety, with some built-in extra capacity to support enhanced levels of work during emergencies and to
retain flexibility for unforeseen breakdowns. Some countries have specific local conditions that require specialist equipment. For example, in Bangladesh the large rivers require specially built catamarans for flow gauging, which need to be of high-quality construction and adequately powered to be able to operate in high-flow conditions. They also require qualified pilots for safe operation.
In countries where travel conditions are physically difficult, for example in jungle or mountain terrain (for example Papua New Guinea or New Zealand) or where large distances are involved (for example in Namibia or India) it is of considerable benefit to use light aircraft or helicopters to transport teams and equipment. This requires high levels of recurrent budget support to cover commercial hire or charter rates. The most important factor is that there be adequate funds for the provision of fuel and that travel and subsistence allowances are adequate and promptly paid. If staff are out-of-pocket, they will find excuses not to carry out field duties, or skimp tasks in order to return to base quickly.
5.5 HYDROLOGICAL DATA RESCUE
5.5.1 Definition of data rescue
Data rescue is the process of archiving data at risk to avoid it being lost due to natural deterioration of the storing media, changes over time of data management agencies or natural hazards. The objective is to allow these data to be stored in a form in which they can be safely preserved for future use and easily accessed. This process may involve the electronic transcription of paper archives that are being degraded by humidity or rodent attack. It may equally involve the resurrec- tion of archives stored on out-of-date media or technologies, the provision for storing data in soft- ware and the digitization or image capture of current and past hydrological data into computer- compatible form for easy current and future access.
In a meeting on international data rescue organized by WMO in 2001, the following definition of data rescue was agreed:
An ongoing process of preserving all data at risk of being lost due to deterioration of the medium, and the digitization of current and past data into computer-compatible form for easy access.
This definition implies that:
(a) Data should be stored as image files onto media that can be regularly renewed to prevent the deterioration of the medium (for example cartridges, CDs and DVDs);
(b) Data already in computer-compatible media should be constantly migrated to storage facili- ties that conform to changing technologies;
(c) If necessary, data should be integrated from existing sources, for example paper archives and year books, in a form that can be used for analyses.
5.5.2 Problems of data archiving
Data archiving is often not given the due impor- tance it deserves. In many countries, data are not well organized but confined to boxes or open shelves in obscure store rooms, and hence subject to moisture, dust, heat and attack by rodents and insects. When space is needed for other activities or office moves take place, the data are often discarded.
Loss of hydrological data amongst WMO Member countries is of major concern. This problem is more serious in developing countries where, due to many constraints, affordability for archiving data is usually limited. Certain efforts have been made to support NHSs in the past through data rescue projects at a very small scale. However, implemen- tation of these projects has revealed that the problem of loss of hydrological data is larger than initially perceived.
Hydrological data are costly, both in terms of effort and resources, to record and collect. Large volumes of data are lost due to being inadequately archived and also due to poor maintenance of data archives.
The problems of hydrological data archiving vary from country to country, but the common factor is the storage of the data on computer and other media that are obsolete and not compatible with modern archiving and processing facilities. Such media include punched cards and magnetic tapes with no systems to read them. Some countries still keep volumes of data as hard copies (paper form), which face the danger of being lost, discarded or of deteriorating. Over time, the technology of data processing has evolved and changed so that consid- erable parts of these data have been irretrievably lost.
Chart recording raingauges and water level record- ers were for many years the mainstay of observing systems for hydrometeorology. Unfortunately, there was a tendency for backlogs of charts to build up without the proper analyses and data summaries being carried out. These charts remain to be proc- essed long after the processing routine within the organization has ceased. Large backlogs of unproc- essed chart data exist in both developed and less-developed NMSs and NHSs, and will require special efforts to digitize to make this valuable source of data available.
5.5.3 Justification for hydrological data
rescue
Data rescue is vital for the preservation of histori- cal records related to hydrology. Historical data provide the observational basis for scientific, engi- neering, and economic decisions for national benefit. Numerous data were collected in the past, in some cases during a period of colonial adminis- tration in countries that are now independent.
Changes in the latter may exacerbate problems because no detailed inventory has survived the change, but similar problems exist in developed countries with major changes in administrative organization. Once lost, these data can never be replaced, and all the effort made by past genera- tions of hydrologists and meteorologists will have been wasted. Now that climate change is of inter- national concern, it is even more vital that long-period, consistent data are available so that baselines and trends can be effectively established.
Comprehensive datasets are critical to ensure that the studies on climate change, variability and extreme events are as correct as possible. Having these valuable data available will contribute to a whole range of societal benefits. In addition, the loss of descriptive station information (metadata) and measurement data can mean that rating curves and the quality of flow records cannot be reassessed.
In the hydrological data rescue survey carried out by WMO in 2007, it was revealed that past activi- ties in this respect have yielded very limited results, and the survey recognized the need for strengthen- ing the capacities in the countries for the development and use of up-to-date data manage- ment systems, including data rescue, that can be used for various societal objectives.
Particular reasons for data rescue include:
(a) Forecasting models are more accurate when longer time series are used;
(b) Extreme hydrological phenomena are studied more thoroughly and accurately;
(c) Design of engineering projects, which critically depend on hydrological information, is more reliable when very long hydrological records are used.
5.5.4 WMO past initiatives
The activity of WMO in the field of data rescue has a long history, beginning in 1979 through the RA I data bank project, which led to the later Belgium data-rescue programme. The Belgium- funded programme assisted over 40 African Members in preserving their meteorological data through a microfilm and microfiche process, and a copy of these data is now kept in each of
CHAPTER 5. MONITORING NETWORKS 5-13
the countries. In 1988, WMO established the Archival Climate History Survey Programme (ARCHISS) to retrieve and make available mete- orological data from the national archives of each country.
The general uptake and progress of these programmes has not, however, been fully adopted and many problems still remain and are becoming increasingly severe as time passes. In June 1999, the WMO Secretariat circulated a questionnaire to the Hydrological Advisers in 39 countries in Africa.
Twenty-three replies (59 per cent) were received.
Based on the replies received, it was clearly indicated that 82 per cent use paper for archiving their data and they requested WMO to assist them in rescuing these data. Following this survey, a pilot project to rescue hydrological data was launched in six English-speaking (Egypt, Eritrea, the Gambia, Ghana, Kenya and the United Republic of Tanzania) and five French-speaking countries (Chad, Congo, Niger, Rwanda and Togo). Under these projects, each country was provided with a computer, a printer, a scanner, and a software package for data processing and management (Hydata for the English-speaking countries and Hydrom for the French-speaking ones). More than 80 national staff were trained in workshops for 10 days on the applications of suitable software for data management, with a view to securing the data in electronic form, in addition to its paper-based status.
As a continuation of this initiative, a survey on hydrological data rescue needs was carried out by WMO in late 2007. It resulted in responses from 57 countries from the different Regions. A full analysis of the responses to the survey has yet to be published. Preliminary results have revealed that past activities have yielded very limited results and that there continues to be a need for strengthening the capacities for the development and use of up-to-date data management systems.
5.5.5 Data rescue and digitization
Hydrological data should be regarded as significant assets with strategic value. It is imperative to convert physical or manual records such as text and images (microfiche) into efficient and digital forms.
Information must be stored in media that ensure the storage is safe and the retrieval is quick. There are different base materials used for digitization, and these are briefly considered in the following points:
Hardcopy:
In many cases, observation books or logbooks are used directly as the source for keying in the data.
These may be handwritten observations, tables with printed data, or strip charts with graphical information. The type of hardcopy material deter- mines to a large extent the digitization method.
There are a number of problems concerning digiti- zation of hydrological data from original logbooks.
These include:
(a) Unreadable data due to decayed material or overwritten documents;
(b) Irregular times of observation;
(c) Measurements made with historical or outdated instruments;
(d) Measurements in historical units requiring conversion to SI units.
Many highly valuable historical records are contained in yearbooks produced by government departments or meteorological archives that ceased publication with the advent of computer- based archives. Very few attempts have been made to digitize these extensive records, with the result that there is a break in continuity and ease of access. As a consequence, these records, which in many cases have a longer duration than the digital database that replaced them, are ignored once the staff with knowledge of pre-digital data sources have retired. The task of digitizing extensive historical records from book sources can be a major undertaking, especially as, to prove useful, they have to be merged with the existing database structure. Simply preserving the old records as scanned format is not satisfactory, as these data cannot be manipulated in the same way as a true digital database.
Hardcopy historical hydrological data are some- times stored as film (or microfiche) when authorities require the information for archive purposes. The quality of the images on films is mostly excellent, especially if high-quality mate- rial such as polyacetate film has been used. Film is an ideal medium for preserving chart data for later digitization, or for maps, plans and diagrams, for example gauging structures and maps of flood extent.
Digital images:
Digital images of data are now generally replacing the use of film, and are obtained by scanning or digitally photographing the hardcopy documents.
Digitally imaged data still have to be entered by keying data into the operational database. Some software systems allow digital images to be displayed alongside the user database on the same computer screen, which allows data entry to be easier than typing data from a paper source.
Nevertheless, transcription always carries the risk of error, so a rigorous checking system has to be adhered to.