WRDS Library [Home]
Digital Library Publications Videos Card Catalog

Chapter V
Testing the Model

This chapter describes the procedures used to test the WIRSOS program after modifications to the model were finished. Testing of the model at incremental stages of the project was also done and will be discussed as it is presented. All modifications to the model and the testing of each modification is discussed.

INTRODUCTION

The WIRSOS model was quite large before any work was done to enhance its capabilities, and each new feature increased the size of the model. The current model has approximately twice the code of the original program model. The increased size of the program has increased its complexity. Unfortunately, it also means an increased chance for error. In an attempt to limit the number of errors, the program was partially "debugged" each time the program was changed. After the model was completed, a data base was modified and every aspect of the completed model was tested.

The features tested were: an initial run to create a comparison file; initial reservoir rights year-to-date distributed by the user; initial reservoir rights year-to-date distributed by the program; minimum release from a reservoir; off-channel storage reservoir; project rights tied to a reservoir right; project rights allowed to withdraw water from two reservoirs; water exchanges for project rights not downstream from their associated reservoir; extreme monthly storage; checking the water rights and station input files; a combination of all options; and a combination of all options with the full data base.

One problem encountered while testing the model, was that there is no way to "check the answer". Instead of checking for a "correct" answer, the model was to drive the model to extreme conditions. The output was then checked to see if it reflected the correct direction expected for the given condition. The final action of the testing plan was to create data bases and output for the current model, so that any future modifications made to the program would have a data base and its corresponding output files to compare to the output files produced by the modified program.

INCREMENTAL TESTING

The size of the model created the problem of how to handle the output files, so testing the model had to be addressed before the program was divided into subroutines. First, the data base used to test the model contained thirteen years of runoff data. The output data files created from thirteen years of runoff data were too large to edit or compare. Therefore, the runoff data base was trimmed to two years of data. Two years of runoff data still produced output files were still too large to efficiently compare the results of separate model runs. In response to this problem, a small program was written that reads the lines of two output files and compares the files line by line. If the files differed in any way, the program would detect the differences. Before making any changes to WIRSOS, the latest version was used to create output files for comparison with output files created by the modified version.

After reviewing the program and studying the output files, it became apparent that Tape 11 (list of rights called out) was the major output file that needed to be compared. Tape 11 was chosen because it is written to by virtually every major section of the model and any changes would show up in this file. After each alteration or addition to the program, the model was recompiled and a new set of output files was produced. The comparison program was then ran on the original Tape 11 output file and the new Tape 11 output file to easily identify any differences between the two files. If there were any differences, the error that caused the difference was located and eliminated. If the difference was not caused by an error, the reason for the change was isolated and it determined if the difference was a desired response. If the difference between the two Tape 11 files was the result of a change in the program and not an error, the original output files were replaced with the new output files. The model was tested in this manner after each modification to WIRSOS.

One problem with the incremental testing, was its inability to test any of the new features added to the program. However, it did verify that the current version of WIRSOS (WIRSOS5) is capable of reproducing the output from the original model. To test the new features, new data bases were created and each function had to be tested separately.

DATA BASES

The original diversion rights data base used in the initial phases of this project was not sorted by priority date (as required by WIRSOS). This data base, the junior project rights, the instream flow rights, and the reservoir rights data bases were sorted by priority date, station number, and permit number. Additionally, the reservoir data file (inp 15), the diversion rights file (inp 4), and the junior project rights file (inp 17) were modified to accept data compatible with WIRSOS5. The modification of these data bases is discussed in the section where testing of each added feature is described. An input file for the efficiency tables (inp 20) was also created.

INITIAL RUN

In addition to testing the added features, WIRSOS5 was ran with all the run options disabled. This produced an initial set of sample output files to compare to files created testing the model. For this initial run, efficiency tables were used and Tapes 12 and 13 were sorted at the end of the model run. Reservoir rights year-to-date were set to zero at the start of the model run and at the beginning of each water year. The initial run was also performed with the sorted diversion data files.

RESERVOIR RIGHTS DIVISION BY MODELER

One of the features added to WIRSOS, was the ability to associate storage in a reservoir with a particular reservoir right. With this feature, the reservoir rights year-to-date (the amount stored by a reservoir right during the current water year) are set equal to the reservoir rights account at the beginning of each water year. The reservoir rights account is all the water stored and removed from a reservoir right during the course of the year. As part of this option, the modeler is given two choices for dividing the initial reservoir volume (volume of the reservoir at the start of the model run) between the reservoir rights. The two choices are initial reservoir rights year-to-date distributed by the modeler and initial reservoir rights year-to-date distributed by the program.

Initial reservoir rights year-to-date distributed by the modeler involves inputting values in the reservoir data file. These values indicate how to divide the initial reservoir volume between the reservoir rights. The modeler can input up to four values (corresponding to the four rights possible for each reservoir). These values should sum to 100%. In this manner, the modeler can chose to assign a percentage of the initial reservoir volume to each right. These values are entered (as a percentage value with no decimal points) in the reservoir data file after the power release goal volume.

A new output file was created to report the reservoir rights accounts for each month of the simulation (Tape 19). In addition to reporting reservoir rights account, the new output file reports reservoir rights year-to-date and the factor that limited reservoir storage at the reservoir right's priority date. This file was used to test the reservoir rights distributed by the modeler option of the WIRSOS5 program.

The data base used to test the model has only one reservoir with multiple reservoir rights (Upper Sunshine). For this test, Upper Sunshine was assumed to store an initial volume of 10,000 acre-feet. The program was directed to assign 65% of this initial volume to the most senior right and 35% to the junior right. A check of Tape 19 after the model run indicated that 3500 acre-feet had been assigned to the junior right. This approach proved conclusively that this option is functioning properly.

RESERVOIR RIGHTS DIVISION BY PROGRAM

The second way to distribute reservoir volume between reservoir rights is to allow the model to distribute the volume between the rights. To distribute the initial reservoir volume in this manner, the modeler sets the reservoir right distribution values in the reservoir data file to zero. The reservoir rights year-to-date are also set equal to the previous water year's reservoir rights account at the start of each water year.

When reading the reservoir data at the beginning of a model run, the program assigns the initial reservoir volume according to the percent of the total permitted volume that each reservoir right controls. For Upper Sunshine, the junior reservoir right controls 6.66% of the permitted reservoir volume. When the model was run, 666 acre-feet of water were assigned the junior reservoir right. This proved that the option was working correctly since 666 is exactly 6.66% of 10,000.

At the beginning of each water year, the reservoir rights year-to-date for the junior right equalled to the reservoir right's account of that right at the end of the previous water year. It was concluded that the model was distributing reservoir rights correctly, and that the reservoir right accounts were functioning properly.

MINIMUM RELEASE FROM A RESERVOIR

Upper Sunshine was again selected to test this model option. A value can be entered in the reservoir data file after the initial storage distribution values for the minimum release from a reservoir. If no minimum release is desired, the minimum release value should be set to zero. The minimum release from a reservoir is either the flow to meet the minimum release value or the flow entering the reservoir at its upstream station. The flow entering the reservoir at its upstream station is used if the flow entering is less than the minimum release value. The flow released is the minimum flow value (or the entering flow) minus the flow below the reservoir. The water released from the reservoir is distributed between the reservoir rights according to the amount of water, stored in the reservoir, that each reservoir right controls. Thus, if 10% of the water stored in a reservoir is controlled by a reservoir right, 10% of the water released will be taken from that right.

To test this option, the minimum release from Upper Sunshine was set at ten cfs. After the model run, the flow (at the river station below the reservoir) was reported in Tape 18 as 10 cfs. The option works correctly.

OFF-CHANNEL STORAGE

To allow off-channel storage reservoirs in WIRSOS, an additional line of data had to be created for each reservoir in the reservoir data file. This extra line contains the ten return flow stations, percent returned at each station, and a return flow delay table number to be used at each station. However, the off-channel storage reservoir diversion station, the reservoir diversion efficiency, the number of return flow stations for the reservoir diversion, and the reservoir diversion canal capacity are located on the second line of data following the minimum release value. The program is directed to model a reservoir as an off-channel storage facility by setting a flag in line one of the reservoir data file.

To test off-channel storage, the reservoir diversion canal capacity was set at ten cfs for Lower Sunshine reservoir. The intent was for the diversion canal capacity to limit reservoir storage. The canal capacity would be the limiting factor reported to Tape 19. This was the case after the model run. Therefore, it is assumed that the off-channel storage option is working correctly.

PROJECT RIGHTS TIED TO A RESERVOIR RIGHT

In order for a project right to be associated with a particular reservoir right, the diversion and junior project rights data bases were modified. Actually, no columns were added to the data bases, but the function of the second column in the first line was changed for each right. In the diversion data base, the second column was used to indicate whether the right was a "normal" diversion or project right. However, this is no longer necessary since the variable DIVTYP performs this function. The second column of the first line of each diversion right is used to indicate whether the right is tied to a particular reservoir right (if the diversion right is a project right). Hence, a value of 0 indicates the right is not tied to a particular reservoir right, and values of 1-4 indicate which reservoir right the project right is associated.

In the junior project rights data base, the second column in the first line of each right previously performed no function. Therefore, making the second column indicate which reservoir right the junior project right was associated did not change the program. In the junior project rights file, a value of 1-4 indicate which reservoir right the junior project right is associated. Any other character indicates the junior project right is not associated with a particular reservoir right.

With this in mind, all project and junior project rights that draw on the Upper Sunshine reservoir were forced to draw on a the junior reservoir right. This was accomplished by putting a `2' in the second column of the first row of each project and junior project right associated with Upper Sunshine. When the model was tested, the project rights released from Upper Sunshine (as reported by Tape 18) decreased. The reservoir released water normally for the first three month when all releases stopped. In addition, Tape 19 indicated that the second right on Upper Sunshine was depleted after the third month of operation. The conclusion reached was that this option was functioning correctly.

PROJECT RIGHT TIED TO TWO RESERVOIRS

Associating a project right with two reservoirs was not as easy as associating a project right to a particular reservoir right. To allow this option to happen, the diversion and junior project rights files had to be modified. In each file, the information for this option is entered at the end of the second line of data (columns 113-116). The information consists of: a flag that indicates that the project right is allowed to withdraw water from a second reservoir, the reservoir from which the water is to be withdrawn, and the reservoir right to which the project right is tied. The flag is either `0' for no second reservoir or `1' to draw from a second reservoir. The reservoir is the reservoir code just like DIVTYP and RESNUM in the diversion and junior project rights data bases, and the reservoir right code is the same as the diversion data base (101 for no particular reservoir right or `1-4' for a particular reservoir right).

Upper Sunshine was chosen as the test reservoir for this model option. Upper Sunshine was chosen because its releases enter the main branches of the Greybull river below Lower Sunshine reservoir. All rights that were physically below both Sunshine reservoirs, could withdraw first on Upper Sunshine and then on Lower Sunshine if the right remained unsatisfied. To test this option work, the river system was stressed beyond "normal" operating limits. To do this, a "dummy" right was added to the diversion and junior project rights data bases. This "dummy" right tried to withdraw over 100 cfs from the river system. While trying this option, the project rights were not associated with a particular reservoir right. When the model ran, Tape 18 showed large (greater than 100 cfs) releases for project rights from Upper Sunshine until that reservoir was empty, and then larger than normal releases for project rights were made from Lower Sunshine until it fell below its minimum volume. In conclusion, this model option appeared to be working correctly.

WATER EXCHANGES

A water exchange occurs when a project right is not physically downstream from its associated reservoir. When this happens, the project right receives its demand from flow in the river. This flow is then replaced by water from the reservoir. To test this option, a right was entered above Lower Sunshine reservoir. This right withdrew water from Lower Sunshine and its demand was 100 cfs per month. When the model was ran, large releases were made from Lower Sunshine and the right was written to the rights called out file (Tape 11). In addition, statements that reported model progress were used to write messages to screen. These messages traced the model progress as the dummy right was processed. It was concluded that the model was working.

EXTREME MONTHLY STORAGE

Extreme monthly storage which allows extreme excess water in the system to be stored out of priority did not require the alteration of any data bases, but did require considerable effort to program into the model. Once it was part of the WIRSOS model, it seemed to operate quite well. The extreme monthly storage option is chosen in the model run information section of the interactive user interface. When this option was initiated, more water was stored in the reservoirs (Tape 18) after their year-to-date rights were met (Tape 19). No water was stored if water available downstream, senior downstream rights, or off-channel diversion capacity limited the amount of water stored by a right during a reservoirs priority date (Tape 19). Tape 11 showed that fewer rights were called out during this simulation, which was expected since there would be more water stored in the reservoirs. In short, the option appeared to be working correctly.

DATA CHECKING

The subroutines that were developed to check the diversion rights, junior project rights, instream flow rights, reservoir rights, and station data files were also tested. For every aspect of a data file that is checked, a data record was deliberately entered incorrectly to test the model. Each time the proper error message was printed to the screen and program operation was stopped.

COMBINED MODEL

As a final check on the model operation, all the options and revised data bases were combined. The model was then tested using these data bases with all options functional. Everything that was checked after this run would take too much space to describe; therefore, it is sufficient to say that everything was operating as expected. In addition to Tape 18 and Tape 19, every output file was visually inspected for any out of place characters or anomalies. Tape 11 was inspected to check all callout messages written to that file to determine whether all aspects of the improved program were functioning. Based on this information, it was concluded that the model was functioning properly.

COMBINED MODEL WITH FULL DATA BASE

At this point, the combined model with the full thirteen years of runoff data was ran. The purpose of running this model was to try and catch any errors overlooked during the previous testing. The model ran without problems.

DATA BASES AND KNOWN OUTPUT

To create the data base and known output files for the current model, the full thirteen years of runoff data were chosen. In addition, no project rights were tied to a particular reservoir right, the off-channel diversion capacity was set at 100 cfs, the minimum release was set at 0.5 cfs, all project rights below both Upper and Lower Sunshine were allowed to call on both reservoirs, reservoir rights year-to-date were set to the previous years reservoir rights account, the beginning volume was divided 35% to the junior right and 65% to the senior right, and the reservoirs were allowed to store any extreme supply of water in the system.

CONCLUSIONS

When testing the model, the first run to test each option often failed. Therefore, much effort was made to track and eliminate the errors discovered. In the end the options worked as expected, and from the output there is a reasonable level of confidence that the model is working properly.


Stroup, 1993 Table of Contents
Theses List
Water Resources Data System Library | Water Resources Data System Homepage