Checking for non-preferred file/folder path names (may take a long time depending on the number of files/folders) ...
This resource contains some files/folders that have non-preferred characters in their name. Show non-conforming files/folders.
This resource contains content types with files that need to be updated to match with metadata changes. Show content type files that need updating.
Authors: |
|
|
---|---|---|
Owners: |
|
This resource does not have an owner who is an active HydroShare user. Contact CUAHSI (help@cuahsi.org) for information on this resource. |
Type: | Resource | |
Storage: | The size of this resource is 163.8 MB | |
Created: | Sep 22, 2020 at 3:47 p.m. | |
Last updated: | Oct 14, 2020 at 7:46 p.m. (Metadata update) | |
Published date: | Oct 14, 2020 at 7:46 p.m. | |
DOI: | 10.4211/hs.282177bd87e7426e92899058f075359a | |
Citation: | See how to cite this resource |
Sharing Status: | Published |
---|---|
Views: | 1716 |
Downloads: | 195 |
+1 Votes: | 1 other +1 this |
Comments: | No comments (yet) |
Abstract
This resource contains 5-minute discharge data from 13 experimental catchments in Central Panama. The Agua Salud Project is managed by the Smithsonian Tropical Research Institute to facilitate research into the ecosystem benefits of various land covers in the humid tropics. The attached README.md includes a more thorough description of this dataset and site specific details. A user can export these data from the HDF archive using the included Python script or access the data directly using a variety of HDF libraries in other languages.
Subject Keywords
Coverage
Spatial
Temporal
Start Date: | |
---|---|
End Date: |
Content
README.md
Export Agua Salud Discharge Data
Use export_discharge.py
to extract discharge data with metadata from the Agua Salud Discharge HDF5 data archive.
Data Description
The AguaSaludDischarge.h5
file contains volumetric discharge data collected as part of the Panama Canal Watershed Experiment: Agua Salud Project. The file contains accumulated 5-minute instantaneous discharge measurements for each experimental watershed.
Collection
We used two-stage V-notch weirs equipped with pressure transducers to collect 5-minute discharge measurements. Each two-stage weir consisted of a concrete short-crested high-flow weir and a metal sharp-crested low-weir. We used non-vented In-Situ LevelTROLL pressure transducers with integrated data loggers to record instantaneous pressure measurements every 5 minutes. A field technician retrieved pressure measurements monthly and measured depth behind each weir using a staff gage. We collected barometric pressure data using an In-Situ BaroTROLL and removed daily barometric pressure fluctuations from the water pressure records. We used a Time Series Data Editor developed by AquaVeo as part of the Watershed Modeling System to convert pressure measurements to stage using field observed staff gage measurements. We used time series decomposition to remove systematic errors due to ephemeral woody debris clogging the weirs and sensor drift. We used discharge coefficients from Ogden et al., 2017 to convert stage measurements to discharge accounting for sedimentation.
Site Description
Dependencies
python 3.6+
h5py
Linux
In a python3 enivronment:
python3 -m pip install h5py
python3 export_discharge.py [options]
Windows
Download and install Anaconda3. Open the export_discharge.py
script in Spyder. Select Run > Configuration per file... Select the checkmark next to Command line options: and enter command line arguments in the text field. Select Run.
CUAHSI JupyterHub
These data are available for download from HydroShare with a HydroShare account. To export these data from a web browser using CUAHSI JupyterHub go to the resource landing page located here after signing into your Hydroshare account. Select Open with... on the right side of the landing page. Select CUAHSI JupyterHub. Read and accept the terms of use to sign-in with your Hydroshare account. Authorize read and write permissions. Select the Python 3.7 - Scientific server and Start. Find the drop down on the right labeled New and select Terminal. Navigate to ~/downloads/282177bd87e7426e92899058f075359a/282177bd87e7426e92899058f075359a/data/contents
. From here you can export data using python3 export_discharge.py [options]
.
Usage
usage: export_discharge.py [-h] [-s S] [-d D] [-f [F]] [-ls] [-ld] [-ht]
[-o O] [-nd ND] [--first FIRST] [--last LAST]
Export binary Agua Salud data to CSV
optional arguments:
-h, --help show this help message and exit
-s S site
-d D dataset
-f [F] HDF5 input file [default: AguaSaludDischarge.h5]
-ls list available sites with engineering code and exit
-ld list datasets for a given site and exit
-ht print input file hierarchy in tree format and exit
-o O optional output filename
-nd ND no data value [default: -9999.9]
--first FIRST [YYYY-MM-DDThh:mmZ] datetime of first measurement to export
--last LAST [YYYY-MM-DDThh:mmZ] datetime of last measurement export
Example
$ python3 export_discharge.py -s FOR -d discharge_sharp --first 2010-01-01T01:00Z --last 2010-01-02T01:00Z
Site: FOR
Dataset: discharge_sharp
Start export: 2010-01-01 01:00:00
End export: 2010-01-02 01:00:00
Exporting 289 measurements...
Wrote output data to FOR_discharge_sharp.csv
Sample output
# site: FOR
# dataset: discharge_sharp
# drainage_area: 144.87 hectares [ha]
# weir_latitude: 9.20893° [wgs84]
# weir_longitude: -79.7795° [wgs84]
# no_data_value: -9999.9
# engr_code: V02ASL
# description: Stream discharge over sharp-crested weir
# datetime: Instantaneous measurement time - ISO 8601 Standard date-time string YYYY-MM-DD hh:mmZ
# discharge: Stream discharge over weir in cubic meters per second [m^3 s^-1]
# quality: Numerical flag indicating sharp-crested weir measurement quality
# -1: No data
# 0 : Depth is within limits of weir box
# 2 : Flow deviation greater than 1 liter per second from short-crested weir
# 3 : Flow depth exceeded weir maximum depth of V-notch
#
#
datetime,discharge,quality
2010-01-01 01:00Z,-9999.9,2
Data hierarchy and available date ranges
Groups
/site_data/ARN/
/site_data/COF/
/site_data/CTD/
/site_data/CUT/
/site_data/FOR/
/site_data/MOS/
/site_data/NAT/
/site_data/PAS/
/site_data/RAS/
/site_data/SAC/
/site_data/SEC/
/site_data/TEK/
/site_data/TKU/
The hierarchical data structure is grouped by site. The top-level site below root is /site_data/
. Below /site_data/
are 13 groups corresponding to each experimental watershed.
Datasets
Each of the site-corresponding groups contains 3 or 6 datasets. depth_short
is a 1-dimensional array containing depth measurements above the invert of a high-flow short-crested weir. depth_sharp
is a 1-dimensional array of depth measurments above the invert of a low-flow sharp-crested weir. discharge_short
is a 1-dimensional array of discharge measurements derived using a weir rating curve and depth_short
measurements. discharge_sharp
is a 1-dimensional array of discharge measurements derived from depth_sharp
. field_discharge_short
and field_discharge_sharp
contain field observed measurements of depth with correspondingly derived discharge for both short and sharp-crested weirs.
$ python3 export_discharge.py -ht
/site_data/
├── ARN/
│ ├── depth_short (2016-04-22T17:30Z to 2018-01-29T15:20Z)
│ ├── depth_sharp (2016-04-22T17:30Z to 2018-01-29T15:20Z)
│ ├── discharge_short (2016-04-22T17:30Z to 2018-01-29T15:20Z)
│ ├── discharge_sharp (2016-04-22T17:30Z to 2018-01-29T15:20Z)
│ ├── field_discharge_short (2016-04-22T17:00Z to 2018-01-29T15:00Z)
│ └── field_discharge_sharp (2016-05-12T17:00Z to 2017-08-08T15:00Z)
├── COF/
│ ├── depth_short (2014-12-10T18:15Z to 2018-02-19T17:35Z)
│ ├── discharge_short (2014-12-10T18:15Z to 2018-02-19T17:35Z)
│ └── field_discharge_short (2014-12-10T18:00Z to 2018-02-19T17:00Z)
├── CTD/
│ ├── depth_short (2015-05-29T15:30Z to 2018-04-09T15:10Z)
│ ├── discharge_short (2015-05-29T15:30Z to 2018-04-09T15:10Z)
│ └── field_discharge_short (2015-05-29T15:00Z to 2018-04-09T15:00Z)
├── CUT/
│ ├── depth_short (2016-04-22T16:00Z to 2018-04-09T15:25Z)
│ ├── depth_sharp (2016-04-22T16:00Z to 2018-04-09T15:25Z)
│ ├── discharge_short (2016-04-22T16:00Z to 2018-04-09T15:25Z)
│ ├── discharge_sharp (2016-04-22T16:00Z to 2018-04-09T15:25Z)
│ ├── field_discharge_short (2016-04-22T16:00Z to 2018-04-09T15:00Z)
│ └── field_discharge_sharp (2016-05-24T17:00Z to 2018-04-09T15:00Z)
├── FOR/
│ ├── depth_short (2009-03-20T14:45Z to 2018-03-22T16:25Z)
│ ├── depth_sharp (2009-03-20T14:45Z to 2018-03-22T16:25Z)
│ ├── discharge_short (2009-03-20T14:45Z to 2018-03-22T16:25Z)
│ ├── discharge_sharp (2009-03-20T14:45Z to 2018-03-22T16:25Z)
│ ├── field_discharge_short (2009-03-20T15:00Z to 2018-03-22T16:00Z)
│ └── field_discharge_sharp (2009-03-20T14:00Z to 2017-03-08T15:00Z)
├── MOS/
│ ├── depth_short (2009-03-27T17:55Z to 2018-03-16T15:50Z)
│ ├── depth_sharp (2009-03-27T17:55Z to 2018-03-16T15:50Z)
│ ├── discharge_short (2009-03-27T17:55Z to 2018-03-16T15:50Z)
│ ├── discharge_sharp (2009-03-27T17:55Z to 2018-03-16T15:50Z)
│ ├── field_discharge_short (2009-03-27T17:00Z to 2018-03-16T15:00Z)
│ └── field_discharge_sharp (2009-04-02T15:00Z to 2017-06-07T16:00Z)
├── NAT/
│ ├── depth_short (2009-03-07T21:30Z to 2018-02-05T17:05Z)
│ ├── depth_sharp (2009-03-07T21:30Z to 2018-02-05T17:05Z)
│ ├── discharge_short (2009-03-07T21:30Z to 2018-02-05T17:05Z)
│ ├── discharge_sharp (2009-03-07T21:30Z to 2018-02-05T17:05Z)
│ ├── field_discharge_short (2009-03-07T21:00Z to 2018-02-05T17:00Z)
│ └── field_discharge_sharp (2009-04-21T15:00Z to 2018-02-05T17:00Z)
├── PAS/
│ ├── depth_short (2009-03-07T19:00Z to 2018-02-07T15:05Z)
│ ├── depth_sharp (2009-03-07T19:00Z to 2018-02-07T15:05Z)
│ ├── discharge_short (2009-03-07T19:00Z to 2018-02-07T15:05Z)
│ ├── discharge_sharp (2009-03-07T19:00Z to 2018-02-07T15:05Z)
│ ├── field_discharge_short (2009-03-07T19:00Z to 2018-02-07T15:00Z)
│ └── field_discharge_sharp (2009-03-07T20:00Z to 2018-02-07T14:00Z)
├── RAS/
│ ├── depth_short (2009-03-20T16:50Z to 2018-01-24T17:45Z)
│ ├── depth_sharp (2009-03-20T16:50Z to 2018-01-24T17:45Z)
│ ├── discharge_short (2009-03-20T16:50Z to 2018-01-24T17:45Z)
│ ├── discharge_sharp (2009-03-20T16:50Z to 2018-01-24T17:45Z)
│ ├── field_discharge_short (2009-03-27T14:00Z to 2018-01-24T17:00Z)
│ └── field_discharge_sharp (2009-03-20T16:00Z to 2009-03-20T16:00Z)
├── SAC/
│ ├── depth_short (2009-06-17T16:55Z to 2018-03-23T16:45Z)
│ ├── depth_sharp (2009-06-17T16:55Z to 2018-03-23T16:45Z)
│ ├── discharge_short (2009-06-17T16:55Z to 2018-03-23T16:45Z)
│ ├── discharge_sharp (2009-06-17T16:55Z to 2018-03-23T16:45Z)
│ ├── field_discharge_short (2009-06-17T16:00Z to 2018-03-23T16:00Z)
│ └── field_discharge_sharp (2009-07-28T21:00Z to 2017-07-28T15:00Z)
├── SEC/
│ ├── depth_sharp (2009-04-23T18:00Z to 2018-02-05T16:55Z)
│ ├── discharge_sharp (2009-04-23T18:00Z to 2018-02-05T16:55Z)
│ └── field_discharge_sharp (2009-04-23T18:00Z to 2018-02-05T16:00Z)
├── TEK/
│ ├── depth_short (2009-01-17T21:20Z to 2018-02-08T16:15Z)
│ ├── depth_sharp (2009-01-17T21:20Z to 2018-02-08T16:15Z)
│ ├── discharge_short (2009-01-17T21:20Z to 2018-02-08T16:15Z)
│ ├── discharge_sharp (2009-01-17T21:20Z to 2018-02-08T16:15Z)
│ ├── field_discharge_short (2009-01-17T21:00Z to 2018-02-08T16:00Z)
│ └── field_discharge_sharp (2009-04-22T15:00Z to 2017-08-28T17:00Z)
└── TKU/
├── depth_short (2009-04-07T16:15Z to 2018-02-08T15:05Z)
├── depth_sharp (2009-04-07T16:15Z to 2018-02-08T15:05Z)
├── discharge_short (2009-04-07T16:15Z to 2018-02-08T15:05Z)
├── discharge_sharp (2009-04-07T16:15Z to 2018-02-08T15:05Z)
├── field_discharge_short (2009-04-14T13:00Z to 2018-02-08T15:00Z)
└── field_discharge_sharp (2009-04-07T16:00Z to 2017-08-28T18:00Z)
These datasets contain arrays of compound data types typically consisting of three components: a null-terminated ASCII character string containing an ISO 8601 formatted datetime, a 32-bit float containing a measurement value, and a 32-bit integer containing a quality code. Associated attributes are added to output CSV files as commented header lines by default when using export_discharge.py
.
Attributes
Each group and dataset has a number of metadata attributes with full descriptions. The data format, contents, and metadata can be explored using h5dump
.
$ h5dump -A -d /site_data/ARN/discharge_sharp AguaSaludDischarge.h5
HDF5 "AguaSaludDischarge.h5" {
DATASET "/site_data/ARN/discharge_sharp" {
DATATYPE H5T_COMPOUND {
H5T_STRING {
STRSIZE 18;
STRPAD H5T_STR_NULLPAD;
CSET H5T_CSET_ASCII;
CTYPE H5T_C_S1;
} "datetime";
H5T_IEEE_F32LE "discharge";
H5T_STD_I32LE "quality";
}
DATASPACE SIMPLE { ( 186311 ) / ( 186311 ) }
ATTRIBUTE "datetime" {
DATATYPE H5T_STRING {
STRSIZE 86;
STRPAD H5T_STR_NULLPAD;
CSET H5T_CSET_ASCII;
CTYPE H5T_C_S1;
}
DATASPACE SCALAR
DATA {
(0): "Instantaneous measurement time - ISO 8601 Standard date-time string YYYY-MM-DD hh:mmZ\000"
}
}
ATTRIBUTE "description" {
DATATYPE H5T_STRING {
STRSIZE 41;
STRPAD H5T_STR_NULLPAD;
CSET H5T_CSET_ASCII;
CTYPE H5T_C_S1;
}
DATASPACE SCALAR
DATA {
(0): "Stream discharge over sharp-crested weir\000"
}
}
ATTRIBUTE "discharge" {
DATATYPE H5T_STRING {
STRSIZE 65;
STRPAD H5T_STR_NULLPAD;
CSET H5T_CSET_ASCII;
CTYPE H5T_C_S1;
}
DATASPACE SCALAR
DATA {
(0): "Stream discharge over weir in cubic meters per second [m^3 s^-1]\000"
}
}
ATTRIBUTE "engr_code" {
DATATYPE H5T_STRING {
STRSIZE 7;
STRPAD H5T_STR_NULLPAD;
CSET H5T_CSET_ASCII;
CTYPE H5T_C_S1;
}
DATASPACE SCALAR
DATA {
(0): "V13ASL\000"
}
}
ATTRIBUTE "no_data_value" {
DATATYPE H5T_IEEE_F32LE
DATASPACE SCALAR
DATA {
(0): -9999.9
}
}
ATTRIBUTE "quality" {
DATATYPE H5T_STRING {
STRSIZE 246;
STRPAD H5T_STR_NULLPAD;
CSET H5T_CSET_ASCII;
CTYPE H5T_C_S1;
}
DATASPACE SCALAR
DATA {
(0): "Numerical flag indicating sharp-crested weir measurement quality
-1: No data
0 : Depth is within limits of weir box
2 : Flow deviation greater than 1 liter per second from short-crested weir
3 : Flow depth exceeded weir maximum depth of V-notch
\000"
}
}
}
}
Related Resources
Title | Owners | Sharing Status | My Permission |
---|---|---|---|
Agua Salud Hydrometric Data | Jason Regina · Fred Ogden | Published | Open Access |
Agua Salud Hydrometric Data | Jason Regina | Published | Open Access |
Agua Salud Hydrometric Data | Jason Regina | Published | Open Access |
Credits
Funding Agencies
This resource was created using funding from the following sources:
Agency Name | Award Title | Award Number |
---|---|---|
Stanley Motta | ||
Hoch Trust | ||
National Science Foundation | Collaborative Research: Planning And Land Management in Tropical Ecosystem; Complexities of land-use and hydrology coupling in the Panama Canal Watershed | 1360305, 1360369, 1360384, 1360391 |
Smithsonian Tropical Research Institute | ||
HSBC Climate Partnership (2008-2012) | ||
Levinson Family Foundation |
How to Cite
This resource is shared under the Creative Commons Attribution CC BY.
http://creativecommons.org/licenses/by/4.0/
Comments
There are currently no comments
New Comment