Checking for non-preferred file/folder path names (may take a long time depending on the number of files/folders) ...
This resource contains some files/folders that have non-preferred characters in their name. Show non-conforming files/folders.
This resource contains content types with files that need to be updated to match with metadata changes. Show content type files that need updating.
New Zealand Hydrological Society Data Workshop 2024: A Python Package for Automating Aquatic Data QA/QC
Authors: |
|
|
---|---|---|
Owners: |
|
This resource does not have an owner who is an active HydroShare user. Contact CUAHSI (help@cuahsi.org) for information on this resource. |
Type: | Resource | |
Storage: | The size of this resource is 159.6 MB | |
Created: | Apr 08, 2024 at 10:59 p.m. | |
Last updated: | Apr 09, 2024 at 2:20 a.m. | |
Citation: | See how to cite this resource |
Sharing Status: | Public |
---|---|
Views: | 327 |
Downloads: | 109 |
+1 Votes: | Be the first one to this. |
Comments: | No comments (yet) |
Abstract
This resource was created for the 2024 New Zealand Hydrological Society Data Workshop in Queenstown, NZ. This resource contains Jupyter Notebooks with examples for conducting quality control post processing for in situ aquatic sensor data. The code uses the Python pyhydroqc package to detect anomalies. This resource consists of 3 example notebooks and associated data files. For more information, see the original resource from which this was derived: http://www.hydroshare.org/resource/451c4f9697654b1682d87ee619cd7924.
Notebooks:
1. Example 1: Import and plot data
2. Example 2: Perform rules-based quality control
3. Example 3: Perform model-based quality control (ARIMA)
4. Example 4: Model-based quality control (ARIMA) with user data
Data files:
Data files are available for 6 aquatic sites in the Logan River Observatory. Each file contains data for one site for a single year. Each file corresponds to a single year of data. The files are named according to monitoring site (FranklinBasin, TonyGrove, WaterLab, MainStreet, Mendon, BlackSmithFork) and year. The files were sourced by querying the Logan River Observatory relational database, and equivalent data could be obtained from the LRO website or on HydroShare. Additional information on sites, variables, and methods can be found on the LRO website (http://lrodata.usu.edu/tsa/) or HydroShare (https://www.hydroshare.org/search/?q=logan%20river%20observatory). Each file has the same structure indexed with a datetime column (mountain standard time) with three columns corresponding to each variable. Variable abbreviations and units are:
- temp: water temperature, degrees C
- cond: specific conductance, μS/cm
- ph: pH, standard units
- do: dissolved oxygen, mg/L
- turb: turbidity, NTU
- stage: stage height, cm
For each variable, there are 3 columns:
- Raw data value measured by the sensor (column header is the variable abbreviation).
- Technician quality controlled (corrected) value (column header is the variable abbreviation appended with '_cor').
- Technician labels/qualifiers (column header is the variable abbreviation appended with '_qual').
There is also a file "data.csv" for use with Example 4. If any user wants to bring their own data file, they should structure it similarly to this file with a single column of datetime values and a single column of numeric observations labeled "raw".
Subject Keywords
Content
Related Resources
The content of this resource is derived from | Jones, A. S. (2022). Hydroinformatics Instruction Module Example Code: Sensor Data Quality Control with pyhydroqc, HydroShare, http://www.hydroshare.org/resource/451c4f9697654b1682d87ee619cd7924, accessed on: 04/08/2024 |
Credits
Funding Agencies
This resource was created using funding from the following sources:
Agency Name | Award Title | Award Number |
---|---|---|
National Science Foundation | Collaborative Research: Elements: Advancing Data Science and Analytics for Water (DSAW) | 1931297 |
Contributors
People or Organizations that contributed technically, materially, financially, or provided general support for the creation of the resource's content but are not considered authors.
Name | Organization | Address | Phone | Author Identifiers |
---|---|---|---|---|
Jeffery S. Horsburgh | Utah State University | Utah, US | 4357972946 | ORCID , ResearchGateID , GoogleScholarID |
Camilo J. Bastidas Pacheco | Utah State University | UT, US | 4357545722 |
How to Cite
This resource is shared under the Creative Commons Attribution CC BY.
http://creativecommons.org/licenses/by/4.0/
Comments
There are currently no comments
New Comment