Supporting data and tools for "Tools for Evaluating, Developing, and Testing Water End Use Disaggregation Algorithms"


Authors:
Owners:
Resource type: Composite Resource
Storage: The size of this resource is 142.3 MB
Created: May 24, 2021 at 6:44 p.m.
Last updated: Jun 10, 2021 at 1:15 a.m.
Citation: See how to cite this resource
Sharing Status: Public
Views: 136
Downloads: 16
+1 Votes: Be the first one to 
 this.
Comments: No comments (yet)

Abstract

The files provided here are the supporting data and code files for the analyses presented in "Tools for Evaluating, Developing, and Testing Water End Use Disaggregation Algorithms," a manuscript submitted to the Journal of Water Resource Planning and Management. The data included in this resource were collected using the CIWS-Logger (https://github.com/UCHIC/CIWS-WM-Logger) data logging device. Cyberinfrastructure for Intelligent Water Supply (CIWS) is an open-source, modular, generalized architecture designed to automate the process from data collection to analysis and presentation of high temporal residential water use data. The CIWS-Logger is a low cost device capable of collecting this type of data on existing, magnetically driven water meters. The code included in this resource (CIWS-Disaggregator) demonstrates a new water end use disaggregation and classification tool that builds on existing end use disaggregation studies and addresses the unavailability of code and data used by prior studies. The tool was developed in Python and can be accessed via any current Python programming environment. It was tested on anonymized, high temporal resolution datasets for five homes selected from a larger dataset for 31 homes located in the Cities of Logan and Providence Utah, USA. Results from different meter types and sizes are presented to demonstrate the accuracy of the tool in disaggregating and classifying high temporal resolution data into individual end use events. The results of this paper are reproducible using openly available code and data, representing an accessible platform for advancing end use disaggregation tools. The tool can be adapted to specific research needs.

Subject Keywords

Deleting all keywords will set the resource sharing status to private.

Resource Level Coverage

Spatial

Coordinate System/Geographic Projection:
WGS 84 EPSG:4326
Coordinate Units:
Decimal degrees
Place/Area Name:
Logan and Providence, Utah, USA
North Latitude
41.7666°
East Longitude
-111.7709°
South Latitude
41.6857°
West Longitude
-111.8650°

Temporal

Start Date:
End Date:

Content

readme.md

Folders in this resource are organized as follows:

1. CIWS_Disaggregator: contains the semi-supervised water end use disaggregation and classification machine learning algorithm used to generate the results in the paper. The folder has three sub-folders, and three files:

  • RawData Folder contains the 4 second temporal resolution water use data collected at each of the 5 houses included in the study for a period of at least two weeks during the summer when outdoor water use was active and two weeks during the winter when no outdoor water use was expected.

  • Results Folder contains the disaggregated and classified water end use events extracted from the high resolution water use data for the 5 residential homes included in the study.

  • Events Folder is an empty folder. Once a user runs the CIWS-Disaggregator code, all of its outputs will be saved automatically in the Events folder.

  • Sites_ResidentialStudy_Info.csv is a .csv file that contains the metadata of the 5 houses included in the study, including the meter size, meter resoultion, number of residents, and irrigation type (sprinkler/hose).

  • TrainingDataset_SS.csv is a .csv file that contains manually labeled water end use events collected by one participating household.

  • CIWS_Disaggregator_SS.py is the python script module used for water end use disaggregation. All water end use disaggregation functions and classes are defined in the module including the filtering, disaggregation, classification, and label assigning functions.

2. Plots: contains the Python and R scripts used to generate figures presented in the Results section of the article. The folder also contains a file All_Events_L.csv that has classified events from all 5 households.

Instructions for reproducing results

Complete the following steps to reproduce the classified events results presented in the article:

  1. Download the complete CIWS_Disaggregator folder.

  2. Leave the files together in the folders to ensure the paths to the files remain correct.

  3. Open the CIWS_Disaggregator_SS_FV.ipynb Jupyter Notebook and execute it.

  4. The Outputs of the tool are .csv files named LabelledEvents_seasonHHID that contain the water end use events extracted from a seasonHHID input file. The results should match the results in the Results folder.

Complete the following steps to reproduce the figures:

  1. Download the complete Plots folder

  2. Leave the files together in the folders to ensure the paths to the files remain correct.

  3. Open the Plots.ipynb Jupyter Notebook and execute it. The Python Juypter Notebook can be used to reproduce Plots 8, 9 and all plots in the Appendix.

Required code libraries

The Python code provided in this resource was developed using Python 3.7.3. The following Python packages are required for running the provided scripts:

  • pandas - Version 1.2.3.
  • matplotlib - Version 3.0.3.
  • seaborn - Version 0.9.0.
  • numpy - Version 1.20.3.
  • sklearn - Version 0.20.3.
  • scipy - Version 1.2-1.

Credits

Funding Agencies

This resource was created using funding from the following sources:
Agency Name Award Title Award Number
National Science Foundation CAREER: Cyberinfrastructure for Intelligent Water Supply (CIWS): Shrinking Big Data for Sustainable Urban Water CBET 1552444

How to Cite

Attallah, N., C. J. Bastidas Pacheco (2021). Supporting data and tools for "Tools for Evaluating, Developing, and Testing Water End Use Disaggregation Algorithms", HydroShare, http://www.hydroshare.org/resource/3143b3b1bdff48e0aaebcb4aedf02feb

This resource is shared under the Creative Commons Attribution CC BY.

 http://creativecommons.org/licenses/by/4.0/
CC-BY

Comments

There are currently no comments

New Comment

required