Reputation System for Crowdsourced Rainfall Networks (RSCRN)


Authors:
Owners: This resource does not have an owner who is an active HydroShare user. Contact CUAHSI (help@cuahsi.org) for information on this resource.
Resource type: Composite Resource
Storage: The size of this resource is 467.7 MB
Created: Feb 28, 2020 at 6:08 p.m.
Last updated: Oct 27, 2021 at 1:51 a.m.
Citation: See how to cite this resource
Sharing Status: Public
Views: 704
Downloads: 85
+1 Votes: Be the first one to 
 this.
Comments: No comments (yet)

Abstract

Dataset and codes used in the manuscript submitted to Water Resources Research "Assessing the Trustworthiness of Crowdsourced Rainfall Networks: A Reputation System Approach" (2021WR029721)

Subject Keywords

Deleting all keywords will set the resource sharing status to private.

Resource Level Coverage

Spatial

Coordinate System/Geographic Projection:
WGS 84 EPSG:4326
Coordinate Units:
Decimal degrees
Place/Area Name:
Houston
North Latitude
30.1291°
East Longitude
-94.9701°
South Latitude
29.4593°
West Longitude
-95.8435°

Content

readme.txt

###

This repository contains codes to generate results for manuscript submitted to Water Resources Research 
"Assessing the Trustworthiness of Crowdsourced Rainfall Networks: A Reputation System Approach" (2021WR029721)

###

1. Input data

# RSCRN input
0_KTXHOUST_meta.csv: metadata of PWSs
1_KTXHOUST_15min_rainfall.csv: 15 minute PWS rainfall of PWSs
2_HCFCD_15min.csv: 15 minute Harris County Flood Control District (HCFCD) rainfall data
3_HCFCD_meta: metadata of HCFCD rainfall stations

# PWS QC method input (generated from R code)
c1_HI_flags.csv: high influx flags for PWSs in cluster 1
c1_FZ_flags.csv: faulty zero flags for PWSs cluster 1
c1_SO_flags.csv: station outlier flags for PWSs in cluster 1
c2_HI_flags.csv: high influx flags for PWSs in cluster 2 
c2_FZ_flags.csv: faulty zero flags for PWSs in cluster 2
c2_SO_flags.csv: station outlier flags for PWSs in cluster 2


2. Code description

Python codes were written in Python 3.x version. Please follow the order because each step uses the output from previous steps.
This code will generate results for cluster 1. To generate results for cluster 2, change every "c1" to "c2" and "445" to "560".

0_cluster.py:

Input: PWS metadata (0_KTXHOUST_meta.csv) and PWS 15 min rainfall (1_KTXHOUST_15min_rainfall.csv)
Output: Clustered sub-dataset (ID_list_c1.csv)

1_consensus_and_score.py:
Input: PWS 15 min rainfall (1_KTXHOUST_15min_rainfall.csv)
Input 2: Clustered subdataset (ID_list_c1.csv)

Output: RSCRN trust score, alpha and beta parameters, weight, robust weight
(RSCRN_var_ff_new_string.csv, var = score, alpha, beta, weight, ff = forgetting factor)

2_find_storm_event.py
Input: HCFCD rainfall (2_HCFCD_15min.csv) and metadata (3_HCFCD_meta.csv)
Output: analyzed storm events (df_storm_event.csv)

3_plot_trust_score_evolution.py:
Input 1: storm event (df_storm_event_st.csv) (for cluster 1 and 2, st = 445 and 560 respectively) 
Input 2: Clustered subdataset (ID_list_c1.csv)
Output: trust score evolution for analyzed storm events (trust_score_evolution.png)

4_comparison_with_PWS_QC.py:
'''
Input 1: PWS rainfall (1_KTXHOUST_15min_rainfall.csv), 
         PWS QC results (HI_flag, FZ_flag, SO_flags.csv)
Input 2: Clustered subdataset (ID_list_c1.csv)
Output 1: Overall assessment of RSCRN and PWS QC methods (df_summary.csv)
Output 2: RSCRN trust scores and PWS QC method assessment for each storm event and each PWS (e.g., Figure 4 in the manuscript)

5_RMSE_comparison.py:
Input 1: PWS rainfall (1_KTXHOUST_15min_rainfall.csv), 
Input 2: HCFCD rainfall (2_HCFCD_15min.csv) and metadata (3_HCFCD_meta.csv)
Input 3: PWS QC results (HI_flag, FZ_flag, SO_flags.csv)
Output : RMSE comparison table (df_RMSE_storm_comparison.csv) (Table 5 and 6 in the manuscript)


### PWS QC method ###
This study compares the RSCRN trust scores with an a PWS Quality Control method (de Vos et al., 2019).
The R codes used to generate the results in this study can be found in the folder "PWS_QC_method".
These codes were modified from the original codes to run PWS data for the case study in this paper. 
Original codes can be found from https://github.com/LottedeVos/PWSQC
For details about the PWS QC method please refer to the original manuscript at https://agupubs.onlinelibrary.wiley.com/doi/full/10.1029/2019GL083731

How to Cite

Chen, A. (2021). Reputation System for Crowdsourced Rainfall Networks (RSCRN), HydroShare, http://www.hydroshare.org/resource/cf7796cdeace42818dbbd7f95f8e1872

This resource is shared under the Creative Commons Attribution CC BY.

http://creativecommons.org/licenses/by/4.0/
CC-BY

Comments

There are currently no comments

New Comment

required