===================================================================
Dry weather flow (DWF) in separate stormwater drainage systems,
Hong Kong: grab sampling and continuous water quality and flow
monitoring, 2024-2026
===================================================================

Version 1.0

This dataset contains multi-parameter dry weather flow (DWF)
monitoring and source sampling data collected from separate stormwater
drainage systems in Hong Kong, comprising field grab sampling, continuous
monitoring, sub-catchment characteristics, and flow
calibration information. The related publication is currently under
preparation.



DIRECTORY OVERVIEW
-------------------------------------------------------------------
Read 00_metadata first; it describes every parameter, method, and sampling
event referenced by the data files.

Folders 00-04 each contain a description.txt giving a detailed,
self-contained guide to that folder: file-by-file contents, column
definitions, units, and reading notes. This README is the overview;
the description.txt files are the reference.

00_metadata/            Data dictionaries and campaign metadata
  description.txt           Detailed guide to this folder
  Sampling_Locations.csv    Sampling and monitoring point details
  Parameter_coverage.csv    Which parameters were measured at each sample
                            type / campaign
  Field_instruments.csv     On-site instruments and methods
  Lab_methods_LOR.csv       Laboratory methods and limits of reporting
  Sonde_sensors.csv         Continuous-monitoring sensors
  Rain_gauge_info.txt       Rain gauge model, resolution, and location
  Cleaning_Steps.csv        Continuous-data cleaning workflow
  Campaign_Timeline.csv     Timeline of sampling and monitoring events
  Known_issues.csv          Documented data gaps, sensor faults, and other
                            known issues affecting specific sites/periods

01_grab_sampling/       Manual grab sampling water-quality data
  description.txt               Detailed guide to this folder
  Grab_samples_outlet.csv       Outlet sampling results
  Grab_samples_source.csv       Potential DWF source sampling results
  Grab_samples_channel.csv      Open-channel sampling results
  Grab_summary_statistics.csv   Outlet and channel sampling results statistics

02_continuous_monitoring/   Continuous monitoring data (1-minute interval)
  description.txt         Detailed guide to this folder
  Raw/                    Fully original sensor records, before any correction.
                          'P1'/'P2' in file name indicates phase 1 and phase 2.
                          The outlet files contain two flow rate columns: the
                          first gives the flow rate as computed by the sensor's
                          native formula in its original unit, and the second
                          gives the same flow rate converted to m3/h.
  Cleaned/                Final series after the complete cleaning workflow
                          listed in 00_metadata/Cleaning_Steps.csv
  Rainfall/               Rainfall records for Phase 1 and Phase 2
                          (gauge details in 00_metadata/Rain_gauge_info.txt)
  Cont_raw_statistics.csv   Coverage and descriptive statistics of raw sensor data

03_site_description/    Study area and drainage network background
  Site_description.pdf          Site overview, maps, sub-catchment
                                characteristics and network schematic
  Pipe_network_attributes.csv   Pipe network physical attributes
  Subcatchment_landuse.csv      Sub-catchment area, land use, and imperviousness

04_flow_calibration/    Flow-rate calibration for the two monitoring phases
  description.txt               Detailed guide to this folder
  Weir_box_property.csv         Phase 1 weir box configuration
  Weir_stage_discharge.csv      Phase 1 weir box stage-discharge data
  Flume_property.csv            Phase 2 Parshall flume configuration and
                                rating equation
  Flume_water_level.pdf         Phase 2 calibration and water-level
                                derivation of Parshall flume

05_code/                Data processing code
  Clean_continuous.py           Automated cleaning script (Steps 6-8, plus
                                Step 2 at the channel site CU)
  Cleaning_README.txt           Cleaning workflow and reproduction notes
  Requirements.txt              Python dependencies


SITE, PHASE AND CAMPAIGN CODES
-------------------------------------------------------------------
  OA, OB, OC, OD   Drainage outlets (Outlets A-D in prose)
  CU, CD           Upstream and downstream open-channel sites
  P1               Phase 1 continuous monitoring (18 Feb - 27 Mar 2025)
  P2               Phase 2 continuous monitoring (Nov 2025 - Mar 2026)
  C1, C2           First (25 Oct 2024) and second (10 Mar 2026) outlet
                   grab-sampling campaigns

Continuous monitoring covered Outlet C in Phase 1, and Outlets A-D and the
upstream channel CU in Phase 2.


CONVENTIONS
-------------------------------------------------------------------
- Time: local time, UTC+08:00.
- Dates: year-month-day (YYYY-MM-DD).
- Coordinates: relative coordinate system.
- Values below the laboratory Limit of Reporting are written with a
  leading "<" (e.g. "<5"); the LORs are listed in
  00_metadata/Lab_methods_LOR.csv.
- Two "EC" definitions are used: grab sampling reports specific conductivity
  at 25 C (Ultrameter II), while continuous monitoring reports actual
  conductivity at in-situ temperature (Aqua TROLL 500). See 00_metadata.


DATA CLEANING
-------------------------------------------------------------------
Continuous data were cleaned through a combined manual and automated
workflow (00_metadata/Cleaning_Steps.csv). Manual steps (removal
of sensor faults, drift correction, flow rescaling) were applied to the raw
data by expert inspection; automated steps (hard thresholds, IQR outlier
removal, short-gap interpolation) are implemented in 05_code. The raw and
cleaned series are both provided so users can assess the net effect of the
workflow. See 05_code/Cleaning_README.txt for details.


DISCLAIMER
-------------------------------------------------------------------
Any opinions, findings, conclusions or recommendations expressed in
this material/event (or by members of the project team) do not reflect
the views of the Government of the Hong Kong Special Administrative
Region, the Innovation and Technology Commission or the Innovation and
Technology Fund Research Projects Assessment Panel.