Validation of the incoming data
How do you know the quality of a water colour measurement?
If you want to use the data it is very important to know the quality of the measurements. Therefore, in the data interpretation and validation phase the collected data are compared to the results of satellite observations and existing knowledge of the waters, be it in the form of maps or results of long-term sampling at fixed stations. The metadata (where/how/what is measured) of Citclops data allow for this comparison.
At present only validation of colour data has been the focus (since this will be the main data source in the project), validation options for fluorenscence and Kduino sensor data may be added later.
Stage 1 Validation of water colour data
The data provided by the user will be transferred to a central data base. Every entry now consists of an image, together with a set of metadata. These metadata can be divided in:
- Boolian information (e.g use of Secchi Disk yes or no)
- Parameter values (eg. Solar Zenith Angle in degrees)
The first step (either called validation, pre-processing or QC procedure) is to perform an in- depth analysis of the image. The ROI identified by the observer is analysed in more detail: the RGB colours are corrected for illumination conditions and the colour-angle in the x,y chromaticity diagram is calculated for each pixel.
Stage 2 Validation of water colour data
The data provided by the user can be compared to previous measurements conducted in the same area by other users, or to GIS layers and satellite FU index estimated for the area. A great difference in number can show that the measurement is incorrect, and can therefore be flagged.
Procedures must be developed and tested to compare the different sources of information that are collected by different observers, different means (satellite, Smart phone,..) at different places and different times.
Procedures must be developed and tested to check if certain parameters (like clouds, wind) or inherent properties of the image (like large standard deviation in the colour angle) are a good indicator of the quality.
Stage 3 Validation against remote sensing data
NOVELTIS will be responsible for putting in place a module for the verification that certain incoming measurements are coherent with values observed in the past. In the initial instance, the measurements expected to be verified are the Forel-Ule colour readings obtained by the mobile application.
The module will take an incoming Forel-Ule colour reading, along with the date and the latitude/longitude and compares the reading with historic values obtained from Satellite imagery.
The historic values will be stored in a raster database.
Verification will be performed by the comparison of the incoming value against the aggregated values for the corresponding time of year and geographic location, resulting in a confidence score for the value. At this stage the exact algorithm to be used is not finalised, however it is expected to involve a statistical analysis of relevant historical values to determine the 'fittedness' of the value in question.
The resulting confidence score will be attached to the original data point and used by the wider decision support system as necessary, for example the system may choose not to publish a point if it is deemed to be an anomaly.