Converting data to information: Coupling lab-level database functionality with primary LTER data archiving systems
Adam M. Kennedy, Suzanne M. Remillard, Donald L. Henshaw, Lawrence A. Duncan, Barbara J. Bond
Last modified: 2008-08-21
Abstract
Developing and operating a data management program to support dynamic terrestrial and aquatic sensor networks is challenging. The database architecture needs to be robust and extensible, and must maintain flexibility in response to frequent changes in sensor array configurations in the field. The objective of this paper is to describe a database application developed for the Forest Ecophysiology and Ecohydrology Laboratory (FEEL) research program at the Andrews Experimental Forest in Oregon, USA. We discuss the fundamentals of a lab-level, web-based, and open source database application, and summarize the database architecture, methods of user-entered metadata, generation and storage of data mappings that provide the flexibility to handle changes in the incoming raw data streams, and methods to couple the lab-level database tables to the archival-level tables for seamless data flow and scheduled updating. This web-based database application enables small labs to handle large and streaming sensor arrays locally. The architecture is flexible and can adjust on-the-fly to changes in data file and field configurations. We detail a robust, user-friendly, and open source database environment that permits metadata generation and handling, low-level sensor tracking, dynamic data streams, general data processing, basic visualization, user-defined queries, and data routing to the primary long-term data repository.