Cleaning and Organizing Environmental Data

question:data-cleaning

_Lead image: Clean Data by Gene Stroman from the [Noun Project](https://thenounproject.com), CC BY_ After you’ve collected environmental data from a sensor, monitor, or other piece of equipment, one of the next steps is to organize and “clean” it! Cleaning includes making sure the dataset is complete and consistent. Organizing the data into a table in a meaningful way gets it ready for making charts, graphs, and other visualizations. Below are some resources on cleaning data, including making tables of tidy data. ### Making tables of tidy data _Images: Illustrations from the [Openscapes](https://www.openscapes.org/) blog “[Tidy Data for reproducibility, efficiency, and collaboration](https://www.openscapes.org/blog/2020/10/12/tidy-data/)” by Julia Lowndes and Allison Horst, [CC BY](http://creativecommons.org/licenses/by/4.0/)_ An example of “tidy data” from an air quality sensor might look like this: _**Each variable forms a column**_: sensor ID number, date, time, and the air quality measurement of particulate matter are individual variables. Each variable gets its own column in the table. The column header at the top lists the variable name and its units of measurement. _**Each observation forms a row**_: this sensor took an air quality measurement every minute. Each measurement gets its own row in the table. _**Each cell is a single measurement**_: each block in the table shows one piece of data---one time, one PM measurement, etc. ### Cleaning data _More to come here!_ ### Questions on organizing and cleaning data Questions tagged with `question:data-cleaning` will appear here [questions:data-cleaning] ### Activities Activity posts tagged with `activity:data-cleaning` will appear here [activities:data-cleaning] ### More resources on organizing and cleaning data + [Formatting Data Tables in Spreadsheets](https://datacarpentry.org/spreadsheets-socialsci/01-format-data/) and [OpenRefine for Data Cleaning](https://datacarpentry.org/openrefine-socialsci/): guidance and exercises from a [workshop session](https://marwahaha.github.io/2019-05-30-nas/) by Data Carpentry. + “[Clean Up Messy Data](https://handsondataviz.org/clean.html),” chapter 4 from the open-access web edition of _Hands-On Data Visualization: Interactive Storytelling from Spreadsheets to Code_, by Jack Dougherty and Ilya Ilyankou. + Wickham, H. 2014. **Tidy Data**. Journal of Statistical Software, 59(10), 1–23. [LINK to paper](https://doi.org/10.18637/jss.v059.i10)....