nmelone/Weather-Project
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
7b30c71b5331eddcb8ab0f7ff51d39af3b8cd560
Weather-Project/Project Final/Preprocessing/preprocess_data.ipynb
nmelone 7b30c71b53 Reconfigured folder structure 
Added Evaluation, Preprocessing, Training, Transformation folders.
Preprocessing is just a rework of the folder for the new structure of the old preprocessing folder.

Training and Transformation are the old project file broken up into two parts and restructured.

Evaluation is for evaluating the predictive power of the model.
2018-09-24 20:48:07 -05:00

2107 lines
85 KiB
Plaintext
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Preprocess the raw data from NOAA\n",
"This notebook is setup to take in the CSV from NOAA and remove the unneccasary data. This will also seperate out each station for later positioning."
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
"import pandas as pd\n",
"import numpy as np\n",
"import os\n",
"from tqdm import tqdm"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [],
"source": [
"#Getting a list of files in raw data folder\n",
"filenames = os.listdir('./full_grid')"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [],
"source": [
"header_wanted = [\n",
" 'HOURLYVISIBILITY',\n",
" 'HOURLYDRYBULBTEMPC',\n",
" 'HOURLYWETBULBTEMPC',\n",
" 'HOURLYDewPointTempC',\n",
" 'HOURLYRelativeHumidity',\n",
" 'HOURLYWindSpeed',\n",
" 'HOURLYWindGustSpeed',\n",
" 'HOURLYStationPressure',\n",
" 'HOURLYPressureTendency',\n",
" 'HOURLYPressureChange',\n",
" 'HOURLYSeaLevelPressure',\n",
" 'HOURLYPrecip',\n",
" 'HOURLYAltimeterSetting']"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [],
"source": [
"usecols = ['DATE','STATION'] + header_wanted"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|██████████████████████████████████████████████████████████████████████████████████| 82/82 [02:07<00:00, 1.55s/it]\n"
]
}
],
"source": [
"#Loading all files into a pandas Dataframe\n",
"tqdm.pandas()\n",
"df = pd.concat([pd.read_csv('./full_grid/{}'.format(x), usecols=usecols, low_memory=False) for x in tqdm(filenames)])"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"406"
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"#Getting the station names\n",
"stations = pd.read_csv(\"./stations_unique.csv\", usecols = ['STATION_ID','LAT_SCALED', 'LON_SCALED'])\n",
"wban_list = stations['STATION_ID'].tolist()\n",
"len(wban_list)"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [],
"source": [
"#removes any letters or oddities that appear with numbers for any of the data\n",
"def remove_letters(headers,dataframes):\n",
" for i in tqdm(headers):\n",
" dataframes[i].replace(regex=True,inplace=True,to_replace=r'\\D',value=r'')\n",
" dataframes[i] = dataframes[i].apply(pd.to_numeric)"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|███████████████████████████████████████████████████████████████████████████████| 13/13 [1:18:33<00:00, 362.54s/it]\n"
]
}
],
"source": [
"remove_letters(header_wanted,df)"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|████████████████████████████████████████████████████████████████████████████████| 406/406 [10:29<00:00, 1.55s/it]\n"
]
}
],
"source": [
"by_station_list = []\n",
"# Divides dataframe into smaller dataframes of a single WBAN and stores in a list in order of wban_list\n",
"for i in tqdm(wban_list):\n",
" by_station_list.append(df.loc[df.STATION == i])\n",
"del df"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"<class 'pandas.core.frame.DataFrame'>\n",
"Int64Index: 46719 entries, 109127 to 155845\n",
"Data columns (total 15 columns):\n",
"STATION 46719 non-null object\n",
"DATE 46719 non-null object\n",
"HOURLYVISIBILITY 43438 non-null float64\n",
"HOURLYDRYBULBTEMPC 46196 non-null float64\n",
"HOURLYWETBULBTEMPC 44790 non-null float64\n",
"HOURLYDewPointTempC 46195 non-null float64\n",
"HOURLYRelativeHumidity 46195 non-null float64\n",
"HOURLYWindSpeed 44852 non-null float64\n",
"HOURLYWindGustSpeed 6657 non-null float64\n",
"HOURLYStationPressure 44796 non-null float64\n",
"HOURLYPressureTendency 0 non-null float64\n",
"HOURLYPressureChange 0 non-null float64\n",
"HOURLYSeaLevelPressure 0 non-null float64\n",
"HOURLYPrecip 2096 non-null float64\n",
"HOURLYAltimeterSetting 46197 non-null float64\n",
"dtypes: float64(13), object(2)\n",
"memory usage: 5.7+ MB\n"
]
}
],
"source": [
"by_station_list[0].info()"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|████████████████████████████████████████████████████████████████████████████████| 406/406 [00:21<00:00, 18.75it/s]\n"
]
}
],
"source": [
"#Sets the date info as the index\n",
"for i in tqdm(range(len(by_station_list))):\n",
" by_station_list[i]['STATION_ID'] = by_station_list[i]['STATION']\n",
" by_station_list[i] = by_station_list[i].set_index(pd.DatetimeIndex(by_station_list[i]['DATE']))"
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>STATION</th>\n",
" <th>DATE</th>\n",
" <th>HOURLYVISIBILITY</th>\n",
" <th>HOURLYDRYBULBTEMPC</th>\n",
" <th>HOURLYWETBULBTEMPC</th>\n",
" <th>HOURLYDewPointTempC</th>\n",
" <th>HOURLYRelativeHumidity</th>\n",
" <th>HOURLYWindSpeed</th>\n",
" <th>HOURLYWindGustSpeed</th>\n",
" <th>HOURLYStationPressure</th>\n",
" <th>HOURLYPressureTendency</th>\n",
" <th>HOURLYPressureChange</th>\n",
" <th>HOURLYSeaLevelPressure</th>\n",
" <th>HOURLYPrecip</th>\n",
" <th>HOURLYAltimeterSetting</th>\n",
" <th>STATION_ID</th>\n",
" </tr>\n",
" <tr>\n",
" <th>DATE</th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>2016-08-01 00:15:00</th>\n",
" <td>WBAN:00184</td>\n",
" <td>2016-08-01 00:15</td>\n",
" <td>1000.0</td>\n",
" <td>245.0</td>\n",
" <td>24.4</td>\n",
" <td>245.0</td>\n",
" <td>100.0</td>\n",
" <td>0.0</td>\n",
" <td>NaN</td>\n",
" <td>2999.0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>3004.0</td>\n",
" <td>WBAN:00184</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2016-08-01 00:35:00</th>\n",
" <td>WBAN:00184</td>\n",
" <td>2016-08-01 00:35</td>\n",
" <td>175.0</td>\n",
" <td>242.0</td>\n",
" <td>24.4</td>\n",
" <td>242.0</td>\n",
" <td>100.0</td>\n",
" <td>0.0</td>\n",
" <td>NaN</td>\n",
" <td>2999.0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>3004.0</td>\n",
" <td>WBAN:00184</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2016-08-01 00:55:00</th>\n",
" <td>WBAN:00184</td>\n",
" <td>2016-08-01 00:55</td>\n",
" <td>400.0</td>\n",
" <td>242.0</td>\n",
" <td>24.4</td>\n",
" <td>242.0</td>\n",
" <td>100.0</td>\n",
" <td>0.0</td>\n",
" <td>NaN</td>\n",
" <td>2999.0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>3004.0</td>\n",
" <td>WBAN:00184</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2016-08-01 01:15:00</th>\n",
" <td>WBAN:00184</td>\n",
" <td>2016-08-01 01:15</td>\n",
" <td>100.0</td>\n",
" <td>240.0</td>\n",
" <td>23.9</td>\n",
" <td>240.0</td>\n",
" <td>100.0</td>\n",
" <td>0.0</td>\n",
" <td>NaN</td>\n",
" <td>2999.0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>3004.0</td>\n",
" <td>WBAN:00184</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2016-08-01 01:35:00</th>\n",
" <td>WBAN:00184</td>\n",
" <td>2016-08-01 01:35</td>\n",
" <td>500.0</td>\n",
" <td>240.0</td>\n",
" <td>23.9</td>\n",
" <td>240.0</td>\n",
" <td>100.0</td>\n",
" <td>0.0</td>\n",
" <td>NaN</td>\n",
" <td>3000.0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>3005.0</td>\n",
" <td>WBAN:00184</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2016-08-01 01:55:00</th>\n",
" <td>WBAN:00184</td>\n",
" <td>2016-08-01 01:55</td>\n",
" <td>700.0</td>\n",
" <td>241.0</td>\n",
" <td>23.9</td>\n",
" <td>241.0</td>\n",
" <td>100.0</td>\n",
" <td>0.0</td>\n",
" <td>NaN</td>\n",
" <td>2999.0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>3004.0</td>\n",
" <td>WBAN:00184</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2016-08-01 02:15:00</th>\n",
" <td>WBAN:00184</td>\n",
" <td>2016-08-01 02:15</td>\n",
" <td>700.0</td>\n",
" <td>241.0</td>\n",
" <td>23.9</td>\n",
" <td>241.0</td>\n",
" <td>100.0</td>\n",
" <td>0.0</td>\n",
" <td>NaN</td>\n",
" <td>2999.0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>3004.0</td>\n",
" <td>WBAN:00184</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2016-08-01 02:35:00</th>\n",
" <td>WBAN:00184</td>\n",
" <td>2016-08-01 02:35</td>\n",
" <td>200.0</td>\n",
" <td>236.0</td>\n",
" <td>23.3</td>\n",
" <td>236.0</td>\n",
" <td>100.0</td>\n",
" <td>0.0</td>\n",
" <td>NaN</td>\n",
" <td>2998.0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>3003.0</td>\n",
" <td>WBAN:00184</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2016-08-01 02:55:00</th>\n",
" <td>WBAN:00184</td>\n",
" <td>2016-08-01 02:55</td>\n",
" <td>50.0</td>\n",
" <td>235.0</td>\n",
" <td>23.3</td>\n",
" <td>235.0</td>\n",
" <td>100.0</td>\n",
" <td>0.0</td>\n",
" <td>NaN</td>\n",
" <td>2999.0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>3004.0</td>\n",
" <td>WBAN:00184</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2016-08-01 03:15:00</th>\n",
" <td>WBAN:00184</td>\n",
" <td>2016-08-01 03:15</td>\n",
" <td>150.0</td>\n",
" <td>237.0</td>\n",
" <td>23.9</td>\n",
" <td>237.0</td>\n",
" <td>100.0</td>\n",
" <td>0.0</td>\n",
" <td>NaN</td>\n",
" <td>2999.0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>3004.0</td>\n",
" <td>WBAN:00184</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2016-08-01 03:35:00</th>\n",
" <td>WBAN:00184</td>\n",
" <td>2016-08-01 03:35</td>\n",
" <td>50.0</td>\n",
" <td>238.0</td>\n",
" <td>23.9</td>\n",
" <td>238.0</td>\n",
" <td>100.0</td>\n",
" <td>0.0</td>\n",
" <td>NaN</td>\n",
" <td>2998.0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>3003.0</td>\n",
" <td>WBAN:00184</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2016-08-01 03:55:00</th>\n",
" <td>WBAN:00184</td>\n",
" <td>2016-08-01 03:55</td>\n",
" <td>75.0</td>\n",
" <td>237.0</td>\n",
" <td>23.9</td>\n",
" <td>237.0</td>\n",
" <td>100.0</td>\n",
" <td>0.0</td>\n",
" <td>NaN</td>\n",
" <td>2998.0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>3003.0</td>\n",
" <td>WBAN:00184</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2016-08-01 04:15:00</th>\n",
" <td>WBAN:00184</td>\n",
" <td>2016-08-01 04:15</td>\n",
" <td>700.0</td>\n",
" <td>233.0</td>\n",
" <td>23.3</td>\n",
" <td>233.0</td>\n",
" <td>100.0</td>\n",
" <td>0.0</td>\n",
" <td>NaN</td>\n",
" <td>2999.0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>3004.0</td>\n",
" <td>WBAN:00184</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2016-08-01 04:35:00</th>\n",
" <td>WBAN:00184</td>\n",
" <td>2016-08-01 04:35</td>\n",
" <td>500.0</td>\n",
" <td>234.0</td>\n",
" <td>23.3</td>\n",
" <td>234.0</td>\n",
" <td>100.0</td>\n",
" <td>0.0</td>\n",
" <td>NaN</td>\n",
" <td>3000.0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>3005.0</td>\n",
" <td>WBAN:00184</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2016-08-01 04:55:00</th>\n",
" <td>WBAN:00184</td>\n",
" <td>2016-08-01 04:55</td>\n",
" <td>1000.0</td>\n",
" <td>240.0</td>\n",
" <td>23.9</td>\n",
" <td>240.0</td>\n",
" <td>100.0</td>\n",
" <td>0.0</td>\n",
" <td>NaN</td>\n",
" <td>2999.0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>3004.0</td>\n",
" <td>WBAN:00184</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2016-08-01 05:15:00</th>\n",
" <td>WBAN:00184</td>\n",
" <td>2016-08-01 05:15</td>\n",
" <td>700.0</td>\n",
" <td>237.0</td>\n",
" <td>23.9</td>\n",
" <td>237.0</td>\n",
" <td>100.0</td>\n",
" <td>0.0</td>\n",
" <td>NaN</td>\n",
" <td>3000.0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>3005.0</td>\n",
" <td>WBAN:00184</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2016-08-01 05:35:00</th>\n",
" <td>WBAN:00184</td>\n",
" <td>2016-08-01 05:35</td>\n",
" <td>500.0</td>\n",
" <td>237.0</td>\n",
" <td>23.9</td>\n",
" <td>237.0</td>\n",
" <td>100.0</td>\n",
" <td>0.0</td>\n",
" <td>NaN</td>\n",
" <td>3001.0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>3006.0</td>\n",
" <td>WBAN:00184</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2016-08-01 05:55:00</th>\n",
" <td>WBAN:00184</td>\n",
" <td>2016-08-01 05:55</td>\n",
" <td>700.0</td>\n",
" <td>239.0</td>\n",
" <td>23.9</td>\n",
" <td>239.0</td>\n",
" <td>100.0</td>\n",
" <td>0.0</td>\n",
" <td>NaN</td>\n",
" <td>3000.0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>3005.0</td>\n",
" <td>WBAN:00184</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2016-08-01 06:15:00</th>\n",
" <td>WBAN:00184</td>\n",
" <td>2016-08-01 06:15</td>\n",
" <td>700.0</td>\n",
" <td>245.0</td>\n",
" <td>24.4</td>\n",
" <td>245.0</td>\n",
" <td>100.0</td>\n",
" <td>0.0</td>\n",
" <td>NaN</td>\n",
" <td>3001.0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>3006.0</td>\n",
" <td>WBAN:00184</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2016-08-01 06:35:00</th>\n",
" <td>WBAN:00184</td>\n",
" <td>2016-08-01 06:35</td>\n",
" <td>1000.0</td>\n",
" <td>249.0</td>\n",
" <td>25.0</td>\n",
" <td>249.0</td>\n",
" <td>100.0</td>\n",
" <td>0.0</td>\n",
" <td>NaN</td>\n",
" <td>3002.0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>3007.0</td>\n",
" <td>WBAN:00184</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2016-08-01 06:55:00</th>\n",
" <td>WBAN:00184</td>\n",
" <td>2016-08-01 06:55</td>\n",
" <td>1000.0</td>\n",
" <td>253.0</td>\n",
" <td>25.6</td>\n",
" <td>253.0</td>\n",
" <td>100.0</td>\n",
" <td>0.0</td>\n",
" <td>NaN</td>\n",
" <td>3002.0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>3007.0</td>\n",
" <td>WBAN:00184</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2016-08-01 07:15:00</th>\n",
" <td>WBAN:00184</td>\n",
" <td>2016-08-01 07:15</td>\n",
" <td>1000.0</td>\n",
" <td>256.0</td>\n",
" <td>25.6</td>\n",
" <td>256.0</td>\n",
" <td>100.0</td>\n",
" <td>0.0</td>\n",
" <td>NaN</td>\n",
" <td>3003.0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>3008.0</td>\n",
" <td>WBAN:00184</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2016-08-01 07:35:00</th>\n",
" <td>WBAN:00184</td>\n",
" <td>2016-08-01 07:35</td>\n",
" <td>1000.0</td>\n",
" <td>265.0</td>\n",
" <td>26.7</td>\n",
" <td>265.0</td>\n",
" <td>100.0</td>\n",
" <td>0.0</td>\n",
" <td>NaN</td>\n",
" <td>3004.0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>3009.0</td>\n",
" <td>WBAN:00184</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2016-08-01 07:55:00</th>\n",
" <td>WBAN:00184</td>\n",
" <td>2016-08-01 07:55</td>\n",
" <td>1000.0</td>\n",
" <td>271.0</td>\n",
" <td>26.8</td>\n",
" <td>267.0</td>\n",
" <td>98.0</td>\n",
" <td>0.0</td>\n",
" <td>NaN</td>\n",
" <td>3004.0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>3009.0</td>\n",
" <td>WBAN:00184</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2016-08-01 08:15:00</th>\n",
" <td>WBAN:00184</td>\n",
" <td>2016-08-01 08:15</td>\n",
" <td>1000.0</td>\n",
" <td>276.0</td>\n",
" <td>27.0</td>\n",
" <td>269.0</td>\n",
" <td>96.0</td>\n",
" <td>0.0</td>\n",
" <td>NaN</td>\n",
" <td>3004.0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>3009.0</td>\n",
" <td>WBAN:00184</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2016-08-01 08:35:00</th>\n",
" <td>WBAN:00184</td>\n",
" <td>2016-08-01 08:35</td>\n",
" <td>1000.0</td>\n",
" <td>283.0</td>\n",
" <td>27.1</td>\n",
" <td>264.0</td>\n",
" <td>89.0</td>\n",
" <td>0.0</td>\n",
" <td>NaN</td>\n",
" <td>3004.0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>3009.0</td>\n",
" <td>WBAN:00184</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2016-08-01 08:55:00</th>\n",
" <td>WBAN:00184</td>\n",
" <td>2016-08-01 08:55</td>\n",
" <td>1000.0</td>\n",
" <td>290.0</td>\n",
" <td>27.3</td>\n",
" <td>269.0</td>\n",
" <td>88.0</td>\n",
" <td>5.0</td>\n",
" <td>NaN</td>\n",
" <td>3004.0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>3009.0</td>\n",
" <td>WBAN:00184</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2016-08-01 09:15:00</th>\n",
" <td>WBAN:00184</td>\n",
" <td>2016-08-01 09:15</td>\n",
" <td>1000.0</td>\n",
" <td>292.0</td>\n",
" <td>27.4</td>\n",
" <td>268.0</td>\n",
" <td>87.0</td>\n",
" <td>0.0</td>\n",
" <td>NaN</td>\n",
" <td>3004.0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>3009.0</td>\n",
" <td>WBAN:00184</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2016-08-01 09:35:00</th>\n",
" <td>WBAN:00184</td>\n",
" <td>2016-08-01 09:35</td>\n",
" <td>1000.0</td>\n",
" <td>296.0</td>\n",
" <td>27.4</td>\n",
" <td>267.0</td>\n",
" <td>84.0</td>\n",
" <td>3.0</td>\n",
" <td>NaN</td>\n",
" <td>3004.0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>3009.0</td>\n",
" <td>WBAN:00184</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2016-08-01 09:55:00</th>\n",
" <td>WBAN:00184</td>\n",
" <td>2016-08-01 09:55</td>\n",
" <td>NaN</td>\n",
" <td>302.0</td>\n",
" <td>27.2</td>\n",
" <td>260.0</td>\n",
" <td>78.0</td>\n",
" <td>0.0</td>\n",
" <td>NaN</td>\n",
" <td>3004.0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>3009.0</td>\n",
" <td>WBAN:00184</td>\n",
" </tr>\n",
" <tr>\n",
" <th>...</th>\n",
" <td>...</td>\n",
" <td>...</td>\n",
" <td>...</td>\n",
" <td>...</td>\n",
" <td>...</td>\n",
" <td>...</td>\n",
" <td>...</td>\n",
" <td>...</td>\n",
" <td>...</td>\n",
" <td>...</td>\n",
" <td>...</td>\n",
" <td>...</td>\n",
" <td>...</td>\n",
" <td>...</td>\n",
" <td>...</td>\n",
" <td>...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2018-07-31 14:35:00</th>\n",
" <td>WBAN:00184</td>\n",
" <td>2018-07-31 14:35</td>\n",
" <td>NaN</td>\n",
" <td>278.0</td>\n",
" <td>NaN</td>\n",
" <td>228.0</td>\n",
" <td>74.0</td>\n",
" <td>0.0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>2995.0</td>\n",
" <td>WBAN:00184</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2018-07-31 14:55:00</th>\n",
" <td>WBAN:00184</td>\n",
" <td>2018-07-31 14:55</td>\n",
" <td>NaN</td>\n",
" <td>277.0</td>\n",
" <td>NaN</td>\n",
" <td>231.0</td>\n",
" <td>76.0</td>\n",
" <td>6.0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>2995.0</td>\n",
" <td>WBAN:00184</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2018-07-31 15:15:00</th>\n",
" <td>WBAN:00184</td>\n",
" <td>2018-07-31 15:15</td>\n",
" <td>NaN</td>\n",
" <td>273.0</td>\n",
" <td>NaN</td>\n",
" <td>229.0</td>\n",
" <td>77.0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>2995.0</td>\n",
" <td>WBAN:00184</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2018-07-31 15:35:00</th>\n",
" <td>WBAN:00184</td>\n",
" <td>2018-07-31 15:35</td>\n",
" <td>NaN</td>\n",
" <td>273.0</td>\n",
" <td>NaN</td>\n",
" <td>237.0</td>\n",
" <td>81.0</td>\n",
" <td>0.0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>2995.0</td>\n",
" <td>WBAN:00184</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2018-07-31 15:55:00</th>\n",
" <td>WBAN:00184</td>\n",
" <td>2018-07-31 15:55</td>\n",
" <td>NaN</td>\n",
" <td>272.0</td>\n",
" <td>NaN</td>\n",
" <td>236.0</td>\n",
" <td>81.0</td>\n",
" <td>3.0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>2996.0</td>\n",
" <td>WBAN:00184</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2018-07-31 16:15:00</th>\n",
" <td>WBAN:00184</td>\n",
" <td>2018-07-31 16:15</td>\n",
" <td>NaN</td>\n",
" <td>274.0</td>\n",
" <td>NaN</td>\n",
" <td>238.0</td>\n",
" <td>81.0</td>\n",
" <td>3.0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>2996.0</td>\n",
" <td>WBAN:00184</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2018-07-31 16:35:00</th>\n",
" <td>WBAN:00184</td>\n",
" <td>2018-07-31 16:35</td>\n",
" <td>NaN</td>\n",
" <td>275.0</td>\n",
" <td>NaN</td>\n",
" <td>237.0</td>\n",
" <td>80.0</td>\n",
" <td>0.0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>2995.0</td>\n",
" <td>WBAN:00184</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2018-07-31 16:55:00</th>\n",
" <td>WBAN:00184</td>\n",
" <td>2018-07-31 16:55</td>\n",
" <td>NaN</td>\n",
" <td>275.0</td>\n",
" <td>NaN</td>\n",
" <td>236.0</td>\n",
" <td>79.0</td>\n",
" <td>0.0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>2995.0</td>\n",
" <td>WBAN:00184</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2018-07-31 17:15:00</th>\n",
" <td>WBAN:00184</td>\n",
" <td>2018-07-31 17:15</td>\n",
" <td>NaN</td>\n",
" <td>274.0</td>\n",
" <td>NaN</td>\n",
" <td>235.0</td>\n",
" <td>79.0</td>\n",
" <td>0.0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>2994.0</td>\n",
" <td>WBAN:00184</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2018-07-31 17:35:00</th>\n",
" <td>WBAN:00184</td>\n",
" <td>2018-07-31 17:35</td>\n",
" <td>NaN</td>\n",
" <td>272.0</td>\n",
" <td>NaN</td>\n",
" <td>238.0</td>\n",
" <td>82.0</td>\n",
" <td>0.0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>2994.0</td>\n",
" <td>WBAN:00184</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2018-07-31 17:55:00</th>\n",
" <td>WBAN:00184</td>\n",
" <td>2018-07-31 17:55</td>\n",
" <td>NaN</td>\n",
" <td>267.0</td>\n",
" <td>NaN</td>\n",
" <td>240.0</td>\n",
" <td>85.0</td>\n",
" <td>3.0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>2994.0</td>\n",
" <td>WBAN:00184</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2018-07-31 18:15:00</th>\n",
" <td>WBAN:00184</td>\n",
" <td>2018-07-31 18:15</td>\n",
" <td>NaN</td>\n",
" <td>262.0</td>\n",
" <td>NaN</td>\n",
" <td>244.0</td>\n",
" <td>90.0</td>\n",
" <td>0.0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>2994.0</td>\n",
" <td>WBAN:00184</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2018-07-31 18:35:00</th>\n",
" <td>WBAN:00184</td>\n",
" <td>2018-07-31 18:35</td>\n",
" <td>NaN</td>\n",
" <td>259.0</td>\n",
" <td>NaN</td>\n",
" <td>249.0</td>\n",
" <td>94.0</td>\n",
" <td>0.0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>2995.0</td>\n",
" <td>WBAN:00184</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2018-07-31 18:55:00</th>\n",
" <td>WBAN:00184</td>\n",
" <td>2018-07-31 18:55</td>\n",
" <td>NaN</td>\n",
" <td>257.0</td>\n",
" <td>NaN</td>\n",
" <td>257.0</td>\n",
" <td>100.0</td>\n",
" <td>0.0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>2995.0</td>\n",
" <td>WBAN:00184</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2018-07-31 19:15:00</th>\n",
" <td>WBAN:00184</td>\n",
" <td>2018-07-31 19:15</td>\n",
" <td>NaN</td>\n",
" <td>257.0</td>\n",
" <td>NaN</td>\n",
" <td>255.0</td>\n",
" <td>99.0</td>\n",
" <td>0.0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>2995.0</td>\n",
" <td>WBAN:00184</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2018-07-31 19:35:00</th>\n",
" <td>WBAN:00184</td>\n",
" <td>2018-07-31 19:35</td>\n",
" <td>NaN</td>\n",
" <td>254.0</td>\n",
" <td>NaN</td>\n",
" <td>254.0</td>\n",
" <td>100.0</td>\n",
" <td>0.0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>2995.0</td>\n",
" <td>WBAN:00184</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2018-07-31 19:55:00</th>\n",
" <td>WBAN:00184</td>\n",
" <td>2018-07-31 19:55</td>\n",
" <td>NaN</td>\n",
" <td>251.0</td>\n",
" <td>NaN</td>\n",
" <td>251.0</td>\n",
" <td>100.0</td>\n",
" <td>0.0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>2996.0</td>\n",
" <td>WBAN:00184</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2018-07-31 20:15:00</th>\n",
" <td>WBAN:00184</td>\n",
" <td>2018-07-31 20:15</td>\n",
" <td>NaN</td>\n",
" <td>250.0</td>\n",
" <td>NaN</td>\n",
" <td>250.0</td>\n",
" <td>100.0</td>\n",
" <td>0.0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>2995.0</td>\n",
" <td>WBAN:00184</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2018-07-31 20:35:00</th>\n",
" <td>WBAN:00184</td>\n",
" <td>2018-07-31 20:35</td>\n",
" <td>NaN</td>\n",
" <td>248.0</td>\n",
" <td>NaN</td>\n",
" <td>248.0</td>\n",
" <td>100.0</td>\n",
" <td>0.0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>2995.0</td>\n",
" <td>WBAN:00184</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2018-07-31 20:55:00</th>\n",
" <td>WBAN:00184</td>\n",
" <td>2018-07-31 20:55</td>\n",
" <td>NaN</td>\n",
" <td>243.0</td>\n",
" <td>NaN</td>\n",
" <td>243.0</td>\n",
" <td>100.0</td>\n",
" <td>0.0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>2996.0</td>\n",
" <td>WBAN:00184</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2018-07-31 21:15:00</th>\n",
" <td>WBAN:00184</td>\n",
" <td>2018-07-31 21:15</td>\n",
" <td>NaN</td>\n",
" <td>240.0</td>\n",
" <td>NaN</td>\n",
" <td>240.0</td>\n",
" <td>100.0</td>\n",
" <td>0.0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>2997.0</td>\n",
" <td>WBAN:00184</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2018-07-31 21:35:00</th>\n",
" <td>WBAN:00184</td>\n",
" <td>2018-07-31 21:35</td>\n",
" <td>NaN</td>\n",
" <td>239.0</td>\n",
" <td>NaN</td>\n",
" <td>239.0</td>\n",
" <td>100.0</td>\n",
" <td>0.0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>2996.0</td>\n",
" <td>WBAN:00184</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2018-07-31 21:55:00</th>\n",
" <td>WBAN:00184</td>\n",
" <td>2018-07-31 21:55</td>\n",
" <td>NaN</td>\n",
" <td>236.0</td>\n",
" <td>NaN</td>\n",
" <td>236.0</td>\n",
" <td>100.0</td>\n",
" <td>0.0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>2997.0</td>\n",
" <td>WBAN:00184</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2018-07-31 22:15:00</th>\n",
" <td>WBAN:00184</td>\n",
" <td>2018-07-31 22:15</td>\n",
" <td>NaN</td>\n",
" <td>235.0</td>\n",
" <td>NaN</td>\n",
" <td>235.0</td>\n",
" <td>100.0</td>\n",
" <td>0.0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>2996.0</td>\n",
" <td>WBAN:00184</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2018-07-31 22:35:00</th>\n",
" <td>WBAN:00184</td>\n",
" <td>2018-07-31 22:35</td>\n",
" <td>NaN</td>\n",
" <td>234.0</td>\n",
" <td>NaN</td>\n",
" <td>234.0</td>\n",
" <td>100.0</td>\n",
" <td>0.0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>2997.0</td>\n",
" <td>WBAN:00184</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2018-07-31 22:55:00</th>\n",
" <td>WBAN:00184</td>\n",
" <td>2018-07-31 22:55</td>\n",
" <td>700.0</td>\n",
" <td>235.0</td>\n",
" <td>23.3</td>\n",
" <td>235.0</td>\n",
" <td>100.0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>2993.0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>2998.0</td>\n",
" <td>WBAN:00184</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2018-07-31 23:15:00</th>\n",
" <td>WBAN:00184</td>\n",
" <td>2018-07-31 23:15</td>\n",
" <td>700.0</td>\n",
" <td>233.0</td>\n",
" <td>23.3</td>\n",
" <td>233.0</td>\n",
" <td>100.0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>2993.0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>2998.0</td>\n",
" <td>WBAN:00184</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2018-07-31 23:35:00</th>\n",
" <td>WBAN:00184</td>\n",
" <td>2018-07-31 23:35</td>\n",
" <td>1000.0</td>\n",
" <td>232.0</td>\n",
" <td>23.3</td>\n",
" <td>232.0</td>\n",
" <td>100.0</td>\n",
" <td>0.0</td>\n",
" <td>NaN</td>\n",
" <td>2993.0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>2998.0</td>\n",
" <td>WBAN:00184</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2018-07-31 23:55:00</th>\n",
" <td>WBAN:00184</td>\n",
" <td>2018-07-31 23:55</td>\n",
" <td>700.0</td>\n",
" <td>233.0</td>\n",
" <td>23.3</td>\n",
" <td>233.0</td>\n",
" <td>100.0</td>\n",
" <td>0.0</td>\n",
" <td>NaN</td>\n",
" <td>2993.0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>2998.0</td>\n",
" <td>WBAN:00184</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2018-07-31 23:59:00</th>\n",
" <td>WBAN:00184</td>\n",
" <td>2018-07-31 23:59</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>WBAN:00184</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"<p>46719 rows × 16 columns</p>\n",
"</div>"
],
"text/plain": [
" STATION DATE HOURLYVISIBILITY \\\n",
"DATE \n",
"2016-08-01 00:15:00 WBAN:00184 2016-08-01 00:15 1000.0 \n",
"2016-08-01 00:35:00 WBAN:00184 2016-08-01 00:35 175.0 \n",
"2016-08-01 00:55:00 WBAN:00184 2016-08-01 00:55 400.0 \n",
"2016-08-01 01:15:00 WBAN:00184 2016-08-01 01:15 100.0 \n",
"2016-08-01 01:35:00 WBAN:00184 2016-08-01 01:35 500.0 \n",
"2016-08-01 01:55:00 WBAN:00184 2016-08-01 01:55 700.0 \n",
"2016-08-01 02:15:00 WBAN:00184 2016-08-01 02:15 700.0 \n",
"2016-08-01 02:35:00 WBAN:00184 2016-08-01 02:35 200.0 \n",
"2016-08-01 02:55:00 WBAN:00184 2016-08-01 02:55 50.0 \n",
"2016-08-01 03:15:00 WBAN:00184 2016-08-01 03:15 150.0 \n",
"2016-08-01 03:35:00 WBAN:00184 2016-08-01 03:35 50.0 \n",
"2016-08-01 03:55:00 WBAN:00184 2016-08-01 03:55 75.0 \n",
"2016-08-01 04:15:00 WBAN:00184 2016-08-01 04:15 700.0 \n",
"2016-08-01 04:35:00 WBAN:00184 2016-08-01 04:35 500.0 \n",
"2016-08-01 04:55:00 WBAN:00184 2016-08-01 04:55 1000.0 \n",
"2016-08-01 05:15:00 WBAN:00184 2016-08-01 05:15 700.0 \n",
"2016-08-01 05:35:00 WBAN:00184 2016-08-01 05:35 500.0 \n",
"2016-08-01 05:55:00 WBAN:00184 2016-08-01 05:55 700.0 \n",
"2016-08-01 06:15:00 WBAN:00184 2016-08-01 06:15 700.0 \n",
"2016-08-01 06:35:00 WBAN:00184 2016-08-01 06:35 1000.0 \n",
"2016-08-01 06:55:00 WBAN:00184 2016-08-01 06:55 1000.0 \n",
"2016-08-01 07:15:00 WBAN:00184 2016-08-01 07:15 1000.0 \n",
"2016-08-01 07:35:00 WBAN:00184 2016-08-01 07:35 1000.0 \n",
"2016-08-01 07:55:00 WBAN:00184 2016-08-01 07:55 1000.0 \n",
"2016-08-01 08:15:00 WBAN:00184 2016-08-01 08:15 1000.0 \n",
"2016-08-01 08:35:00 WBAN:00184 2016-08-01 08:35 1000.0 \n",
"2016-08-01 08:55:00 WBAN:00184 2016-08-01 08:55 1000.0 \n",
"2016-08-01 09:15:00 WBAN:00184 2016-08-01 09:15 1000.0 \n",
"2016-08-01 09:35:00 WBAN:00184 2016-08-01 09:35 1000.0 \n",
"2016-08-01 09:55:00 WBAN:00184 2016-08-01 09:55 NaN \n",
"... ... ... ... \n",
"2018-07-31 14:35:00 WBAN:00184 2018-07-31 14:35 NaN \n",
"2018-07-31 14:55:00 WBAN:00184 2018-07-31 14:55 NaN \n",
"2018-07-31 15:15:00 WBAN:00184 2018-07-31 15:15 NaN \n",
"2018-07-31 15:35:00 WBAN:00184 2018-07-31 15:35 NaN \n",
"2018-07-31 15:55:00 WBAN:00184 2018-07-31 15:55 NaN \n",
"2018-07-31 16:15:00 WBAN:00184 2018-07-31 16:15 NaN \n",
"2018-07-31 16:35:00 WBAN:00184 2018-07-31 16:35 NaN \n",
"2018-07-31 16:55:00 WBAN:00184 2018-07-31 16:55 NaN \n",
"2018-07-31 17:15:00 WBAN:00184 2018-07-31 17:15 NaN \n",
"2018-07-31 17:35:00 WBAN:00184 2018-07-31 17:35 NaN \n",
"2018-07-31 17:55:00 WBAN:00184 2018-07-31 17:55 NaN \n",
"2018-07-31 18:15:00 WBAN:00184 2018-07-31 18:15 NaN \n",
"2018-07-31 18:35:00 WBAN:00184 2018-07-31 18:35 NaN \n",
"2018-07-31 18:55:00 WBAN:00184 2018-07-31 18:55 NaN \n",
"2018-07-31 19:15:00 WBAN:00184 2018-07-31 19:15 NaN \n",
"2018-07-31 19:35:00 WBAN:00184 2018-07-31 19:35 NaN \n",
"2018-07-31 19:55:00 WBAN:00184 2018-07-31 19:55 NaN \n",
"2018-07-31 20:15:00 WBAN:00184 2018-07-31 20:15 NaN \n",
"2018-07-31 20:35:00 WBAN:00184 2018-07-31 20:35 NaN \n",
"2018-07-31 20:55:00 WBAN:00184 2018-07-31 20:55 NaN \n",
"2018-07-31 21:15:00 WBAN:00184 2018-07-31 21:15 NaN \n",
"2018-07-31 21:35:00 WBAN:00184 2018-07-31 21:35 NaN \n",
"2018-07-31 21:55:00 WBAN:00184 2018-07-31 21:55 NaN \n",
"2018-07-31 22:15:00 WBAN:00184 2018-07-31 22:15 NaN \n",
"2018-07-31 22:35:00 WBAN:00184 2018-07-31 22:35 NaN \n",
"2018-07-31 22:55:00 WBAN:00184 2018-07-31 22:55 700.0 \n",
"2018-07-31 23:15:00 WBAN:00184 2018-07-31 23:15 700.0 \n",
"2018-07-31 23:35:00 WBAN:00184 2018-07-31 23:35 1000.0 \n",
"2018-07-31 23:55:00 WBAN:00184 2018-07-31 23:55 700.0 \n",
"2018-07-31 23:59:00 WBAN:00184 2018-07-31 23:59 NaN \n",
"\n",
" HOURLYDRYBULBTEMPC HOURLYWETBULBTEMPC \\\n",
"DATE \n",
"2016-08-01 00:15:00 245.0 24.4 \n",
"2016-08-01 00:35:00 242.0 24.4 \n",
"2016-08-01 00:55:00 242.0 24.4 \n",
"2016-08-01 01:15:00 240.0 23.9 \n",
"2016-08-01 01:35:00 240.0 23.9 \n",
"2016-08-01 01:55:00 241.0 23.9 \n",
"2016-08-01 02:15:00 241.0 23.9 \n",
"2016-08-01 02:35:00 236.0 23.3 \n",
"2016-08-01 02:55:00 235.0 23.3 \n",
"2016-08-01 03:15:00 237.0 23.9 \n",
"2016-08-01 03:35:00 238.0 23.9 \n",
"2016-08-01 03:55:00 237.0 23.9 \n",
"2016-08-01 04:15:00 233.0 23.3 \n",
"2016-08-01 04:35:00 234.0 23.3 \n",
"2016-08-01 04:55:00 240.0 23.9 \n",
"2016-08-01 05:15:00 237.0 23.9 \n",
"2016-08-01 05:35:00 237.0 23.9 \n",
"2016-08-01 05:55:00 239.0 23.9 \n",
"2016-08-01 06:15:00 245.0 24.4 \n",
"2016-08-01 06:35:00 249.0 25.0 \n",
"2016-08-01 06:55:00 253.0 25.6 \n",
"2016-08-01 07:15:00 256.0 25.6 \n",
"2016-08-01 07:35:00 265.0 26.7 \n",
"2016-08-01 07:55:00 271.0 26.8 \n",
"2016-08-01 08:15:00 276.0 27.0 \n",
"2016-08-01 08:35:00 283.0 27.1 \n",
"2016-08-01 08:55:00 290.0 27.3 \n",
"2016-08-01 09:15:00 292.0 27.4 \n",
"2016-08-01 09:35:00 296.0 27.4 \n",
"2016-08-01 09:55:00 302.0 27.2 \n",
"... ... ... \n",
"2018-07-31 14:35:00 278.0 NaN \n",
"2018-07-31 14:55:00 277.0 NaN \n",
"2018-07-31 15:15:00 273.0 NaN \n",
"2018-07-31 15:35:00 273.0 NaN \n",
"2018-07-31 15:55:00 272.0 NaN \n",
"2018-07-31 16:15:00 274.0 NaN \n",
"2018-07-31 16:35:00 275.0 NaN \n",
"2018-07-31 16:55:00 275.0 NaN \n",
"2018-07-31 17:15:00 274.0 NaN \n",
"2018-07-31 17:35:00 272.0 NaN \n",
"2018-07-31 17:55:00 267.0 NaN \n",
"2018-07-31 18:15:00 262.0 NaN \n",
"2018-07-31 18:35:00 259.0 NaN \n",
"2018-07-31 18:55:00 257.0 NaN \n",
"2018-07-31 19:15:00 257.0 NaN \n",
"2018-07-31 19:35:00 254.0 NaN \n",
"2018-07-31 19:55:00 251.0 NaN \n",
"2018-07-31 20:15:00 250.0 NaN \n",
"2018-07-31 20:35:00 248.0 NaN \n",
"2018-07-31 20:55:00 243.0 NaN \n",
"2018-07-31 21:15:00 240.0 NaN \n",
"2018-07-31 21:35:00 239.0 NaN \n",
"2018-07-31 21:55:00 236.0 NaN \n",
"2018-07-31 22:15:00 235.0 NaN \n",
"2018-07-31 22:35:00 234.0 NaN \n",
"2018-07-31 22:55:00 235.0 23.3 \n",
"2018-07-31 23:15:00 233.0 23.3 \n",
"2018-07-31 23:35:00 232.0 23.3 \n",
"2018-07-31 23:55:00 233.0 23.3 \n",
"2018-07-31 23:59:00 NaN NaN \n",
"\n",
" HOURLYDewPointTempC HOURLYRelativeHumidity \\\n",
"DATE \n",
"2016-08-01 00:15:00 245.0 100.0 \n",
"2016-08-01 00:35:00 242.0 100.0 \n",
"2016-08-01 00:55:00 242.0 100.0 \n",
"2016-08-01 01:15:00 240.0 100.0 \n",
"2016-08-01 01:35:00 240.0 100.0 \n",
"2016-08-01 01:55:00 241.0 100.0 \n",
"2016-08-01 02:15:00 241.0 100.0 \n",
"2016-08-01 02:35:00 236.0 100.0 \n",
"2016-08-01 02:55:00 235.0 100.0 \n",
"2016-08-01 03:15:00 237.0 100.0 \n",
"2016-08-01 03:35:00 238.0 100.0 \n",
"2016-08-01 03:55:00 237.0 100.0 \n",
"2016-08-01 04:15:00 233.0 100.0 \n",
"2016-08-01 04:35:00 234.0 100.0 \n",
"2016-08-01 04:55:00 240.0 100.0 \n",
"2016-08-01 05:15:00 237.0 100.0 \n",
"2016-08-01 05:35:00 237.0 100.0 \n",
"2016-08-01 05:55:00 239.0 100.0 \n",
"2016-08-01 06:15:00 245.0 100.0 \n",
"2016-08-01 06:35:00 249.0 100.0 \n",
"2016-08-01 06:55:00 253.0 100.0 \n",
"2016-08-01 07:15:00 256.0 100.0 \n",
"2016-08-01 07:35:00 265.0 100.0 \n",
"2016-08-01 07:55:00 267.0 98.0 \n",
"2016-08-01 08:15:00 269.0 96.0 \n",
"2016-08-01 08:35:00 264.0 89.0 \n",
"2016-08-01 08:55:00 269.0 88.0 \n",
"2016-08-01 09:15:00 268.0 87.0 \n",
"2016-08-01 09:35:00 267.0 84.0 \n",
"2016-08-01 09:55:00 260.0 78.0 \n",
"... ... ... \n",
"2018-07-31 14:35:00 228.0 74.0 \n",
"2018-07-31 14:55:00 231.0 76.0 \n",
"2018-07-31 15:15:00 229.0 77.0 \n",
"2018-07-31 15:35:00 237.0 81.0 \n",
"2018-07-31 15:55:00 236.0 81.0 \n",
"2018-07-31 16:15:00 238.0 81.0 \n",
"2018-07-31 16:35:00 237.0 80.0 \n",
"2018-07-31 16:55:00 236.0 79.0 \n",
"2018-07-31 17:15:00 235.0 79.0 \n",
"2018-07-31 17:35:00 238.0 82.0 \n",
"2018-07-31 17:55:00 240.0 85.0 \n",
"2018-07-31 18:15:00 244.0 90.0 \n",
"2018-07-31 18:35:00 249.0 94.0 \n",
"2018-07-31 18:55:00 257.0 100.0 \n",
"2018-07-31 19:15:00 255.0 99.0 \n",
"2018-07-31 19:35:00 254.0 100.0 \n",
"2018-07-31 19:55:00 251.0 100.0 \n",
"2018-07-31 20:15:00 250.0 100.0 \n",
"2018-07-31 20:35:00 248.0 100.0 \n",
"2018-07-31 20:55:00 243.0 100.0 \n",
"2018-07-31 21:15:00 240.0 100.0 \n",
"2018-07-31 21:35:00 239.0 100.0 \n",
"2018-07-31 21:55:00 236.0 100.0 \n",
"2018-07-31 22:15:00 235.0 100.0 \n",
"2018-07-31 22:35:00 234.0 100.0 \n",
"2018-07-31 22:55:00 235.0 100.0 \n",
"2018-07-31 23:15:00 233.0 100.0 \n",
"2018-07-31 23:35:00 232.0 100.0 \n",
"2018-07-31 23:55:00 233.0 100.0 \n",
"2018-07-31 23:59:00 NaN NaN \n",
"\n",
" HOURLYWindSpeed HOURLYWindGustSpeed \\\n",
"DATE \n",
"2016-08-01 00:15:00 0.0 NaN \n",
"2016-08-01 00:35:00 0.0 NaN \n",
"2016-08-01 00:55:00 0.0 NaN \n",
"2016-08-01 01:15:00 0.0 NaN \n",
"2016-08-01 01:35:00 0.0 NaN \n",
"2016-08-01 01:55:00 0.0 NaN \n",
"2016-08-01 02:15:00 0.0 NaN \n",
"2016-08-01 02:35:00 0.0 NaN \n",
"2016-08-01 02:55:00 0.0 NaN \n",
"2016-08-01 03:15:00 0.0 NaN \n",
"2016-08-01 03:35:00 0.0 NaN \n",
"2016-08-01 03:55:00 0.0 NaN \n",
"2016-08-01 04:15:00 0.0 NaN \n",
"2016-08-01 04:35:00 0.0 NaN \n",
"2016-08-01 04:55:00 0.0 NaN \n",
"2016-08-01 05:15:00 0.0 NaN \n",
"2016-08-01 05:35:00 0.0 NaN \n",
"2016-08-01 05:55:00 0.0 NaN \n",
"2016-08-01 06:15:00 0.0 NaN \n",
"2016-08-01 06:35:00 0.0 NaN \n",
"2016-08-01 06:55:00 0.0 NaN \n",
"2016-08-01 07:15:00 0.0 NaN \n",
"2016-08-01 07:35:00 0.0 NaN \n",
"2016-08-01 07:55:00 0.0 NaN \n",
"2016-08-01 08:15:00 0.0 NaN \n",
"2016-08-01 08:35:00 0.0 NaN \n",
"2016-08-01 08:55:00 5.0 NaN \n",
"2016-08-01 09:15:00 0.0 NaN \n",
"2016-08-01 09:35:00 3.0 NaN \n",
"2016-08-01 09:55:00 0.0 NaN \n",
"... ... ... \n",
"2018-07-31 14:35:00 0.0 NaN \n",
"2018-07-31 14:55:00 6.0 NaN \n",
"2018-07-31 15:15:00 NaN NaN \n",
"2018-07-31 15:35:00 0.0 NaN \n",
"2018-07-31 15:55:00 3.0 NaN \n",
"2018-07-31 16:15:00 3.0 NaN \n",
"2018-07-31 16:35:00 0.0 NaN \n",
"2018-07-31 16:55:00 0.0 NaN \n",
"2018-07-31 17:15:00 0.0 NaN \n",
"2018-07-31 17:35:00 0.0 NaN \n",
"2018-07-31 17:55:00 3.0 NaN \n",
"2018-07-31 18:15:00 0.0 NaN \n",
"2018-07-31 18:35:00 0.0 NaN \n",
"2018-07-31 18:55:00 0.0 NaN \n",
"2018-07-31 19:15:00 0.0 NaN \n",
"2018-07-31 19:35:00 0.0 NaN \n",
"2018-07-31 19:55:00 0.0 NaN \n",
"2018-07-31 20:15:00 0.0 NaN \n",
"2018-07-31 20:35:00 0.0 NaN \n",
"2018-07-31 20:55:00 0.0 NaN \n",
"2018-07-31 21:15:00 0.0 NaN \n",
"2018-07-31 21:35:00 0.0 NaN \n",
"2018-07-31 21:55:00 0.0 NaN \n",
"2018-07-31 22:15:00 0.0 NaN \n",
"2018-07-31 22:35:00 0.0 NaN \n",
"2018-07-31 22:55:00 NaN NaN \n",
"2018-07-31 23:15:00 NaN NaN \n",
"2018-07-31 23:35:00 0.0 NaN \n",
"2018-07-31 23:55:00 0.0 NaN \n",
"2018-07-31 23:59:00 NaN NaN \n",
"\n",
" HOURLYStationPressure HOURLYPressureTendency \\\n",
"DATE \n",
"2016-08-01 00:15:00 2999.0 NaN \n",
"2016-08-01 00:35:00 2999.0 NaN \n",
"2016-08-01 00:55:00 2999.0 NaN \n",
"2016-08-01 01:15:00 2999.0 NaN \n",
"2016-08-01 01:35:00 3000.0 NaN \n",
"2016-08-01 01:55:00 2999.0 NaN \n",
"2016-08-01 02:15:00 2999.0 NaN \n",
"2016-08-01 02:35:00 2998.0 NaN \n",
"2016-08-01 02:55:00 2999.0 NaN \n",
"2016-08-01 03:15:00 2999.0 NaN \n",
"2016-08-01 03:35:00 2998.0 NaN \n",
"2016-08-01 03:55:00 2998.0 NaN \n",
"2016-08-01 04:15:00 2999.0 NaN \n",
"2016-08-01 04:35:00 3000.0 NaN \n",
"2016-08-01 04:55:00 2999.0 NaN \n",
"2016-08-01 05:15:00 3000.0 NaN \n",
"2016-08-01 05:35:00 3001.0 NaN \n",
"2016-08-01 05:55:00 3000.0 NaN \n",
"2016-08-01 06:15:00 3001.0 NaN \n",
"2016-08-01 06:35:00 3002.0 NaN \n",
"2016-08-01 06:55:00 3002.0 NaN \n",
"2016-08-01 07:15:00 3003.0 NaN \n",
"2016-08-01 07:35:00 3004.0 NaN \n",
"2016-08-01 07:55:00 3004.0 NaN \n",
"2016-08-01 08:15:00 3004.0 NaN \n",
"2016-08-01 08:35:00 3004.0 NaN \n",
"2016-08-01 08:55:00 3004.0 NaN \n",
"2016-08-01 09:15:00 3004.0 NaN \n",
"2016-08-01 09:35:00 3004.0 NaN \n",
"2016-08-01 09:55:00 3004.0 NaN \n",
"... ... ... \n",
"2018-07-31 14:35:00 NaN NaN \n",
"2018-07-31 14:55:00 NaN NaN \n",
"2018-07-31 15:15:00 NaN NaN \n",
"2018-07-31 15:35:00 NaN NaN \n",
"2018-07-31 15:55:00 NaN NaN \n",
"2018-07-31 16:15:00 NaN NaN \n",
"2018-07-31 16:35:00 NaN NaN \n",
"2018-07-31 16:55:00 NaN NaN \n",
"2018-07-31 17:15:00 NaN NaN \n",
"2018-07-31 17:35:00 NaN NaN \n",
"2018-07-31 17:55:00 NaN NaN \n",
"2018-07-31 18:15:00 NaN NaN \n",
"2018-07-31 18:35:00 NaN NaN \n",
"2018-07-31 18:55:00 NaN NaN \n",
"2018-07-31 19:15:00 NaN NaN \n",
"2018-07-31 19:35:00 NaN NaN \n",
"2018-07-31 19:55:00 NaN NaN \n",
"2018-07-31 20:15:00 NaN NaN \n",
"2018-07-31 20:35:00 NaN NaN \n",
"2018-07-31 20:55:00 NaN NaN \n",
"2018-07-31 21:15:00 NaN NaN \n",
"2018-07-31 21:35:00 NaN NaN \n",
"2018-07-31 21:55:00 NaN NaN \n",
"2018-07-31 22:15:00 NaN NaN \n",
"2018-07-31 22:35:00 NaN NaN \n",
"2018-07-31 22:55:00 2993.0 NaN \n",
"2018-07-31 23:15:00 2993.0 NaN \n",
"2018-07-31 23:35:00 2993.0 NaN \n",
"2018-07-31 23:55:00 2993.0 NaN \n",
"2018-07-31 23:59:00 NaN NaN \n",
"\n",
" HOURLYPressureChange HOURLYSeaLevelPressure \\\n",
"DATE \n",
"2016-08-01 00:15:00 NaN NaN \n",
"2016-08-01 00:35:00 NaN NaN \n",
"2016-08-01 00:55:00 NaN NaN \n",
"2016-08-01 01:15:00 NaN NaN \n",
"2016-08-01 01:35:00 NaN NaN \n",
"2016-08-01 01:55:00 NaN NaN \n",
"2016-08-01 02:15:00 NaN NaN \n",
"2016-08-01 02:35:00 NaN NaN \n",
"2016-08-01 02:55:00 NaN NaN \n",
"2016-08-01 03:15:00 NaN NaN \n",
"2016-08-01 03:35:00 NaN NaN \n",
"2016-08-01 03:55:00 NaN NaN \n",
"2016-08-01 04:15:00 NaN NaN \n",
"2016-08-01 04:35:00 NaN NaN \n",
"2016-08-01 04:55:00 NaN NaN \n",
"2016-08-01 05:15:00 NaN NaN \n",
"2016-08-01 05:35:00 NaN NaN \n",
"2016-08-01 05:55:00 NaN NaN \n",
"2016-08-01 06:15:00 NaN NaN \n",
"2016-08-01 06:35:00 NaN NaN \n",
"2016-08-01 06:55:00 NaN NaN \n",
"2016-08-01 07:15:00 NaN NaN \n",
"2016-08-01 07:35:00 NaN NaN \n",
"2016-08-01 07:55:00 NaN NaN \n",
"2016-08-01 08:15:00 NaN NaN \n",
"2016-08-01 08:35:00 NaN NaN \n",
"2016-08-01 08:55:00 NaN NaN \n",
"2016-08-01 09:15:00 NaN NaN \n",
"2016-08-01 09:35:00 NaN NaN \n",
"2016-08-01 09:55:00 NaN NaN \n",
"... ... ... \n",
"2018-07-31 14:35:00 NaN NaN \n",
"2018-07-31 14:55:00 NaN NaN \n",
"2018-07-31 15:15:00 NaN NaN \n",
"2018-07-31 15:35:00 NaN NaN \n",
"2018-07-31 15:55:00 NaN NaN \n",
"2018-07-31 16:15:00 NaN NaN \n",
"2018-07-31 16:35:00 NaN NaN \n",
"2018-07-31 16:55:00 NaN NaN \n",
"2018-07-31 17:15:00 NaN NaN \n",
"2018-07-31 17:35:00 NaN NaN \n",
"2018-07-31 17:55:00 NaN NaN \n",
"2018-07-31 18:15:00 NaN NaN \n",
"2018-07-31 18:35:00 NaN NaN \n",
"2018-07-31 18:55:00 NaN NaN \n",
"2018-07-31 19:15:00 NaN NaN \n",
"2018-07-31 19:35:00 NaN NaN \n",
"2018-07-31 19:55:00 NaN NaN \n",
"2018-07-31 20:15:00 NaN NaN \n",
"2018-07-31 20:35:00 NaN NaN \n",
"2018-07-31 20:55:00 NaN NaN \n",
"2018-07-31 21:15:00 NaN NaN \n",
"2018-07-31 21:35:00 NaN NaN \n",
"2018-07-31 21:55:00 NaN NaN \n",
"2018-07-31 22:15:00 NaN NaN \n",
"2018-07-31 22:35:00 NaN NaN \n",
"2018-07-31 22:55:00 NaN NaN \n",
"2018-07-31 23:15:00 NaN NaN \n",
"2018-07-31 23:35:00 NaN NaN \n",
"2018-07-31 23:55:00 NaN NaN \n",
"2018-07-31 23:59:00 NaN NaN \n",
"\n",
" HOURLYPrecip HOURLYAltimeterSetting STATION_ID \n",
"DATE \n",
"2016-08-01 00:15:00 NaN 3004.0 WBAN:00184 \n",
"2016-08-01 00:35:00 NaN 3004.0 WBAN:00184 \n",
"2016-08-01 00:55:00 NaN 3004.0 WBAN:00184 \n",
"2016-08-01 01:15:00 NaN 3004.0 WBAN:00184 \n",
"2016-08-01 01:35:00 NaN 3005.0 WBAN:00184 \n",
"2016-08-01 01:55:00 NaN 3004.0 WBAN:00184 \n",
"2016-08-01 02:15:00 NaN 3004.0 WBAN:00184 \n",
"2016-08-01 02:35:00 NaN 3003.0 WBAN:00184 \n",
"2016-08-01 02:55:00 NaN 3004.0 WBAN:00184 \n",
"2016-08-01 03:15:00 NaN 3004.0 WBAN:00184 \n",
"2016-08-01 03:35:00 NaN 3003.0 WBAN:00184 \n",
"2016-08-01 03:55:00 NaN 3003.0 WBAN:00184 \n",
"2016-08-01 04:15:00 NaN 3004.0 WBAN:00184 \n",
"2016-08-01 04:35:00 NaN 3005.0 WBAN:00184 \n",
"2016-08-01 04:55:00 NaN 3004.0 WBAN:00184 \n",
"2016-08-01 05:15:00 NaN 3005.0 WBAN:00184 \n",
"2016-08-01 05:35:00 NaN 3006.0 WBAN:00184 \n",
"2016-08-01 05:55:00 NaN 3005.0 WBAN:00184 \n",
"2016-08-01 06:15:00 NaN 3006.0 WBAN:00184 \n",
"2016-08-01 06:35:00 NaN 3007.0 WBAN:00184 \n",
"2016-08-01 06:55:00 NaN 3007.0 WBAN:00184 \n",
"2016-08-01 07:15:00 NaN 3008.0 WBAN:00184 \n",
"2016-08-01 07:35:00 NaN 3009.0 WBAN:00184 \n",
"2016-08-01 07:55:00 NaN 3009.0 WBAN:00184 \n",
"2016-08-01 08:15:00 NaN 3009.0 WBAN:00184 \n",
"2016-08-01 08:35:00 NaN 3009.0 WBAN:00184 \n",
"2016-08-01 08:55:00 NaN 3009.0 WBAN:00184 \n",
"2016-08-01 09:15:00 NaN 3009.0 WBAN:00184 \n",
"2016-08-01 09:35:00 NaN 3009.0 WBAN:00184 \n",
"2016-08-01 09:55:00 NaN 3009.0 WBAN:00184 \n",
"... ... ... ... \n",
"2018-07-31 14:35:00 NaN 2995.0 WBAN:00184 \n",
"2018-07-31 14:55:00 NaN 2995.0 WBAN:00184 \n",
"2018-07-31 15:15:00 NaN 2995.0 WBAN:00184 \n",
"2018-07-31 15:35:00 NaN 2995.0 WBAN:00184 \n",
"2018-07-31 15:55:00 NaN 2996.0 WBAN:00184 \n",
"2018-07-31 16:15:00 NaN 2996.0 WBAN:00184 \n",
"2018-07-31 16:35:00 NaN 2995.0 WBAN:00184 \n",
"2018-07-31 16:55:00 NaN 2995.0 WBAN:00184 \n",
"2018-07-31 17:15:00 NaN 2994.0 WBAN:00184 \n",
"2018-07-31 17:35:00 NaN 2994.0 WBAN:00184 \n",
"2018-07-31 17:55:00 NaN 2994.0 WBAN:00184 \n",
"2018-07-31 18:15:00 NaN 2994.0 WBAN:00184 \n",
"2018-07-31 18:35:00 NaN 2995.0 WBAN:00184 \n",
"2018-07-31 18:55:00 NaN 2995.0 WBAN:00184 \n",
"2018-07-31 19:15:00 NaN 2995.0 WBAN:00184 \n",
"2018-07-31 19:35:00 NaN 2995.0 WBAN:00184 \n",
"2018-07-31 19:55:00 NaN 2996.0 WBAN:00184 \n",
"2018-07-31 20:15:00 NaN 2995.0 WBAN:00184 \n",
"2018-07-31 20:35:00 NaN 2995.0 WBAN:00184 \n",
"2018-07-31 20:55:00 NaN 2996.0 WBAN:00184 \n",
"2018-07-31 21:15:00 NaN 2997.0 WBAN:00184 \n",
"2018-07-31 21:35:00 NaN 2996.0 WBAN:00184 \n",
"2018-07-31 21:55:00 NaN 2997.0 WBAN:00184 \n",
"2018-07-31 22:15:00 NaN 2996.0 WBAN:00184 \n",
"2018-07-31 22:35:00 NaN 2997.0 WBAN:00184 \n",
"2018-07-31 22:55:00 NaN 2998.0 WBAN:00184 \n",
"2018-07-31 23:15:00 NaN 2998.0 WBAN:00184 \n",
"2018-07-31 23:35:00 NaN 2998.0 WBAN:00184 \n",
"2018-07-31 23:55:00 NaN 2998.0 WBAN:00184 \n",
"2018-07-31 23:59:00 NaN NaN WBAN:00184 \n",
"\n",
"[46719 rows x 16 columns]"
]
},
"execution_count": 13,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"by_station_list[0]"
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|████████████████████████████████████████████████████████████████████████████████| 406/406 [00:06<00:00, 64.90it/s]\n"
]
}
],
"source": [
"#Some stations take reading multiple times per hour this reduces to one reading per hour by taking the mean of readings\n",
"#in an hour\n",
"for i in tqdm(range(len(by_station_list))):\n",
" by_station_list[i] = by_station_list[i].resample('60T').mean()"
]
},
{
"cell_type": "code",
"execution_count": 15,
"metadata": {},
"outputs": [],
"source": [
"#making a list of filenames by removing the colon \n",
"wban_filename = []\n",
"for x in wban_list:\n",
" wban_filename.append(x[:4] + x[5:])"
]
},
{
"cell_type": "code",
"execution_count": 16,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"'WBAN00184'"
]
},
"execution_count": 16,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"wban_filename[0]"
]
},
{
"cell_type": "code",
"execution_count": 17,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"'WBAN:00184'"
]
},
"execution_count": 17,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"wban_list[0]"
]
},
{
"cell_type": "code",
"execution_count": 18,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|███████████████████████████████████████████████████████████████████████████████| 406/406 [00:00<00:00, 619.85it/s]\n"
]
}
],
"source": [
"#adding back WBAN station since it was absorbed when data was set as the index\n",
"for i in tqdm(range(len(by_station_list))):\n",
" by_station_list[i]['STATION'] = wban_list[i] "
]
},
{
"cell_type": "code",
"execution_count": 19,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"17520"
]
},
"execution_count": 19,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"len(by_station_list[0])"
]
},
{
"cell_type": "code",
"execution_count": 34,
"metadata": {},
"outputs": [],
"source": [
"rows = 24 * 365 * 2 \n",
"stations_used = []\n"
]
},
{
"cell_type": "code",
"execution_count": 35,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|████████████████████████████████████████████████████████████████████████████████| 406/406 [02:04<00:00, 3.27it/s]\n"
]
}
],
"source": [
"for i in tqdm(range(len(wban_filename))):\n",
" if(len(by_station_list[i]) >= rows):\n",
" by_station_list[i].to_csv('../Project Final/Transformation/processed_data/{}.csv'.format(wban_filename[i]))\n",
" stations_used.append(stations.loc[stations.STATION_ID == wban_list[i]])"
]
},
{
"cell_type": "code",
"execution_count": 42,
"metadata": {},
"outputs": [],
"source": [
"stations_used_df = pd.concat(stations_used)"
]
},
{
"cell_type": "code",
"execution_count": 44,
"metadata": {},
"outputs": [],
"source": [
"stations_used_df.to_csv('../Project Final/Transformation/stations_mask.csv')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Current Issues\n",
"\n",
"1) data is not synced across time-zone and all entries start at midnight local time. \n",
"\n",
"2) some stations have multiple entries per hour and need to be reduced.\n",
"\n",
"Solutions\n",
"\n",
"Remove rows from data based on timezone to sync times\n",
"limit only 1 entry per hour for a station"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.6.4"
}
},
"nbformat": 4,
"nbformat_minor": 2
}
Reference in New Issue View Git Blame Copy Permalink