Weather Data Science

I became interested in the question: what weather factors influence the temperature of my pool water? And by how much? It seemed clear to me that things like air temperature, solar radiation, and wind would have an effect. But as discussed in the previous post in this series, my sensors only provided me with temperature and relative humidity. I entertained the notion of getting a home weather station like those available from Ambient Weather or AcuRite, but they seemed a bit too expensive. Plus, I saw on Weather Underground that a neighbor about 200 yards away from my backyard had a home weather station, and I could get historical data from that station uploaded to Weather Underground. I started using this data, but it was inconvenient because I had to scrape the data from a webpage¹. And frankly, the quality of that data seemed a little suspect - e.g., I think the wind vane may have been stuck. Plus, that particular weather station did not measure solar radiation, which I thought would be important for my purposes.

Visual Crossing

After some web searches, I hit the jackpot on obtaining (fairly) localized weather data when I discovered Visual Crossing. This site allows me to download all the historical local weather data I needed, in a convenient CSV format, for free.

As any data scientist might tell you, a lot of the work in data science is just obtaining, formatting, cleaning, cross-checking, and curating the data. This groundwork has to happen before you can apply any fancy data analysis techniques. I created several Jupyter notebooks just to read and sanity-check the data. This effort was well spent because it pointed out several errors that I made in getting the data standardized and formatted properly - I’m glad these mistakes were caught early. Here are some plots of the data that I obtained from Visual Crossing, using data from ‘official’ weather stations a few miles from my home, over the same time range for which I have collected pool temperature data. (Click on the plots to enlarge them.)

Of special interest is the solar radiation and cloud cover percentage data shown in this plot:

Obviously the solar radiation is reduced on cloudy days. Also, if you look at the maximum solar radiation for each clear day, you can see that the solar radiation decreases into the fall and winter, as it should.

It is interesting to examine the heat map² of the correlations between the various weather variables:

In the heat map, darker colors for a square mean that the weather variables for the row and column for that square show a correlation between them, either positive (blue) if they vary together or negative (red) if they vary in opposite directions. A correlation near zero means that the variables have no obvious (linear) relationship between them. Some comments on the heat map:

• Air temperature and barometric pressure are negatively correlated to a fairly high degree: when the pressure is high, the temperature is low, and vice versa. The physics here is that cold air is denser than warm air and so exerts a higher pressure.

• Cloud cover and humidity are positively correlated, which makes sense - high humidity usually means cloudy.

• One might expect there to be a high negative correlation between cloud cover and solar radiation, but there is only a weak correlation because solar radiation is always zero at night, regardless of the cloud cover. If we examined the correlation between cloud cover and solar radiation only during the day, it would be much higher.

• Interestingly, there is still a higher negative correlation between humidity and solar radiation, even though humidity and cloud cover are positively correlated but cloud cover and solar radiation are not terribly correlated. This is because at night when the solar radiation is zero, the humidity is generally higher because of reduced air temperatures. So at night humidity increases but solar radiation decreases (to zero), preserving some negative correlation.

Cross-Check

Since I had a sensor in my backyard for air temperature and relative humidity, and those quantities were also provided in the Visual Crossing data, I decided to do a cross-check. Before plotting and comparing the data, I had to reconcile the time scales. While the raw data from my sensor had a resolution of 10 minutes, the Visual Crossing data I downloaded had a resolution of 1 hour.³ So I converted the sensor data to 1-hour resolution using a rolling average. This hourly data will be used for the rest of this series of posts.

I’m happy to report that the relative humidity data had very good agreement. However, there seemed to be an issue with the temperature data. The sensor systematically reported air temperatures significantly higher than those from Visual Crossing in the afternoons, but only on sunny days. This plot shows the sensor and Visual Crossing air temperature, along with the solar radiation over the span of a few days:

The span was chosen to include a cloudy day, where the solar radiation was much lower, to show that the air temperatures agreed much better on cloudy days.

So while I took care to put the backyard sensor in the shade to avoid direct sunlight hitting it, apparently the hot Texas sun warmed up the local environment around the sensor anyway. ‘Official’ weather station thermometers are usually shielded from sunlight with a Stevenson screen or similar setup to avoid this problem. I decided to use the Visual Crossing data for air temperature and relative humidity for the rest of the analysis to avoid this potential systematic error with my sensor data and to get all the weather data from the same source. This perhaps sacrificed some ’local-ness’ of the weather data for better data consistency.

Evaporation

Because it is somewhat interesting by itself, and because I thought it would have a large effect on pool temperature, I wanted to estimate the amount of evaporation from the pool as a function of other weather variables. I found an empirical formula for the amount of water lost per day, expressed in kg per square meter of pool surface per day:

\(h_w\) and \(h_a\) are the humidity ratios at the water surface and in the air, which can be calculated from the vapor pressures at the water surface and in the air according to

and similarly for \(h_a\). The vapor pressures \(P_w\) and \(P_a\) can themselves be estimated from temperature using Tetens’ approximation and relative humidity (for air). \(P_{atm}\) is the atmospheric pressure, but it has only a weak effect on the overall evaporation rate.

\(v\) is the wind velocity (in mph) at the surface of the pool. Note that the Visual Crossing wind speed is measured at some height (usually either 1.5 m or 10 m) from the ground, but the wind speed at the surface of the pool will be significantly less than this, as it is actually slightly below ground and sheltered by trees, houses, and fences. After some web searches, I estimated that the wind at the pool surface is only about 30% of the ‘official’ measured wind speed, and used that reduced wind speed in the calculation. Let’s call this estimated wind-shielding factor \(\alpha_{wind} = 0.3\).

So the evaporation rate is a function of air and water temperatures, wind speed, humidity, and (weakly) barometric pressure. Here then is a plot of the calculated amount of evaporation from my pool, as a function of all the relevant weather variables, measured in kg of water lost per square meter of pool, per day:

Here is that same evaporation plot plotted together with several different weather variables:

As can be seen from the graphs, dry windy days cause water to evaporate quickly from the pool, as expected.

Another sanity-check calculation is possible here. On the ‘worst’ days, it seems to me like the water level of my pool can drop by about 1 inch per day. To check these numbers, I needed to estimate the surface area of my pool, which I did by measuring on Google Maps.⁴ I got a value of about 49 square meters. From the graphs, on the worst days the evaporation averages about 18 kg per square meter per day. So, converting kg of water to gallons, the evaporation rate from the entire pool is

18 \(\cdot\) 49 / 3.785 \(\approx\) 233 gallons/day

Which is in pretty good agreement with this online pool evaporation calculator.

An inch of water in my pool is a volume of water that is about

0.0254 \(\cdot\) 49 \(\approx\) 1.2446 m^3

A cubic meter of water contains about 264.172 gallons, so losing an inch of water corresponds to

1.2446 \(\cdot\) 264.172 \(\approx\) 329 gallons

So the predicted evaporation rate from the pool would be about

233 / 329 \(\approx\) 0.7 inches per day

Which is in pretty good agreement with my rough observation of “about an inch” of pool water lost on the worst days.⁵⁶

I pay the city about $8 for every 1000 gallons of water that I use, so it costs me about $2 to replace the water that evaporates from my pool on those dry windy days.

Evaporative Cooling

It takes significant heat energy to evaporate water, a fact that underpins evaporative cooling for phones (see Vaporizing is Cool), among other phenomena. So evaporation from a pool can cause a significant heat flow out of the pool. From the relationship

examined in the previous post, if the heat injected \(H\) is negative, you would expect evaporation to produce a negative \(\frac{dT}{dt}\) for the pool.

To better see the trends through the daily variations, I further condensed the hourly evaporation rate and the \(\frac{dT}{dt}\) for the pool to daily averages, and plotted them:

When plotted at this resolution, the hourly fluctuations (and possible glitches discussed in the previous post) are smoothed over, and an inverse relationship between the evaporation rate and the pool \(\frac{dT}{dt}\) can be seen - high evaporation rates mean more evaporative cooling, which means more negative \(\frac{dT}{dt}\) values.

But there are other factors that also influence the heat loss from the pool. Armed with the pool temperature data, the weather data, a way to calculate evaporative losses, and a strong hint that evaporative heat loss may be important, we are now in a position to make a more detailed model of how weather affects my pool temperature.