Ever since I retired my 1-Wire weather station and moved to the Davis Vantage Vue, I have been missing my solar and lightning sensors. I started looking into some DIY solutions using readily available parts. I also wanted the sensors to be wireless and have low power consumption. After doing some Internet research I came across the ESP32 SOC and ESPHome.

ESP32 is a family of low cost and low power microcontrollers that integrate both Wi-Fi as well as Bluetooth. The microcontrollers had more than sufficient processing power for the sensors I had planned. They had implementations of both I2C and SPI for communicating with common sensor chips. Since they had Wi-Fi built in, they could accommodate OTA updates and could even run simple web server software. This meant I could administer and update my new sensors remotely. While not being completely familiar with all the different variations of the ESP32 family of microcontrollers, I went ahead and ordered a few inexpensive development boards. This is the development board I ordered: Teyleten Robot ESP32S Development Board Dual-Core WiFi +Bluetooth. At the same time, I also ordered an AS3935 lightning sensor board, an AHT21 temperature and humidity sensor board, and a VEML7700 ambient light sensor module.

I chose to start by building the solar sensor. I laid out all the parts on a solderless breadboard and connected everything up using jumpers. Interfacing the ESP32 to the VEML7700 solar sensor was easy enough using I2C. I didn’t really need to, but I daisy chained in a temperature/humidity sensor for good measure. I had planned to use the Arduino IDE or the Espressif-IDF add-in for Visual Studio Code to program the ESP32. But then I learned about ESPHome, which already provided everything I wanted to do with the solar sensor. All I needed to do was create a simple YAML file containing some specifics about the ESP32 and the sensors attached to it. I decided to try it out. I knew I could always go back and write the code myself if ESPHome didn’t work out for me.

Here is a picture of the breadboard. This is after fixing some wires that my son’s cat helped to rearrange for me.

To setup the ESPHome development environment, I followed these instructions. Installing ESPHome Manually – ESPHome – Smart Home Made Simple. This installed Python and ESP home and provided the ESPHome command line application. Next, I created a configuration file, which I named ESPHomeConfig. Running the command “esphome dashboard ESPHomeConfig” started a web server and web-based application on my local computer. I accessed this application in my web browser at http://localhost:6052. This browser-based application permitted me to connect to my ESP32 through a USB connection and create the YAML file that specified what sensors were connected, how the ESP32 would communicate with them, and how to connect to my wireless network.

Creating this YAML file using examples from the Internet turned out to be very simple. I flashed the program to the ESP32 on the solar sensor and watched the logs as the ESP32 restarted and connected to my wireless network. It was able to read the temperature, humidity, and solar data from the attached sensors. I attempted to connect to the ESP32 via my web browser and was greeted with a page that displayed my data as well as a scrolling log of messages from the chip. Since the sensor was now online, I tried doing an OTA update. It worked with no issues.

Now that the solar sensor was operating properly, I needed a way to get the data to my weather station server. I saw that ESPHome provides a method for publishing data to an MQTT broker. This seemed like an easy way to transfer the data. The solar sensor would publish the data to MQTT, and my weather station server could subscribe to it. I added the appropriate configuration to the YAML file and just like that the solar sensor data was published to MQTT. It was fairly easy to add a MQTT data collector to the weather station server software. The software needed to do a few computations using the published solar data, but the equations are provided by the AS3935 chip manufacturer.

Now that the solar sensor was working, the data was being published to MQTT, and my weather server was able to retrieve the data and post it to my web site, I needed to work on packaging the sensor. The first thing was to wire everything more permanently, as the circuit was on a solderless breadboard with wire jumpers. I purchased some solderable breadboard PCB boards, laid out the chips, soldered them in place and wired everything back up. Upon testing I found that the permanent layout was working fine. Here is the solar sensor on the solderable breadboard.

I started working on a 3D printed case for the circuit board, but that is still a work in progress. Here is how the solderable breadboard will fit in the case. This is just a prototype to make sure I got the dimensions correct and put the locating pins in the correct arrangement.

The only work left on this sensor is finishing the case for it and finding a suitable location to install it. I also need to build a lightning sensor to replace my old 1-wire version. That will be my next post.

I use Nagios software to monitor my home network, including the hardware and software services. When there is an issue with anything being monitored Nagios sends me an email alert. I recently received an alert that the computer (Raspberry Pi 1B) that runs my weather station was offline. I checked and the computer was powered up but was not reachable on the network. I rebooted the computer and it came back online, but all of the software services that run on that computer were not running.

I attempted several times to reboot the computer and connect to it over the network with no success. I use an external hard drive with that Raspberry Pi and it appeared that the Raspberry Pi would start to boot from the SD card and then fail when it attempted to access the external drive. I removed the drive and connected it to another Raspberry Pi and ran “fsck” on the drive. When I saw the number of errors that scrolled across the screen I knew the drive was badly corrupted. In the end there were no files left on the drive and everything that could be salvaged was in the lost+found directory.

While the fsck command was running I decided to reimage the SD card that boots the computer. This was because I needed to update to the most recent version of Raspbian and thought this was a good time since I was going to have to create a new bootable hard drive. I booted the Pi from the SD card and it came back to life. I connected another external drive to the PI and it was not recognized. I attempted replacing the drive with several other external drives with the same result. All of the external drives worked with another Raspberry Pi I had so the drives weren’t the main problem.

In the end I connected the Davis Vantage Vue console to another (and slightly newer) Raspberry Pi that was already up and running. I took all of the weather station software and data files I had salvaged using fsck and installed them on that computer. Then I brought the weather station service online on the new host computer. This is most likely a temporary solution but for now everything is operational again.

Once again satellite images were not showing up on my satellite page. I did some quick research and found a new set of links to the satellite images. I believed that I was using an approved API to get the images, but apparently I was mistaken. Either that or the API was changed or removed. So now the satellite images are working again. I’ll just have to keep chasing them around the NOAA site or else find a different source for them.

My Web hosting provider has added an SSL certificate to my web hosting plan. This certificate permits me to use SSL encryption on all my web pages. If you don’t know what that means, I can now use the encrypted https protocol rather than unencrypted http. In the past there was a performance penalty for using SSL, but nowadays this penalty is minimal and it is generally recommended to use SSL wherever possible. Without a certificate I was unable to use SSL encryption on my web site. But with the addition of the certificate to my account I was able to make the switch.

The first thing I did was to check using my browser to see if all the pages loaded when I typed https://www.cloppermillweather.org. All my pages loaded, but none of the images were showing. When I switched back to the unencrypted protocol everything loaded fine. After a lot of head scratching and a bit of searching I learned that my web site had a feature called Hotlink protection turned on. What Hotlink does is prevent other web sites from linking directly to resources on your web site.This prevents the other site from using your images in their web pages and in effect stealing your bandwidth. I don’t recall turning on this feature for my web site so I’m guessing it was on by default. It blocks all links to images except for sites with an exception set in the software. The problem is that there was only an exception for my unencrypted web site. I added an exception for the encrypted site and the images were now visible.

The next change I wanted to make was to redirect requests from the unencrypted site to the encrypted one. This would force anybody connecting to my site using http to be automatically switched to https. I added the following lines to the .htaccess file for my site and now all the http requests changed to https.

# redirect to ssl page if request is for the non-SSL page
RewriteCond %{HTTPS} off
# Redirect to HTTPS
RewriteRule .* https://%{HTTP_HOST}%{REQUEST_URI} [L,R=301]

I made these changes to all my web sites hosted with this provider, and as far as I can tell they all are working fine. I haven’t done any concrete measuring, but I am unable to observe a performance penalty. I’ll consider this a success so far.

On July 8 2018 we had an historic rain event that triggered flash flooding and water rescues all over the area. I was excited to see what my rain gauge had recorded, and as soon as I had an opportunity I checked my web site. I was shocked to see that no rainfall had been recorded that day. I checked the console for the Vantage Vue and it also read no rainfall that day., We had some rain a couple of days earlier and that had registered, but nothing for the big event that was still occurring as I checked.

This obviously upset me, and I went out in the rain to check my weather station. I assumed that something was blocking the gauge and preventing the water from entering it. When I went outside to look, I saw that nothing was blocking the gauge and that the rain was entering and exiting the gauge normally. At this point the only thing I could think of was to take down the weather station and attempt to service it in my dry garage.

Once I got the gauge down from its’ pole and into the garage I started inspecting it. I learned (and should have remembered) that the rain and temperature sensors were designed to be easily removed and cleaned. I removed both of those sensors and gave them a good cleaning. I didn’t find anything clogging the rain gauge but I cleaned it out anyway. Then I reassembled the station and reinstalled it on the pole.

When I went back inside and checked the weather station console it said it was not receiving the weather station at all. I was unable to get the station and console to pair, so I took the station back inside to try and pair them up in a place where it was nice and dry. I eventually got them paired back up and reinstalled the weather station back on its’ pole for hopefully the last time.

Going back inside and looking at the console I saw rain being recorded once again. Success finally.I The moral to the story is that I need to do regular maintenance on the weather station. Davis designed it to make it easy to clean. I just hadn’t done it for a long time. I’m going to add a reminder to my calendar to clean out the gauges annually at a minimum.

A few days ago my Web hosting provider upgraded the server hosting my web site to PHP version 7.2. I haven’t made any attempts at running my web site under anything newer than PHP 5.6. At least not until now. Most of the site still worked fine. The only page that failed is the advisories page that displays the NWS discussion of the current weather conditions. I was using some functionality that has been deprecated and no longer works in PHP 7.2.

I made some updates that wouldn’t have been difficult if I had kept my limited PHP development skills up to date. Once I had refreshed myself on the PHP side of my code the updates weren’t very difficult. I needed to replace a few preg_replace calls with the newer preg_replace_callback. Once that was completed the site worked again.

I have uploaded the updated code to the site. This may also serve to motivate me to update some of my supporting jar files since the versions on them are likely quite old. That’s what I get for not keeping my software up to date.

Today I noticed that none of my satellite images were showing.  I’m not certain how long this was broken.  NASA had changed their naming conventions on the GOES images.  Fortunately they provided an API to permit embedding the images on web pages.  I modified my satellite page to use the API.  I also added some images from the National Hurricane Center.  I may still make some changes to the images page to make it more customizable.  But for now it is fixed.  I will upload the updated source code for anybody that might be interested.

We experienced a cold spell where temperatures fell below zero degrees Fahrenheit.  Unfortunately my scheme for decoding the Davis Vantage Vue data didn’t handle negative numbers properly.  I did some quick research and found that Java offers a class called a ByteBuffer that could handle the conversion and also handle any endian issues that might occur with the conversion.  I made a quick change in the code and everything appears to be working OK.  The true test will be when the temperature drops down below zero again.

It took me a few days, but I finally relocated my Raspberry Pi and set up the revised weather station software.  Everything appears to be working just fine.  I plan to just let things run as they are for now and see what develops.  Hopefully the weather station is all set for the winter.  I also need to post the updated software to my Web site in case somebody plans to use it.  I’ll revise this post once it has been uploaded.

Well, I sort of predicted there would be some kind of hardware failure in my future.  The predicted failure finally occurred.  After some diagnosis it looked like my 1-wire hub failed.  Also, my barometer became intermittent, occasionally returning a reading of zero.  Some research proved hat the hub was no longer available, and support for 1-wire in general wasn’t very good anymore.  Hobby Boards, which was the source of most of my hardware was even preparing to go out of business.  It was becoming clear to me that 1-wire was probably not the best choice to purchase as new hardware.

After some investigating, I decided that my best choice was a Davis weather station, either the Vantage Pro or Vantage Vue.  They both are solar powered and wireless, which was a big step up from my previous weather station.  The Pro would be my preference, but cost drove me to the Vue.  The two features I wish the Vue had are a fan aspirated solar shield and the ability to separate the anemometer from the rest of the station for better siting.  But when I looked at the cost I realized that my current station has all the sensors mounted together, even though I could separate them.  And I could figure out a way to add a fan if solar heating became an issue.  So I placed my order for a Vantage Vue and the Weatherlink hardware/software.

Once the new hardware arrived, I set it up indoors for testing.  After I verified that it was working I mounted the hardware sensors outside and made sure the console was receiving a good signal.  I had no issues and everything worked perfectly.  Then I installed the Weatherlink module and connected it to my computer in order to test that interface and modify my weather software to work with the Davis hardware.  After a bit of experimentation and reading the Davis documentation I got the software talking to the Weatherlink module.

At the moment the revised software is running in my development environment, which is a Windows 10 desktop computer.  I plan to migrate everything back to the Raspberry Pi in the next few days.  Since the Pi isn’t running my weather station software at the moment I took this opportunity to use apt-get to upgrade any software running on the Pi.  It hadn’t been upgraded, powered down, or rebooted in well over a year, so it was long overdue.  I plan to relocate the Pi to another room in my house, so it will take me some time to make room for everything.  I hope to have everything running back on the Pi within the next few days.