My first Artificial Intelligence project.

I’ve had the idea for Project Leroy for some time. The pandemic has provided the time. The inspiration came from my Grandfather Leroy. He and my Grandmother loved birds. I remember visiting Grandma and Grandpa’s house and seeing bird-watching books by the window. Just outside was a collection of different feeders. They also enjoyed cuckoo clocks. They had one in their home that sang a different song every hour.

Project Leroy is applying Artificial Intelligence to bird watching. The goal is to be able to identify any bird species that comes to the feeder.

Raspberry Pi 3, Google Coral, Camera Module

The first step was getting the hardware. I already had a Raspberry Pi 3 laying around for the main system. I picked up the Google Coral USB module for the model processing. Lucky enough, Google Coral had a bird feeder example project that became my starting point for the code. Last addition was the Raspberry camera module.

The second step was setting up the software. Upgrading the raspberry pi from python 2 to python 3. Installing the Google Coral example project and the like.

The third step was using a prebuilt model to see what I could identify. It came with the ability to identify 1000 different objects. I ran the program and watched the logs to see what it was picking up. I was able to verify birds using my kids stuffed animal in the shape of a chicken.

The fourth step was getting birds in front of the camera. I set up a feeder, but it takes time for birds to start using it. I tried moving the camera to place in my yard that had frequent bird visits, but it yielded nothing. I learned that for a bird to be recognized it had to be really close to the camera.

Sparrows started showing up at the feeder. While they were feeding I captured several hundred photos that I could use to train the model. I also took photos of the feeder with no birds on it so the model could tell the difference. This took some trial and error. The same photos at different times of day had different shadows and coloring that could fool the model. For example, if I only used feeder photos when the sun was overhead, a feeder with an afternoon shadow may be categorized as a bird. My training data eventually had the feeder and several bird feedings from throughout the day.

The fifth step was updating the code to only save the best photos. I settled on treating roughly 80% confidence for “sparrow” as marking a keeper photo. Even scores as high as 70% would still give me false positives. This will improve as I get better at tuning the model. My guess is that any bird will get categorized as a sparrow initially. But with each new species I can create a new category with its own training data and slowly expand to all species that use the feeder.

This is just the beginning, but I am excited. It will provide insights into the local bird ecosystem. When birds feed, what species are around, and what seasons certain species are more prevalent.

A final touch on my first iteration was to add a Twitter account. When I get a photo that registers with above 90% accuracy the code will autonomously post a photo to the Project Leroy twitter account.

You can find the Project Leroy code on Github.

I leave you with Project Leroy’s first autonomous tweet!