Remote Farm camera using Raspberry Pi – alpha test version

At The Oak Tree we have acquired two bullocks/steers, and it would be good to be able to monitor these guys to check they are okay.

cattle
cattle

The obvious way is a GSM trail camera, which has got some merit. However, these are typically tripped by movement, and you get a lot of that outside. It might work to use a trail camera with a 15 minute trigger lockout – it’d be tripped all the time but wouldn’t spam you senseless with the 15 min delay. But it’s going to be dear to run as they tend to use MMS – giffgaff charge 16p per MMS for instance, whereas their data rates are 20p per 20Mb . CCTV solutions have the same problem – they are event triggered, whereas we want to be able to see a picture taken sometime in the last 15 minutes. Well, obviously we’d really like realtime streaming video by a pic at most 15 minutes old is a decent second best 🙂

What we ideally want is a camera you can send a message to to take a picture.

I used to have a Youthnet V900 which you could send an SMS to take a picture, but it was the most maddening hunk of junk I have ever purchased. Configuring the settings was done by SMS, and I only ever got it to work with Virgin mobile once. Then Virgin must have changed something and I never got this to work again, either with Virgin or with Giffgaff. In the end I put a hammer through it so it wouldn’t waste any more of my time and money. The satisfaction of doing that was the most value I got out of it.

I took a look at this trail cam and recognised the same commands as the Youthnet . I’m just not going to go there again if it uses the same software. Plus there’s the fact that it doesn’t really suit what I want it to do.

The Pi’s USB subsystem isn’t good enough to run a 3g dongle

So it’s time to break out the Raspberry Pi. Unfortunately the USB subsystem of the Pi means I couldn’t use Gammu directly with a Huawei E156G USB 3G dongle. It would work for a while and then conk out, and only come good again when the Pi was power cycled, not even a restart would fix it. That’s a real shame – this would save me a lot of power drain. Maybe it’s specific to this dongle, but there is always a world of hurt when it comes to the Pi and USB because of the power problem, and a 3G dongle has a high power usage when connected. Even running the E156G off an approved USB separately powered hub and the Pi off an approved PSU didn’t make it any better.

Go MiFi

Obviously if your site has WiFi you don’t need to wrangle with 3G, but there is a lively market in boxes to put a WiFi hotspot using 3G, called MiFi devices. You need to configure these to make use of your 3G dongle (and make sure they are compatible), the type I used is a TP-Link MR-3220

1501_TL-MR3020-V1-03

So I used a MICRONEXT  MN-WD552B WiFi dongle in the A’s single USB port, which seems reliable. Of course I now eat the power drain of the Pi, WiFi is never low power and the TP-Link 3020 MiFi device gives me another power hit. But it works, and I needed results fast rather than good, so it’ll do for now.

The Raspberry Pi Camera module is really rather excellent

The camera module that the Pi foundation sell for of the Pi is rather good – the larger version of the pic above is already scaled down by half in the camera to reduce the file size. And I compressed it a fair bit. You can control the camera in Python too, using Picamera. Python seems to be the default language for the RPi.

Although the Raspberry Pi is credit card sized, it’s remarkable how big it gets if you want to do anything with it. I used a box I already had – if I decide this works well enough for a permanent version then I will use a PICE waterproof case, but for the prototype a temporary rig is fine.

the Pi gets big fast when you hang enough onto it to make it useful
the Pi gets big fast when you hang enough onto it to make it useful

At the moment both are powered up all the time, and a cron job takes a picture and SFTPs it up every 15 minutes during the day. The Pi and the TP3020 are powered via one of those dual-output 12V car to USB converters, the power drain at 12V is 250mA which is horrible, meaning using 40Ah of a 80Ah leisure battery will take 40*4=160 hours, nearly a week. Which is nasty, but livable with. There is massive opportunity to cut the power drain by bringing up the TP-Link 30210 and the Pi only once every 15 minutes when it’s light – they can be up and do their stuff in 5 minutes so I’d get a 3x drop in power drain from that, and not running in the night would save me another third.

The installation is bush-league, but this is going to need a bit of optimisation anyway

the camera is at the top left of this IBC
the camera is at the top left of this IBC

and a close-up shows just how bad it is – waterproofing by electrical tape is living on borrowed time. But that’s where those nice people at PICE come in, so there’s no point in putting any real work in that.

best not let the IEE see that else they'll revoke C.Eng for disgraceful and outrageous bodgery of the first order ;)
disgraceful and outrageous bodgery of the first order 😉 The PICE case has solved most of the mechanical challenges kludged together here.

This shows one of the hazards of this location, in that the crows sit on the bar surveying their domain and crap on the IBC. There’s no crap on the bars though, so I might get away with it. Some anti-bird spikes might be in order.

 #!/usr/bin/python
#$Id: takepic.py 53 2014-06-26 16:30:17Z richard $
import time
import picamera
import paramiko
import os
import socket
import datetime
# import RPi.GPIO as GPIO

# GPIO.setmode(GPIO.BOARD) # USE Pi BOARD pins, not the BCM ver
# GPIO.setup(7, GPIO.IN, pull_up_down=GPIO.PUD_UP) # 7 is next to gnd on pin 9, so set pull up

# defs
camerafail=False;
DIR='/home/pi/mine/'
imagename='cam01.jpg'
remotename='mysite.com' # assuming this is reachable by ssh and www



try :
    with picamera.PiCamera() as camera:
        #camera.resolution = (2592, 1944)
        # The following is equivalent
        #camera.resolution = camera.MAX_IMAGE_RESOLUTION
        # run half res to test out connectivity etc and save money
        #camera.led = False
        camera.resolution = camera.MAX_IMAGE_RESOLUTION
        #camera.resolution = (1296, 972) # do half real to eliminate Bayer softness and save TX bandwidth
        camera.start_preview()
        time.sleep(2)
        camera.capture(DIR+imagename, resize=(800,600), format='jpeg', quality=15)
except picamera.PiCameraError,e :
    print e
    camerafail=True
finally :
        camera.close()
time.sleep(10) # hopefully nw is up by now   

if not(camerafail) :


    timedout=False
    connected=False
    counter=0

    while (not timedout) and not connected :
        try :
            s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
            s.connect((remotename,80))
            print(s.getsockname()[0])
            connected=True
        except socket.error,e :
            
            counter += 1
            print counter
        finally: 
            s.close()
        time.sleep(5)
        if counter >= 5:
            timedout=True
            print 'Failed to connect to ',remotename,' ',datetime.datetime.now().strftime("%y/%m/%d %H:%M")
    
    # Ideally, this gets called by cron once every quarter of an hour. 

     
    #upload
    if not timedout: 
        print 'ftp image starting ',datetime.datetime.now().strftime("%y/%m/%d %H:%M")
        try :
            ssh = paramiko.SSHClient()
            ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
            ssh.connect(remotename, port=2222, username='username', password='password')
            sftp = ssh.open_sftp();
            sftp.put(DIR+imagename, '/my/remote/path/'+imagename)
            sftp.close()
            print "closed SFTP"
        except paramiko.AuthenticationException,e :
            print e
        except socket.error,e :
            print e

The cron job

This is then called by a cron job on the Raspberry Pi. You must to this with root privileges because you are accessing the hardware, so use

sudo crontab -e

and mine looks like

*/15 4-22 * * * /home/pi/takepic.py > /home/pi/log.txt

which means every 15 minutes between 4am and 10pm run takepic.py and save the error messages etc to log.txt. Before running this as a cron job you want to make sure that if you manually run

sudo /home/pi/takepic.py

you get a picture uploaded to where you want it 🙂

Goose Incubator Camera

Getting a camera in an incubator has a lot of the aah factor and lets you see what’s going on without opening the incubator up all the time, which is good for the incubator, the eggs and the reliability of hatching.

You have to take into account that the incubator is a fairly harsh environment. There are lots of ways of making an incubator camera in principle, but  the heat and humidity rule out a lot of them, like webcams and trail cams.

Five requirements of a Incubator camera

  • sealed against moisture
  • elevated temperature running
  • physically small
  • capable of shortening the close focus to ~ 5cm
  • externally powered

I used an outdoor bullet camera I got from RF Concepts – it’s an old variant of this one.

camera
camera

It used to be on top of a fence post looking at my bird feeder, and stood the test of time against the rain. I removed the square part on the front and tried replacing the lens with a wide angle lens, which stuck out too much and cracked the glass.Fortunately replacement mineral watch crystals are to be had – a 24.8mm was a suiable replacement. Replacing the original lens and unscrewing a bit to get the short focus closer worked a treat, and the replacement glass and O rings have held so far.

A single channel Vivotek 8100 CCTV webserver turns the analogue CCTV signal into a IP-compatible one and thei nbuilt FTP client uploads a picture every so often to make it publically viewable. The 8100 could also stream video but I don’t have a good enough connection for that.

Goose hatching camera (inactive most of the time!)

Real IP CCTV cameras are still on the expensive side and a little bit too big for this job.

It’s Goose incubation time again

Goose eggs – pure aggression in a small package. Apparently geese spew the eggs out, about 30 per goose per year, then collect them all up and sit on ’em. Last year we incubated some eggs, and because geese are waterfowl, getting the humidity right is key to improving the hit-rate.

look out, here comes trouble
look out, here comes trouble

Last year I used Xively for the display, but the Cosm/Xively TCP stack is ratty. Certainly for an Arduino, which has only just enough horsepower to run a TCP/IP stack, but the Xively linux library wasn’t that reliable on the Raspberry Pi that I now use as a concentrator.

goose eggs
goose eggs
Incubator temperature

I now use RRDtool, which can run on the Raspberry Pi,

sudo apt-get update
sudo apt-get upgrade
sudo apt-get install rrdtool

and there is a Python library that can inject data into the RRD databases called python-rrdtool

I really didn’t get on with python-rrdtool, I invoke RRDtool explicity

getstatusoutput("/usr/bin/rrdtool update "+DIR+"AB N:1234")

where DIR is the full directory path

 

 

 

 

Humidity and Temperature sensor on Cosm IoT

The trouble with geese is that they are waterfowl, ie used to hatching near water, unlike chickens, so you tend to have to control the humidity of the incubator as well as the temperature. Humidity is terribly counterintuitive – unlike temperature humans have no real sense for it. So I was looking at how to make a humidity and temperature sensor.

Goslings
Goslings. Like most animals, geese don’t look so mean when young

The problem starts with our cheap Chinese import incubator. There’s no real calibration on the temperature dial, and you know it’s a bad sign when the manual tells you ‘when your small poultry is ready’. I was really sceptical about how well this would stabilise the temperature. Quite unfairly as it happens. Continue reading “Humidity and Temperature sensor on Cosm IoT”

Fix for ratty behaviour of Wiznet 5200 with the OpenKontrol Gateway

I’m a fan of Ciseco’s OpenKontrol Gateway. It doesn’t get anywhere near enough love on their site compared to the flighty Eve. Where it scores is that it’s a cheap way of making stuff happen and I like that.

Ciseco OKG

It makes a dandy data logger, basically get the RTC kit and the SD card socket, slam one of their XRF radios in the left hand socket plus a LLAP thermistor device within 50 yards and you’re all set 🙂 You can add more thermistor devices while the datalogger is running and they get logged too without bumping it. And all for less than about £80 total system cost, what’s not to like? Okay in an ideal world the power consumption could come down a little bit as the OKG is in receive only mode, but it’s livable with – using 8 NiMH AAs gave me a couple of days runtime.

thermistor board with XRF

thermistor board in box

 

This saved our tail with finding out what the soil temperature was like in and outside a polytunnel earlier this year

reading of soil and air temp inside and outside polytunnel

What you can also see in the OKG picture is a Wiznet 5200 Ethernet card. What I didn’t realise is that this damn thing has a microcontroller inside. And it’s reset at the same time as the Arduino, so you get some sort of evil who-dares-wins race to get out of the starting blocks when that reset line goes High. The horse you want to win is the Arduino, so as the Wiznet’s ready to get initialised by the Arduino, but it doesn’t seem to happen that way.

Looks like with some configs the wiznet wins the race, and presumably gets confused when it sees data and stuff. I never got it to work reliably with the SD card datalogging. I the got another OKG, this time without the SD card and RTC option, just for posting LLAP devices from the XRF to Cosm. And this damn thing was about 50% reliable, it  gave me a world of hurt until I discovered this natty fix for this sucker over at Open EnergyMonitor. I am so grateful to them. Basically you dis the reset on the Wiznet and ground the sonofabitch once the Arduino powers up, so you know where it starts from rather than whatever it got to see on the data lines as everything Arduino and SD card and XRF and what have you fired up. Sounds good to me, and it works. I had considered chucking a monostable at this to ground the line for a sec so the Arduino can sort itself out, but caught the Wiznet getting its knickers in a twist after a while having started okay. So the option to deck it if it returns more than five -ve values for etherclient post is necessary. I went off pin 3 of the RH socket which is arduino pin 6 ISTR, but I need to see if there is a genuinely spare pin because one day I might want to use the OKG as an RF to RF gateway which would mean getting both slots into service. I’m open to pinching the red status LED if necessary 😉

The Wiznet seems actually reliable now, and if it does get itself stuffed in an IoT application returning a negative value for a post five times in a row I simply reset it and reinit the device. It’s not the end of the world if five temperature readings get lost, as long as recovery is possible 😉

You still have to patch the Arduino Ethernet library as per this wiznet blog to make it go.

Logging soil and air temperature in a polytunnel

I have used two Dallas DS18B20 sensors to log the soil temperature inside and outside a polytunnel, as well as a free-standing RF thermistor probe for the air temperature.

reading of soil and air temp inside and outside polytunnel
reading of soil and air temp inside and outside polytunnel

The air temperature reading clearly has issues in the day, but the soil temperature readings show the value of a polytunnel. The blue trace, inside, shows the soil temperature to always remain above 5C, which is apparently important for seed germination. The temperature excursions to be smaller, ranging over about 8C rather than 12 for the outside soil temperature, and the outdie soil temperature falls to 0 at one point.

The air temperature suffers in the day from the problem of the greenhouse effect, which is of course the whole point of a polytunnel, but the sensor additionally suffers from not being in thermal balance in the day.

Measuring temperature in a room is easy, and the one-box LLAP sensors do that fine. However, setting them up where the sun shines on them means the sensor gets heated by the sun and no longer accurately reflects the ambient air temperature, because the black case is heated up faster than it radiates / convects heat away, so it’s no longer in thermal equilibrium.

The way to address this is to use an aspirated thermomenter, which has a thermal screen from above to stop solar radiation heating the sensor and a fan to waft air across the sensor – it doesn’t need a high fan speed but it needs some.

However, at night solar radiation isn’t a problem, so the air temperature reading is useful in the night, where it shows the air temperature falling below zero inside the polytunnel from about 9pm to 6am.

The soil sensors are about 5cm below the surface, which is typically where seeds would be planted.

Project Overview – what I’m trying to do with the sensors

I have two applications for sensor networks. One is at home, where I’ve focused on temperature monitoring. I am trying to reduce gas costs by using a 6kW wood burner in the living room to heat the house, and it’s interesting to see how the heat propagates to the various rooms. Insulation is good and I achived this last year. It’s a fairly standard application of the IoT idea, so probably not of much wider interest.

The second is for gathering data over a 12 acre smallholding, the Oak Tree Low Carbon Farm. This is more challenegin as it is an island site without mains power or data communications. Remote sensing has a lot to offer a smallholding – temperature and light logging are interesting. Monitoring the state of the various 12V batteries used on site could make life easier, and control applications for managing the irrigation pumps may be feasible.

Initially I am using a Ciseco OpenKontrol Gateway to log the data. I’m not particularly using it as a gateway, more a generic Arduino kit connected to the sensors that happens to have a SD card interface and RTC on it. Eventually that could be used as its name indicates, to groom more significant status changes and alarm via SMS if necessary, for instance low battery or low water for the chickens.

At the moment I am using it as a straight datalogger for temperatures at home, to gain familiarity with the sensor and Arduino stuff. I will probably move to an EVE device for that, and move the OKG to the farm.

The first thing to do is get the OKG datalogging working right for a sensor network, in particular writing the data in a format that is usable for me. When I loaded the original version of the datalogger it didn’t work as I’d expected, specifically it didn’t log data. Some hacking was required.