Mail server with local and virtual users

OK, so I wanted an Eierlegendewollmilchsau (Egg laying woolly milk pig). But Linux obliges (hooray), with postfix, dovecot, spamassassin and clamav. Postfix is the basic sendmail transport layer. Dovecot is the imap mailbox handler.

To add to complexity, my ISP doesn’t allow incoming traffic on port 25, so I use Ghettosmtp to relay mail on a non-conventional port to my home. Then I wrapped everything up in ssl using Letsencrypt.

Ports to forward on the router:

2525 => 25 (smtp)
143 => 143 (imap)
993 => 993 (imaps)
587 => 587 (smtps)

I ask Ghettosmtp to forward my mail to my server on port 2525. I configure my nameserver to point its MX records to ghettosmtp as per their specifications. It takes 24 hours for them to honour the request.

Sending mail is another hurdle. I setup postfix to relay outbound e-mail through my service provider.

My main resource for this setup task was the appropriate Ubuntu community help page. Some fidly bits include adding some “standard” folders that were ommited from the add-virtual-user scripts: Archives and Templates – although it seems that Thunderbird will cause auto-creation of the archive folder (and a year folder inside it) when you archive something. I also added a couple of stanzas to the dovecot.conf to allow local unix users to be hosted as well as virtual users, and SSL is not covered by the linked help document.

Finally, when I noticed that the spam-bots of the world had discovered me after only 4 days of operation, I added spamassassin to my setup. Details for setup including configuration files.

Posted in technical | Leave a comment

Add Development tools to the PiServr

As a developer, I would like continuous integration set up on my home server. I am most familiar with Jenkins, and that is part of the debian distro – so I decided to go that route.


$ sudo apt-get install jenkins

Connect via localhost, and configure security (before the next part). use local database, and matrix based permissions. Make sure to “allow users to sign up”, and add read overall permission to anonymous before adding another user and giving them full permissions. Then add yourself (as the full permissions user). Then you can go back and disable allowing users to sign up.

Edited /etc/default/jenkins to enable listening on the actual network interface so you can connect from your network (and port-forward your router to connect from the internet).

Posted in technical | Leave a comment

Privacy, security, low cost – The Self Hosted PiServr.

Your very own slice of Raspberry Pi can host all your personal cloud needs, with real privacy, for less than $100 all told.

piservr

Pictured above is a Raspberry Pi single board computer sitting atop a Western Digital Pi Drive – in my case a 1TB disc, which hosts this blog and a few other services.

A complete server solution, including MariaDB (open source SQL) and Apache web server. Not on the cloud. In your living room, or your office shelf. Consuming a few measly watts – cents per month. No Amazon bills, none of your data need be in the hands of the data giants. Now you can have your Pi and eat it.

Hardware costs in USD:

Raspberry Pi                          35.00
WD PiDrive Foundation edition 250 GB  28.99
Blue Square enclosure                 14.99
3A Power supply                       12.99
                                     ======
Total                                 91.97

The 1TB drive I use is available for a mere $30 more.

What can it do? I have more than 600 GB of files (mostly movies and music) on it. It hosts my blog sites, and a mediawiki site which I use for project documentation. I have two printers (an HP laser and an Epson inkjet) connected to it, so we can print from anywhere in the house (including from my phone or Chromebook). I can play my music through a connected stereo. I can even access the Pi remotely, using either ssh and a command line or using X11 forwarding to run graphical software from any X Window capable computer (running Linux or Cygwin X-Window).

This project will make available a downloadable image, which you copy to a micro-SD card. This image boots your Pi. Then you connect to your Pi using a web browser (Chrome, Firefox) and enter some simple configuration details to enable each web service. Modify your router’s forwarding to connect your Pi’s services to the internet. Then use a dynamic DNS service to locate your server from the outside world.

Posted in technical | Leave a comment

Now super secure-ish!

Whoopie! I have just successfully installed strong ssl encryption (https) for this WordPress blog. Thanks entirely to the great folks at letsencrypt – with a generous G+ prompt in my feed from +Steven Vaughan Nichols. I say Secure-ish because I hate to tempt fate (or hackers) with over confidence, and as has been said, WordPress has a reputation for insecurity.

Posted in technical | Leave a comment

Ram 1500 ecodiesel review – 6 months, 30,000 km

I have owned Big Blue, my 2016 Outdoorsman ecodiesel for just over 6 months, and driven just over 30,000 km. I have made one significant trip – driving from Hamilton to Orlando and back for a disney vacation, made my daily 65 km commute, and towed Marshmallow (an 11 year old Percheron Mare) 3 or 4 times in her gooseneck trailer. My daughter is a frequent rider in the back of the Quad cab, as well as a couple who sometimes ride-share with me on the work-commute.

Throw in sundry trips to the hardware store (4 x 8 ply, 12′ cedar planks etc), grocery shopping, taking the dogs to the park and you get the life of Big Blue.

She is the Outdoorsman rig, with 4×4, skid plates, tow hooks, trailer mirrors, trailer tow brakes, limited slip diff, Quad cab (like extended in other brands), 6’4″ bed, in dark blue (Blue Pearl) with black lower panels and grill. I took the Comfort package (I hate the cold wheel in the winter), and the deluxe package (which had the multimedia options I consider necessary these days), and the electric sliding rear window. The only option I am not convinced of is the rear window. I did not take the backup camera (and it might have saved me a replacement taillight – cost 175). One serious gripe I have is with the ordering process. Both the online customer facing build tool, and the salesman’s ordering tool, showed that the outdoorsman package would come with the low ratio rear axle (for greater towing capacity). However, the higher ratio was delivered, and we were never able to resolve why, or make some satisfactory arrangement to compensate for what I felt was a mistake on their part.

I have not babied the truck – I haven’t really washed or waxed her since owning her, and I have made small “bondo” patches to the rear after scraping it a couple of times (and busting a taillight) due to my underestimating her size.

My average MPG is 25.56 Miles per US gallon (30.7 miles UK gallon), or 9.2 l/100 km overall. This includes all the driving types I mentioned, over a total of 30,302 km. My mileage has been tracked on Fuelly.com, a handy web site that enables you to track mileage fill-up by fill up.

So far, I have had 2 oil changes – every 12000 km. These have been quite expensive – the first at $175, second was around 350 as they did some other maintenance work – tire rotation, brake, exhaust and suspension inspection. I am left not terribly happy about the second service – I will double check the invoice for the work done. That said, the truck has performed perfectly, with 0 problems. I had a gooseneck hitch fitted (by Hitch City) a couple months ago for trailering our horse, which cost around $1000. The next thing I really need to do is to get the truck lined. I plan to buy a bucket of the liner stuff and paint it myself (being a cheap Yorkshireman as my wife says). I have found a source for oil and filters online which would cost me about 150 per oil change – not much saving, but I sometimes like to do it myself anyway. Before our disney trip, I purchased a soft tonneau cover online for about 275.

So now for the driving. The truck rides really well – smooth and well behaved, not jittery and “truck like”, thanks to the all round coil springs. Under load, or towing she rides even better – towing the gooseneck with Marshmallow aboard with great aplomb – all in a days work, and averaging under 15 l/100 km while towing her. The bed lights are great for fishing stuff in and out of the back at night. There is sufficient room in the back even for occasional adults, and I am happy that I chose the Quad rather than the Crew cab – the bed length is good. If the truck was any longer, parking would be next to impossible on my street. As it is, I have to park within an inch to avoid sticking out fore or aft over my neighbour’s driveways.

Acceleration is somewhat modest from start (I was driving a Fiat 500 before this truck), but passing at speed seems fine, and towing power feels good. Turbo lag is somewhat pronounced (compare to my wife’s VW diesel), but not a problem when you get used to it. The brakes are great. The tires are pretty grippy in poor conditions – she felt really sure footed in the tail of of Winter in March earlier this year. The driving position is great, with a commanding view of the road, and reasonable over the shoulder visibility. The mirrors are really good, almost eliminating the blind spot. The seats – firm fabric – are comfortable and very adjustable including lumbar support – good for my back considering how much time I spend in this truck – about 3 hours daily.

The Disney trip was a good test at 2 months and about 7,000 km in. I got my best fuel tank on that trip, making 8.3 l/100km (28.3 mpg US or 34 mpg UK) on a stretch coming back up the coast through South Carolina and Virginia. She pulled up and down the Apalachians like a champ, the 8-speed gearbox switching away handily, but never making me feel unnecessarily. The ride was comfortable (as a 10 hour a day drive can be). On my daily commute, I still average about 9 l/100 km, on an average speed of about 80 km/h (mostly 100 but some start-stop in traffic).

As a cost-benefit analysis of the diesel, I look at Fuelly.com for my consumption. I have paid $2,359 in fuel over 30,302 km. If I had purchased the 5.7 Hemi (the pentastar would not have met my towing needs), with an average consumption of 14.7 l/100km, I would have spent about $4,000. So I saved about $1600 in 6 months. Factor in the extra cost of the diesel – $5,700 – 1400 for the hemi = 3300. It will take me less than 3 years to pay off the initial investment. However, since I financed the truck anyway, my monthly fuel savings are way more than the extra monthly billing for the diesel option. In other words, I actually have about $200 extra in my pocket every month when adding both the monthly payments and the fuel costs together. Mind you – I don’t have a hemi – but I consider that a small price to pay next to the savings I am enjoying.

Posted in technical | Leave a comment

Upgrading Mediawiki

So very recently mediawiki became “no longer supported” in debian Jessie. This coincides with the fact that the version in Jessie is a venerable 1.19 (current version is 1.26). I discovered this while trying to get a flexible banner extension working (WikidataPageBanner), and it wouldn’t play nice. So I decided to upgrade my wiki.

Hmmm.

Here is the Wiki about upgrading the Wiki! (Proof that it worked).

The one thing I wish I had done before starting is to prevent my ssh connection from timing out. It ended up backgrounding the database backup task and doing stuff every minute just to make sure the shell stayed open.

Posted in technical | Leave a comment

Choosing a php Framework

I have an old (10 years) web application that was designed to build, track and print characters and create spell lists for the 3rd edition of Dungeons and Dragons.

The application is written in php, without a framework, with a MySQL backend. It still works, and prints out nice pdf character sheets, but I don’t play 3.x anymore. We have (happily) transitioned into 5th edition, and I miss the functionality. So my main task consists of a re-write for 5th edition. At the same time, I want to use a framework, both in order to learn something new (I know, sucker for punishment), and to make it more maintainable going forward.

I did a bit of comparison “shopping”, and ended up taking a short dive into zend. Unfortunately, I found zend to be a little too nuts and bolts. Although I was coming from hand coding the whole thing, I have played with Rails, so I was expecting a little more framework than zend seemed to offer (and frankly, better or easier to follow docs). I am now trying Symfony, and I am much more impressed. It seems to have a very rails-like philosophy and layout, and I actually prefer php syntax to ruby. To me, ruby seems like a programming language written by someone who has a grudge against C, whereas php is unabashedly rooted in C like syntax, which is familiar to me.

I just noticed one weird side effect – all my (self hosted) sites now show up with the Symfony logo on the tab. Must be an apache configuration screw up!

Watch this space for details of how this progresses.

Posted in technical | Leave a comment

The TissueCam

An idea just came to me how I could package the dashcam case – in a tissue box! I could make one of those open-bottom tissue box covers that can sit on the dash (many cars come with a recess to hold just such a box).

That way, the Pi can be subtly hidden in plain sight. The box would be slightly (the thickness of the Pi board) wider towards the windshield, It would hold a normal box of tissues. The streaming interface would be used to view the camera in real time to adjust its position. The main challenges would be connecting to power inconspicuously – although it could be a handy USB hub, and keeping it stable and not sliding around. Some inconspicuous rubber feet, and some solid (laser cut plywood) construction should do the trick, I think, as well as look pretty good.

The size of a low profile tissue box look like 4 3/4″ x 9 1/2″x 2 7/8″ tall inside dimensions, which should hold a Raspberry Pi nicely when mounted vertically. The outside dimensions of my box would be about 6 1/4″ x 10″ x 3″ when constructed with 1/4″ ply.

View from below (not to scale):

+----( )-----------+
|    cam   ---Pi++ |
+------------------+
|                  |
|    (tissues)     |
|                  |
+------------------+

I would have enough room for the power supply board and supplementary battery. The Lens would fit into a hole on the front edge (more to the centre than my picture above). I will post a better diagram once I have sketched out the plan to scale.

Posted in technical | Leave a comment

Pi Dashcam Software

Having pieced together the hardware for my development system (Raspberry Pi Model A, Pi Camera, Microstack GPS module, SunAir power supply, Edisoft WiFi), it is now time for me to create the software.

Here is my feature wish list, in roughly priority order.

  • Forward facing camera
  • Save video recording to USB (or server via WiFi hotspot)
  • Save up to preset maximum video, in a ring buffer unless record button pressed, which will save previous clip plus next 2 clips permanently
  • GPS and timestamp to video
  • Rear facing camera (USB)
  • Recorded video playback on network device.
  • Live stream capture to a cell phone display.
  • Automatic data transfer to server via network.
  • Local battery, smart charging.
  • Accelerometer activated record mode.
  • Unattended car mode.

Forward facing camera.

The forward facing camera is the official Pi Camera, so the python picamera module will do all the heavy lifting (did I mention that I intend to write everything in python?

On startup, the unit should check for USB storage available (and if so, create directory tree if non-existent), and do all recording to USB. Otherwise, recording should happen to local (SD) recording directory.

Implementing a simple ring buffer should be straightforward. I will capture video in 20 second chunks continuously. When the ring buffer is nearly full, I will delete the oldest chunk and continue. If a record event occurs, I will link the previous and the current chunk to the save folder, and setup the next chunk to be linked to the save folder, until the stop recording button is pressed. It occurs to me that the start and stop buttons can be the same button, and that a double press will work to save the previous current and next (3 segments of video). An LED to show that we are saving video would be a nice addition to the hardware. I could use the camera LED to indicate this, as long as I make the LED visible to the driver.

GPS and Timestamp.

Using the GPSD daemon and it’s python module, I can get the current Lat/Long and I can use the GPS time to set my system clock (since the Pi lacks a real time clock).

The picamera module include the ability to add a line of text overlay using the annotation feature. We can use this to provide a timestamp and GPS position information (and speed) to the recording.

Rear facing camera.

To use a USB camera (cheap webcam) from python, OpenCV has a suitable python module. It is (perhaps) a pity that we need two different APIs one for each camera, but such are the vicissitudes of life.

Recorded video playback on network device.

Use recording to network stream, or file mounted by NFS?

Live stream capture to a cell phone display.

Hmm. Tricky? It seems we can create a custom output stream which wants a write and a flush method. We can use this to write to a file, and to a network stream with the same frame. I will need to play with this to see how it works (and how well it works).

Automatic data transfer to server via network.

On connection to local WiFi, a dashcam script can copy all recorded video chunks to a predefined server location (may use ownCloud for this).

Local battery, smart charging.

Using the SunAir card, I can connect a mini-usb for normal power, and a LiPO cellphone battery for backup. The SunAir will charge the battery when the car is running. The SunAir will power the Pi from either the USB or the battery (when the ignition is off). I hope to find a signal from the SunAir card that will tell me when USB power is interrupted so I can perform a smooth shutdown of the Pi.

Accelerometer activated record mode.

Automatic recording when a significant accelerometer event occurs (strong braking, sudden change in direction etc). There is a python module for using the accelerometer.

Unattended car mode.

Mode to record video when a vibration occurs. Perhaps this will be normal operation with file save triggered by a very low threshold accelerometer event, or even a sound event (captured by webcam microphone).

Posted in technical | Leave a comment

Minimal Pi Dashcam

Here I am creating a dashcam with simple software features and a minimal UI. One may think that this is pointless given the plethora of cheap devices already available. Indeed, the simple designs I have seen can be had for as little as $30. However, a two camera design competes with dashcams that cost over $200, and with added GPS, the home build becomes competitive. The new release of the Pi zero now has added camera support! I will continue prototyping with the Model A, and switch to the zero when it arrives (albeit I had to subscribe to the magazine to get it).

Main component pricelist:

Raspberry Pi zero      5    Raspberry Pi (via magazine)
Pi Camera             25    Raspberry Pi (on hand)
GPS module            25    Microstack (on hand)
WiFi adapter          10    Edisoft (on hand)
USB camera            10    (Tiger Direct)
Wide angle lens       10    Aukey kit (Amazon)
Power supply          25    SunAir (on hand)
Battery               free  Old cell phone (on hand)

Obviously these are retail module prices, and a finished manufacturing design would have much cheaper components – but that is unlikely to happen, since this is just a hobby and I lack the resources to compete with manufacturing companies.

At this point, it may or may not have a display (I have a handy 2.8″ 320×200 display from Adafruit). As a headless design, it could connect using WiFi, and push chosen video to a cloud drive (such as google drive, or ownCloud), or simply save to a USB key. It could use a model A (with WiFi or a USB stick), or the latest model 3 B (which has WiFi included), or even the elusive Pi Zero. A benefit of a headless design is that the only component that would have to be visible would be the camera (and it’s cable).

Conceptually, it will take video in x second chunks, in a loop of y segments. On the “save” button press, it will save the last y segments, and continue saving for another z segments – where x, y and z would be programmable parameters. A rear facing camera would be a nice addition (using USB since the Pi only supports a single CSI camera interface). Audio would be another option (requiring a microphone – but the rear USB cam might have one included). Conveniently, I think my truck would make mounting a rear camera fairly easy.

The user can view video using their existing mobile device or computer screen, either from the USB stick, over WiFi from the cam, or their cloud of choice.

I intend to use this guide as the starting point. I would gladly credit the author, but I can’t establish his name.

So far, I have installed minibian on an 8 GB SD card. I have expanded the filesystem, renamed the system to “dashcam”, enabled the camera and also SPI and I2C (required for the Adafruit screen), and updated the software. I made the necessary modifications (from Adafruit’s website) to the kernel and boot config to drive the display. I also installed fbcp – a tool which copies the internal GPU frame buffer to the SPI console frame buffer – required for the SPI driven screen to show the Pi Camera output in real time. This may go away if I switch to headless design. Initial tests show the video from the camera on the display. I installed omxplayer to allow local replay of the h264 video produced by the raspivid tool.

In terms of hardware I have ordered a set of camera lenses from Aukey to get a wider angle on the Pi Camera. I could have gone cheaper, but I wanted glass lenses. I have a small camera bracket, and a couple of simple Pi cases I can use until I have settled on a configuration and mounting method.

I want at least two buttons – one to trigger saving video, and one to initiate shut down. If I go with the touchscreen, it has optional physical buttons already. Otherwise I will use buttons on GPIO lines and project them through the case.

I have a GPS module which I can integrate, software can overlay the time and coordinates on the video. If I take this route, I am strongly considering adding navigation software and a larger screen (7″ Pi Display) to make full use of it. NAVIT sounds like a strong contender for this. However, I shall continue building from small and see how it works out. Another piece of hardware to consider is an INU. The accelerometer can be used to trigger autosaving – in the event of a crash for example – although I suspect a simple tilt sensor would also work for this purpose. As well, an INU can be used to augment or smooth GPS data using a Kalman filter – although I am not sure whether (or how hard it would be) to integrated that into NAVIT.

I successfully hooked up my gps module (a microstack board), by connecting the 3.3, ground and TX/RX lines to my Pi headers. One issue I ran into with gps is the gpsd package does not much like systemd. On initial install, it works fine, but as soon as I reboot, it does not start. running dpkg-reconfigure gpsd starts it up again, but again it fails on reboot. I followed the instructions here, and replaced systemd with tradition sysvinit support, and my problems miraculously vanished.

I then discovered another problem – raspbian jessie comes with libgps21 and the dashcam scripts were built for version 2.2. Anyhow – I think I need to go beyond the scripts from the pidashcam guide. I think I will write a python application from the ground up, using the picamera module, gpsd.

Last but not least, I need to look at the power supply. Car cigarette lighter chargers usually drop abruptly as soon as they engine is stopped – in order to protect the car battery. This could be disruptive to data collection, and possibly injurious to the SD card. Ideally, I would have a constant 5 v supply available, and a notification when ignition goes off, in order to shutdown gracefully. I am considering tapping into the overhead cabin light supply (as this is available after ignition off), and find another line in the vicinity to tell me when ignition is off.

On further consideration, I realize I have a SunAir Solar Power Board kicking around – I bought it on a kickstarter some time ago. It is a $25 board that controls charge to a 3.7 V LiOn battery, and supplies 5 V to Raspberry Pi-like devices. It expects 6 V from a solar panel, or a mini usb connector to charge the battery / power the device. This, coupled with a cell-phone battery or two will ensure I have a smooth shutdown, or ignition-off power available. The downside of this device is that it supplies only 1000 mA, which would only supply the model A + camera safely.

Posted in technical | Leave a comment