Configuring iBeacons

iBeacons are a great way to locate you within your apartment without draining the battery too much. An iBeacon is basically a Bluetooth-Low-Energy device that just broadcasts an identifier in periodic intervals. iOS and Android devices that receive these broadcasts can identify the iBeacon and even calculate the distance to it.

For in-door navigation, the cheapest iBeacons works best as their range is limited. This makes it easier to locate you without using complex triangulation algorithms. When you move, you will just leave a cell when you are too far away for the iBeacon.

As most existing iBeacons use a Texas instruments chipset they are not extremely cheap. Even when ordering directly from China, it is hard to find them for less than $10. One iBeacon that is relatively cheap is the Ghostyu. It consists of a small PCB and a relatively large battery on the back.

ghostyu

Using a single piece of these is simple. You can just use them as they come. If there are neighbours or people on your way to work that use the same beacon, you might run into trouble as the basic configuration is the same for all of these devices. The basic configuration is:

UUID: E2C56DB5-DFFB-48D2-B060-D0F5A71096E0
Major: 0
Minor: 0

All 3 IDs can be used, but the UUID is always being used (you can ignore major and minor).

Therefore I recommend not only to set major and minor to different values than the default, but also UUID. The UUID is an 128-bit ID, you can just randomly select one without having to fear that somebody else in the world will use this. The easiest way is using an online UUID generator.

But how do you change the UUID? For the Ghostyu, there is an iOS app called LightBeacon. Unfortunately it is completely in Chinese. However, clicking around a bit should bring you to the following configuration dialog:

lightbeacon

Here, you can not only define UUID, major and minor ID, but also the advertising interval. This can have a major impact on the battery life. If you want to use iBeacons to turn the light on when you enter a room, 0.5s might still be ok. If you just want to track you and turn the light off when you left the room for some time, even the longest interval of 4 seconds is more than enough. In many use cases, 4 seconds will work fine and result in a longer battery life.  Now just store the value to the iBeacon. It will reboot and advertise the new UUID, major and minor.

Power supplies for LED installations

If you look around on the Internet you will find a lot of interesting DIY LED installations. Most projects are stripes for lighting and Ambilight clones. Last week I stumbled across a very good guide how to build your own Ambilight clone. I won’t link to this guide as it had one big problem: safety!

Many projects use power supplies similar to this one:

pl8886134-short_circuit_protection_standard_led_display_power_supply_100w_5v_20a_ip20_60hz_en1122

Why? Because they are available quite cheap from many Chinese suppliers. I can’t say if a specific power supply itself is safe. With this type of power supply, the build quality is usually quite good because you can see most parts and people wouldn’t buy them if you can already see that parts are not soldered correctly. Note that this doesn’t say anything about the quality of the parts. But let’s just say, the manufacturer designed this power supply well and it is safe. Why shouldn’t you use something like this? Why isn’t it safe?

Many DIY installations install power supplies in places where they are hidden, but can still be accessed  from the outside. Check out this picture:

powersupply-unsafe

You won’t see the power supply from the front, however, the mains is still relatively easy accessible. Even if you don’t have kids at home, this is unsafe!

There is an easy way to deal with this problem: You should use fully capsulated power supplies with standard AC cables like this one:

laptop-adapter-ac-adapter

This type of power supplies is also available in many different voltages with a power range of up to 100W – more than enough for usual LED installations.

Bluetooth problems on the Raspberry Pi 3

The Bluetooth tracker in Home Assistant is a cool device as it allows you to track if people are home based on their mobile device. With the Raspberry Pi 3’s onboard bluetooth chip, this is a great features as it does not even require additional hardware. However, I’ve had major problems getting it running. Bluetooth was enabled on the mobile phone, but Home Assistant just did not find any device.

Ok, checking from command line:

[email protected]:~# hcitool scan
Scanning ...

No bluetooth devices, even if there are bluetooth enabled devices available. Let’s try with “blutoothctl”:

[email protected]:~# sudo bluetoothctl
[NEW] Controller B8:27:EB:76:70:7C raspberrypi [default]
[bluetooth]# devices
[bluetooth]# scan on
Discovery started
[CHG] Controller B8:27:EB:76:70:7C Discovering: yes
[NEW] Device 04:69:F8:xx:xx:xx 04-69-F8-xx-xx-xx
[NEW] Device 88:C6:26:xx:xx:xx 88-C6-26-xx-xx-xx
[bluetooth]# quit
[DEL] Controller B8:27:EB:76:70:7C raspberrypi [default]

Clearly, blutoothctl sees other devices. Maybe a daemon isn’t running?

[email protected]:~# systemctl status bluetooth
● bluetooth.service - Bluetooth service
   Loaded: loaded (/lib/systemd/system/bluetooth.service; enabled)
   Active: active (running) since Mon 2016-06-13 15:53:36 UTC; 1h 54min ago
     Docs: man:bluetoothd(8)
 Main PID: 729 (bluetoothd)
   Status: "Running"
   CGroup: /system.slice/bluetooth.service
           └─729 /usr/lib/bluetooth/bluetoothd

Jun 13 15:53:36 raspberrypi systemd[1]: Starting Bluetooth service...
Jun 13 15:53:36 raspberrypi bluetoothd[729]: Bluetooth daemon 5.23
Jun 13 15:53:36 raspberrypi systemd[1]: Started Bluetooth service.
Jun 13 15:53:36 raspberrypi bluetoothd[729]: Starting SDP server
Jun 13 15:53:36 raspberrypi bluetoothd[729]: Bluetooth management interface 1.10 initialized
Jun 13 15:53:36 raspberrypi bluetoothd[729]: Sap driver initialization failed.
Jun 13 15:53:36 raspberrypi bluetoothd[729]: sap-server: Operation not permitted (1)

Ok, this looks strange. While bluetoothd is running, there is an error message from sap-server. Looking for the error message, I found a long thread on raspberypi.org. The solution: add a command line option to the Bluetooth daemon

Edit the file /etc/systemd/system/bluetooth.target.wants/bluetooth.service and add the option “–noplugin=sap” to the ExecStart line:

[Unit]
Description=Bluetooth service
Documentation=man:bluetoothd(8)

[Service]
Type=dbus
BusName=org.bluez
ExecStart=/usr/lib/bluetooth/bluetoothd --noplugin=sap
NotifyAccess=main
#WatchdogSec=10
#Restart=on-failure
CapabilityBoundingSet=CAP_NET_ADMIN CAP_NET_BIND_SERVICE
LimitNPROC=1

[Install]
WantedBy=bluetooth.target
Alias=dbus-org.bluez.service

Ok, let’s check the daemon again:

[email protected]:~# systemctl status bluetooth
● bluetooth.service - Bluetooth service
   Loaded: loaded (/lib/systemd/system/bluetooth.service; enabled)
   Active: active (running) since Mon 2016-06-13 17:56:35 UTC; 9s ago
     Docs: man:bluetoothd(8)
 Main PID: 19106 (bluetoothd)
   Status: "Running"
   CGroup: /system.slice/bluetooth.service
           └─19106 /usr/lib/bluetooth/bluetoothd --noplugin=sap

Jun 13 17:56:35 raspberrypi bluetoothd[19106]: Bluetooth daemon 5.23
Jun 13 17:56:35 raspberrypi bluetoothd[19106]: Starting SDP server
Jun 13 17:56:35 raspberrypi bluetoothd[19106]: Excluding (cli) sap
Jun 13 17:56:35 raspberrypi bluetoothd[19106]: Bluetooth management interface 1.10 initialized
Jun 13 17:56:35 raspberrypi systemd[1]: Started Bluetooth service.

This looks better. However, even after a reboot,

hcitool scan

still does not show any Bluetooth device.

To be continued …

Battery powered ESP8266 sensor

The ESP8266 is a powerful and inexpensive device. There is almost no need anymore to use proprietary RF protocols like the nRF24 chip that was popular some years ago. However, there is one major problem when using WLAN: power consumption.

When connected to a WLAN, the ESP8266 consumes at least 80mA – sometimes even more. If you use 3 2400mAh that means, they will be empty in a bit more than a day. This doesn’t sound like a good deal.

Luckily, for sensor applications, you don’t have to be connected to the LAN the whole time. You don’t even have to have the ESP itself running. To save as much power as possible, you have to do the following:

  1. Collect your data. If this takes time, do it before connecting to WLAN. If it takes just a few millesconds, it doesn’t matter much.
  2. Connect to WLAN.
  3. Send data.
  4. Go to deep sleep mode. You use the ESP.deepSleep() method for this.

You have to look at a few things when using the ESP.deepSleep() method:

  1. To wake up after sleeping, GPIO16 of the ESP8266 has to be connected to RESET. Otherwise the device won’t wake up again.
  2. The delay parameter is defined in microseconds (not seconds or milliseconds). If it looks like your device isn’t sleeping at all, you might have used a value that is too small (you won’t even notice if the device sleeps for 3600 microseconds)
  3. There is a second parameter that can have the values WAKE_RF_DEFAULT, WAKE_RFCAL,WAKE_NO_RFCAL, WAKE_RF_DISABLED. The WAKE_RF_DISABLED looks good, but beware to use it. Using this does not mean that the RF subsystem is disabled during sleep (it always is), but that it is disabled when waking up again. Your ESP8266 won’t be able to do any network communication anymore.

A few other tips to save energy:

  1. Do not turn on LEDs for a longer time. Flashing them a few milliseconds is usually a good visual feedback and will consume only a tiny bit of energy.
  2. If you need to have LEDs on for a longer time, dimm them. analogWrite is your friend for this.

Check out this sample code. It uses WiFiManager which helps to connect to configure your WLAN SSID/password and even the name/IP address of the MQTT broker.

#include         // Generic ESP8266 WiFi routines
#include        // MQTT client
#include           // Local DNS Server used for redirecting all requests to the configuration portal
#include    // Local WebServer used to serve the configuration portal
#include         // WiFi Configuration Magic


const char* mqtt_server = "192.168.1.88";
const char* mqtt_channel_up = "active";
const char* mqtt_channel_data = "sensor/%d/adc";

const int pin_red = 15;
const int pin_green = 12;
const int pin_blue = 13;
const int pin_button = 4;
const int analog_ldr = 0;

const int sleepSeconds = 60 * 15;

WiFiManager wifiManager;
WiFiClient espClient;
PubSubClient client(espClient);


void send_sensor_data(void) {
  Serial.println("Sending sensor data\n");
  int data = analogRead(A0);
  char s_channel[50];
  char s_id[10];
  char s_data[10];
  sprintf(s_channel, mqtt_channel_data, ESP.getChipId());
  sprintf(s_id, "%d", ESP.getChipId());
  sprintf(s_data, "%d", data);

  digitalWrite(pin_blue, HIGH);
  client.connect("ESP8266Client");
  digitalWrite(pin_blue, LOW);
  if (! client.connected()) {
    Serial.println("Could not connect to MQTT broker, doing nothing");
  } else {
    digitalWrite(pin_green, HIGH);
    client.publish(mqtt_channel_up, s_id); // show that this device is up
    client.publish(s_channel, s_data); // send sensor data
    digitalWrite(pin_green, LOW);
  }
}

void setup() {
  delay(500); // This makes it easier to upload new firmware and/or press the "clean" button

  pinMode(pin_red, OUTPUT);
  pinMode(pin_green, OUTPUT);
  pinMode(pin_blue, OUTPUT);

  digitalWrite(pin_red, HIGH);

  // Setup Serial interface for debugging
  Serial.begin(115200);

  //  Bring up WLAN
  WiFiManager wifiManager;

  //sets timeout until configuration portal gets turned off
  //useful to make it all retry or go to sleep
  //in seconds
  wifiManager.setTimeout(60);

  WiFiManagerParameter custom_mqtt_server("server", "mqtt server", mqtt_server, 40);
  wifiManager.addParameter(&custom_mqtt_server);

  if (digitalRead(pin_button) == 0) {
    // reset settings
    wifiManager.resetSettings();
  }

  //fetches ssid and pass and tries to connect
  //if it does not connect it starts an access point with the specified name
  //here  "AutoConnectAP"
  //and goes into a blocking loop awaiting configuration
  if (!wifiManager.autoConnect()) {
    Serial.println("failed to connect and hit timeout");
    delay(100);
    // do nothing: deep sleep, but longer than usual. Something might be wrong and
    // we should save power
    ESP.deepSleep(sleepSeconds * 10 * 1000000, WAKE_RF_DISABLED);
  }

  mqtt_server = custom_mqtt_server.getValue();

  Serial.println("");
  Serial.println("WiFi connected");
  Serial.println("IP address: ");
  Serial.println(WiFi.localIP());

  // Setup MQTT client
  client.setServer(mqtt_server, 1883);

  digitalWrite(pin_red, LOW);
}

void loop(void) {
  send_sensor_data();
  Serial.printf("Sleeping deep for %i seconds...", sleepSeconds);
  ESP.deepSleep(sleepSeconds * 1000000);
  delay(100);
}

Program an ESP8266 from Arduino on MacOS

There are a lot cheap ESP8266 boards available on the market. The modules are quite powerful and come with WiFi. While this is a problem for ultra-low-power applications (there are tricks to deal with this), it is a great platform for simple sensors or notification applications. You can even program them with the Arduino IDE. This guide shows how to do this.

Install the Arduino IDE

You can download the IDE from the Arduino web site

Connect the ESP8266 module to your computer with an USB cable

Many ESP8266 modules already come with an integrated USB interface. Just connect the module with your Mac.
Note: Some modules have a USB port on the board that can only be used to power the board but isn’t made for communication.

Check that the device is recognised

Open the “About this Mac” dialog and click on “System report”
systemreport

You should now see the a USB2.0 serial device. usbserial

Install the driver

If you now start the Arduino IDE, you can select a device like /dev/cu.usbmodem147741 (the numbers can be different) as the serial interface. However, if you try to communicate with the board, it won’t work. While MacOS recognised that the device is a serial interface, it does not come with the driver for the cheap Chinese CH340G USB/serial converters (in the past some companies used fake FTDI chips until FTDI implemented counter-measurements in their drivers that disabled some fakes). However, installing the driver is not complicated. Download it, and install it. After a reboot, you Arduino IDE should now offer an devices named like /dev/wchusbserial14460. This is the correct serial interface device now.

Install the ESP8266 modules for the Arduino IDE

Arduino does not come with out-of-the-box support for the ESP chip. To install the software, go to the preference dialog and enter
http://arduino.esp8266.com/versions/2.2.0/package_esp8266com_index.json
under “Additional board manager URLs”.

arduino-settings

Now click on “Tools/Board/Board Manager” and install the ESP8266 modules from the “Contributed section”.

espcontrib

That’s it. You have successfully installed all necessary tools to start programming your new IoT device.

If you can’t program your ESP device from the Arduino GUI, have a look at esptool.py. This seems to work better than esptool bundled with Arduino. I did not find a simple way to integrate it directly into Arduino. However, starting it from command line is not very complicated. Just check the command line arguments.

Install an MQTT broker on your Raspberry Pi

Not only for home automation purposes, a central MQTT broker can be a good system that connects sensors and control applications. MQTT has been designed especially to be very lightweight. Therefore, it will also work very well on a Raspberry Pi. Installing it is very simple:
sudo apt-get install -y mosquitto mosquitto-clients

Your Mosquitto server should be already running now. You can test this easily.
mosquitto_sub -h localhost -v -t test

This starts a MQTT subscriber waiting for incmoming messages on the “test” channel.

Now open a new connection to your Raspberry Pi and type the following:
mosquitto_pub -h localhost -t test -m "Hello world, Mosquitto"

The first process should now display the “Hello world” message.

Control EDIMAX SP-1101 or SP-2101 from Home Assistant

The Edimax SP-1101W and SP2101W are supported in a standard home assistant installation. However, you need to configure a user name. During setup of the smart switch you can set a new password, but never a user name. However, it is easy: the username is “admin”.
To control an Edimax switch from Home assistant, just add the following block and set the correct host, password and name parameters for your installation.

switch:
  platform: edimax
  host: 
  username: admin
  password: 
  name: 

How secure is KNX?

Looking at modern home automation communication technologies (like Z-Wave or Zigbee), it seems that most of them have some security issues. These may or may not been fixed in the future. If these modern architectures already show a lot of problems, what about KNX? Let’s make it quick: from an computer security point of view, KNX is not just insecure, but the worst kind of protocol. All devices communicate on a shared medium, there is no encryption, no authentication, no authorisation. If you have access to the KNX bus, you can do everything.

Luckily, there is one good thing about KNX: You need access to the bus. If somebody has physical access to your bus, he can do crazy things. But this means, this person is already in your flat. Even without home automation, a person that has physical access to a light switch can turn it on and off. This means that for a residential installation, cabled KNX installations are perfectly fine, if they are running standalone. However, modern installations aren’t standalone. Nobody wants to install an home automation system anymore that is not somehow connected to a network.

Here are some tips how to make sure this system is still secure.

  1. Don’t simply connect your KNX network to your home LAN (that might be even allow guest WLAN access).
    KNX/IP interfaces are cool products and you will need one. However, be clear that these do not have any security built in. That means everybody that has access to your network can do everything on your home automation installation. This might not be the best idea. I would recommend some gateway that makes sure only authorised system can access your KNX bus.
  2. If you use any commercial product to connect your KNX bus to your LAN, (there are a lot of products available) make sure, the supplier provides regular updates and reacts on security incidents. If the last firmware you can find is 17 months old, the supplier most likely doesn’t do a good job. Most of these gateways are Linux based and you will need regular updates for it.
  3. If you implement some kind of gateway/firewall by yourself, make sure it is designed with security in mind and you also update it regularly.

Remark 1: The concept of physical security can be very critical in environments where multiple parties share a KNX installation. This can be a problem not only in office buildings, but also apartment complexes. If you live in an apartment with some kind of “smart” technology, have a look how it is implemented and who might be able to access it.

Remark 2: With ETS 5.5 the KNX association now supports some kind of encryption. I haven’t looked into it. Why? Because all old devices do not have any idea about it. It might become more popular in the future, but in today’s practical KNX installations, it is usually not supported by the devices that are already in place.

Draw your own LCARS home automation user interface with Javascript and SVG

Some time ago I wrote already why I like LCARS as a user interface design for home automation. Looking around on the internet you will find several frameworks to build your own LCARS-styled UI. Some of them are quite old and I did not even look into them. I hope nobody is using Flash for new UIs anymore. Others use a lot of image files. I don’t like this approach as the controls are really simple and you don’t need to load a graphics file for a simple rectangle.

The project I liked most was the LCARS SDK. It is using Javascript in the browser and

blocks to build a UI. However as the documentation wasn’t always clear it took me a lot of time to build even a simple UI.

But why using

blocks? There is an even better way: SVG. Scalable vector graphics are part of the web standard since more then 10 years, but they aren’t used widely yet. Fortunately all modern browsers support them and it is very simple to draw some rectangles and circles. You can also add styling using CSS, modify them with Javascript and add Javascript event. Rendering is very lightweight as the number of objects is usually relatively small (between 50 and 500 objects on screen). This makes this approach also ideal for browsers on tablets and mobile phones.

Programming a simple control frame doesn’t need a lot of code:

function home() {
    lcarsgen.container=document.getElementById("canvas");
    lcarsgen.next_x = 10
    lcarsgen.next_y = 10
    lcarsgen.knee({x:10, height2: 40})
    save_x = lcarsgen.next_x
    lcarsgen.button({x:10, text:"Light", id:"b_light"})
    lcarsgen.button({x:10, text:"Heating", id:"b_heating"})
    lcarsgen.button({x:10, text:"Blinds", id:"b_blinds"})
    lcarsgen.button({x:10, text:"Weather", id:"b_weather"})
    lcarsgen.button({x:10, text:"Movies", id:"b_movies"})
    lcarsgen.button({x:10, text:"Music", id:"b_music"})
    lcarsgen.button({x:10, text:"Config", id:"b_config"})
    lcarsgen.knee({x:10, height1: 120, height2: 40, \
                   type:"bottomleft"})
    save_y = lcarsgen.next_y - lcarsgen.spacing - 40
 
    // top bar
    lcarsgen.next_x=save_x
    lcarsgen.button({y: 10, width: 310, height: 40} )
    lcarsgen.button({y: 10, width: 310, height: 40} )

    // save for text display
    tx = lcarsgen.next_x-lcarsgen.spacing
    lcarsgen.button({y: 10, width: 55, height: 40, \
                     type:"rounded_right"} )

    // bottom bar
    lcarsgen.next_x=save_x
    lcarsgen.button({y: save_y, width: 310, height: 40} )
    lcarsgen.button({y: save_y, width: 310, height: 40} )
    lcarsgen.button({y: save_y, width: 55, height: 40, \
                     type:"rounded_right"} )

    lcarsgen.text({x:tx, y: 50, height: 40, text:"H.11 CONTROL", \
                   text_align: "right"} )
}

The result look already pretty cool:
Screenshot 2016-05-12 12.15.52

Interfacing a KNX bus with Python

If you are looking for frameworks that allow you to interface a KNX bus using an IP interface, you will find a lot of tools. Many people still use eibd. However, looking at the eibd page you will see that eibd is no longer maintained.

If you read the KNX specification, you will notice that KNX packets are quite small with a simple structure. No XML stuff with namespaces as often used in modern APIs. So why not implementing the communication in a small script? Unfortunately it is a bit more complicated than just sending a packet to the KNX/IP interface an wait for the answer.

While KNX itself is connectionless, the KNX/IP interface isn’t. This means you first have to initialise a control connection to the KNX/IP interface and use this for data transmissions. Also you have to acknowledge every packet, otherwise the KNX/IP interface will drop the connection. Does this seem complicated? It isn’t.

A simple version of a KNX/IP communications stack (with very limited functionality) can be implemented in less than 400 lines of code. This even implements caching. This means the daemon actively listens to the KNX bus and stored the state of every object internally. Just reading the value of an object than does not need any KNX communication when the value has been seen on the bus already.

Using this simple interface, it is very easy to exchange messages with KNX group addresses:

from knx.ip import KNXIPTunnel
import time
import logging

def main():
    logging.basicConfig(format='%(levelname)s:%(message)s', level=logging.DEBUG)

    tunnel = KNXIPTunnel("192.168.1.128",3671)
    tunnel.connect()
    
    while (True):
        # Toggle the value of group address 0/0/1
        tunnel.group_toggle(1)
        
        # display the values of group addresses 0/0/1 to 0/0/5
        for i in range(1,6):
            v=tunnel.group_read(i)
            print("{} = {}".format(i,v))

        # delay
        time.sleep(12)
        
if __name__ == '__main__':
    main()

References: