Flask and server-sent events

I recently discovered the existence of the HTML5 server-sent events standard. Although it lacks the bidirectional communications of a websocket, SSE is perfect for the publish-subscribe networking pattern. This pattern just so happens to fit in conveniently with writing software to remotely monitor hardware that many people might want to check in on at the same time.

In order to try SSE out within a Flask framework, I put together a simple demo app using gevent. The core of the demo on the Python side looks like this:

app = Flask(__name__)

def event():
    while True:
        yield 'data: ' + json.dumps(random.rand(1000).tolist()) + '\n\n'

def index():
    return render_template('index.html')

@app.route('/stream/', methods=['GET', 'POST'])
def stream():
    return Response(event(), mimetype="text/event-stream")

This can be run either using gevent's WSGI server or gunicorn using gevent workers.

Update 2016-04-21: There is now a very nice Flask extension called Flask-SSE which handles all of this for you. It additionally supports the concept of channels in order to fine tune what notifications a given client receives.

more ...

Using websockets with Flask via Tornado

I've been working on some projects for the lab that involve remotely controlling hardware to perform various tasks. Since the hardware in question is shared between different experiments, some sort of asynchronous solution is needed, and a web-based client coupled with websockets seemed to be the best bet (this also leaves the option open in the future to write a standalone client that is not browser-based if desired).

There is no shortage of web frameworks for Python. Some of the more popular ones are Django, Flask, Tornado, and Pyramid. Of these, I greatly prefer Flask for a number of reasons:

  • Very thorough and easy to read documentation, including "snippets" with helpful tips and a very helpful community.
  • Extreme ease of use for both small and large projects.
  • Great use of decorators to further ease development.
  • A large number of extensions to build up a complex project without requiring overhead for simple projects.

This is not to say that the other options are bad, but having looked at all of them, Flask suits me best. The one problem: only Tornado directly supports websockets since it is both an HTTP server and a web framework in one, whereas the others utilize WSGI for deployment.

Luckily, it is possible to leverage both the excellent asynchronous features of Tornado and the power and ease of use of Flask through Tornado's ability to serve WSGI apps with tornado.wsgi.WSGIContainer. The Flask documentation shows a very simple example on how to do just that.

Integrating …

more ...

ssh-agent for sudo authentication with a passwordless account

For best security on a public system, it is generally best to disable password-based logins with ssh and instead require authorized keys. However, this complicates things if you want to use sudo with a regular user account, since by default it uses the standard system password to verify the user is authorized to run commands as root.

Enter pam_ssh_agent_auth. This module allows using regular ssh keys and ssh-agent to verify the user has the proper authorization to use sudo.


You'll want to start by ensuring you have generated ssh keys for your user and are using ssh-agent. To generate the keys:

$ ssh-keygen

Then just accept the defaults, but make sure to set a password for your new key pair. Add the public key to $HOME/.ssh/authorized_keys.


Since the PAM module isn't in Debian, first grab the build dependencies:

# apt-get install build-essential checkinstall libssl-dev libpam0g-dev

Next, grab the source and build:

# wget http://downloads.sourceforge.net/project/pamsshagentauth/pam_ssh_agent_auth/v0.10.2/pam_ssh_agent_auth-0.10.2.tar.bz2
# tar -xvjf pam_ssh_agent_auth-0.10.2.tar.bz2
# cd pam_ssh_agent_auth-0.10.2
# ./configure --libexecdir=/lib/security --with-mantype=man
# make
# checkinstall

Note that the libexecdir option to the configure script is set since apparently Debian keeps PAM modules in a different place than pam_ssh_agent_auth expects by default.


Edit the file /etc/pam.d/sudo and add the following line before any other auth or @include commands:

auth sufficient pam_ssh_agent_auth.so file=~/.ssh/authorized_keys

Run visudo to edit /etc/sudoers and add this …

more ...

Raspberry Pi as a USB to Ethernet Gateway


One of the most convenient ways of communicating with experimental devices (such as oscilloscopes, frequency generators, pulse generators, etc.) is via ethernet. The advantages of this over other forms of communication such as GPIB, RS-232 serial ports, etc., is that, provided the device receives a fixed IP address or some sort of dynamic DNS service is used, it doesn't matter where it is located and specialty cabling can be kept to a minimum. Luckily, most of these devices, even if they are not equipped with ethernet capability, can be made to work over ethernet with some sort of device server (e.g., there are device servers such as those made by Moxa which can "convert" RS-232 serial port communications to ethernet).

A lot of modern devices come equipped with a USB port on the back which complies with the USBTMC (USB test and measurement class) specifications. Even fairly inexpensive equipment which lacks an ethernet port are likely to have a USB port for USBTMC communications (e.g., the popular and inexpensive Rigol DS1000D series digital oscilloscopes). There exists a USBTMC Linux kernel module which allows for communication with USBTMC devices via /dev/usbtmcNNN device files. This module, coupled with the versatile socat command, can thus allow for transparent communications over ethernet with a USBTMC device as if it were connected via ethernet itself. The rest of this note describes the process for using a Raspberry Pi as a USBTMC to ethernet adapter.

Compiling the RPi kernel

The RPi's default …

more ...