Multiline lambdas

Although Python has anonymous lambda functions, they lack the flexibility that some languages such as Javascript or even modern C++ have. In Python, lambda functions are limited to a single statement, which is often interpreted as meaning that it can only do one thing. This is not strictly true, however, since constructing a tuple is considered a single statement. In other words, we can cheat a little and call two independent functions in one lambda like this:

(lambda: (foo(), bar()))()

This is still somewhat constraining since variables cannot be defined within the lambda expression, so cases where this trick is useful are limited. One instance where this is particularly nice though is when defining callbacks for a GUI. Typically when a user clicks on a button, there might be several actions that should be triggered, such as starting an experiment, updating a GUI label, etc. Below is a simple example to illustrate this method:

import sys
from PyQt5.QtWidgets import *
from PyQt5.QtGui import *

class MainWindow(QWidget):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)

        self.setWindowTitle('Multiline lambdas')

        self.label = QLabel("Click the button")

        self.button = QPushButton("Click me!")
        self.button.clicked.connect(lambda: (
            self.button.setText("You clicked me!"),

        layout = QVBoxLayout()

if __name__ == "__main__":
    app = QApplication(sys.argv)
    win = MainWindow()
more ...

Sharing data between processes with SQLite

Because of the global interpreter lock in CPython, it is sometimes beneficial to use separate processes to handle different tasks. This can pose a challenge for sharing data: it's generally best to avoid sharing memory between processes for reasons of safety1. One common approach is to use pipes, queues, or sockets to communicate data from one process to another. This approach works quite well, but it can be a bit cumbersome to get right when there are more than two processes involved and you just need to share a small amount of infrequently changing data (say some configuration settings that are loaded after worker processes have already been spawned). In such cases, using a file that each process can read is a simple solution, but may have problems if reading and writing happen simultaneously. Thankfully, SQLite can handle this situation easily!

I have created a small module (Permadict) which utilizes SQLite to persist arbitrary (picklable) Python objects to a SQLite database using a dict-like interface. This is not a new idea, but it was fun and simple to utilize only the Python standard library to accomplish this. A basic usage example:

>>> from permadict import Permadict
>>> d = Permadict("db.sqlite")
>>> d["key"] = "value"
>>> print(d["key"])

Because context managers are great, you can also use permadicts that way:

>>> with Permadict("db.sqlite") as d:
...     d["something"] = 1.2345
>>> with Permadict("db.sqlite") as d:
...     print(d["something"])

  1. Of course, Python allows you to share memory among processes …

more ...

Simplifying argparse usage with subcommands

One of the best things about Python is its standard library: it's frequently possible to create complex applications while requiring few (if any) external dependencies. For example, command line interfaces can be easily built with the argparse module. Despite this, there exist several alternative, third-party modules (e.g., docopt, click, and begins). These all tend to share similar motivations: while argparse is powerful, it is by inherently verbose and is therefore cumbersome to use for more complex CLIs which use advanced features such as subcommands. Nevertheless, I tend to prefer sticking with argparse in part because I am already familiar with the API and because using it means I don't need to bring in another dependency from PyPI just to add a small bit of extra functionality. The good news is that with a simple decorator and a convenience function, writing CLIs with subcommands in argparse is pretty trivial and clean.

Start by creating a parser and subparsers in

from argparse import ArgumentParser

cli = ArgumentParser()
subparsers = cli.add_subparsers(dest="subcommand")

Note that we are storing the name of the called subcommand so that we can later print help if either no subcommand is given or if an unrecognized one is. Now we can define a decorator to turn a function into a subcommand:

def subcommand(args=[], parent=subparsers):
    def decorator(func):
        parser = parent.add_parser(func.__name__, description=func.__doc__)
        for arg in args:
            parser.add_argument(*arg[0], **arg[1])
    return decorator

What this does …

more ...

Javascript for Python programmers

Unless you're just writing a simple HTTP API server, any amount of web programming in Python will likely require at least a little bit of Javascript. Like it or not (and I will try to argue in this post that you should like it for what it's good at), Javascript is really the only game in town when it comes to client-side scripting on a web page. Sure, there are a number of Python-to-Javascript transpilers out there, but using these just tends to limit the ability to use new Javascript features as they are rolled out to browsers and may limit the ability to use third-party Javascript libraries. At the very least, using one these transpilers introduces added complexity to deploying a web app1.

In this post, I will describe some things I've learned about Javascript from the perspective of someone who prefers to use Python as much as possible. This guide is mainly aimed at scientists and others who are not primarily programmers but who may find it useful to make a web app for their main work. It is assumed that the reader is at least moderately familiar with Javascript (Mozilla has a nice tutorial to get you up to speed if not).

Namespaces, encapsulation, modularization, and bundling

Modules in Python make it very easy to encapsulate components without polluting the global namespace. In contrast, Javascript in the browser will make everything a global if you are not careful2. The good news is that it doesn't …

more ...

Getting Matplotlib's colors in order

Matplotlib can be very easy to use at times, especially if you just want to make a simple "y vs. x" type of plot. But when it comes to specialized customization, it can be a bit challenging to find the proper solution. The situation is not helped by the fact that a lot of times, an obscure answer on Stack Overflow no longer works because the API changed.

One common need is to color things in the same way. For example, say you want to plot two dependent variables with widely different scales that share an independent variable. This is often represented by having two separate vertical axes which are colored to match the lines or markers of each data set. The most basic approach is to manually assign colors for the lines and axes, but if using a custom style, such as the ggplot style, we need a way to access the color cycle used if we want to remain consistent with the selected style. Here is the best way I have found which works at least in version 1.5:

colors = plt.rcParams["axes.prop_cycle"].by_key()["color"]

colors should now be a list which contains the colors defined in order by the current style.

more ...