Posts from October 2019

pydscheck -- A quick hack that keeps slowly growing

4 min read

Something I always try to do when I'm coding is be consistent. I feel this is important. While people's coding standards may differ, I think different approaches are easier to handle if someone has been consistent with their style across all of their code.

This also stands for documentation too.

In my current position, I do a lot of Python coding, and one of the things I like about Python (there are things I don't like too, but that's not for now) is that it has doc-strings (just like my favourite language). I use them extensively, ensuring every function and method has some form of documentation, and generally I use Sphinx to generate documentation from those doc-strings.

Early on I was bothered by the fact that, just by the simple act of making typos, I wasn't keeping the form of the doc-strings consistent. And in this case it was a really simple thing that was bugging me. Normally, if I'm writing a single-line doc-string, I'll write like this:

def one_liner():
    """Here is a one-line doc-string."""

So far, so good. But, if the doc-string is a multi-liner, I prefer the ending quotes to be on a line of their own, like this:

def multi_liner():
    """Here is the first line.
    Here is another line.
    Here is the final line.
    """"

But, sometimes, by accident, I'd leave a doc-string like this:

def multi_liner():
    """Here is the first line.
    Here is another line.
    Here is the final line.""""

While it's really not a big deal, it would bug me and every time I found one like this I'd "fix" it.

Eventually, it bugged me enough that I decided I was going to write a little tool to find all such instances in my code and report them. My first approach was to think "I could just do this with some regexp magic", which was really a bad idea. Then I though, I know, I should use this as an excuse to to play with Python's ast library.

That worked really well! I had the first version of the code up and running in no time. It was simple but did the job. It ran through Python code I threw at it and alerted me to both missing doc-strings, and doc-strings with the ending I didn't like.

That served me for a while, until one day I realised that it wasn't quite doing the job correctly; it was only really looking at top-level functions and top-level methods in classes. Sometimes, not often, but sometimes, I'll define functions within functions, and I feel they deserve documentation too. So then I modified the code to ensure it walked every part of the AST.

Since then, when I've run into new things and had new ideas, pydscheck has grown and grown. I've added checks that all mentioned parameters have a type; I've added checks that any function/method that returns something actually documents the return value; I've added checks that any documentation of a returned value includes its type; I've added checks that any function or method that yields a value documents that fact; I've added checks that ensure that every parameter is documented in some way.

Each time I've done this it's helped uncover issues in my code's documentation that could be cleaner, and it's also given me a pet project to slowly better understand Python's AST.

It could be that there are better tools out there, I'd have thought that a good doc-string linting tool would be something someone had already written. But this time around I was happy to NIH it because I needed a fun learning exercise that would also have some benefits for my day-to-day work.

I'll caveat this with the fact that it's very particular to how I work and how I like my documentation to look, but if it sounds useful, here it is: https://github.com/davep/pydscheck.

There's still lots I could do with it. First off I should really properly package it up so it can be installed as a command line tool via pip. Other things that would be handy would be to allow some form of customisation of how it works. I'm sure there's other fun things I can do with it too.

That's part of the fun of having a pet project: you can tinker when you like and also get benefits from it as you use it.

Why I really like fish abbreviations

3 min read

I'm filing this as a TIL because, while it wasn't T, I did L it very recently and it was a new trick that impacted on around 25 years if prior working practice.

I think it must have been around 1991 when I first encountered 4DOS. While I'd used the odd Unix shell here and there previously, it'd only been in passing. It was 4DOS that first introduced me to the power of aliases on the command line. Many of the aliases I set up and used in 4DOS still remain with me to this day, on GNU/Linux and macOS, in some form or another.

I'm sure I don't need to tell anyone reading this why aliases and cool and handy and pretty much vital if you do lots of work on the command line.

And then, a couple or so weeks ago, as a very recent convert to fish, I discovered the abbr command. At first glance it didn't seem to make much sense. It was like alias, only it expanded what you typed rather than acted as a command in its own right.

I did a bit of digging and some of it started to make sense. One thing that really won me over -- and while it's something that doesn't directly impact on me -- was the argument that it allows for a far more transparent command history; especially if you're likely to use a transcript of a shell session in a place where people might not know or have access to your aliases.

Imagine being in a position where you have loads of handy and cool aliases, but you also need to record what you've done so other people can follow your work (does it show that I sit amongst people who maintain lab notebooks?); it seems like it would be a bit of a bother needing to record all of the aliases in your own work environment up front. Without that information few people will be able to make sense of the recorded commands, with that information they'd still need to double-check what each command does.

So imagine an alias that, when used, expands in place. Then you'd get all of the benefit of aliases while also having a full and readable record of what you actually did.

Seems neat!

Here's a silly example. For a long time I've carried around an alias called greedy that runs something like this:

du -hs * | sort -rh

It's pretty straightforward: I'm using du to get a sense of which directories are using what space, and then using sort to make a worst-to-best-offender list out of it. So I could use an alias:

alias greedy="du -hs * | sort -rh"

The only downside to this is that, any time I run it, if I were to record the shell session and make it available for someone else to read, they'd just see:

~/develop$ greedy
1.1G    JavaScript
824M    C
699M    rust
 93M    python
 33M    fonts
 33M    elisp
3.4M    zsh
3.0M    misc
1.1M    bash
840K    ocaml
428K    C++
316K    lisp
172K    Swift
152K    git
132K    ruby
 28K    ObjC

Now, with an abbreviation rather than an alias, I'd type greedy but as soon as I hit Enter it'd get expanded to something anyone could read and follow:

~/develop$ du -hs * | sort -rh
1.1G    JavaScript
824M    C
699M    rust
 93M    python
 33M    fonts
 33M    elisp
3.4M    zsh
3.0M    misc
1.1M    bash
840K    ocaml
428K    C++
316K    lisp
172K    Swift
152K    git
132K    ruby
 28K    ObjC

This is far from the only benefit of abbreviations; for most people it probably isn't one of the most important ones, but I find it neat and compelling and this alone drove me to rework almost all of my aliases as abbreviations.

Having done that, I get other benefits too. For example, fish (like other shells) has good support for argument completion for well-known commands. The problem is, if you alias such a command, you don't get that completion. With an abbreviation though you do! All you need to do is type the abbreviation, hit space and it'll expand to the underlying command and then the full range of completion can happen.

There's also one last reason why I like abbreviations over aliases, and it's kind of a silly one, but in a good way. It's actually fun to see what you type magically expand as you do things, it makes you look like you can type even faster than you normally can! ;-)

PS: If you've never tried fish before and you're curious, it's easy to try in your browser.

gitweb.el -- Quickly visit a repo's forge from Emacs

1 min read

gh.fish, which I wrote about yesterday, actually sprang from something I initially wrote for Emacs. I'm often spending my time switching between Emacs and the command line (which is fast and easy -- I normally work on macOS and have Emacs and iTerm2 running full screen, and I can switch between them without ever taking my hands off the keyboard), so it makes sense to have some handy commands repeated in both places.

So, originally, I'd written gitweb.el to open the current repo's "forge" in the web browser.

As with the fish version, how it works is quite simple. I use shell-command-to-string to call git and find the origin URL for the current repo, and then manipulate it a bit to turn it into a normal browser-friendly URL. Finally, if I get something workable, I use browser-url to have the resulting page open in the browsing environment of choice.

I have the command bound to a key combination that's similar to the ones I use with magit and forge, so in terms of muscle-memory it's easy for me to remember what to press when I quickly want to skip over from a magit view to the repo forge itself.

Similar to what I wrote a couple of days back, I think this again illustrates how handy Emacs is as a work environment. While it's absolutely true that there are other development environments out there that offer similar extensibility, Emacs is the one I'm comfortable with, and it has a long history of offering this.

gh.fish -- Quickly visit a repo's forge

1 min read

These days fish is my shell of choice. I started out with bash back in the 1990s, went through a bit of a zsh/oh-my-zsh phase, but earlier this year finally settled on fish.

At some point I might write a post about my fish config, and why fish works well for me. But that's an idea for another time.

In this post I thought I'd share a little snippet of code that can come in handy now and again.

Sometimes I find myself inside a git repo, in the shell, and I want to get to the "forge" for that repo. This is most often either on GitHub, or in a company-local installation of GitLab. To get there quickly I wrote gh.fish:

##############################################################################
# Attempt go visit the origin hub for the current repo.

function gh -d "Visit the repo in its origin hub"

    # Check that there is some sort of origin.
    set origin (git config --get remote.origin.url)

    # If we didn't get anything...
    if not test "$origin"
        # ...complain and exit.
        echo "This doesn't appear to be a git repo with an origin"
        return 1
    end

    # Open in the browser.
    open "https://"(string replace ":" "/" (string replace -r '\.git$' "" (string split "@" $origin)[ 2 ]))

end

### gh.fish ends here

The idea is pretty simple: I see if the repo has an origin of some description and, if it has, I slice and dice it into something that looks like the URL you'd expect to find for a GitHub or GitLab repo. Finally I use open to open the URL in the environment's browser of choice.

pypath.el -- A little Emacs hack to help with Django

2 min read

One of the things I really like about coding with Emacs is how I can easily identify a repeated task and turn it into a command in my environment, saving me a load of work down the line.

pypath.el is one such example.

In my day job I write a lot of Django code. As part of that, I write a good number of unit tests too. Sometimes I'll write the tests as I'm writing the code they test, other times I'm writing them afterwards; it all really depends on where my head's at and how the code is flowing.

When I'm writing those tests, I often want to test them as I go. Given that starting up a test session can take a while, and given that running all the tests in the system can take a while, it's really handy if I can run that single test I'm working on.

This is easy enough with Django. In my work environment it's normally something like:

$ pipenv run ./manage.py test -v 2 app.test.some.sub.module.TestClass.test_method

Only... typing out the:

app.test.some.sub.module.TestClass.test_method

part is a bit of a pain. Sure, once you've typed it the once you can use your shell of choice (mine being fish and on occasion eshell) to recall it from history, but typing it out the first time is the annoying part.

So this was the point where I took 1/2 hour or so to code up pypath.el to solve the problem for me. It gives me two commands:

  • pypath: which simply places the dotted path of the current "defun", within the context of being part of a Django system, into the clipboard buffer.
  • pypath-django-test: which works similar to the above but places the whole Django testing command into the clipboard.

With the above, I can work on a test, hit the latter command above, flip to my command line, paste and I'm running the test.

Of course, I'm sure there's plenty of other handy ways to do this. Doubtless there's work environments where the test can be run right there, in the edit buffer, without flipping away, and which takes into account the fact that there's a pipenv-managed virtual environment involved, etc. If there is, that's great, but I don't think it'd work with how I work.

And that's one of the things I really love about Emacs, and why it's still my work environment after almost 25 years of on and off use: with very little work on my part I can create a couple of commands that work exactly how I need them to. While it's great to create generally-useful code for Emacs that lots of people benefit from, sometimes the real value is that you can code up your own particular quirk and just get on with stuff.

To conclude: this post isn't to show off pypath.el; really this post is to sing the praises of Emacs and why it still works so well for me after all these years.

Time to move on

2 min read

It's well over a year since I last wrote something on this blog. As mentioned in the last post (and the one before), it's not for bad reasons or anything like that. Being in a new job, which actually isn't all that new now, has kept me busy in all the best ways possible.

There's been other stuff going on too which has drawn on my attention and the time and motivation to blog, either random stuff, or more development-related stuff, just hasn't been there.

Also... blogging via GitHub, using Jekyll, has lost a lot of its shine. It sort of makes sense, well, sort of made sense, but in the end it felt like more work than it should. Whereas most blogging systems tend to encourage just diving in and banging on the keyboard, there's just a bit more faff with the GitHub pages approach.

So, with that in mind, and with no desire right now to roll my own (which would be fun, it has to be said), I'm going to skip off over to Hashnode's blogging system. It seems to have everything I'd want and I can slap it on a domain of mine.

Most of my random musings about random things really happen on Twitter, so I can't imagine I'll be wanting to blog about normal/mundane things. What I would like to do is write about development-related things from time to time. So that would seem to fit even better.

Anyway, enough of all this waffle. If you land here and it looks kind of quiet, that's because it has been quiet for a while and I'm now going to try and concentrate elsewhere, with a wish to do some coding-related writing now and again.