by phildini on June 13, 2017
There are many challenges to running an Open Source organization, but the one that I have personally felt the pain of again and again is that our tooling is awful. Github (and realistically we’re all using Github at this point) still feels in many ways like a tool designed around the idea that all the action is going to happen in one repo. This may not be entirely the fault of Github. Git itself is very tightly coupled to the idea that anything you care about for a particular action is going to happen in one, and only one, repository.
When Github released Organizations, the world rejoiced, because we could now map permissions and team members in our source repository the way they were mapped in the real world. Every new feature Github adds to its Organizations product causes more rejoicing, because so many teams work across multiple repos, and the tooling around multiple repos is still awful.
The awfulness of this tooling is probably a strong factor in the current trend towards “microservice, monorepo” code organization, but that’s another post.
I’ve been the equivalent of a core contributor for a half dozen Github organizations, and I’ve noticed that one area where the tooling is especially lacking is around labels. I’ve seen labels used to designate team or individual ownership, indicate the status of pull requests, signal that certain issues are friendly for beginners, and even used as deploy targets for chunks of code. It’s fair to say that labels form a core tool in the infrastructure of every team I’ve seen using Github, and yet the tooling Github exposes for labels is painfully lacking.
I could go on and on about this, but my goal here isn’t to necessarily make Github feel bad. I hope they’re working on better label tooling, and if they want ideas, boy am I willing to give them. But there is one label-specific wall I kept banging my head against, and that is label consistency across all the repos of an Organization.
Some of you read that and feel remembered pain. I feel that pain with you, and we are here for each other. Some of you might have no idea what I’m talking about, so I’ll explain a bit more.
Let’s say you want to add a “beginner-friendly” label to all the repos in your Open Source Organization, so that new contributors can find issues to start with. Right now on Github, you would need to go into every repo, click into the Issues page, click into the Labels tab, and manually create that label. There are no “Org-wide labels”, and no tool for easily creating and updating labels across all the repos of an organization.
Introducing Epithet, a Python-based command line tool for managing labels across an organization. You give it a Github key, organization, and label name, and it will make sure that label exists across all the repos in your org. Give it a color, and it’ll make the color of that label consistent across all repos as well. Have you decided you’re done with a particular label? Epithet can delete it from all your repos for you. Are you using Github Enterprise? Epithet supports that too.
Epithet exists to fill a very particular need in open (and closed) source Github organizations, and it’s still pretty alpha. We use it for the BeeWare project, and it might be used soon for syncing labels in the Ragtag organization. You can start using it today by checking out the (sadly small) documentation, and if there’s a feature missing you’d like to see, I’m happy to work with you on getting a PR submitted.
Managing Open Source organizations is hard. My hope is Epithet makes it a little bit easier.
WordFugue is independent, and we will never run traditional ads. If you like what we're doing, consider donating to phildini's Patreon, or buy a book from our affiliate store. This week we're reading Patrick Rothfuss' "The Name of the Wind".
Special thanks to Katie Cunningham and Kenneth Love for reviewing this post.
by phildini on May 26, 2015
Hello! Welcome to the first in a series of posts about my experiences making Django apps Python 3 compatible. Through these posts I'll start with a Django app that is currently written for Python 2.7, and end up with something can be run on Python 3.4 or greater.
Some quick notes before we begin:
- Why am I doing this? Because we have 5 years until Python 2.7 goes end-of-life, and I want to be as ready as possible for making that change in the code that I write for my job. To prep for that, I'm converting all the Django apps I can find, from side-projects and Open Source projects.
- Why 5 years? Because that's the time outlined in PEP-0373, and based on Guido's keynote at PyCon 2015, that's the timeline we all should be sticking to. It's also recently been brought to my attention that further Python 2.7 releases are really the responsibility of one person, the inimitable Benjamin Peterson, and if he for any reason decides to stop making updates that 2020 timeline may get drastically shortened. It's better to be prepared now.
- Why "Python 3 compatible"? Why not fully Python 3? Because I believe the best way forward for the next 5 years will be writing polyglot code that can be run in either Python 2.7 or Python3.4+ environments. (I'm going to start shortening those to py2 and py3 for the rest of this post.) So I won't be using 2to3, but I will be using six.
With those pieces in mind, let's begin!
I started with Cards Against Django, a Django implementation of Cards Against Humanity that I wrote with some friends a couple years ago. We didn't own Cards Against Humanity, and hilariously thought it would be easier to build it than to buy it. (We also may have just wanted the challenge of building a usable Django app from scratch). The end result was a game that could be played with an effectively unlimited number of players, each on their own device, and which was partially optimized for mobile play. To get a sense of what the code was like before I started the migration, browse the Github repo at this commit.
Now it turns out I made one assumption right at the beginning of this port that made things a bit harder, and may have distracted from the original mission. The assumption was that Django 1.5 is not py3 compatible, when in fact it was the first py3-compatible version. Had I found and read this Python 2 to 3 porting guide for Django, I may have saved myself some headache. You now get the benefit of a free mini-lesson on upgrading from Django 1.5 to Django 1.8.
Real quick, I'm going to go through how my environment was set up at the beginning of this project, based on the starting commit listed above.
This snippet will setup a virtual environment using mkvirtualenv, install the local requirements for the app, and initialize the db using the local settings.
Ok, let's upgrade to Django 1.8
$ pip install -U Django
..and naively try to run the dev server.
Well that's a bummer, but fairly expected that I wouldn't be able to make the jump to 1.8 easily. What's interesting about this error is that it's not my code that seems to be the problem -- it looks like the problem is in django-nose.
$ pip install -U django-nose nose
Try runserver again...
Hmm... obviously the API for transactions changed between Django 1.5 and Django 1.8. Here I looked at the Django release notes, and noticed that 'commit_on_success' was deprecated in 1.8. Digging in to the new transaction API, it looked like 'transaction.atomic' was pretty much the behavior I wanted, so I went with that.
Third time's the charm, yes?
Apparently not. This one was weird to me, because I didn't have South in my installed apps. Through a sense of intuition that I can't really explain, I suspected django-allauth, the authentication package this project uses. I wondered if an older version of django-allauth was trying to do South-style migrations.
$ pip install -U django-allauth
Sure enough, an old version of allauth was the culprit, and an upgraded version allowed the runserver to launch successfully.
So now I have the development server running, but I've got that warning about needing to run migrations. This is the part of this upgrade that I knew was coming, and I was most worried about. I already have the database initialized from Django 1.5's 'syncdb' -- what will happen when I run 'migrate'?
It turns out, not a whole lot. Running this command gave me a 'table already exists' DatabaseError. Googling for this issue left me a little stumped, so eventually I turned to the #django channel on Freenode IRC. (If you're curious how to get a persistent connection to IRC, check out this post.) I was able to get some great help there, and it was suggested I try the one-two punch of:
That '--fake' bit did the trick, convincing Django I had run the migrations (since the tables were already correctly created), and silencing the warning.
With the development server running on Django 1.8 (including the very limited test suite), I'm feeling confident about the migration to Python 3. Is my confidence misplaced? Find out in part 2!
If you'd like to see the totality of the work required to migrate this Django app from 1.5 to 1.8, check out this commit.
If you have feedback about what I did wrong or right, or have questions about what's here, leave a comment, and I'll respond as soon as I'm able!
by phildini on December 2, 2014
This is the first Homage for the Holidays project. It may help to read the previous blog post for context.
Let's follow the logic train on this one. XOXO was awesome, and inspired me to want to do projects not for fame or money, but just because they were interesting to me and hopefully pushed the envelope in some way. I started by looking at what the organizers, Andy Baio and Andy McMillan, had done previous to and alongside XOXO for inspiration. Digging through Andy Baio's profile, I found his work with Inform 7, and Playfic.
Briefly: Inform 7, often shortened just to Inform, is a language and tool for creating and running interactive fiction. If you think you don't know what interactive fiction is, think back to really old text based games - "go north you are eaten by a grue" type stuff. Playfic is a site where you can run and play interactive fiction works in a web browser. Playfic was created by Andy Baio, because he wanted to -- well, I'll let him tell it:
Andy loves interactive fiction and wanted to make a game, but found it to hard to share his work-in-progress online. In an epic tale of yak shaving, he built Playfic before writing his first game.
From the PlayFic Website
I deeply empathized with this level of yak shaving, and Playfic/Inform seemed like a great way give homage to one of the people who had inspired me. Additionally, in my head Inform and I had unfinished business. If my education had gone according to plan, I would have taken a course on interactive fiction (a subject that really excites me) and spent three months doing a deep dive into Inform. My school career didn't exactly follow a linear path, so I never got a chance to play with Inform.
Here is what I have done, although started may be a better word. I have rebuilt, as much as possible, the XOXO 2014 experience in Inform 7, hosted on Playfic.
The project is called "from Portland get XOXO".
Is it finished? Nope, although I'll be working on it more this week, and posting updates when I can. What is there is the bones, including pretty much every major location and most of the events from the four days of fun.
What would I like to happen? Well, mostly I want people to play it. And have a reaction to it that hopefully isn't boredom. I also want people to change it, make it better, make it crazier, basically take what I've done as a base and push the limits of it.
So I've put the whole thing in a github repo, and I am actively soliciting issues and pull requests. Anyone who contributes will get my eternal thanks, and a call-out on twitter plus this blog.
Play it, have fun with, and make it your own.
Thanks, and look for more updates about the project this week, plus more Homage for Holiday posts in the coming weeks