Getting Along With Python

http://gawp.sashahart.net

I really enjoy using Python.

But I've noticed that a lot of people suffer unnecessarily when using Python.

Because of things nobody told them.

What sorts of things?

The stuff "around" the programming

  • Tools that make life easier
  • Practices that help keep you out of trouble
  • Idioms that help you go with the flow

Python courses and tutorials rarely talk about this kind of thing.

This kind of practical Python lore is usually obtained from peers, mentors... or excessively many years of hard personal experience.

Even when something is recommended, the "why" is rarely explained.

Beginners can't be blamed for not following practices that just seem like a pain in the ass - but it may hurt them for years.

... And come up over and over again as fake reasons why Python sucks.

We should do a better job of explaining "why" instead of either abandoning practice recommendations, or shaming people into following them without understanding.

Even people who know Python suffer common problems for years before learning there was a better way all along.

That's not OK.

I found that I kept explaining and recommending the same tools...

(things "everyone" uses, like virtualenv)

So I wrote a book* to help orient people in Python's tools ecosystem.

* well, a website. Save the trees.

Too much material to walk through here

So this presentation is an overview, and if you want more detail you can go check out the book.

Have no fear: I'm never going to charge for the site, and your using it benefits me in no way whatsoever. Current URL:

http://gawp.sashahart.net

And I arbitrarily deem this "beta", so I would appreciate your feedback if you get use out of GAWP!

Important Disclaimers

Before we get properly started, I want to get a few things out of the way

Don't knock it till you've really tried it

If you are looking for help, please try these recommendations for a little while - do not immediately quit when you feel some initial frustration.

I know all too well that a lot of tools and practices seem hard and unnecessary at first - until you've been burned a few times.

But doing everything by hand, or going shopping for alternatives before you know what's what, are recipes for chronic frustration and burnout.

I will express opinions

In trying to encourage people to take advantage of Python's tools and idiomatic practices, I can't avoid making recommendations that some will disagree with. I do try to be ecumenical where it makes sense.

But If I tried to avoid all controversy, I wouldn't recommend anything and beginners would be back at square one. That's not OK.

I really am just trying to help

I'm trying to help people who are less conversant with Python's tools and idioms by giving them reasonable, general-purpose, commonly-used, and mostly uncontroversial starting points which will either help them, or help them develop informed opinions.

Please don't cloud the waters

If you are more experienced, please don't take this as a forum for an old flamewar or to shill for something new.

Topics

Use pip as your default package installer

Life without it

  • installing things in any number of broken and confusing ways
  • reinventing wheels because you're afraid of dependencies
  • only being able to use distro packages (greatly limiting your choices)
  • suffering the clumsiness of python setup.py and easy_install
  • puzzling over corner cases of non-standard installers
  • Life with it

  • installing any version of most Python packages with little fuss
  • working seamlessly with things like virtualenv and wheel
  • mostly ignoring $PYTHONPATH, sys.path or site-packages
  • Initial downside of pip

    Installing pip the first time seems intimidating and might take a few minutes. :`(

    Pain mitigation Start with GAWP's pip installation hints or pip's own install instructions

    Manage Python deps with virtualenv

    What it does makes directories containing mini-installs of Python that your code can run in, so projects with conflicting requirements and Python versions won't confuse you

    Life without it

    • you can only use one Python (the system's Python),
    • you can only use the system's library versions,
    • everything breaks as those versions change,
    • you break everything when you install different versions,
    • using sudo for installs causes awful messes,
    • your stuff ONLY "works on my machine"

    Life with virtualenv

    (for bash-like shells which support source):

    $ python --version
    Python 2.7.6
    $ virtualenv -p python3.4 ex
    Running virtualenv with interpreter /usr/bin/python3.4
    Using base prefix '/usr'
    New python executable in ex/bin/python3.4
    Also creating executable in ex/bin/python
    Installing setuptools, pip...done.
    $ source ex/bin/activate
    $ python --version
    Python 3.4.0
    $ pip install -q django  # only installed in ex/
    $ django-admin.py startproject inex
    $ pip freeze > requirements.txt
    $ cat requirements.txt 
    Django==1.7
    $ deactivate
    $ python --version
    Python 2.7.6
    

    Was that really so bad? Check out GAWP's quick guide to virtualenv for more orientation. And there are other tools to make it even easier.

    Use version control religiously

    Life with version control

    • keep track of changes you (or others) make
    • 'rewind' and 'undo' at will across the whole project
    • it's not insanely hard to work together with others

    Life without version control

    • Sh*t, I just deleted months of work
    • Sh*t, which directory is the newest version
    • Sh*t, merging these directories is totally insane

    TL;DR the programming life without version control is mostly sh*t.

    No, really, use version control

    If learning something like git seems too scary, remember:

    • Long run, it is still ~500% easier than not using version control
    • You don't have to learn everything to get started well
    • It's unlikely that anybody will see or use your work otherwise

    I don't really want to shame you but I'll do it if I have to: this is a matter of basic competence to work with others.

    Also see GAWP's cursory introduction to version control or any other tutorial

    Apply community style

    Get used to writing code per PEP 8, with PEP 257 docstrings.

    Life without it

    • never-ending bikeshedding due to lack of accepted standards
    • getting upset while reading almost anyone else's Python code
    • pointlessly annoying your Pythonistic coworkers

    Life with it

    • easy checking with tools like flake8 or any decent IDE
    • you gradually discover the practical reasons for PEP 8

    Run static checkers

    Purpose Reduce the amount of debugging you have to do, and find code which you obviously wrote while drunk (not that I'm judging...)

    pylint does a pretty good job of flagging places where you started to get sloppy and need a rethink. And if you really don't want a certain message, you can just turn that message off.

    flake8 is fast enough that you can set your editor to run it every time you change a file. But it's not very picky.
    So it's not a terrible choice for commit hooks, if you use those.

    There's no substitute for careful review by humans.

    Automate your testing

    Purpose to save you the trouble of manually testing over and over
    or fixing an endless Sisyphean trickle of embarassing bugs AFTER people saw them

    I don't really care what you use. Python unittest is pretty standard and comes with a simple test runner. I like py.test and tox.

    Though testing is a deep discipline with its own internal flamewars, basic testing is probably easier than you think, and the return on investment can be outstanding. You can find a practical approach with examples in GAWP's testing chapter

    Use logging

    Stop having to choose between spammy print statements that nobody can suppress, and not having information on what just happened.

    Stop introducing bugs that crash important code as you add and remove print statements everywhere.

    Instead, use the logging system, and then you or anyone can control the spam level, for each type of message, from outside your code, as needed.

    So you have the information when you need it, even in production. But you have complete control over the spam level and destination of your log messages.

    logging has several confusing nuances, so you might want to consult GAWP's logging chapter

    Learn a debugger

    Purpose investigate the causes of bugs more efficiently

    Okay, not everybody wants to use them, and you can get along without them up to a certain point. But when you actually need one for some particularly bad bug it'll save you a lot of time to already know one rather than flailing.

    So it's probably a good idea to practice pdb or pudb or winpdb or whatever before you have a big problem.

    There's a basic introduction to pdb in GAWP's chapter on debuggers.

    Build documentation with Sphinx

    A typical project will maintain three kinds of documentation:

    • comments
    • docstrings
    • narrative documentation

    Comments and docstrings are up to you. But Sphinx can generate nice-looking, searchable docs in multiple formats, slurp up your PEP 257 RST docstrings, and also let you explain things rather than just dumping a list of classes at users. (Actually, I used it to write GAWP.)

    And you can host your public project's docs for free on http://readthedocs.org.

    There's a quick guide to documentation practice in GAWP's chapter on docs.

    Create Python packages

    It's not really that hard, you mostly end up copying the same setup.py and modifying it slightly for each new project.

    You can learn it in an afternoon.
    Here you go: GAWP on making Python packages

    You can use tools like pip to install instead of clumsy manual instructions or ad hoc scripts. Works great for command line tools, too. And you can publish stuff to PyPI if you want to. But you don't have to.

    If you want happiness,
    cultivate wisdom

    • Invest your time Taking a little extra time to do things right up front means less frustration later. Learning better habits hurts once, but pays off many times.
    • Go with the flow If you insist on using tools contrary to their design and flouting conventions, don't be surprised that you get tired swimming upstream.

    The End

    Slides: http://slides.sashahart.net/gawp

    Book: http://gawp.sashahart.net

    Feedback to: gawp@sashahart.net