=========================
 Coding Style Guidelines
=========================

Main Goals
==========

    * Code should be compatible with Python 2.4 and above (using 2to3 for
      conversion to Python 3.x). This may not be possible in the short term
      for Theano-dependent code.

    * Code should be easy to read, understand and update by developers and
      users.

    * Code should be well-documented and well-tested.

Python Coding Guidelines
========================

Official Guidelines
-------------------

Source Material
~~~~~~~~~~~~~~~

The four main documents describing our Python coding guidelines are:
    * `PEP 8 -- Style Guide for Python Code
      <http://www.python.org/dev/peps/pep-0008>`_
    * `PEP 257 -- Docstring Conventions
      <http://www.python.org/dev/peps/pep-0257>`_
    * `Numpy Docstring Standard
      <http://projects.scipy.org/numpy/wiki/CodingStyleGuidelines#docstring-standard>`_
    * `Google Python Style Guide
      <http://google-styleguide.googlecode.com/svn/trunk/pyguide.html>`_


However, there are a few points mentioned in those documents that we decided
to do differently:

    * Use only one space (not two) after a sentence-ending period in comments.

    * You do not need to add an extra blank line before the closing quotes of
      a multi-line docstring.

      .. code-block:: python

        # Good.
        """This is a multi-line docstring.

        Which means it has more than one line.
        """

        # Bad.
        """This is a multi-line docstring.

        Which means it has more than one line.

        """

Excerpts
~~~~~~~~

We emphasize here a few important topics that are found in the official
guidelines:

    * Only use ASCII characters in code files.

    * Code indent must be done with four blank characters (no tabs).

    * Limit lines to 79 characters.

    * Naming conventions: ``ClassName``, ``TOP_LEVEL_CONSTANT``,
      ``everything_else``.

    * Comments should start with a capital letter (unless the first word is a
      code identifier) and end with a period (short inline comments may skip
      the period at the end).

    * Imports should be listed in alphabetical order. It makes it easier to
      verify that something is imported, and avoids duplicated imports.

    * Use absolute imports only. This is compatible across a wider range of
      Python versions, and avoids confusion about what is being
      imported.

    * Avoid using lists if all you care about is iterating on something. Using
      lists:

         - uses more memory (and possibly more CPU if the code may break out of
           the iteration),
         - can lead to ugly code when converted to Python 3 with 2to3,
         - can have a different behavior if evaluating elements in the list has
           side effects (if you want these side effects, make it explicit by
           assigning the list to some variable before iterating on it).

      +------------------------+------------------------+
      | Iterative version      |    List version        |
      +========================+========================+
      | .. code-block:: python | .. code-block:: python |
      |                        |                        |
      |   my_dict.iterkeys     |   my_dict.keys         |
      |   my_dict.itervalues   |   my_dict.values       |
      |   my_dict.iteritems    |   my_dict.items        |
      +------------------------+------------------------+
      | .. code-block:: python | .. code-block:: python |
      |                        |                        |
      |   itertools.ifilter    |   filter               |
      |   itertools.imap       |   map                  |
      |   itertools.izip       |   zip                  |
      +------------------------+------------------------+
      | .. code-block:: python | .. code-block:: python |
      |                        |                        |
      |   xrange               |   range                |
      +------------------------+------------------------+

      Code example with ``map``:

      .. code-block:: python

        # Good.
        for f_x in imap(f, x):
            ...
        all_f_x = map(f, x)
        map(f, x)   # f has some side effect.
        # Bad.
        for element in map(f, x):
            ...
        imap(f, x)

    * Generally prefer list comprehensions to ``map`` / ``filter``, as the former are
      easier to read.

      .. code-block:: python

        # Good.
        non_comments = [line.strip() for line in my_file.readlines()
                                     if not line.startswith('#')]
        # Bad.
        non_comments = map(str.strip,
                           ifilter(lambda line: not line.startswith('#'),
                                   my_file.readlines()))
 
    * Use ``in`` on container objects instead of using class-specific methods:
      it is easier to read and may allow you to re-use your code with different
      container types.

      .. code-block:: python

        # Good.
        has_key = key in my_dict
        has_substring = substring in my_string
        # Bad.
        has_key = my_dict.has_key(key)
        has_substring = my_string.find(substring) >= 0

    * Do not use mutable arguments as default values. Instead, use a helper
      function (conditional expressions are forbidden at this point, see
      below).

      .. code-block:: python

        # Good.
        def f(array=None):
            array = pylearn.if_none(array, [])
            ...
        # Bad.
        def f(array=[]): # Dangerous if `array` is modified down the road.
            ...

    * Use a leading underscore '_' in names of internal attributes / methods,
      but avoid the double underscore '__' unless you know what you are
      doing.


Additional Recommendations
--------------------------

Things you should do even if they are not listed in official guidelines:

    * No conditional expression (not supported in Python 2.4). These are
      expressions of the form ``x = y if condition else z``.

    * Use ``//`` for integer division and ``/ float(...)`` if you want the
      floating point operation (for readability and compatibility across all
      versions of Python).

      .. code-block:: python

        # Good.
        n_samples_per_split = n_samples // n_splits
        mean_x = sum(x) / float(len(x))
        # Bad.
        n_samples_per_split = n_samples / n_splits
        mean_x = sum(x) / len(x)

    * Always raise an exception with ``raise MyException(args)`` where ``MyException``
      inherits from ``Exception``. This is required for compatibility across
      all versions of Python.

      .. code-block:: python

        # Good.
        raise NotImplementedError('The Pylearn team is too lazy.')
        # Bad.
        raise NotImplementedError, 'The Pylearn team is too lazy.'
        raise 'The Pylearn team is too lazy to implement this.'

    * Use either ``try ... except`` or ``try ... finally``, but do not mix
      ``except`` with ``finally`` (which is not supported in Python 2.4).
      You can however embed one into the other to mimic the ``try ... except ...
      finally`` behavior.

      .. code-block:: python
    
        # Good.
        try:
            try:
                something_that_may_fail()
            except SomeError:
                do_something_if_it_failed()
        finally:
            always_do_this_regardless_of_what_happened()
        # Bad.
        try:
            something_that_may_fail()
        except SomeError:
            do_something_if_it_failed()
        finally:
            always_do_this_regardless_of_what_happened()

    * Do not use the ``all`` and ``any`` builtin functions (they are not supported
      in Python 2.4). Instead, import them from ``theano.gof.python25`` (or
      use ``numpy.all`` / ``numpy.any`` for array data).

    * Do not use the ``hashlib`` module (not supported in Python 2.4). We will
      probably provide a wrapper around it to be compatible with all Python
      versions.

    * Avoid backslashes whenever possible. They make it more
      difficult to edit code, and they are ugly (as well as potentially
      dangerous if there are trailing white spaces).

      .. code-block:: python

        # Good.
        if (cond_1 and
            cond_2 and
            cond_3):
            ... 
        # Bad.
        if cond_1 and \
           cond_2 and \
           cond_3:
            ...

    * When indenting multi-line statements like lists or function arguments,
      keep elements of the same level aligned with each other.
      The position of the first
      element (on the same line or a new line) should be chosen depending on
      what is easiest to read (sometimes both can be ok).

      .. code-block:: python

        # Good.
        for my_very_long_variable_name in [my_foo, my_bar, my_love,
                                           my_everything]:
            ...
        for my_very_long_variable_name in [
                my_foo, my_bar, my_love, my_everything]:
            ...
        # Good iff the list needs to be frequently updated or is easier to
        # understand when each element is on its own line.
        for my_very_long_variable_name in [
                my_foo,
                my_bar,
                my_love,
                my_everything,
                ]:
            ...
        # Good as long as it does not require more than two lines.
        for my_very_long_variable_name in [my_foo,
                                           my_bar]:
            ...
        # Bad.
        for my_very_long_variable_name in [my_foo, my_bar, my_love,
                my_everything]:
            ...
        for my_very_long_variable_name in [my_foo,
                                           my_bar,
                                           my_love,
                                           my_everything]:
            ...

    * Use the ``key`` argument instead of ``cmp`` when sorting (for Python 3
      compatibility).

      .. code-block:: python

        # Good.
        my_list.sort(key=abs)
        # Bad.
        my_list.sort(cmp=lambda x, y: cmp(abs(x), abs(y)))

    * Whenever you read / write binary files, specify it in the mode ('rb' for
      reading, 'wb' for writing). This is important for cross-platform and
      Python 3 compatibility (e.g. when pickling / unpickling objects).

      .. code-block:: python

        # Good.
        cPickle.dump(obj, open('my_obj.pkl', 'wb', protocol=-1))
        # Bad.
        cPickle.dump(obj, open('my_obj.pkl', 'w', protocol=-1))

    * Avoid tuple parameter unpacking as it can lead to very ugly code when
      converting to Python 3.

      .. code-block:: python

        # Good.
        def f(x, y_z):
            y, z = y_z
            ...
        # Bad.
        def f(x, (y, z)):
            ...

    * Only use ``cPickle``, not ``pickle`` (except for debugging purpose since
      error messages from ``pickle`` are sometimes easier to understand).

    * A script's only top-level code should be something like:

      .. code-block:: python

        if __name__ == '__main__':
            sys.exit(main())


The ``logging`` Module vs. the ``warning`` Module
=================================================

The ``logging`` Module
----------------------

A central logging facility for Python capable of logging messages of various
categories/urgency and choosing with some granularity which messages are
displayed/suppressed, as well as where they are displayed or written. This
includes an ``INFO`` level for innocuous status information, a ``WARNING`` level
for unexpected state that is still recoverable, ``DEBUG`` for detailed
information which is only really of interest when things are going wrong, etc.

In addition to the `library documentation`_, see this helpful tutorial,
`Python Logging 101`_.

.. _library documentation: http://docs.python.org/library/logging.html
.. _Python Logging 101: http://plumberjack.blogspot.com/2009/09/python-logging-101.html

The ``warning`` Module
----------------------

The ``warning`` module in the standard library and its main interface, the
``warn()`` function, allows the programmer to issue warnings in situations where
they wish to alert the user to some condition, but the situation is not
urgent enough to throw an exception. By default, a warning issued at a given
line of the code will only be displayed the first time that line is executed.
By default, warnings are written to ``sys.stderr`` but the ``warning`` module
contains flexible facilities for altering the defaults, redirecting, etc.

Which? When?
------------

It is our feeling that the ``logging`` module's ``WARNING`` level be used to log
warnings more meant for *internal*, *developer* consumption, to log situations
where something unexpected happened that may be indicative of a problem but
is several layers of abstraction below what a user of the library would
care about.

By contrast, the warning module should be used for warnings intended for user
consumption, e.g. alerting them that their version of Pylearn is older than
this plugin requires, so things may not work as expected, or that a given
function/class/method is slated for deprecation in a coming release (early
in the library's lifetime, ``DeprecationWarning`` will likely be the most common
case). The warning message issued through this facility should avoid
referring to Pylearn internals.

Code Sample
===========

The following code sample illustrates many of the coding guidelines one should
follow in Pylearn.

.. code-block:: python

    import os, sys, time