Note

This section is optional, the ROS2 tutorial starts at ROS2 Installation.

(Murilo’s) Python Best Practices

Warning

This tutorial expects prior knowledge in Python and object-oriented programming. As such, this section is not meant to be a comprehensive Python tutorial. You have better resources made by smarter people available online, e.g. The Python Tutorial.

Terminology

Let’s go through the Python terminology used in this tutorial. This terminology is not necessarily uniform with other sources/tutorials you might find elsewhere. It is based on my interpretation of The Python Tutorial on Modules, the Python Glossary, and my own experience.

(Murilo’s) Python Glossary

Term	Book Definition	Use in the wild
script	A Python file that can be executed.	Any Python file meant to be executed.
module	A file with content that is meant to be imported by other modules and scripts.	This term is used very loosely and can basically mean any Python file, but usually a Python file meant to be imported from.
package	A collection of modules.	A folder with an __init__.py, even if it doesn’t have more than one module. When people say Python Packaging it refers instead to making your package installable (e.g. with a setup.py or pyproject.toml), so be ready for that ambiguity.

Use a venv

We already know that it is a good practice to When you want to isolate your environment, use venv. So, let’s turn that into a reflex and do so for this whole section.

cd ~
source ros2tutorial_venv/bin/activate

Minimalist package: something to start with

In this step, we’ll work on these.

python/minimalist_package/
  └── minimalist_package/
        └── __init__.py

First, let’s make a folder for our project

Hint

The -p option for mkdir creates all parent folders as well, when they do not exist.

mkdir -p ~/ros2_tutorials_preamble/python/minimalist_package

Then, let’s create a folder with the same name within it for our package. A Python package is a folder that has an __init__.py, so for now we add an empty __init__.py by doing so

cd ~/ros2_tutorials_preamble/python/minimalist_package
mkdir minimalist_package
cd minimalist_package
touch __init__.py

The (empty) package is done!

Hint

In PyCharm, open the ~/ros2_tutorials_preamble/python/minimalist_package folder to correctly interact with this project.

Warning

It is confusing to have two nested folders with the same name. However, this is quite common and starts to make sense after getting used to it (it is also the norm in ROS2). The first folder is supposed to be how your file system sees your package, i.e. the project folder, and the other contains the actual Python package, with the __init__.py and other source code.

Minimalist script

In this step, we’ll work on this.

python/minimalist_package/
  └── minimalist_package/
        └── __init__.py
        └── minimalist_script.py

Let’s start with a minimalist script that prints a string periodically, as follows. Create a file in ~/ros2_tutorials_preamble/python/minimalist_package/minimalist_package called minimalist_script.py with the following contents.

minimalist_script.py

#!/bin/python3
import time


def main() -> None:
    """An example main() function that prints 'Howdy!' twice per second."""
    try:
        while True:
            print("Howdy!")
            time.sleep(0.5)
    except KeyboardInterrupt:
        pass
    except Exception as e:
        print(e)


if __name__ == "__main__":
    """When this module is run directly, it's __name__ property will be '__main__'."""
    main()

Running a Python script on the terminal

There are a few ways to run a script/module in the command line. Without worrying about file permissions, specifying that the file must be interpreted by Python (and which version of Python) is the most general way to run a script

cd ~/ros2_tutorials_preamble/python/minimalist_package/minimalist_package
python3 minimalist_script.py

which will output

Hint

You can end the minimalist_script.py by pressing CTRL+C in the terminal in which it is running.

Howdy!
Howdy!
Howdy!

Another way to run a Python script is to execute it directly in the terminal. This can be done with

cd ~/ros2_tutorials_preamble/python/minimalist_package/minimalist_package
./minimalist_script.py

which will result in

bash: ./minimalist_script.py: Permission denied

because our file does not have the permission to run as an executable. To give it that permission, we must run ONCE

cd ~/ros2_tutorials_preamble/python/minimalist_package/minimalist_package
chmod +x minimalist_script.py

and now we can run it properly with

cd ~/ros2_tutorials_preamble/python/minimalist_package/minimalist_package
./minimalist_script.py

resulting in

Howdy!
Howdy!
Howdy!

Note that for this second execution strategy to work, we MUST have the #!, called shebang, at the beginning of the first line. The path after the shebang specifies what program will be used to interpret that file. In general, differently from Windows, Ubuntu does not guess the file type by the extension when running it.

#!/bin/python3

If we remove the shebang line and try to execute the script, it will return the following errors, because Ubuntu doesn’t know what to do with that file.

./minimalist_script.py: line 2: import: command not found
./minimalist_script.py: line 5: syntax error near unexpected token `('
./minimalist_script.py: line 5: `def main() -> None:'

When using if __name__=="__main__":, just call the real main()

There are multiple ways of running a Python script. In the one we just saw, the name of the module becomes __main__, but in others that does not happen, meaning that the if can be completely skipped. So, write the main() function of a script as something standalone and, in the condition, just call it and do nothing else, as shown below

if __name__ == "__main__":
    """When this module is run directly, it's __name__ property will be '__main__'."""
    main()

It’s dangerous to go alone: Always wrap the contents of main function on a try–except block

It is good practice to wrap the contents of main() call in a try--except block with at least the KeyboardInterrupt clause. This allows the user to shutdown the module cleanly either through the terminal or through PyCharm. We have done so in the example as follows

def main() -> None:
    """An example main() function that prints 'Howdy!' twice per second."""
    try:
        while True:
            print("Howdy!")
            time.sleep(0.5)
    except KeyboardInterrupt:
        pass
    except Exception as e:
        print(e)

This is of particular importance when hardware is used, otherwise, the connection with it might be left in an undefined state causing difficult-to-understand problems at best and physical harm at worst.

The Exception clause in our example is very broad, but a MUST in code that is still under development. Exceptions of all sorts can be generated when there is a communication error with the hardware, software (internet, etc), or other issues.

This broad Exception clause could be replaced for a less broad exception handling if that makes sense in a given application, but that is usually not necessary nor safe. When handling hardware, it is, in general, IMPOSSIBLE to test the code of all combinations of inputs and states. As they say,

Be wary, for overconfidence is a slow and insidious [source for terrible bugs and failed demos]

Hint

Catching all Exceptions might make debugging more difficult in some cases. At your own risk, you can remove this clause temporarily when trying to debug a stubborn bug, at the risk of forgetting to put it back and ruining your hardware.

Minimalist class: Use classes profusely

In this step, we’ll work on these.

python/minimalist_package/
  └── minimalist_package/
        └── __init__.py
        └── minimalist_script.py
        └── _minimalist_class.py

As you are familiar with object-oriented programming, you know that classes are central to this paradigm. As a memory refresher, let’s make a class that honestly does nothing really useful but illustrates all the basic points in a Python class.

Create a file in ~/ros2_tutorials_preamble/python/minimalist_package/minimalist_package called _minimalist_class.py with the following contents.

_minimalist_class.py

class MinimalistClass:
    """
    A minimalist class example with the most used elements.
    https://docs.python.org/3/tutorial/classes.html
    """
    # Attribute reference, accessed with MinimalistClass.attribute_reference
    attribute_reference: str = "Hello "

    def __init__(self,
                 attribute_arg: float = 10.0,
                 private_attribute_arg: float = 20.0):  # With a default value of 20.0
        """The __init__ works together with __new__ (not shown here) to
        construct a class. Loosely it is called the Python 'constructor' in
        some references, although it is officially an 'initializer' hence
        the name.
        https://docs.python.org/3/reference/datamodel.html#object.__init__
        It customizes an instance with input arguments.
        """
        # Attribute that can be accessed externally
        self.attribute: float = attribute_arg

        # Attribute that should not be accessed externally
        # a name prefixed with an underscore (e.g. _spam) should be treated
        # as a non-public part of the API (whether it is a function, a method or a data member).
        # It should be considered an implementation detail and subject to change without notice.
        self._private_attribute: float = private_attribute_arg

    def method(self) -> float:
        """Methods with 'self' should use at least one statement in which 'self' is required."""
        return self.attribute + self._private_attribute

    def set_private_attribute(self, private_attribute_arg: float) -> None:
        """If a private attribute should be writeable, define a setter."""
        self._private_attribute = private_attribute_arg

    def get_private_attribute(self) -> float:
        """If a private attribute should be readable, define a getter."""
        return self._private_attribute

    @staticmethod
    def static_method():
        """
        Methods that do not use the 'self' should be decorated with the @staticmethod.
        It will only have access to attribute references.
        https://docs.python.org/3.10/library/functions.html#staticmethod
        """
        return MinimalistClass.attribute_reference + "World!"

then, let’s modify the __init__.py with the following contents

__init__.py

"""
Having an __init__.py file within a directory turns it into a Python Package.
A package within a package is called a subpackage.
https://docs.python.org/3/tutorial/modules.html#packages
"""
from minimalist_package._minimalist_class import MinimalistClass

Note

When adding imports to the __init__.py, the folder that we use to open in Pycharm and that we call to execute the scripts is extremely relevant. When packages are deployed (e.g. in PyPI or ROS2), the “correct” way to import in __init__.py is to use import <PACKAGE_NAME>.<THING_TO_IMPORT>, which is why we’re doing it this way.

Note

Relative imports such as .<THING_TO_IMPORT> might work in some cases, and that is fine. It is a supported and valid way to import. However, don’t be surprised when it doesn’t work in ROS2, PyPI packages, etc, and generates a lot of frustration.

Not a matter of taste: Code style

It might be parsing through jibber-jabber code in l__tcode lessons with weird C-pointer logic and nested dereference operators that gets you through the door into one of those fancy companies with no dress code and free snacks, perks that I’m totally not envious of one bit. In the ideal world, at least, writing easy-to-understand code with the proper style is what should keep you in that job.

So, always pay attention to the naming of classes (PascalCase), files and functions (snake_case), etc.

Thankfully, Python has a bunch of style rules builtin the language and PEP, such as PEP8. Take this time to read it and get inspired by The Zen of Python

Beautiful is better than ugly.

Explicit is better than implicit.

Simple is better than complex.

Complex is better than complicated.

Flat is better than nested.

Sparse is better than dense.

Readability counts.

Special cases aren’t special enough to break the rules.

Although practicality beats purity.

Errors should never pass silently.

Unless explicitly silenced.

In the face of ambiguity, refuse the temptation to guess.

There should be one– and preferably only one –obvious way to do it.

Although that way may not be obvious at first *unless you’re Dutch*.

Now is better than never.

Although never is often better than *right now.*

If the implementation is hard to explain, it’s a bad idea.

If the implementation is easy to explain, it may be a good idea.

Namespaces are one honking great idea – let’s do more of those!

Take the (type) hint: Always use type hints

Note

For more info, check out the documentation on Python typing and the type hints cheat sheet

Before you flood my inbox with complaints, let me vent for you. A preemptive vent.

But, you know, one of the cool things in Python is that we don’t have to explicitly type variables. Do you want to turn Python into C?? Why do you love C++ so much you unpythonic Python hater????

The dynamic typing nature of Python is, no doubt, a strong point of the language. Note that adding type hints does not impede your code to be used with other types as arguments. Type hints are, to no one’s surprise, hints to let users (and some automated tools) know what types your functions were made for, e.g. to allow your favorite IDE to help you with code suggestions.

In these tutorials, we are not going to use any complex form of type hints. We’re basically going to attain ourselves to the simplest two forms, the (attribute, argument, etc) type, and the return types.

For attributes we use <attribute>: type, as shown below

        self.attribute: float = attribute_arg

For method arguments we use <argument>: <type> and for return types we use def <method>(<params>) -> <type>, as shown below in our example

    def set_private_attribute(self, private_attribute_arg: float) -> None:
        """If a private attribute should be writeable, define a setter."""
        self._private_attribute = private_attribute_arg

Document your code with Docstrings

You do not need to document every single line you code, that would in fact be quite obnoxious

# c stores the sum of a and b
c = a + b

# d stores the square of c
d = c**2

# check if d is zero
if d == 0:
   # Print warning
   print("Warning")

But, on the other side of the coin, it doesn’t take too long for us to forget what the parameters of a function mean. Take the (type) hint: Always use type hints helps a lot, but additional information is always welcome. If you get used to using docstrings for every new method, your programming will be better in general because documenting your code makes you think about it.

The example below shows a quick explanation of what the class does using a docstring

class MinimalistClass:
    """
    A minimalist class example with the most used elements.
    https://docs.python.org/3/tutorial/classes.html
    """

The PEP 257 talks about docstrings but does not define too much beyond saying that we should use it. My recommendation as of now would be the Sphinx markup, because of the many Python libraries using it for Sphinx documentation/tutorials like this one.

The sample code shown in this section has docstrings everywhere, but they are being used to explain the general usage of some Python syntax. When documenting your code, obviously, the documentation should be about what the method/class/attribute does.

Hint

Ideally, all documentation is perfect from the start. In reality, however, that rarely ever happens so some documentation is always better than none. My advice would be to write something as it goes and possibly adjust it to more stable or cleaner documentation when the need arises.

Unit tests: always test your code

Note

For a comprehensive tutorial on unit testing go through the unittest docs.

In this step, we’ll work on these.

python/minimalist_package/
  └── minimalist_package/
        └── __init__.py
        └── minimalist_script.py
        └── _minimalist_class.py
  └── test/
        └── test_minimalist_class.py

Unit testing is a flag that has been waved by programming enthusiasts and is often a good measurement of code maturity.

The elephant in the room is that writing unit tests is boring. Yes, we know, very boring.

Unit tests are boring because they are an investment. Unit testing won’t necessarily make your code […] better, faster, […] right now. However, without tests, don’t be surprised after some point if your implementations make you drown in tech debt. Dedicating a couple of minutes now to make a couple of tests when your codebase is still in its infancy makes it more manageable and less boresome.

Back to the example, a good practice is to create a folder name test at the same level as the packages to be tested, like so

cd ~/ros2_tutorials_preamble/python/minimalist_package
mkdir test

Then, we create a file named test_minimalist_class.py with the contents below in the test folder.

Note

The prefix test_ is important as it is used by some frameworks to automatically discover tests. So it is better not to use that prefix if that file does not contain a unit test.

test_minimalist_class.py

import unittest
from minimalist_package import MinimalistClass


class TestMinimalistClass(unittest.TestCase):
    """For each `TestCase`, we create a subclass of `unittest.TestCase`."""

    def setUp(self):
        self.minimalist_instance = MinimalistClass(attribute_arg=15.0,
                                                   private_attribute_arg=35.0)

    def test_attribute(self):
        self.assertEqual(self.minimalist_instance.attribute, 15.0)

    def test_private_attribute(self):
        self.assertEqual(self.minimalist_instance._private_attribute, 35.0)

    def test_method(self):
        self.assertEqual(self.minimalist_instance.method(), 15.0 + 35.0)

    def test_get_set_private_attribute(self):
        self.minimalist_instance.set_private_attribute(20.0)
        self.assertEqual(self.minimalist_instance.get_private_attribute(), 20.0)

    def test_static_method(self):
        self.assertEqual(MinimalistClass.static_method(), "Hello World!")


def main():
    unittest.main()

Running the tests

For a quick jolt of instant gratification, let’s run the tests before we proceed with the explanation.

There are many ways to run tests written with unittest. The following will run all tests found in the folder test

cd ~/ros2_tutorials_preamble/python/minimalist_package
python -m unittest discover -v test

which will output

test_attribute (test_minimalist_class.TestMinimalistClass) ... ok
test_get_set_private_attribute (test_minimalist_class.TestMinimalistClass) ... ok
test_method (test_minimalist_class.TestMinimalistClass) ... ok
test_private_attribute (test_minimalist_class.TestMinimalistClass) ... ok
test_static_method (test_minimalist_class.TestMinimalistClass) ... ok

----------------------------------------------------------------------
Ran 5 tests in 0.000s

OK

Yay! We’ve done it!

Start with use unittest

Note

ROS2 uses pytest as default, but that doesn’t mean you also have to use it in every Python code you ever write.

There are many test frameworks for Python. Nonetheless, the unittest module is built into Python so, unless you have a very good reason not to use it, just [use] it.

We import the unittest module along with the class that we want to test, namely MinimalistClass.

import unittest
from minimalist_package import MinimalistClass

Test them all

Note

Good unit tests will not only let you know when something broke but also where it broke. A failed test of a high-level function might not give you too much information, whereas a failed test of a lower-level (more fundamental) function will allow you to pinpoint the issue.

Unit tests are somewhat like insurance. The more coverage you have, the better. In this example, we test all the elements in the class. Each test will be based on one or more asserts. For more info check the unittest docs.

In a few words, we make a subclass of unittest.TestCase and create methods within it that test one part of the code, hence the name unit tests.

    def test_attribute(self):
        self.assertEqual(self.minimalist_instance.attribute, 15.0)

    def test_private_attribute(self):
        self.assertEqual(self.minimalist_instance._private_attribute, 35.0)

    def test_method(self):
        self.assertEqual(self.minimalist_instance.method(), 15.0 + 35.0)

    def test_get_set_private_attribute(self):
        self.minimalist_instance.set_private_attribute(20.0)
        self.assertEqual(self.minimalist_instance.get_private_attribute(), 20.0)

    def test_static_method(self):
        self.assertEqual(MinimalistClass.static_method(), "Hello World!")

If one of the asserts fails, then the related test will fail, and the test framework will let us know which one.

The test’s main function

Generally, a test script based on unittest will have the following main function. It will run all available tests in our test class. For more info and alternatives check the unittest docs.

def main():
    unittest.main()