Themes

• Namming styles
• Truthly and Falsy
• Loop
• Exception handling
• Python command line
• Shell Args
• Variables declaration
• Environment variables
• Execute shell process with subprocess module (Python 3.5)

String

• String flavors
• String Declaration
• String formatting

Functions

• Built-in functions
• print() function
• Function definition and call

Object features

• Method overriding
• Constructor

Types

str -> « dog » or ‘dog’
int -> 50
float -> 50.55
bool -> True or False
list -> []
empty : []
list of strings : [« foo »]
list of numbers [1,2,5]
dictionary -> {}
tuple: (a,b,c)
empty : {}
not empty : {‘jack’: 1234, ‘john’: 1111}

Namming styles

General

From Google python conventions :
module_name, package_name, ClassName, method_name, ExceptionName, function_name, GLOBAL_CONSTANT_NAME, global_var_name, instance_var_name, function_parameter_name, local_var_name.

function/methods specificities

_single_leading_underscore: weak « internal use » indicator. E.g. from M import * does not import objects whose names start with an underscore.

single_trailing_underscore_: used by convention to avoid conflicts with Python keyword, e.g :
tkinter.Tllevel(master, class_='ClassName')

__double_leading_underscore: when naming a class attribute, invokes name mangling, e.g:
inside class FooBar, __boo becomes _FooBar__boo;

__double_leading_and_trailing_underscore__: « magic » objects or attributes that live in user-controlled namespaces. E.g.
__init__, __import__ or __file__.
Never invent such names; only use them as documented.

Truthly and Falsy

Indicate whether a variable contains « something » or « nothing ».
For example for these built-in types, these are falsy (and all others is so truthly) :
None, [], 0, False, {}, '', "", ()
The truth implementation relies on __bool__(self) if defined. If not defined, __len__() (>0 comparison) is used. If not defined either, that is considered truthly.

Built-in functions

print(*objects, sep=' ', end='n', file=sys.stdout, flush=False)
Print objects to the text stream file, separated by sep and followed by end.
sep, end, file and flush, if present, must be given as keyword arguments.
About non-keyword arguments : these are converted to strings like str() does and written to the stream, separated by sep and followed by end.
About keyword arguments :
sep and end must be strings; they can also be None, which means to use the default values.
file must be an object with a write(string) method; if it is not present or None, sys.stdout will be used.

len(s)
Return the length (the number of items) of an object.
The argument may be :
– a sequence (such as a string, bytes, tuple, list, or range)
– a collection (such as a dictionary, set, or frozen set).

dir([object])
describe module, class or object attributes according to the parameter passed to
– Without arguments: return the list of names in the current local scope.
– With an argument: attempt to return a list of valid attributes for that object.

The default dir() mechanism behaves differently with different types of objects, as it attempts to produce the most relevant, rather than complete, information:
– If the object is a module object, the list contains the names of the module’s attributes.
– If the object is a type or class object, the list contains the names of its attributes, and recursively of the attributes of its bases.
– Otherwise, the list contains the object’s attributes’ names, the names of its class’s attributes, and recursively of the attributes of its class’s base classes.

example: list modules available in the current scope:

>>> dir()
['__annotations__', '__builtins__', '__doc__', '__loader__', '__name__', '__package__', '__spec__']

list attributes for that object(here a module):

>>> import struct
>>> dir(struct)
['Struct', '__all__', '__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__', '_clearcache', 'calcsize', 'error', 'iter_unpack', 'pack', 'pack_into', 'unpack', 'unpack_from']

list attributes for that object(here a function):

>>> dir(struct.pack)
['__call__', '__class__', '__delattr__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__name__', '__ne__', '__new__', '__qualname__', '__reduce__', '__reduce_ex__', '__repr__', '__self__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__text_signature__']

show documentation for that object(here a module)

>>> help(struct)
...

show documentation for that object(here a function)

>>> help(struct.pack)
Help on built-in function pack in module _struct:
 
pack(...)
    pack(format, v1, v2, ...) -> bytes
 
    Return a bytes object containing the values v1, v2, ... packed according
    to the format string.  See help(struct) for more on format strings.

Here the shape class to illustrate how to document our own code:

class Shape:
    """the shape class is an example class created to see how the documentation works"""
    def __dir__(self):
        """blob able"""
        return ['area', 'perimeter', 'location']
    def foo(self):
        """it is a wonderful function"""
        print('foo')

show the documentation of the Shape class:

>>> help(Shape)
Help on class Shape in module __main__:
 
class Shape(builtins.object)
 |  the shape class is an example class created to see how the documentation works
 |
 |  Methods defined here:
 |
 |  __dir__(self)
 |      blob able
 |
 |  foo(self)
 |      it is a wonderful function
 |

list attributes of the instance of the Shape class(we can see that the result is the call of the dir() function):

>>> s = Shape()
>>> dir(s)
['area', 'location', 'perimeter']

String flavors

f-strings : string literals that are prefixed by the letter ‘f’ or ‘F’.
it allows to format a string with some variable without the requirement to use concatenation and string conversion.
ex:
print(f'ex={ex}')'

raw string: string literals that are prefixed by the letter ‘r’ or ‘R’.
It treats backslashes \ as literal characters. Raw strings are useful when handling strings that use a lot of backslashes, such as Windows paths and regular expression patterns.
ex:

path = 'C:\\Windows\\system32\\cmd.exe'
rpath = r'C:\Windows\system32\cmd.exe'
print(path == rpath)
# True

combine raw and formated strings
ex:

file='cmd.exe'
rpath = fr'C:\Windows\system32\{file}'

String declaration

To declare a string on several lines that preserves any blanks/newlines between the delimiters, enclose it with triple  » or triple ‘ :

s = """Some chars here  
   and some others there"""

To declare a string on several lines that does not preserve any spaces between the delimiters :

s = ("this list is written "
         "on multiple"
         "line! Yes"
    )

String formatting

String concatenation with + operator : NOT RECOMMENDED

Rare enough to be mentioned : in Python String concatenations with + operator are not implicit/straight at all contrary to other modern languages.
For example in Java, we can concatenate any primitive or Object with String(s).
Indeed for object variables we concatenate with a String, the compiler generates whenever required the toString() method on the objects.
So in Java that is perfectly valid at compile time (and runtime):

String myString = "foo" + 3 + Arrays.asList("one", "two");
System.out.println(myString); // output: foo3[one, two]

In Python, the dumb equivalent will fail at runtime at the first statement:

myString: str = "foo" + 3 + ["one", "two"] 
# TypeError: can only concatenate str (not "int") to str
print(str)

To achieve such a thing, each operand to concatenate has to be first convert to a string with the str() function such as :

myString: str = "foo" + str(3) + str(["one", "two"])
print(myString)# output:  foo3['one', 'two']

Or better don’t use string concatenation but instead of the format() function or still better the f-string.

String formatting function : str.format(*args, **kwargs)

The string on which this method is called can contain literal text and replacement fields delimited by braces {}.
Each replacement field contains either the numeric index of a positional argument, or the name of a keyword argument.
Examples :

name = "David"
age = 120
# default numeric index by order appearance
print("He said his name is {} and that he is {} old.".format(name, age))
# output: He said his name is David and that he is 120 old.
 
# numeric index
print("{0} said his name is {0} and that he is {1} old.".format(name, age))
# output: David said his name is David and that he is 120 old.
 
# both keyword argument and numeric index
print("{name} said his name is {name} and that he is {0} old.".format(age, name=name))
# output: David said his name is David and that he is 120 old.

f-strings, or formatted string literals : THE BEST WAY !

str.format() is powerful but quite cumbersome, so Python 3.6 introduces a new elegant while flexible way to format/concatenate Strings and other objects into String.
Formatted string literals are prefixed with ‘f’ and are similar to the format strings accepted by str.format() and replacement fields are surrounded by curly braces.
When use that ?
When we want to format an inline string (one shot use).
Not relevant if we need to define a reusable string with some token to replace in distinct ways according to the requirement.
Here some examples :

import decimal
 
#Basic example with string and number fields
name = "David"
age = 120
print(f"He said his name is {name} and that he is {age} old.")
# output: He said his name is David and that he is 120 old.
 
# More elaborated example with subfields that we use in str.format() such as width and precision
#Example with width
width = 8
idPerson = 1
print(f"formatted idPerson is {idPerson:{width}}")
# output: formatted idPerson is        1
 
idPerson = 100
print(f"formatted idPerson is {idPerson:{width}}")
# output: formatted idPerson is      100
 
#Example with precision
precision = 4
value = decimal.Decimal("12.34567")
print(f"formatted value is {value:.{precision}}")
# output: formatted value is 12.35
 
#Example with both width and precision
width = 8
precision = 4
value = decimal.Decimal("12.34567")
print(f"formatted value is {value:{width}.{precision}}")
# output: formatted value is    12.35

print function

aaa

Function definition and call

Python 3.8 : rich while quite complex arguments flavors

To be short :
A function argument can be only-positional, only-keyword or both at the same time.
A function argument can be optional or required but never both at the same time.
Below, syntax and example for positional-only, keyword-only and both.
In each scenario, we will illustrate an example with a function that takes a required and an optional parameter. Indeed in the examples, foo is mandatory and bar is optional.

Default flavor : parameters both positional and keyword

Syntax : nothing.

def do(foo, bar="defaultbar"):
    print(foo, bar)

Call examples

# positional work
do("dog")  # dog defaultbar
do("dog", "cat")  # dog cat
 
# keyword work
do(foo="dog", bar="cat")  # dog cat
 
# mixed work
do(foo="dog")  # dog defaultbar
do("dog", bar="cat")  # dog cat
 
# pass args as a dictionary
do(**{'foo': 'dog', 'bar': 'cat'})

The last example works only for params that are not marked as positional-only.

Positional-only parameters

Syntax : add a / parameter after parameters to consider as positional-only.
Example :

def do_with_required_and_optional_positional_only_args(foo, bar="defaultbar", /):
    print(foo, bar)

Call examples

# work 
do_with_required_and_optional_positional_only_args("dog")  # dog defaultbar
do_with_required_and_optional_positional_only_args("dog", "cat")  # dog cat
 
# TypeError: cannot pass  the 'bar' positional-only argument as a keyword argument !
do_with_required_and_optional_positional_only_args("dog", bar="cat")
# TypeError: Same thing with 'foo' !
do_with_required_and_optional_positional_only_args(foo="cat")

Keyword-only parameters

Syntax : add a * parameter before parameters to consider as keyword-only.
Example :

def do_with_required_and_optional_keyword_only_args(*, foo, bar="defaultbar"):
    print(foo, bar)

Call examples

# work
do_with_required_and_optional_keyword_only_args(foo="dog")  # dog defaultbar
do_with_required_and_optional_keyword_only_args(foo="dog", bar="cat")  # dog cat
 
# TypeError : cannot pas 'bar' or 'foo' as positional arg !
do_with_required_and_optional_keyword_only_args("dog", "cat")

Order of passed parameters

The order has to be :

callFoo(POSITIONAL_ARG(S), KEYWORD_ARG(S)).

Variable number arguments : *args and **kwargs

*args and **kwargs is a common idiom to allow arbitrary number of arguments to functions

*args will give you all function parameters as a tuple: Example :

def foo(*args):
    for a in args:
        print(a)        
 
foo(1)
# 1
 
foo(1,2,3)
# 1
# 2
# 3

**kwargs give all keyword arguments except for those corresponding to a formal parameter as a dictionary.
Example :

def bar(**kwargs):
    print(kwargs['name'], kwargs['age'])
    for a in kwargs:
        print(a, kwargs[a])  
 
bar(name='one', age=27)
bar(**{'name':'one', 'age':27})
# age 27
# name one

unpack iterable (list, tuple, …) or dictionary

example:

def print_data(a: str, b: str, c: str):
    print(f'a={a}')
    print(f'b={b}')
    print(f'c={c}')
 
 
t = ("foo", "bar", "foobar1")
print_data(*t)
print('')
 
t = ["foo", "bar", "foobar1"]
print_data(*t)
print('')
 
d = {'a': 'foo', 'b': 'bar', 'c': 'foobar2'}
print_data(**d)

Output:

a=foo
b=bar
c=foobar1
 
a=foo
b=bar
c=foobar1
 
a=foo
b=bar
c=foobar2

Object features

Methods of Object

__repr__(self) :
Called by the repr() built-in function to compute the “official” string representation of an object.
If possible, this should look like a valid Python expression that could be used to recreate an object with the same value (given an appropriate environment).
If this is not possible, a string of the form <...some useful="" description=""> should be returned.

__str__(self) :
Called by str(object) and the built-in functions format() and print() to compute the “informal” or nicely printable string representation of an object.
The return value must be a string object.
Difference with object.__repr__() : no expectation that __str__() return a valid Python expression: a more convenient or concise representation can be used.
Default implementation : object.__repr__().

Method overriding

Ex: override __str__(self, *args, **kwargs) defined in object.

class Foo(object):
    def __str__(self) -> str:
        return self.title + " released the " + str(self.release_date)

Constructor

Declare it :

from datetime import date
 
 
class Movie(object):
    def __init__(self, title: str, release_date: date):
        self.release_date = release_date
        self.title = title

Instantiate an object:

book = Movie("star wars episode 1", date(1999, 10, 13))

Exception handling

Catch any exception and capture the textual message or the stacktrace:

try:
    input_parameters = 'boy'.index(77)
except Exception as e:
    print('Not good :', e)
    logging.error(traceback.format_exc())
    do_other_thing(traceback)

To catch several exceptions :

 except (TypeError, Exception, ...) as e:

Python command line

Execute a python file :

python3 foo.py

Execute python code on a shell :

python3 -c "foo python code"

To list installed modules :

python3 -c "help('modules')"

Shell Args

Pass arguments when we execute a script

We have several flavors for args (optional and positional).
Ex for 1 positional and two optional args :
python3 foo.py foo --arg2=bar -arg3=foobar

Native way to get/set arguments passed to a script

The list of command line arguments passed to a Python script are stored in sys.argv[].
0th arg: script name
1st arg: first passed arg
2nd arg: second passed arg
...
We can get and set the sys.argv whenever.

User-friendly command-line module: argparse

That program defines required arguments and argparse figure out how to parse those out of sys.argv.
It also automatically generates help and usage messages and issues errors when users give the program invalid arguments.
We could define : 
- positional args (required and not key prefixed)
- optional args (no required and key prefixed)
- both required and key prefixed  args (mix between the two above)

import argparse
 
 
def foo():
    # Get args from the script + a help for script clients:
    parser = argparse.ArgumentParser(description='Foo usage help')
    #  Positional : key to not specify
    parser.add_argument('name', help='name that..')
    #  Optional (-- or -) : key to specify
    parser.add_argument('--gender', help='gender that...')
    #  Not positional but required (--) : key to specify
    parser.add_argument('--taste', help='taste that...', required=True)
    #  Not positional but required (-) :  key to specify
    parser.add_argument('-mind', help='mind that...', required=True)
    #  and so for ....
    args = parser.parse_args()
    #
    print(args)
 
 
if __name__ == '__main__':
    foo()

Examples of use

With the taste optional arg :
python3 Foo.py --taste=burger -mind=open davidxxx
Namespace(gender=None, mind='open', name='davidxxx', taste='burger')

With all possible args :
python3 Foo.py --taste=burger -mind=open --gender=Extra davidxxx
Namespace(gender='Extra', mind='open', name='davidxxx', taste='burger')

With a required but missing arg (-mind) :
python3 Foo.py --taste=burger --gender=Extra davidxxx
usage: Test.py [-h] [--gender GENDER] --taste TASTE -mind MIND name
Test.py: error: the following arguments are required: -mind

variables declaration

global variables

When use it ?
In classes files we declare instance field(s) to store state of the instance.
In no-class python files we declare global variables as workaround.

How to declare and use it ?
Define a variable in the outer scope make it global :

# define main_logger as a global variable
main_logger: logging.Logger # can optionally be set with a default value

A global variable may also be assigned to a default value during its declaration such as :

foo = 'foo value'

To modify or init a global variable after its declaration, we need to explicitly declare the variable as global :

def bar():
    global main_logger
    main_logger = logging.getLogger("elastic-delete-indices")

To access to the global variable wherever :

def foo():
    main_logger.info("yeah!")

Env variables

Environment variables :
available in the os.environ dictionary.

List all environment variables (key-value) :

for e in os.environ:
    print(e, '=', os.environ[e])

Set an environment variable : .
os.environ['FOO'] = 'VALUE'