Comprehension

A concise way to create a new list or a new dictionary from an existing iterable or sequence.
Syntax :
It consists of brackets (for list comprehension) and of braces(for dict comprehension) containing an expression followed by a for clause, then zero or more for or if clauses.

list comprehension

very basic syntax:

foo = [x for x in foo_iterable ]

syntax with a condition:

foo = [x for x in foo_iterable if x > 10 ]

the result expression (here x variable) at the beginning of the comprehension expression represents each element that we will add in the list created.
We can use the elements of the initial iterable such as or we can modify them(adding, substracting, concat…)
Beware: the conditional statement is evaluated before the result expression

Examples :

No nested and without condition.
Create a list with all squares from 0 to 10 :

squares = [x ** 2 for x in range(10)]
# [0, 1, 4, 9, 16, 25, 36, 49, 64, 81]

No nested and with condition.
Create a list with all squares from 0 to 10 for input that are even:

squares = [x ** 2 for x in range(10) if (x % 2 == 0)]
# [0, 4, 16, 36, 64]

More complex case: reuse in the result a value computed in the conditional statement
We can avoid repeating the statement to create the value (That is in the conditional statement and in the result statement)by using an assignment expression in the conditional statement.
Example:
For each element of the initial sequence, we want to get the stripped string of the first group of a pattern match. But if the pattern doesn’t match with the element we just want to skip that element in the final result.
We can see that the match variable is initialized in the conditional statement and reused in the result statement:

    pattern: Pattern = re.compile(r'self\.(.*?)=')
    fields_list: List[str] = [match.group(1).strip() for line in lines
                              if (match := re.match(pattern, line.strip())) is not None]

Dictionary comprehensions

we have two main cases:
– create the new dictionary from a list
– create the new dictionary from a dictionary

the syntax is very close to the syntax used for the list comprehension.

create the new dictionary from a list
example: dictionary created from the expression x: x*2 and the Iterable (2, 4, 6)

dic = {x: x * 2 for x in (2, 4, 6)}
print(f'dic={dic}')

output:

    dic={2: 4, 4: 8, 6: 12}

create the new dictionary from a dictionary
example: dictionary created from the expression k: v.capitalize() and the dictionary : {'name': 'david', 'size': 'big'}

original_dic = {'name': 'david', 'size': 'big'}
dic = {k: v.capitalize() for k, v in original_dic.items()}
print(f'dic={dic}')

output:

    dic={'name': 'David', 'size': 'Big'}

call of function dynamically

in python,passing a function as a parameter is very straight.
we just need to pass the name of the function without the parenthesis
we will illustrate with the 4 main cases:
– call the function without parameter.
– call the function with specific parameters.
– call the function with *args parameter.
– call the function with *kwargs parameter.

here are the functions we want to call:

def add(a: int, b: int) -> int:
    return a + b
 
def print_hello():
    print('hello')

call the function without parameter

function = print_hello
function()#hello

call the function with specific parameters

function = add
print(f'function(1,5)={function(1, 5)}')

call the function with *args parameter or with *kwargs parameter

1) Call the function directly

result = add(*(1, 3))
print(f'result={result}')
 
result = add(**{'a': 4, 'b': 2})
print(f'result={result}')

2)Or we can also pass by a common function to call dynamically any other function:

def function_that_can_call_other_function(function, *args, **kwargs):
    if args:
        return function(*args)
    elif kwargs:
        return function(**kwargs)
    else:
        return function()

Here the calls to that function with the different cases:
The syntax is the same as above.
call with no parameter:

function_that_can_call_other_function(print_hello)
    #hello

call with kwargs parameter:

return_value = function_that_can_call_other_function(add, **{'a': 2, 'b': 5})
print(f'return_value={return_value}') # return_value=7
# or simplified syntax
return_value = function_that_can_call_other_function(add, a=2, b=5)
print(f'return_value={return_value}')# return_value=7

call with args parameter:

return_value = function_that_can_call_other_function(add, *(2, 5))
print(f'return_value={return_value}')# return_value=7
# or simplified syntax
return_value = function_that_can_call_other_function(add, 2, 5)
print(f'return_value={return_value}')# return_value=7

Define a function as parameter

– No typed way:

def do_apply(text: str, filter_function) -> str:
          return filter_function(text)

Advantages: very short syntax.
Example:

result = do_apply('john', str.upper)
print(f'result={result}')#JOHN

Drawbacks: no type checking of parameters or of the return type of the function
Example:
We pass a function taking 3 parameters and returning an int while the function executed in do_apply() doesn’t expect them.

result = do_apply('john', str.count)

It fails at runtime:TypeError: count() takes at least 1 argument (0 given)

– Typed way:

def do_apply_robust(text: str, filter_function: Callable[[str], str]) -> str:
    return filter_function(text)

Here the function expects a string parameter and return a string.
Drawbacks: more verbose syntax
Advantages: warnings if type in caller and type in definition doesn’t match and that before the execution
Example ok (It works as in the previous case):

result = do_apply_robust('john', str.upper)
print(f'result={result}')

Example ko:

result = do_apply_robust('john', str.count)

now we have a warning in pycharm before the execution on the statement str.upper :

Expected type '(str) -> str', got '(x: str, __start: Optional[SupportsIndex], __end: Optional[SupportsIndex]) -> int' instead

lambdas

Lambda expressions are used to create anonymous functions.
syntax: lambda parameters: expression.
It yields a function object behaving like a function object defined with:

def <lambda>(parameters):
    return expression

use case: define in a compact/private way a function and use it

example with an argument:
Add 1 to the argument and return the result:

x = lambda a : a + 1
print(x(5)) # 6
print(x(8)) # 9

example with two arguments:
sum the arguments and return the result:

x = lambda a,b : a + b
print(x(1,9)) # 10

Limitations of chaining lambda with functions in python

Python is not at all a functional language, so chaining the stream functions is not as straight as with languages providing more functional features such as scala or even java, javascript…
In functional paradigm, we have generally a fluent API, that is we invoke a function on an object referencing an interface and the function returns in its contract the same interface in the general case, so we can chain calls obj.a().b().c().
Python that has a more procedural paradigm requires to wrap calls such as: a(b(c())), which is of course less simple and straight to read and to write.

Lambda use cases: make a function with a parameter as function that callers will define their own implementation as lambda

That function can be used for filtering,adding,computing,collecting and so for…

example with a filtering function

The filtering function takes a string as parameter and return a boolean:

def filter_animals(animals: List[str], filter_function) -> List[str]:
    return [x for x in animals if filter_function(x)]

Alternatively, we can define the function with the callable interface as seen above to allow stronger type checking:

def filter_animals(animals: List[str], filter_function: Callable[[str], bool]) -> List[str]:
    return [x for x in animals if filter_function(x)]

How to call this function
– We can specify a parameter in the lambda expression and use it such as:

animals = filter_animals(['dog', 'cat', 'lion', 'jaguar'], lambda x: len(x) == 3)
print(f'animals={animals}')  # animals=['dog', 'cat']
 
animals = filter_animals(['dog', 'cat', 'lion', 'jaguar'], lambda x: str(x).count('a') >= 1)
print(f'animals={animals}')  # animals=['cat', 'jaguar']

– Or we can provide no value for the parameter of the lambda expression and instead of use a hardcoded value or a constant.
But we are forced to specify the lambda parameter (with _) even if it is empty.
For example, we define lambda where we hard code the value of the parameter in a way where the lambda return is always true:

animals = filter_animals(['dog', 'cat', 'lion', 'jaguar'], lambda _: len('foo') == 3)
print(f'animals={animals}')  # animals=['dog', 'cat', 'lion', 'jaguar']

example with a creator function

In our example, the called function doesn’t require any parameter, it is just a basic creator function:

def get_if_valid(condition: bool, creator_function: Callable[[], str]) -> str:
    # basic condition just too illustrate
    if condition:
        return creator_function()

Usage:

text = get_if_valid(True, lambda: 'Super item')
print(f'text={text}')#text=Super item
text = get_if_valid(False, lambda: 'Another item')
print(f'text={text}')#text=None

Stream like features

The filter(function,iterable) function:
Accept as arguments an iterable and a predicate function,and return an iterator containing items which are true for the predicate.
generally we want to transform the returned iterator into a more handy object like a list or a tuple.
example:

animals = ['dog', 'cat', 'lion', 'jaguar']
animals_filtered = list(filter(lambda x: len(x) == 3, animals))
print(f'animals_filtered={animals_filtered}')    #animals=['dog', 'cat']

The next(iterator, default) function:
It is quite similar to the findFirst() in java stream.
Accept as arguments an iterator and return the next item from the iterator by calling its __next__() method.
If the iterator is exhausted, StopIterationis raised or default is returned if argument was provided.

Example where an element is found:

animals = ['dog', 'cat', 'lion', 'jaguar']
animal_filtered = next(filter(lambda x: str(x).count('a') >= 1, animals))
print(f'animal_filtered={animal_filtered}')#animal_filtered=cat

Example where an element is found and no default return is provided as parameter:

animals = ['dog', 'cat', 'lion', 'jaguar']
next(filter(lambda x: str(x).count('x') >= 1, animals))

As expected, and exception is raised:

Traceback (most recent call last):
  File "C:\Users\david\AppData\Roaming\JetBrains\PyCharmCE2021.3\scratches\scratch.py", line 103, in <module>
    next(filter(lambda x: str(x).count('x') >= 1, animals))
StopIteration

Example where an element is found but a default return is provided as parameter:

animals = ['dog', 'cat', 'lion', 'jaguar']
animal_filtered =next(filter(lambda x: str(x).count('x') >= 1, animals),'pig')
print(f'animal_filtered={animal_filtered}')#pig

As expected, the default value is returned: ‘pig’

The map(function,iterables) function
Accepts as argument a function and one or multiple iterables.

– Case where we have only one iterable:
Return an iterator that applied function(x) to every item of iterable.

Example:

animals = ['dog', 'cat', 'lion', 'jaguar']
mapped_animals = list(map(lambda x: x.upper(), animals))
print(f'mapped_animals={mapped_animals}')
# mapped_animals=['DOG', 'CAT', 'LION', 'JAGUAR']

We can see that the function is applied for each element of the iterable provided: we have 4 elements in input and so we have 4 elements in output.

– Case where we have multiple iterables:
Return an iterator that applied function(x,y,...) to every item of iterables where the function has as many as parameters that iterables parameter provided.
The map() function works quite similarly to the zip() function:
The iterator stops when the shortest iterable is exhausted.

Example:
we will illustrate with a case where the iterable objects don’t have the same length in order to show that we have less elements in output in this case.

animals = ['dog', 'cat', 'lion', 'jaguar']
colors = ['blue', 'red', 'black']
mapped_animals_with_color = list(map(lambda a, c: c + ' ' + a, animals, colors))
print(f'mapped_animals_with_color={mapped_animals_with_color}')
# mapped_animals_with_color=['blue dog', 'red cat', 'black lion']