From previous experience, you should be familiar with defining simple functions that take a set of input parameters and potentially return some value. When calling such a function from somewhere in your Python code, you must provide values (or expressions that evaluate to some value) for each of these parameters, and these values are then accessible under the names of the respective parameters in the code that makes up the body of the function.
However, from working with different tool functions provided by arcpy and different functions from the Python standard library, you also already know that functions can also have optional parameters (often denoted by italics or within {} or 'opt' in documentation and are generally indexed after all of the required parameters), and you can use the names of such parameters to explicitly provide a value for them when calling the function. In this section, we will show you how to write functions with such keyword arguments and functions that take an arbitrary number of parameters, and we will discuss some more details about passing different kinds of values as parameters to a function.
The parameters we have been using so far, for which we only specify a name in the function definition, are called positional parameters or positional arguments because the value that will be assigned to them when the function is called depends on their position in the parameter list: The first positional parameter will be assigned the first value given within the parentheses (…) when the function is called, and so on. Here is a simple function with two positional parameters, one for providing the last name of a person and one for providing a form of address. The function returns a string to greet the person with.
def greet(lastName, formOfAddress):
return 'Hello {0} {1}!'.format(formOfAddress, lastName)
print(greet('Smith', 'Mrs.'))
Output Hello Mrs. Smith!
Note how the first value used in the function call (“Smith”) in line 6 is assigned to the first positional parameter (lastName) and the second value (“Mrs.”) to the second positional parameter (formOfAddress). Nothing new here so far.
The parameter list of a function definition can also contain one or more so-called keyword arguments. A keyword argument appears in the parameter list as
<argument name> = <default value>
A keyword argument can be provided in the function by again using the notation
It can also be left out, in which case the default value specified in the function definition is used. This means keyword arguments are optional. Here is a new version of our greet function that now supports English and Spanish, but with English being the default language:
def greet(lastName, formOfAddress, language = 'English'):
greetings = { 'English': 'Hello', 'Spanish': 'Hola' }
return '{0} {1} {2}!'.format(greetings[language], formOfAddress, lastName)
print(greet('Smith', 'Mrs.'))
print(greet('Rodriguez', 'Sr.', language = 'Spanish'))
Output Hello Mrs. Smith! Hola Sr. Rodriguez!
Compare the two different ways in which the function is called in lines 8 and 10. In line 8, we do not provide a value for the ‘language’ parameter, so the default value ‘English’ is used when looking up the proper greeting in the dictionary stored in variable greetings. In the second version in line 10, the value ‘Spanish’ is provided for the keyword argument ‘language,’ so this is used instead of the default value and the person is greeted with “Hola” instead of "Hello." Keyword arguments can be used like positional arguments meaning the second call could also have been
print(greet('Rodriguez', 'Sr.', 'Spanish'))
without the “language =” before the value.
Things get more interesting when there are several keyword arguments, so let’s add another one for the time of day:
def greet(lastName, formOfAddress, language = 'English', timeOfDay = 'morning'):
greetings = { 'English': { 'morning': 'Good morning', 'afternoon': 'Good afternoon' },
'Spanish': { 'morning': 'Buenos dias', 'afternoon': 'Buenas tardes' } }
return '{0}, {1} {2}!'.format(greetings[language][timeOfDay], formOfAddress, lastName)
print(greet('Smith', 'Mrs.'))
print(greet('Rodriguez', 'Sr.', language = 'Spanish', timeOfDay = 'afternoon'))
Output Good morning, Mrs. Smith! Buenas tardes, Sr. Rodriguez!
Since we now have four different forms of greetings depending on two parameters (language and time of day), we now store these in a dictionary in variable greetings that for each key (= language) contains another dictionary for the different times of day. For simplicity reasons, we left it at two times of day, namely “morning” and “afternoon.” In line 7, we then first use the variable language as the key to get the inner dictionary based on the given language and then directly follow up with using variable timeOfDay as the key for the inner dictionary.
The two ways we are calling the function in this example are the two extreme cases of (a) providing none of the keyword arguments, in which case default values will be used for both of them (line 10), and (b) providing values for both of them (line 12). However, we could now also just provide a value for the time of day if we want to greet an English person in the afternoon:
print(greet('Rogers', 'Mrs.', timeOfDay = 'afternoon'))
Output Good afternoon, Mrs. Rogers!
This is an example in which we have to use the prefix “timeOfDay =” because if we leave it out, it will be treated like a positional parameter and used for the parameter ‘language’ instead which will result in an error when looking up the value in the dictionary of languages. For similar reasons, keyword arguments must always come after the positional arguments in the definition of a function and in the call. However, when calling the function, the order of the keyword arguments doesn’t matter, so we can switch the order of ‘language’ and ‘timeOfDay’ in this example:
print(greet('Rodriguez', 'Sr.', timeOfDay = 'afternoon', language = 'Spanish'))
Of course, it is also possible to have function definitions that only use optional keyword arguments in Python.
Let us continue with the “greet” example, but let’s modify it to be a bit simpler again with a single parameter for picking the language, and instead of using last name and form of address we just go with first names. However, we now want to be able to not only greet a single person but arbitrarily many persons, like this:
greet('English', 'Jim', 'Michelle')
Output: Hello Jim! Hello Michelle!
greet('Spanish', 'Jim', 'Michelle', 'Sam')
Output: Hola Jim! Hola Michelle! Hola Sam!
To achieve this, the parameter list of the function needs to end with a special parameter that has a * symbol in front of its name. If you look at the code below, you will see that this parameter is treated like a list in the body of the function:
def greet(language, *names):
greetings = { 'English': 'Hello', 'Spanish': 'Hola' }
for n in names:
print('{0} {1}!'.format(greetings[language], n))
What happens is that all values given to that function from the one corresponding to the parameter with the * on will be placed in a list and assigned to that parameter. This way you can provide as many parameters as you want with the call and the function code can iterate through them in a loop. Please note that for this example we changed things so that the function directly prints out the greetings rather than returning a string.
We also changed language to a positional parameter because if you want to use keyword arguments in combination with an arbitrary number of parameters, you need to write the function in a different way. You then need to provide another special parameter starting with two stars ** and that parameter will be assigned a dictionary with all the keyword arguments provided when the function is called. Here is how this would look if we make language a keyword parameter again:
def greet(*names, **kwargs):
greetings = { 'English': 'Hello', 'Spanish': 'Hola' }
language = kwargs['language'] if 'language' in kwargs else 'English'
for n in names:
print('{0} {1}!'.format(greetings[language], n))
If we call this function as
greet('Jim', 'Michelle')
the output will be:
Hello Jim! Hello Michelle!
And if we use
greet('Jim', 'Michelle', 'Sam', language = 'Spanish')
we get:
Hola Jim! Hola Michelle! Hola Sam!
Yes, this is getting quite complicated, and it’s possible that you will never have to write functions with both * and ** parameters, still here is a little explanation: All non-keyword parameters are again collected in a list and assigned to variable names. All keyword parameters are placed in a dictionary using the name appearing before the equal sign as the key, and the dictionary is assigned to variable kwargs. To really make the ‘language’ keyword argument optional, we have added line 5 in which we check if something is stored under the key ‘language’ in the dictionary (this is an example of using the ternary "... if ... else ..." operator). If yes, we use the stored value and assign it to variable language, else we instead use ‘English’ as the default value. In line 9, language is then used to get the correct greeting from the dictionary in variable greetings while looping through the name list in variable names.
Lesson content developed by Jan Wallgrun and James O’Brien
In this section, we are going to introduce a new and very powerful concept of Python (and other programming languages), namely the idea that functions can be given as parameters to other functions similar to how we have been doing so far with other types of values like numbers, strings, or lists. You see examples of this near the end of the lesson with the pool.starmap(...) function. A function that takes other functions as arguments is often called a higher order function.
Let us immediately start with an example: Let’s say you often need to apply certain string functions to each string in a list of strings. Sometimes you want to convert the strings from the list to be all in upper-case characters, sometimes to be all in lower-case characters, sometimes you need to turn them into all lower-case characters but have the first character capitalized, or apply some completely different conversion. The following example shows how one can write a single function for all these cases and then pass the function to apply to each list element as a parameter to this new function:
def applyToEachString(stringFunction, stringList): myList = [] for item in stringList: myList.append(stringFunction(item)) return myList allUpperCase = applyToEachString(str.upper, ['Building', 'ROAD', 'tree'] ) print(allUpperCase)
As you can see, the function definition specifies two parameters; the first one is for passing a function that takes a string and returns either a new string from it or some other value. The second parameter is for passing along a list of strings. In line 7, we call our function with using str.upper for the first parameter and a list with three words for the second parameter. The word list intentionally uses different forms of capitalization. upper() is a string method that turns the string it is called for into all upper-case characters. Since this a method and not a function, we have to use the name of the class (str) as a prefix, so “str.upper”. It is important that there are no parentheses () after upper because that would mean that the function will be called immediately and only its return value would be passed to applyToEachString(…).
In the function body, we simply create an empty list in variable myList, go through the elements of the list that is passed in parameter stringList, and then in line 4 call the function that is passed in parameter stringFunction to an element from the list. The result is appended to list myList and, at the end of the function, we return that list with the modified strings. The output you will get is the following:
['BUILDING', 'ROAD', 'TREE']
If we now want to use the same function to turn everything into all lower-case characters, we just have to pass the name of the lower() function instead, like this:
allLowerCase = applyToEachString(str.lower, ['Building', 'ROAD', 'tree'] ) print(allLowerCase)
Output ['building', 'road', 'tree']
You may at this point say that this is more complicated than using a simple list comprehension that does the same, like:
[ s.upper() for s in ['Building', 'ROAD', 'tree'] ]
That is true in this case but we are just creating some simple examples that are easy to understand here. For now, trust us that there are more complicated cases of higher-order functions that cannot be formulated via list comprehension.
For converting all strings into strings that only have the first character capitalized, we first write our own function that does this for a single string. There actually is a string method called capitalize() that could be used for this, but let’s pretend it doesn’t exist to show how to use applyToEachString(…) with a self-defined function.
def capitalizeFirstCharacter(s): return s[:1].upper() + s[1:].lower() allCapitalized = applyToEachString(capitalizeFirstCharacter, ['Building', 'ROAD', 'tree'] ) print(allCapitalized)
Output ['Building', 'Road', 'Tree']
The code for capitalizeFirstCharacter(…) is rather simple. It just takes the first character of the given string s and turns it into upper-case, then takes the rest of the string and turns it into lower-case, and finally puts the two pieces together again. Please note that since we are passing a function as parameter not a method of a class, there is no prefix added to capitalizeFirstCharacter in line 4.
Lesson content developed by Jan Wallgrun and James O’Brien
In a case like this where the function you want to use as a parameter is very simple like just a single expression and you only need this function in this one place in your code, you can skip the function definition completely and instead use a so-called lambda expression. A lambda expression basically defines a function without giving it a name using the format (there's a good first principles discussion on Lambda functions here [1] at RealPython).
lambda <parameters>: <expression for the return value>
For capitalizeFirstCharacter(…), the corresponding lamba expression would be this:
lambda s: s[:1].upper() + s[1:].lower()
Note that the part after the colon does not contain a return statement; it is always just a single expression and the result from evaluating that expression automatically becomes the return value of the anonymous lambda function. That means that functions that require if-else or loops to compute the return value cannot be turned into lambda expression. When we integrate the lambda expression into our call of applyToEachString(…), the code looks like this:
allCapitalized = applyToEachString(lambda s: s[:1].upper() + s[1:].lower(), ['Building', 'ROAD', 'tree'] )
Lambda expressions can be used everywhere where the name of a function can appear, so, for instance, also within a list comprehension:
[(lambda s: s[:1].upper() + s[1:].lower())(s) for s in ['Building', 'ROAD', 'tree'] ]
We here had to put the lambda expression into parenthesis and follow up with “(s)” to tell Python that the function defined in the expression should be called with the list comprehension variable s as parameter.
So far, we have only used applyToEachString(…) to create a new list of strings, so the functions we used as parameters always were functions that take a string as input and return a new string. However, this is not required. We can just as well use a function that returns, for instance, numbers like the number of characters in a string as provided by the Python function len(…). Before looking at the code below, think about how you would write a call of applyToEachString(…) that does that!
Here is the solution.
wordLengths = applyToEachString(len, ['Building', 'ROAD', 'tree'] ) print(wordLengths)
len(…) is a function so we can simply put in its name as the first parameter. The output produced is the following list of numbers:
Output [8, 4, 4]
With what you have seen so far in this lesson the following code example should be easy to understand:
def applyToEachNumber(numberFunction, numberList): l = [] for item in numberList: l.append(numberFunction(item)) return l roundedNumbers = applyToEachNumber(round, [12.3, 42.8] ) print(roundedNumbers)
Right, we just moved from a higher-order function that applies some other function to each element in a list of strings to one that does the same but for a list of numbers. We call this function with the round(...) function for rounding a floating point number. The output will be:
Output [12.0, 43.0]
If you compare the definition of the two functions applyToEachString(…) and applyToEachNumber(…), it is pretty obvious that they are exactly the same, we just slightly changed the names of the input parameters! The idea of these two functions can be generalized and then be formulated as “apply a function to each element in a list and build a list from the results of this operation” without making any assumptions about what type of values are stored in the input list. This kind of general higher-order function is already available in the Python standard library. It is called map(…) and it is one of several commonly used higher-order functions defined there. In the following, we will go through the three most important list-related functions defined there, called map(…), reduce(…), and filter(…).
Lesson content developed by Jan Wallgrun and James O’Brien
Like our more specialized versions, map(…) takes a function (or method) as the first input parameter and a list as the second parameter. It is the responsibility of the programmer using map(…) to make sure that the function provided as a parameter is able to work with whatever is stored in the provided list. In Python 3, a change to map(…) has been made so that it now returns a special map object rather than a simple list. However, whenever we need the result as a normal list, we can simply apply the list(…) function to the result like this:
l = list(map(…, …))
The three examples below show how we could have performed the conversion to upper-case and first character capitalization, and the rounding task with map(...) instead of using our own higher-order functions:
map(str.upper, ['Building', 'Road', 'Tree']) map(lambda s: s[:1].upper() + s[1:].lower(), ['Building', 'ROAD', 'tree']) # uses lambda expression for only first character as upper-case map(round, [12.3, 42.8])
Map is actually more powerful than our own functions from above in that it can take multiple lists as input together with a function that has the same number of input parameters as there are lists. It then applies that function to the first elements from all the lists, then to all second elements, and so on. We can use that to, for instance, create a new list with the sums of corresponding elements from two lists as in the following example. The example code also demonstrates how we can use the different Python operators, like the + for addition, with higher-order functions: The operator module [2] from the standard Python library contains function versions of all the different operators that can be used for this purpose. The one for + is available as operator.add(...).
import operator map(operator.add, [1,3,4], [4,5,6])
Output [5, 8, 10]
As a last map example, let’s say you instead want to add a fixed number to each number in a single input list. The easiest way would then again be to use a lambda expression:
number = 11 map(lambda n: n + number, [1,3,4,7])
Output [12, 14, 15, 18]
Lesson content developed by Jan Wallgrun and James O’Brien
The goal of the filter(…) higher-order function is to create a new list with only certain items from the original list that all satisfy some criterion by applying a boolean function to each element (a function that returns either True or False) and only keeping an element if that function returns True for that element.
Below we provide two examples for this, one for a list of strings and one for a list of numbers. The first example uses a lambda expression that uses the string method startswith(…) to check whether or not a given string starts with the character ‘R’. Here is the code:
newList = filter(lambda s: s.startswith('R'), ['Building', 'ROAD', 'tree'])
print(newList)
Output ['ROAD']
In the second example, we use is_integer() from the float class to take only those elements from a list of floating point numbers that are integer numbers. Since this is a method, we again need to use the class name as a prefix (“float.”).
newList = filter(float.is_integer, [12.4, 11.0, 17.43, 13.0]) print(newList)
Output [11.0, 13.0]
Lesson content developed by Jan Wallgrun and James O’Brien
import operator from functools import reduce result = reduce(operator.add, [234,3,3], 0) # sum print(result)
Output 240
import operator from functools import reduce result = reduce(operator.mul, [234,3,3], 1) # product print(result)
Output 2106
Other things reduce(…) can be used for are computing the minimum or maximum value of a list of numbers or testing whether or not any or all values from a list of booleans are True. We will see some of these use cases in the practice exercises of this lesson. Examples of the higher-order functions discussed in this section will occasionally appear in the examples and walkthrough code of the remaining lessons.
Lesson content developed by Jan Wallgrun and James O’Brien