Now that you’re familiar with Elixir’s basic building blocks, it’s time to look at some typical low-level idioms of the language. In this chapter, we’ll deal with conditionals and loops. As you’ll see, these work differently than in many imperative languages.
Classical conditional constructs, such as if and case, are often replaced with multiclause functions, and there are no loop statements, such as while. However, you can still solve problems of arbitrary complexity in Elixir, and the resulting code is no more complicated than a typical object-oriented solution.
All this may sound a bit radical, which is why conditionals and loops receive detailed treatment in this chapter. But before we start discussing branching and looping, you need to learn about the important underlying mechanism: pattern matching.
As mentioned in chapter 2, the = operator isn’t an assignment. In the expression a = 1, we bind the variable a to the value 1. The = operator is called the match operator, and the assignment-like expression is an example of pattern matching.
Pattern matching is an important concept in Elixir. It’s a feature that makes manipulations with complex variables (e.g., tuples and lists) a lot easier. Less obviously, it allows you to write elegant, declarative-like conditionals and loops. You’ll see what this means by the end of the chapter; in this section, we’ll look at the basic mechanical workings of pattern matching.
Let’s begin by looking at the match operator.
So far, you’ve seen the most basic use of the match operator:
iex(1)> person = {"Bob", 25}
We treated this as something akin to an assignment, but in reality, something more complex is going on here. At run time, the left side of the = operator is matched to the right side. The left side is called a pattern, whereas on the right side, there is an expression that evaluates to an Elixir term.
In the example, you match the variable person to the right-side term {"Bob", 25}. A variable always matches the right-side term, and it becomes bound to the value of that term. This may seem a bit theoretical, so let’s look at a slightly more complex use of the match operator that involves tuples.
The following example demonstrates basic pattern matching of tuples:
iex(1)> {name, age} = {"Bob", 25}
This expression assumes that the right-side term is a tuple of two elements. When the expression is evaluated, the variables name and age are bound to the corresponding elements of the tuple. You can now verify that these variables are correctly bound:
iex(2)> name "Bob" iex(3)> age 25
This feature is useful when you call a function that returns a tuple and you want to bind individual elements of that tuple to separate variables. The following example calls the Erlang function :calendar.local_time/0 to get the current date and time:
iex(4)> {date, time} = :calendar.local_time()
The date and time are also tuples, which you can further decompose as follows:
iex(5)> {year, month, day} = date
iex(6)> {hour, minute, second} = time
What happens if the right side doesn’t correspond to the pattern? The match fails and an error is raised:
iex(7)> {name, age} = "can't match"
** (MatchError) no match of right hand side value: "can't match"
Note We haven’t yet covered the error-handling mechanisms—they’ll be discussed in chapter 8. For now, suffice it to say that raising an error works somewhat like the classical exception mechanisms in mainstream languages. When an error is raised, control is immediately transferred to code somewhere up the call chain, which catches the error (assuming such code exists).
Finally, it’s worth noting that, just like any other expression, the match expression also returns a value. The result of a match expression is always the right-side term you’re matching against:
iex(8)> {name, age} = {"Bob", 25} ❶
{"Bob", 25} ❷
❷ Result of the match expression
Matching isn’t confined to destructuring tuple elements to individual variables. Surprisingly enough, even constants are allowed on the left side of the match expression:
iex(1)> 1 = 1 1
Recall that the match operator = tries to match the right-side term to the left-side pattern. In the example, you try to match the pattern 1 to the term 1. Obviously, this succeeds, and the result of the entire expression is the right-side term. This example doesn’t have much practical benefit, but it illustrates that you can place constants to the left of =, which proves = is not an assignment operator.
Constants are much more useful in compound matches. For example, tuples are sometimes used to group various fields of a record. The following snippet creates a tuple that holds a person’s name and age:
iex(2)> person = {:person, "Bob", 25}
The first element is a constant atom :person, which you use to denote that this tuple represents a person. Later, you can rely on this knowledge and retrieve individual attributes of the person:
iex(3)> {:person, name, age} = person
{:person, "Bob", 25}
Here, you expect the right-side term to be a three-element tuple, with its first element having a value of :person. After the match, the remaining elements of the tuple are bound to the variables name and age, which you can easily verify:
iex(4)> name "Bob" iex(5)> age 25
This is a common idiom in Elixir. Many functions from Elixir and Erlang return either {:ok, result} or {:error, reason}. For example, imagine your system relies on a configuration file and expects it to always be available. You can read the file contents with the help of the File.read/1 function:
{:ok, contents} = File.read("my_app.config")
In this single line of code, three distinct things happen:
An attempt to open and read the file my_app.config takes place.
If the attempt succeeds, the file contents are extracted to the variable contents.
If the attempt fails, an error is raised. This happens because the result of File.read is a tuple in the form {:error, reason}, so the match to {:ok, contents} fails.
By using constants in patterns, you tighten the match, ensuring some part of the right side has a specific value.
Whenever a variable name exists in the left-side pattern, it always matches the corresponding right-side term. Additionally, the variable is bound to the term it matches.
Occasionally, we aren’t interested in a value from the right-side term, but we still need to match on it. For example, let’s say you want to get the current time of day. You can use the function :calendar.local_time/0, which returns a tuple: {date, time}. But you aren’t interested in a date, so you don’t want to store it to a separate variable. In such cases, you can use the anonymous variable (_):
iex(1)> {_, time} = :calendar.local_time()
iex(2)> time
{20, 44, 18}
When it comes to matching, the anonymous variable works just like a named variable: it matches any right-side term. But the value of the term isn’t bound to any variable.
You can also add a descriptive name after the underscore character:
iex(1)> {_date, time} = :calendar.local_time()
The _date is regarded as an anonymous variable because its name starts with an underscore. Technically speaking, you could use that variable in the rest of the program, but the compiler will emit a warning.
Patterns can be arbitrarily nested. Taking the example further, let’s say you only want to retrieve the current hour of the day:
iex(3)> {_, {hour, _, _}} = :calendar.local_time()
iex(4)> hour
20
A variable can be referenced multiple times in the same pattern. In the following expressions, you expect an RGB triplet with the same number for each component:
iex(5)> {amount, amount, amount} = {127, 127, 127} ❶
{127, 127, 127}
iex(6)> {amount, amount, amount} = {127, 127, 1} ❷
** (MatchError) no match of right hand side value: {127, 127, 1}
❶ Matches a tuple with three identical elements
❷ Fails because the tuple elements aren’t identical
Occasionally, you’ll need to match against the contents of the variable. For this purpose, the pin operator (^) is provided. This is best explained with an example:
iex(7)> expected_name = "Bob" ❶ "Bob" iex(8)> {^expected_name, _} = {"Bob", 25} ❷ {"Bob", 25} iex(9)> {^expected_name, _} = {"Alice", 30} ❷ ** (MatchError) no match of right hand side value: {"Alice", 30}
❶ Matches anything and then binds to the expected_name variable
❷ Matches to the content of the expected_name variable
Using ^expected_name in patterns indicates you expect the value of the variable expected_name to be in the appropriate position in the right-side term. In this example, it would be the same as if you used the hardcoded pattern ({"Bob", _} = ...). Therefore, the first match succeeds, but the second fails.
Notice that the pin operator doesn’t bind the variable. You expect that the variable is already bound to a value, and you try to match against that value.
List matching works similarly to tuples. The following example decomposes a three-element list:
iex(1)> [first, second, third] = [1, 2, 3] [1, 2, 3]
And of course, the previously mentioned pattern techniques work as well:
[1, second, third] = [1, 2, 3] ❶ [first, first, first] = [1, 1, 1] ❷ [first, second, _ ] = [1, 2, 3] ❸ [^first, second, _ ] = [1, 2, 3] ❹
❶ The first element must be 1.
❷ All elements must have the same value.
❸ You don’t care about the third element, but it must be present.
❹ The first element must have the same value as the variable first.
Matching lists is more often done by relying on their recursive nature. Recall from chapter 2 that each nonempty list is a recursive structure that can be expressed in the form [head | tail]. You can use pattern matching to put each of these two elements into separate variables:
iex(3)> [head | tail] = [1, 2, 3] [1, 2, 3] iex(4)> head 1 iex(5)> tail [2, 3]
If you need only one element of the [head, tail] pair, you can use the anonymous variable. Here’s an inefficient way of calculating the smallest element in the list:
iex(6)> [min | _] = Enum.sort([3,2,1]) iex(7)> min 1
First, you sort the list, and then, with the pattern [min | _], you take only the head of the (sorted) list. Note that this could also be done with the hd function mentioned in chapter 2. In fact, in this case, hd would be more elegant. The pattern [head | _] is more useful when pattern-matching function arguments, as you’ll see in section 3.2.
To match a map, the following syntax can be used:
iex(1)> %{name: name, age: age} = %{name: "Bob", age: 25}
%{age: 25, name: "Bob"}
iex(2)> name
"Bob"
iex(3)> age
25
When matching a map, the left-side pattern doesn’t need to contain all the keys from the right-side term:
iex(4)> %{age: age} = %{name: "Bob", age: 25}
iex(5)> age
25
You may be wondering about the purpose of such a partial-matching rule. Maps are frequently used to represent structured data. In such cases, you’re often interested in only some of the map’s fields. For example, in the previous snippet, you just want to extract the age field, ignoring everything else. The partial-matching rule allows you to do exactly this.
Of course, a match will fail if the pattern contains a key that’s not in the matched term:
iex(6)> %{age: age, works_at: works_at} = %{name: "Bob", age: 25}
** (MatchError) no match of right hand side value
We won’t deal with bitstrings and pure binaries much in this book, but it’s worth mentioning some basic matching syntax. Recall that a bitstring is a chunk of bits, and a binary is a special case of a bitstring that’s always aligned to the byte size.
To match a binary, you use syntax similar to creating one:
iex(1)> binary = <<1, 2, 3>>
<<1, 2, 3>>
iex(2)> <<b1, b2, b3>> = binary ❶
<<1, 2, 3>>
iex(3)> b1
1
iex(4)> b2
2
iex(5)> b3
3
This example matches on a three-byte binary and extracts individual bytes to separate variables.
The following example takes the binary apart by taking its first byte into one variable and the rest of the binary into another:
iex(6)> <<b1, rest :: binary>> = binary <<1, 2, 3>> iex(7)> b1 1 iex(8)> rest <<2, 3>>
rest::binary states that you expect an arbitrarily sized. You can even extract separate bits or groups of bits. The following example splits a single byte into two four-bit values:
iex(9)> <<a :: 4, b :: 4>> = << 155 >> << 155 >> iex(10)> a 9 iex(11)> b 11
Pattern a::4 states that you expect a four-bit value. In the example, you put the first four bits into variable a and the other four bits into variable b. Because the number 155 is represented as 10011011 in binary, you get values of 9 (1001 in binary) and 11 (1011 in binary).
Matching bitstrings and binaries is immensely useful when you’re trying to parse packed binary content that comes from a file, an external device, or a network. In such situations, you can use binary matching to extract separate bits and bytes elegantly.
As mentioned, the examples in this book won’t need this feature. Still, you should take note of binaries and pattern matching, in case the need arises at some point.
Recall that strings are binaries, so you can use binary matches to extract individual bits and bytes from a string:
iex(13)> <<b1, b2, b3>> = "ABC" "ABC" iex(13)> b1 65 iex(14)> b2 66 iex(15)> b3 67
The variables b1, b2, and b3 hold corresponding bytes from the string you matched on. This isn’t very useful, especially if you’re dealing with Unicode strings. Extracting individual characters is better done using functions from the String module.
A more useful pattern is to match the beginning of the string:
iex(16)> command = "ping www.example.com"
"ping www.example.com"
iex(17)> "ping " <> url = command ❶
"ping www.example.com"
iex(18)> url
"www.example.com"
In this example, you construct a string that holds a ping command. When you write "ping " <> url = command, you state the expectation that a command variable is a binary string starting with "ping ". If this matches, the rest of the string is bound to the variable url.
You’ve already seen this, but let’s make it explicit. Patterns can be arbitrarily nested, as in the following contrived example:
iex(1)> [_, {name, _}, _] = [{"Bob", 25}, {"Alice", 30}, {"John", 35}]
In this example, the term being matched is a list of three elements. Each element is a tuple representing a person, consisting of two fields: the person’s name and age. The match extracts the name of the second person in the list.
Another interesting feature is match chaining. Before you see how that works, let’s discuss match expressions in more detail.
A match expression has this general form:
pattern = expression
As you’ve seen in examples, you can place any expression on the right side:
iex(2)> a = 1 + 3 4
Let’s break down what happens here:
The resulting value is matched against the left-side pattern.
The result of the match expression is the result of the right-side term.
An important consequence of this is that match expressions can be chained:
iex(3)> a = (b = 1 + 3) 4
In this (not so useful) example, the following things happen:
Consequently, both a and b have the value 4.
Parentheses are optional, and many developers omit them in this case:
iex(4)> a = b = 1 + 3 4
This yields the same result because the = operator is right-associative.
Now, let’s look at a more useful example. Recall the function :calendar.local_ time/0:
iex(5)> :calendar.local_time()
{{2023, 11, 11}, {21, 28, 41}}
Let’s say you want to retrieve the function’s total result (datetime) as well as the current hour of the day. Here’s the way to do it in a single compound match:
iex(6)> date_time = {_, {hour, _, _}} = :calendar.local_time()
You can even swap the ordering. It still gives the same result (assuming you call it in the same second):
iex(7)> {_, {hour, _, _}} = date_time = :calendar.local_time()
In any case, you get what you wanted:
iex(8)> date_time
{{2023, 11, 11}, {21, 32, 34}}
iex(9)> hour
21
This works because the result of a pattern match is always the result of the term being matched (whatever is on the right side of the match operator). You can successively match against the result of that term and extract different parts you’re interested in.
We’re almost done with basic pattern-matching mechanics. We’ve worked through a lot of examples, so let’s try to formalize the behavior a bit.
The pattern-matching expression consists of two parts: the pattern (left side) and the term (right side). In a match expression, there is an attempt to match the term to the pattern.
If the match succeeds, all variables in the pattern are bound to the corresponding values from the term. The result of the entire expression is the entire term you matched. If the match fails, an error is raised.
Therefore, in a pattern-matching expression, you perform two different tasks:
You assert your expectations about the right-side term. If these expectations aren’t met, an error is raised.
You bind some parts of the term to variables from the pattern.
Finally, it’s worth mentioning that we haven’t covered all possible patterns here. For the detailed reference, you can refer to the official documentation (https://mng.bz/dd6o).
The match operator = is just one example in which pattern matching can be used. Pattern matching powers many other kinds of expressions, and it’s especially powerful when used in functions.
The pattern-matching mechanism is used in the specification of function arguments. Recall the basic function definition:
def my_fun(arg1, arg2) do ... end
The argument specifiers arg1 and arg2 are patterns, and you can use standard matching techniques.
Let’s see this in action. As mentioned in chapter 2, tuples are often used to group related fields together. For example, if you do a geometry manipulation, you can represent a rectangle with a tuple, {a, b}, containing the rectangle’s sides. The following listing shows a function that calculates a rectangle’s area.
Listing 3.1 Pattern matching function arguments (rect.ex)
defmodule Rectangle do
def area({a, b}) do ❶
a * b
end
end
Notice how you pattern-match the argument. The function Rectangle.area/1 expects that its argument is a two-element tuple. It then binds corresponding tuple elements into variables and returns the result.
You can see whether this works from the shell. Start the shell, and then load the module:
$ iex rect.ex
iex(1)> Rectangle.area({2, 3})
6
What happens here? When you call a function, the arguments you provide are matched against the patterns specified in the function definition. The function expects a two-element tuple and binds the tuple’s elements to variables a and b.
When calling functions, the term being matched is the argument provided to the function call. The pattern you match against is the argument specifier—in this case, {a, b}.
Of course, if you provide anything that isn’t a two-element tuple, an error will be raised:
iex(2)> Rectangle.area(2) ** (FunctionClauseError) no function clause matching in Rectangle.area/1
Pattern matching function arguments is an extremely useful tool. It underpins one of the most important features of Elixir: multiclause functions.
Elixir allows you to overload a function by specifying multiple clauses. A clause is a function definition specified by the def expression. If you provide multiple definitions of the same function with the same arity, it’s said that the function has multiple clauses.
Let’s see this in action. Extending the previous example, let’s say you need to develop a Geometry module that can handle various shapes. You’ll represent shapes with tuples and use the first element of each tuple to indicate which shape it represents:
rectangle = {:rectangle, 4, 5}
square = {:square, 5}
circle = {:circle, 4}
Given these shape representations, you can write the following function to calculate a shape’s area.
Listing 3.2 Multiclause function (geometry.ex)
defmodule Geometry do
def area({:rectangle, a, b}) do ❶
a * b
end
def area({:square, a}) do ❷
a * a
end
def area({:circle, r}) do ❸
r * r * 3.14
end
end
As you can see, you provide three clauses of the same function. Depending on which argument you pass, the appropriate clause is called. Let’s try this from the shell:
iex(1)> Geometry.area({:rectangle, 4, 5})
20
iex(2)> Geometry.area({:square, 5})
25
iex(3)> Geometry.area({:circle, 4})
50.24
When you call the function, the runtime goes through each of its clauses, in the order they’re specified in the source code, and attempts to match the provided arguments. The first clause that successfully matches all arguments is executed.
Of course, if no clause matches, an error is raised:
iex(4)> Geometry.area({:triangle, 1, 2, 3})
** (FunctionClauseError) no function clause matching in Geometry.area/1
From the caller’s perspective, a multiclause function is a single function. You can’t directly reference a specific clause. Instead, you always work on the entire function. Recall from chapter 2 that you can create a function value with the capture operator, &:
&Module.fun/arity
If you capture Geometry.area/1, you capture all of its clauses:
iex(4)> fun = &Geometry.area/1 ❶
iex(5)> fun.({:circle, 4})
50.24
iex(6)> fun.({:square, 5})
25
❶ Captures the entire function
This proves that the function is treated as a whole, even if it consists of multiple clauses.
Sometimes, you’ll want a function to return a term indicating a failure, rather than raising an error. You can introduce a default clause that always matches. Let’s do this for the area function. The next listing adds a final clause that handles any invalid input.
Listing 3.3 Multiclause function (geometry_invalid_input.ex)
defmodule Geometry do
def area({:rectangle, a, b}) do
a * b
end
def area({:square, a}) do
a * a
end
def area({:circle, r}) do
r * r * 3.14
end
def area(unknown) do ❶
{:error, {:unknown_shape, unknown}}
end
end
❶ Additional clause that handles invalid input
If none of the first three clauses match, the final clause is called. This is because a variable pattern always matches the corresponding term. In this case, you return a two-element tuple, {:error, reason}, to indicate something has gone wrong.
iex(1)> Geometry.area({:square, 5})
25
iex(2)> Geometry.area({:triangle, 1, 2, 3})
{:error, {:unknown_shape, {:triangle, 1, 2, 3}}}
tip For this to work correctly, it’s important to place the clauses in the appropriate order. The runtime tries to select the clauses, using the order in the source code. If the area(unknown) clause were defined first, you’d always get the error result.
Notice that the area(unknown) clause works only for area/1. If you pass more than one argument, this clause won’t be called. Recall from chapter 2 that functions differ in name and arity. Because functions with the same name but different arities are two different functions, there’s no way to specify an area clause that’s executed, regardless of how many arguments are passed.
One final note: you should always group clauses of the same function together, instead of scattering them in various places in the module. If a multiclause function is spread all over the file, it becomes increasingly difficult to analyze the function’s complete behavior. Even the compiler complains about this by emitting a compilation warning.
Let’s say you want to write a function that accepts a number and returns an atom :negative, :zero, or :positive, depending on the number’s value. This isn’t possible with the simple pattern matching you’ve seen so far. Elixir gives you a solution for this in the form of guards.
Guards are an extension of the basic pattern-matching mechanism. They allow you to state additional broader expectations that must be satisfied for the entire pattern to match.
A guard can be specified by providing the when clause after the arguments list. This is best illustrated via an example. The following code tests whether a given number is positive, negative, or zero.
Listing 3.4 Using guards (test_num.ex)
defmodule TestNum do
def test(x) when x < 0 do
:negative
end
def test(x) when x == 0 do
:zero
end
def test(x) when x > 0 do
:positive
end
end
The guard is a logical expression that adds further conditions to the pattern. In this example, we have three clauses with the same pattern (x), that would normally always match. The additional guard refines the pattern, making sure the clause is invoked only if the given condition is satisfied, as demonstrated in this shell session:
iex(1)> TestNum.test(-1) :negative iex(2)> TestNum.test(0) :zero iex(3)> TestNum.test(1) :positive
Surprisingly enough, calling this function with a nonnumber yields strange results:
iex(4)> TestNum.test(:not_a_number) :positive
What gives? The explanation lies in the fact that Elixir terms can be compared with the operators < and >, even if they’re not of the same type. In this case, the type ordering determines the result:
number < atom < reference < fun < port < pid < tuple < map < list < bitstring (binary)
A number is smaller than any other type, which is why TestNum.test/1 always returns :positive if you provide a nonnumber. To fix this, you must extend the guard by testing whether the argument is a number.
Listing 3.5 Using guards (test_num2.ex)
defmodule TestNum do
def test(x) when is_number(x) and x < 0 do
:negative
end
def test(x) when x == 0 do
:zero
end
def test(x) when is_number(x) and x > 0 do
:positive
end
end
This code uses the function Kernel.is_number/1 to test whether the argument is a number. Now, TestNum.test/1 raises an error if you pass a nonnumber:
iex(1)> TestNum.test(-1) :negative iex(2)> TestNum.test(:not_a_number) ** (FunctionClauseError) no function clause matching in TestNum.test/1
The set of operators and functions that can be called from guards is very limited. In particular, you may not call your own functions, and most of the other functions won’t work. These are some examples of operators and functions allowed in guards:
Boolean operators (and and or) and negation operators (not and !)
Type-check functions from the Kernel module (e.g., is_number/1, is_atom/1, and so on)
You can find the complete, up-to-date list at https://mng.bz/rjVJ.
In some cases, a function used in a guard may cause an error to be raised. For example, length/1 makes sense only on lists. Imagine you have the following function, which calculates the smallest element of a nonempty list:
defmodule ListHelper do
def smallest(list) when length(list) > 0 do
Enum.min(list)
end
def smallest(_), do: {:error, :invalid_argument}
end
You may think that calling ListHelper.smallest/1 with anything other than a list will raise an error, but this won’t happen. If an error is raised from inside the guard, it won’t be propagated, and the guard expression will return false. The corresponding clause won’t match, but some other clause might.
In the preceding example, if you call ListHelper.smallest(123), you’ll get the following result: {:error, :invalid_argument}. This demonstrates that an error in the guard expression is internally handled.
Anonymous functions (lambdas) may also consist of multiple clauses. First, recall the basic way of defining and using lambdas:
iex(1)> double = fn x -> x * 2 end ❶ iex(2)> double.(3) ❷ 6
The general lambda syntax has the following shape:
fn
pattern_1, pattern_2 ->
... ❶
pattern_3, pattern_4 ->
... ❷
...
end
❶ Executed if pattern_1 matches the first argument and pattern_2 matches the second argument
❷ Executed if pattern_3 matches the first argument and pattern_4 matches the second argument
Let’s see this in action by reimplementing the test/1 function that inspects whether a number is positive, negative, or zero:
iex(3)> test_num =
fn
x when is_number(x) and x < 0 -> :negative
x when x == 0 -> :zero
x when is_number(x) and x > 0 -> :positive
end
Notice there’s no special ending terminator for a lambda clause. The clause ends when the new clause is started (in the form pattern ->) or the lambda definition is finished with end.
Note Because all clauses of a lambda are listed under the same fn expression, the parentheses for each clause are omitted by convention. In contrast, each clause of a named function is specified in a separate def (or defp) expression. As a result, parentheses around named function arguments are recommended.
iex(4)> test_num.(-1) :negative iex(5)> test_num.(0) :zero iex(6)> test_num.(1) :positive
Multiclause lambdas come in handy when using higher-order functions, as you’ll see later in this chapter. But for now, we’re done with the basic theory behind multiclause functions. They play an important role in conditional runtime branching, which is our next topic.
Elixir provides some standard ways of doing conditional branching, with expressions such as if and case. Multiclause functions can be used for this purpose as well. In this section, we’ll cover all the branching techniques, starting with multiclause functions.
You’ve already seen how to use conditional logic with multiclauses, but let’s see it once more:
defmodule TestNum do def test(x) when x < 0, do: :negative def test(0), do: :zero def test(x), do: :positive end
The three clauses constitute three conditional branches. In a typical imperative language, such as JavaScript, you could write something like the following:
function test(x){
if (x < 0) return "negative";
if (x == 0) return "zero";
return "positive";
}
Arguably, both versions are equally readable. But with multiclauses, you can reap all the benefits of pattern matching, such as branching, depending on the shape of the data. In the following example, a multiclause is used to test whether a given list is empty:
defmodule TestList do def empty?([]), do: true def empty?([_|_]), do: false end
The first clause matches the empty list, whereas the second clause relies on the head | tail representation of a nonempty list.
By relying on pattern matching, you can implement polymorphic functions that do different things depending on the input type. The following example implements a function that doubles a variable. The function behaves differently depending on whether it’s called with a number or a binary (string):
iex(1)> defmodule Polymorphic do
def double(x) when is_number(x), do: 2 * x
def double(x) when is_binary(x), do: x <> x
end
iex(2)> Polymorphic.double(3)
6
iex(3)> Polymorphic.double("Jar")
"JarJar"
The power of multiclauses becomes evident in recursions. The resulting code seems declarative and is devoid of redundant ifs and returns. Here’s a recursive implementation of a factorial, based on multiclauses:
iex(4)> defmodule Fact do
def fact(0), do: 1
def fact(n), do: n * fact(n - 1)
end
iex(5)> Fact.fact(1)
1
iex(6)> Fact.fact(3)
6
A multiclause-powered recursion is also used as a primary building block for looping. This will be thoroughly explained in the next section, but here’s a simple example. The following function sums all the elements of a list:
iex(7)> defmodule ListHelper do
def sum([]), do: 0
def sum([head | tail]), do: head + sum(tail)
end
iex(8)> ListHelper.sum([])
0
iex(9)> ListHelper.sum([1, 2, 3])
6
The solution implements the sum by relying on the recursive definition of a list. The sum of an empty list is always 0, and the sum of a nonempty list equals the value of its head plus the sum of its tail.
Everything that can be done with classical branching expressions can be accomplished with multiclauses. However, the underlying pattern-matching mechanism can often be more expressive, allowing you to branch depending on values, types, and shapes of function arguments. In some cases, though, the code looks better with the classical, imperative style of branching. Let’s look at the other branching expressions we have in Elixir.
Multiclause solutions may not always be appropriate. Using them requires creating a separate function and passing the necessary arguments. Sometimes, it’s simpler to use a classical branching expression in the function, and for such cases, the expressions if, unless, cond, and case are provided. These work roughly as you might expect, although there are a couple of twists. Let’s look at each of them.
The if expression has a familiar syntax:
if condition do ... else ... end
This causes one or the other branch to execute, depending on the truthiness of the condition. If the condition is anything other than false or nil, you end up in the main branch; otherwise, the else part is called.
You can also condense this into a one-liner, much like a def expression:
if condition, do: something, else: another_thing
Recall that everything in Elixir is an expression that has a return value. The if expression returns the result of the executed block (that is, of the block’s last expression). If the condition isn’t met and the else clause isn’t specified, the return value is the atom nil:
iex(1)> if 5 > 3, do: :one :one iex(2)> if 5 < 3, do: :one nil iex(3)> if 5 < 3, do: :one, else: :two :two
Let’s look at a more concrete example. The following code implements a max function that returns the larger of two elements (according to the semantics of the > operator):
def max(a, b) do if a >= b, do: a, else: b end
The unless expression is also available, which is the equivalent of if not .... Consider the following if expression:
if result != :error, do: send_notification(...)
unless result == :error, do: send_notification(...)
The cond expression can be thought of as equivalent to an if-else-if pattern. It takes a list of expressions and executes the block of the first expression that evaluates to a truthy value:
cond do
expression_1 ->
...
expression_2 ->
...
...
end
The result of cond is the result of the corresponding executed block. If none of the conditions are satisfied, cond raises an error.
The cond expression is a good fit if there are more than two branching choices:
def call_status(call) do
cond do
call.ended_at != nil -> :ended
call.started_at != nil -> :started
true -> :pending ❶
end
end
❶ Equivalent of a default clause
In this example, you’re computing the status of a call. If the ended_at field is populated, the call has ended. Otherwise, if the started_at field is populated, the call has started. If neither of these two fields is populated, the call is pending. Notice the final clause: (true -> :pending). Since the condition of this clause (true) is always satisfied, this effectively becomes the fallback clause that is invoked if none of the previously stated conditions in the cond expression are met.
The general syntax of case is as follows:
case expression do
pattern_1 ->
...
pattern_2 ->
...
...
end
The term pattern here indicates that it deals with pattern matching. In the case expression, the provided expression is evaluated, and then the result is matched against the given clauses. The first one that matches is executed, and the result of the corresponding block (its last expression) is the result of the entire case expression. If no clause matches, an error is raised.
The case-powered version of the max function would then look like this:
def max(a,b) do
case a >= b do
true -> a
false -> b
end
end
The case expression is most suitable if you don’t want to define a separate multiclause function. Other than that, there are no differences between case and multiclause functions. In fact, the general case syntax can be directly translated into the multiclause approach:
defp fun(pattern_1), do: ... defp fun(pattern_2), do: ... ...
This must be called using the fun(expression).
You can specify the default clause by using the anonymous variable to match anything:
case expression do
pattern_1 -> ...
pattern_2 -> ...
...
_ -> ... ❶
end
❶ The default clause that always matches
As you’ve seen, there are different ways of doing conditional logic in Elixir. Multiclauses offer a more declarative feel to branching, but they require you to define a separate function and pass all the necessary arguments to it. Classical expressions, like if and case, seem more imperative but can often prove simpler than the multiclause approach. Selecting an appropriate solution depends on the specific situation as well as your personal preferences.
The final branching expression we’ll discuss is the with expression, which can be very useful when you need to chain a couple of expressions and return the error of the first expression that fails. Let’s look at a simple example.
Suppose you need to process registration data submitted by a user. The input is a map, with keys being strings ("login", "email", and "password"). Here’s an example of one input map:
%{
"login" => "alice",
"email" => "some_email",
"password" => "password",
"other_field" => "some_value",
"yet_another_field" => "...",
...
}
Your task is to normalize this map into a map that contains only the fields login, email, and password. Usually, if the set of fields is well defined and known up front, you can represent the keys as atoms. Therefore, for the given input, you can return the following structure:
%{login: "alice", email: "some_email", password: "password"}
However, some required field might not be present in the input map. In this case, you want to report the error, so your function can have two different outcomes. It can return either the normalized user map or an error. An idiomatic approach in such cases is to make the function return {:ok, some_result} or {:error, error_reason}. In this exercise, the successful result is the normalized user map, whereas the error reason is descriptive text.
Start by writing the helper functions for extracting each field:
defp extract_login(%{"login" => login}), do: {:ok, login}
defp extract_login(_), do: {:error, "login missing"}
defp extract_email(%{"email" => email}), do: {:ok, email}
defp extract_email(_), do: {:error, "email missing"}
defp extract_password(%{"password" => password}), do: {:ok, password}
defp extract_password(_), do: {:error, "password missing"}
Here, you’re relying on pattern matching to detect the field’s presence.
Now, you need to write the top-level extract_user/1 function, which combines these three functions. Here’s one way to do it with case:
def extract_user(user) do
case extract_login(user) do
{:error, reason} ->
{:error, reason}
{:ok, login} ->
case extract_email(user) do
{:error, reason} ->
{:error, reason}
{:ok, email} ->
case extract_password(user) do
{:error, reason} ->
{:error, reason}
{:ok, password} ->
%{login: login, email: email, password: password}
end
end
end
end
This is quite noisy, given that the code composes three functions. Each time you fetch something, you need to branch depending on the result, and you end up with three nested cases. In real life, you usually must perform many more validations, so the code can become quite nasty pretty quickly.
This is precisely where with can help you. The with special form allows you to use pattern matching to chain multiple expressions, verify that the result of each conforms to the desired pattern, and return the first unexpected result.
In its simplest form, with has the following shape:
with pattern_1 <- expression_1,
pattern_2 <- expression_2,
...
do
...
end
You start from the top, evaluating the first expression and matching the result against the corresponding pattern. If the match succeeds, you move to the next expression. If all the expressions are successfully matched, you end up in the do block, and the result of the with expression is the result of the last expression in the do block.
If any match fails, however, with will not proceed to evaluate subsequent expressions. Instead, it will immediately return the result that couldn’t be matched.
iex(1)> with {:ok, login} <- {:ok, "alice"},
{:ok, email} <- {:ok, "some_email"} do
%{login: login, email: email}
end
%{email: "some_email", login: "alice"}
Here, you go through two pattern matches to extract the login and the email. Then, the do block is evaluated. The result of the with expression is the last result of the expression in the do block. Superficially, this is no different from the following:
{:ok, login} = {:ok, "alice"}
{:ok, email} = {:ok, "email"}
%{login: login, email: email}
The benefit of with is that it returns the first term that fails to be matched against the corresponding pattern:
iex(2)> with {:ok, login} <- {:error, "login missing"},
{:ok, email} <- {:ok, "email"} do
%{login: login, email: email}
end
{:error, "login missing"}
In your case, this is precisely what’s needed. Armed with this new knowledge, refactor the top-level extract_user function.
Listing 3.6 with-based user extraction (user_extraction.ex)
def extract_user(user) do
with {:ok, login} <- extract_login(user),
{:ok, email} <- extract_email(user),
{:ok, password} <- extract_password(user) do
{:ok, %{login: login, email: email, password: password}}
end
end
As you can see, this code is much shorter and clearer. You extract desired pieces of data, moving forward only if you succeed. If something fails, you return the first error. Otherwise, you return the normalized structure. The complete implementation can be found in user_extraction.ex. Try it out:
$ iex user_extraction.ex
iex(1)> UserExtraction.extract_user(%{})
{:error, "login missing"}
iex(2)> UserExtraction.extract_user(%{"login" => "some_login"})
{:error, "email missing"}
iex(3)> UserExtraction.extract_user(%{
"login" => "some_login",
"email" => "some_email"
})
{:error, "password missing"}
iex(4)> UserExtraction.extract_user(%{
"login" => "some_login",
"email" => "some_email",
"password" => "some_password"
})
{:ok, %{email: "some_email", login: "some_login",
password: "some_password"}}
The with special form has a couple more features not presented here. I recommend studying it in more detail at https://mng.bz/VRxy.
This concludes our tour of the branching expressions in Elixir. Now, it’s time to look at how you can perform loops and iterations.
Looping in Elixir works very differently than in mainstream languages. Constructs such as while and do...while aren’t provided. Nevertheless, any serious program needs to do some kind of dynamic looping. So how do you go about it in Elixir? The principal looping tool in Elixir is recursion, so next, we’ll take a detailed look at how to use it.
Note Although recursion is the basic building block of any kind of looping, most production Elixir code uses it sparingly. That’s because there are many higher-level abstractions that hide the recursion details. You’ll learn about many of these abstractions throughout the book, but it’s important to understand how recursion works in Elixir because most of the complex code is based on this mechanism.
Note Most of the examples in this section deal with simple problems, such as calculating the sum of all the elements in a list—tasks Elixir allows you to do in an effective and elegant one-liner. The point of the examples, however, is to understand the different aspects of recursion-based processing on simple problems.
Let’s say you want to implement a function that prints the first n natural numbers (positive integers). Because there are no loops, you must rely on recursion. The basic approach is illustrated in the following listing.
Listing 3.7 Printing the first n natural numbers (natural_nums.ex)
defmodule NaturalNums do
def print(1), do: IO.puts(1)
def print(n) do
print(n - 1)
IO.puts(n)
end
end
This code relies on recursion, pattern matching, and multiclause functions. If n is equal to 1, you print the number. Otherwise, you print the first n - 1 numbers and then the nth one.
Trying it in the shell gives satisfying results:
iex(1)> NaturalNums.print(3) 1 2 3
You may have noticed that the function won’t work correctly if you provide a negative integer or a float. This could be resolved with additional guards and is left for you as an exercise.
The code in listing 3.7 demonstrates the basic way of doing a conditional loop. You specify a multiclause function, first providing the clauses that stop the recursion. This is followed by more general clauses that produce part of the result and call the function recursively.
Next, let’s look at computing something in a loop and returning the result. You’ve already seen this example when dealing with conditionals, but let’s repeat it. The following code implements a function that sums all the elements in a given list.
Listing 3.8 Calculating the sum of a list (sum_list.ex)
defmodule ListHelper do
def sum([]), do: 0
def sum([head | tail]) do
head + sum(tail)
end
end
This code looks very declarative:
The sum of all the elements of a nonempty list equals the list’s head plus the sum of the list’s tail.
iex(1)> ListHelper.sum([1, 2, 3]) 6 iex(2)> ListHelper.sum([]) 0
You probably know from other languages that a function call will lead to a stack push and, therefore, will consume some memory. A very deep recursion might lead to a stack overflow and crash the entire program. This isn’t necessarily a problem in Elixir because of the tail-call optimization.
If the last thing a function does is call another function (or itself), you’re dealing with a tail call:
def original_fun(...) do
...
another_fun(...) ❶
end
Elixir (or, more precisely, Erlang) treats tail calls in a specific manner by performing a tail-call optimization. In this case, calling a function doesn’t result in the usual stack push. Instead, something more like a goto or jump statement happens. You don’t allocate additional stack space before calling the function, which, in turn, means the tail function call consumes no additional memory.
How is this possible? In the previous snippet, the last thing done in original_fun is calling another_fun. The final result of original_fun is the result of another_fun. This is why the compiler can safely perform the operation by jumping to the beginning of another_fun without doing additional memory allocation. When another_fun finishes, you return to whatever place original_fun was called from.
Tail calls are especially useful in recursive functions. A tail-recursive function—that is, a function that calls itself at the very end—can run virtually forever without consuming additional memory.
The following function is the Elixir equivalent of an endless loop:
def loop_forever(...) do ... loop_forever(...) end
Because tail recursion doesn’t consume additional memory, it’s an appropriate solution for arbitrarily large iterations.
In the next listing, you’ll convert the ListHelper.sum/1 function to the tail-recursive version.
Listing 3.9 Tail-recursive sum of the first n natural numbers (sum_list_tc.ex)
defmodule ListHelper do
def sum(list) do
do_sum(0, list)
end
defp do_sum(current_sum, []) do
current_sum
end
defp do_sum(current_sum, [head | tail]) do
new_sum = head + current_sum
do_sum(new_sum, tail)
end
end
The first thing to notice is that you have two functions. The exported function sum/1 is called by the module clients, and on the surface, it works just like before.
The recursion takes place in the private do_sum/2 function, which is implemented as tail recursive. This is a two-clause function, and we’ll analyze it clause by clause. The second clause is more interesting, so we’ll start with it. Here it is in isolation:
defp do_sum(current_sum, [head | tail]) do new_sum = head + current_sum ❶ do_sum(new_sum, tail) ❷ end
❶ Computes the new value of the sum
This clause expects two arguments: the nonempty list to operate on and the sum you’ve calculated so far (current_sum). It then calculates the new sum and calls itself recursively with the remainder of the list and the new sum. Because the call happens at the very end, the function is tail recursive, and the call consumes no additional memory.
The variable new_sum is introduced here just to make things more obvious. You could also inline the computation:
defp do_sum(current_sum, [head | tail]) do do_sum(head + current_sum, tail) end
This function is still tail recursive because it calls itself at the very end.
The final thing to examine is the first clause of do_sum/2:
defp do_sum(current_sum, []) do current_sum end
This clause is responsible for stopping the recursion. It matches on an empty list, which is the last step of the iteration. When you get here, there’s nothing else to sum, so you return the accumulated result.
Finally, you have the function sum/1:
def sum(list) do do_sum(0, list) end
This function is used by clients and is also responsible for initializing the value of the current_sum parameter that’s passed recursively in do_sum.
You can think of tail recursion as a direct equivalent of a classical loop in imperative languages. The parameter current_sum is a classical accumulator: the value for which you incrementally add the result in each iteration step. The do_sum/2 function implements the iteration step and passes the accumulator from one step to the next. Elixir is an immutable language, so you need this trick to maintain the accumulated value throughout the loop. The first clause of do_sum/2 defines the ending point of the iteration and returns the accumulator value.
In any case, the tail-recursive version of the list sum is now working, so you can try it from the shell:
iex(1)> ListHelper.sum([1, 2, 3]) 6 iex(2)> ListHelper.sum([]) 0
As you can see, from the caller’s point of view, the function works exactly the same way. Internally, you rely on the tail recursion and can, therefore, process arbitrarily large lists without requiring extra memory for this task.
Tail calls can take different shapes. You’ve seen the most obvious case, but there are a couple of others. A tail call can also happen in a conditional expression:
def fun(...) do
...
if something do
...
another_fun(...) ❶
end
end
The call to another_fun is a tail call because it’s the last thing the function does. The same rule holds for unless, cond, case, and with expressions.
But the following code isn’t a tail call:
def fun(...) do
1 + another_fun(...) ❶
end
This is because the call to another_fun isn’t the last thing done in the fun function. After another_fun finishes, you must increment its result by 1 to compute the final result of fun.
All this may seem complicated, but it’s not too hard. If you’re coming from imperative languages, it’s probably not what you’re used to, and it will take some time to get accustomed to the recursive way of thinking combined with the pattern-matching facility. You may want to take some time to experiment with recursion yourself. Here are a couple of functions you can write for practice:
A range/2 function that takes two integers, from and to, and returns a list of all integer numbers in the given range
A positive/1 function that takes a list and returns another list that contains only the positive numbers from the input list
Try to write these functions first in the non-tail-recursive form, and then convert them to the tail-recursive version. If you get stuck, the solutions are provided in the recursion_practice.ex and recursion_practice_tc.ex files (for the tail-recursive versions).
Recursion is the basic looping technique, and no loop can be done without it. Still, you won’t need to write explicit recursion all that often. Many typical tasks can be performed using higher-order functions.
A higher-order function is a type of function that takes one or more functions as its input or returns one or more functions (or both). The word function here refers to the function value.
You already made first contact with higher-order functions in chapter 2, when you used Enum.each/2 to iterate through a list and print all of its elements. Let’s recall how to do this:
iex(1)> Enum.each(
[1, 2, 3],
fn x -> IO.puts(x) end ❶
)
1
2
3
❶ Passing a function value to another function
The function Enum.each/2 takes an enumerable (in this case, a list) and a lambda. It iterates through the enumerable, calling the lambda for each of its elements. Because Enum.each/2 takes a lambda as its input, it’s called a higher-order function.
You can use Enum.each/2 to iterate over enumerable structures without writing the recursion. Under the hood, Enum.each/2 is powered by recursion; there’s no other way to do loops and iterations in Elixir. However, the complexity of writing the recursion, repetitive code, and intricacies of tail recursion is hidden from you.
Enum.each/2 is just one example of an iteration powered by a higher-order function. Elixir’s standard library provides many other useful iteration helpers in the Enum module. You should spend some time researching the module documentation (https://hexdocs.pm/elixir/Enum.xhtml). Here, we’ll look at some of the most frequently used Enum functions.
One manipulation you’ll often need is a one-to-one transformation of a list to another list. This is why Enum.map/2 is provided. It takes an enumerable and a lambda that maps each element to another element. The following example doubles every element in the list:
iex(1)> Enum.map(
[1, 2, 3],
fn x -> 2 * x end
)
[2, 4, 6]
Recall from chapter 2 that you can use the capture operator, &, to make the lambda definition a bit denser:
iex(2)> Enum.map(
[1, 2, 3],
&(2 * &1)
)
The &(...) denotes a simplified lambda definition, where you use &n as a placeholder for the nth argument of the lambda.
Another useful function is Enum.filter/2, which can be used to extract only some elements of the list, based on certain criteria. The following snippet returns all odd numbers from a list:
iex(3)> Enum.filter(
[1, 2, 3],
fn x -> rem(x, 2) == 1 end
)
[1, 3]
Enum.filter/2 takes an enumerable and a lambda. It returns only those elements for which the lambda returns true.
Of course, you can use the capture syntax as well:
iex(3)> Enum.filter(
[1, 2, 3],
&(rem(&1, 2) == 1)
)
[1, 3]
Let’s play a bit more with Enum. Recall the example from section 3.3.3, where you used with to verify that the login, email, and password are submitted. In that example, you returned the first encountered error. Armed with this new knowledge, you can improve that code to report all missing fields immediately.
To briefly recap, your input is a map, and you need to fetch the keys "login", "email", and "password" and then convert them into a map in which keys are atoms. If a required field isn’t provided, you need to report an error. In the previous version, you simply reported the first missing field. A better user experience would be to return a list of all missing fields.
This is something you can do easily with the help of Enum.filter/2. The idea is to iterate through the list of required fields and take only those fields that aren’t present in the map. You can easily check for the presence of a key with the help of Map.has_key?/2. The sketch of the solution then looks like the next listing.
Listing 3.10 Reporting all missing fields (user_extraction_2.ex)
case Enum.filter( ❶ ["login", "email", "password"], ❶ &(not Map.has_key?(user, &1)) ❷ ) do [] -> ❸ ... missing_fields -> ❹ ... end
There are two possible outcomes of Enum.filter/2. If the result is an empty list, all the fields are provided, and you can extract the data. Otherwise, some fields are missing, and you need to report an error. The code for each branch is omitted here for the sake of brevity, but you can find the complete solution in user_extraction_2.ex.
The most versatile function from the Enum module is likely Enum.reduce/3, which can be used to transform an enumerable into anything. If you’re coming from languages that support first-class functions, you may already know reduce under the name inject or fold.
Reducing is best explained with an example. You’ll use reduce to sum all the elements in a list. Before doing it in Elixir, let’s see how you could do this task in an imperative manner. Here’s an imperative JavaScript example:
var sum = 0; ❶ [1, 2, 3].forEach(function(element) { sum += element; ❷ })
This is a standard imperative pattern. You initialize an accumulator (the variable sum) and then do some looping, adjusting the accumulator value in each step. After the loop is finished, the accumulator holds the final value.
In a functional language, you can’t change the accumulator, but you can still calculate the result incrementally by using Enum.reduce/3. The function has the following shape:
Enum.reduce(
enumerable,
initial_acc,
fn element, acc ->
...
end
)
Enum.reduce/3 takes an enumerable as its first argument. The second argument is the initial value for the accumulator—what you compute incrementally. The final argument is a lambda that’s called for each element. The lambda receives the element from the enumerable and the current accumulator value. The lambda’s task is to compute and return the new accumulator value. When the iteration is done, Enum.reduce/3 returns the final accumulator value.
Let’s use Enum.reduce/3 to sum up elements in the list:
iex(4)> Enum.reduce(
[1, 2, 3],
0, ❶
fn element, sum -> sum + element end ❷
)
6
❶ Sets the initial accumulator value
❷ Incrementally updates the accumulator
That’s all there is to it! Coming from an imperative background myself, it helps me to think of the lambda as the function that’s called in each iteration step. Its task is to add a bit of the information to the result.
You may recall I mentioned that many operators are functions, and you can turn an operator into a lambda by calling &+/2, &*/2, and so on. This combines nicely with higher-order functions. For example, the sum example can be written in a more compact form:
iex(5)> Enum.reduce([1,2,3], 0, &+/2) 6
It’s worth mentioning that there’s a function, called Enum.sum/1, that works exactly like this snippet. The point of the sum example was to illustrate how to iterate through a collection and accumulate the result.
Let’s work a bit more with reduce. The previous example works only if you pass a list that consists exclusively of numbers. If the list contains anything else, an error is raised (because the + operator is defined only for numbers). The next example can work on any type of list and sums only its numeric elements:
iex(6)> Enum.reduce(
[1, "not a number", 2, :x, 3],
0,
fn ❶
element, sum when is_number(element) -> ❷
sum + element
_, sum ->
sum ❸
end
)
This example relies on a multiclause lambda to obtain the desired result. If the element is a number, you add its value to the accumulated sum. Otherwise, you return whatever sum you have at the moment, effectively passing it unchanged to the next iteration step.
Personally, I tend to avoid writing elaborate lambdas. If there’s a bit more logic in the anonymous function, it’s a sign that it will probably look better as a distinct function. In the following snippet, the lambda code is pushed to a separate private function:
defmodule NumHelper do
def sum_nums(enumerable) do
Enum.reduce(enumerable, 0, &add_num/2) ❶
end
defp add_num(num, sum) when is_number(num), do: sum + num ❷
defp add_num(_, sum), do: sum
end
❶ Captures the add_num/2 to lambda
This is more or less similar to the approach you saw earlier. This example moves the iteration step to the separate, private function add_num/2. When calling Enum.reduce, you pass the lambda that delegates to that function, using the capture operator &.
Notice how when capturing the function, you don’t specify the module name. That’s because add_num/2 resides in the same module, so you can omit the module prefix. In fact, because add_num/2 is private, you can’t capture it with the module prefix.
This concludes our basic showcase of the Enum module. Be sure to review the other available functions because you’ll find many useful helpers for simplifying loops, iterations, and manipulations of enumerables.
The cryptic comprehensions name denotes another expression that can help you iterate and transform enumerables. The following example uses a comprehension to square each element of a list:
iex(1)> for x <- [1, 2, 3] do
x*x
end
The comprehension iterates through each element and runs the do...end block. The result is a list that contains all the results returned by the do...end block. In this basic form, for is no different from Enum.map/2.
Comprehensions have several other features, which often make them elegant compared to Enum-based iterations. For example, it’s possible to perform nested iterations over multiple collections. The following example takes advantage of this feature to calculate a small multiplication table:
iex(2)> for x <- [1, 2, 3], y <- [1, 2, 3], do: {x, y, x*y}
[
{1, 1, 1}, {1, 2, 2}, {1, 3, 3},
{2, 1, 2}, {2, 2, 4}, {2, 3, 6},
{3, 1, 3}, {3, 2, 6}, {3, 3, 9}
]
In this example, the comprehension performs a nested iteration, calling the provided block for each combination of input collections.
Just like functions from the Enum module, comprehensions can iterate through anything that’s enumerable. For example, you can use ranges to compute a multiplication table for single-digit numbers:
iex(3)> for x <- 1..9, y <- 1..9, do: {x, y, x*y}
In the examples so far, the result of the comprehension has been a list, but comprehensions can return anything that’s collectable. Collectable is an abstract term for a functional data type that can collect values. Some examples include lists, maps, MapSet, and file streams; you can even make your own custom type collectable (more on that in chapter 4).
In more general terms, a comprehension iterates through enumerables, calling the provided block for each value and storing the results in some collectable structure. Let’s see this in action.
The following snippet makes a map that holds a multiplication table. Its keys are tuples of factors {x,y}, and the values contain products:
iex(4)> multiplication_table =
for x <- 1..9,
y <- 1..9,
into: %{} do ❶
{{x, y}, x*y}
end
iex(5)> Map.get(multiplication_table, {7, 6})
42
Notice the into option, which specifies what to collect. In this case, it’s an empty map %{} that will be populated with values returned from the do block. Notice how you return a {factors, product} tuple from the do block. You use this format because the map “knows” how to interpret it. The first element will be used as a key, and the second will be used as the corresponding value.
Another interesting comprehension feature is that you can specify filters. This enables you to skip some elements from the input. The following example computes a nonsymmetrical multiplication table for numbers x and y, where x is never greater than y:
iex(6)> multiplication_table =
for x <- 1..9,
y <- 1..9,
x <= y, ❶
into: %{} do
{{x, y}, x*y}
end
iex(7)> Map.get(multiplication_table, {6, 7})
42
iex(8)> Map.get(multiplication_table, {7, 6})
nil
The comprehension filter is evaluated for each element of the input enumerable, prior to block execution. If the filter returns true, the block is called and the result is collected. Otherwise, the comprehension moves on to the next element.
As you can see, comprehensions are an interesting feature, allowing you to do some elegant transformations of the input enumerable. Although this can be done with Enum functions, most notably Enum.reduce/3, the resulting code is often more elegant when comprehensions are used. This is particularly true when you must perform a Cartesian product (cross join) of multiple enumerables or traverse a nested collection to produce a flat result.
Note Comprehensions can also iterate through a binary. The syntax is somewhat different, and we won’t treat it here. For more details, it’s best to look at the official for documentation at https://mng.bz/xj2d.
A stream is a special kind of enumerable that can be useful for doing lazy composable operations over anything enumerable. To see what this means, let’s look at one shortcoming of standard Enum functions.
Let’s say you have a list of employees and need to print each one, prefixed by their position in the list:
1. Alice 2. Bob 3. John ...
This is fairly simple to perform by combining various Enum functions. For example, there’s a function, Enum.with_index/1, that takes an enumerable and returns a list of tuples, where the first element of the tuple is a member from the input enumerable and the second element is its zero-based index:
iex(1)> employees = ["Alice", "Bob", "John"]
["Alice", "Bob", "John"]
iex(2)> Enum.with_index(employees)
[{"Alice", 0}, {"Bob", 1}, {"John", 2}]
You can now feed the result of Enum.with_index/1 to Enum.each/2 to get the desired output:
iex(3)> employees
|> Enum.with_index()
|> Enum.each(fn {employee, index} ->
IO.puts("#{index + 1}. #{employee}")
end)
1. Alice
2. Bob
3. John
Here, you rely on the pipe operator to chain together various function calls. This saves you from having to use intermediate variables and makes the code a bit cleaner.
So what’s the problem with this code? The Enum.with_index/1 function goes through the entire list to produce another list with tuples, and then Enum.each performs another iteration through the new list. It would be better if you could do both operations in a single pass without building another list. This is where streams can help.
Streams are implemented in the Stream module (https://hexdocs.pm/elixir/Stream.xhtml), which, at first glance, looks similar to the Enum module, containing functions like map, filter, and take. These functions take any enumerable as an input and give back a stream: an enumerable with some special powers.
A stream is a lazy enumerable, which means it produces the actual result on demand. Let’s look at what this means.
The following snippet uses a stream to double each element in a list:
iex(4)> stream = Stream.map([1, 2, 3], fn x -> 2 * x end) ❶ #Stream<[enum: [1, 2, 3], ❷ funs: [#Function<44.45151713/1 in Stream.map/2>]]> ❷
❷ The result of Stream.map/2 is a stream.
Because a stream is a lazy enumerable, the iteration over the input list ([1, 2, 3]) and the corresponding transformation (multiplication by 2) haven’t yet happened. Instead, you get the structure that describes the computation.
To make the iteration happen, you need to pass the stream to an Enum function, such as each, map, or filter. You can also use the Enum.to_list/1 function, which converts any kind of enumerable into a list:
iex(5)> Enum.to_list(stream) ❶
[2, 4, 6]
❶ At this point, stream iteration takes place.
Enum.to_list/1 (and any other Enum function, for that matter) is an eager operation. It immediately starts iterating through the input and creates the result. In doing so, Enum.to_list/1 requests the input enumerable to start producing values. This is why the output of the stream is created when you send it to an Enum function.
The laziness of streams goes beyond iterating the list on demand. Values are produced one by one when Enum.to_list requests another element. For example, you can use Enum.take/2 to request only one element from the stream:
iex(6)> Enum.take(stream, 1) [2]
Because Enum.take/2 iterates only until it collects the desired number of elements, the input stream doubles only one element in the list. The others are never visited.
Returning to the example of printing employees, using a stream allows you to print employees in a single go. The change to the original code is simple enough. Instead of using Enum.with_index/1, you can rely on its lazy equivalent, Stream.with_index/1:
iex(7)> employees
|> Stream.with_index() ❶
|> Enum.each(fn {employee, index} ->
IO.puts("#{index + 1}. #{employee}")
end)
1. Alice
2. Bob
3. John
❶ Performs a lazy transformation
The output is the same, but the list iteration is done only once. This becomes increasingly useful when you need to compose several transformations of the same list. The following example takes the input list and only prints the square root of elements representing a nonnegative number, adding an indexed prefix at the beginning:
iex(1)> [9, -1, "foo", 25, 49]
|> Stream.filter(&(is_number(&1) and &1 > 0))
|> Stream.map(&{&1, :math.sqrt(&1)})
|> Stream.with_index()
|> Enum.each(fn {{input, result}, index} ->
IO.puts("#{index + 1}. sqrt(#{input}) = #{result}")
end)
1. sqrt(9) = 3.0
2. sqrt(25) = 5.0
3. sqrt(49) = 7.0
This code is dense, and it illustrates how concise you can be by relying only on functions as the abstraction tool. You start with the input list and filter only positive numbers. Next, you transform each such number into an {input_number, square_root} tuple. Then, you index the resulting tuples using Stream.with_index/1. Finally, you print the result.
Even though you stack multiple transformations, everything is performed in a single pass when you call Enum.each. In contrast, if you used Enum functions everywhere, you’d need to run multiple iterations over each intermediate list, which would incur a memory-usage penalty.
This lazy property of streams can prove useful for consuming a slow and potentially large enumerable input. A typical case is when you need to parse each line of a file. Relying on eager Enum functions means you must read the entire file into memory and then iterate through each line. In contrast, using streams makes it possible to read and immediately parse one line at a time. For example, the following function takes a filename and returns the list of all lines from that file that are longer than 80 characters:
def large_lines!(path) do File.stream!(path) |> Stream.map(&String.trim_trailing(&1, "\n")) |> Enum.filter(&(String.length(&1) > 80)) end
Here, you rely on the File.stream!/1 function, which takes the path of a file and returns a stream of its lines. Because the result is a stream, the iteration through the file happens only when you request it. After File.stream! returns, no byte from the file has been read yet. Then, you remove the trailing newline character from each line, again in a lazy manner. Finally, you eagerly take only long lines, using Enum.filter/2. This is when the iteration happens. The consequence is that you never read the entire file in memory; instead, you work on each line individually.
Note There are no special tricks in the Elixir compiler that allow these lazy enumerations. The real implementation is fairly involved, but the basic idea behind streams is simple and relies on anonymous functions. In a nutshell, to make a lazy computation, you need to return a lambda that performs the computation. This makes the computation lazy because you return its description rather than its result. When the computation needs to be materialized, the consumer code can call the lambda.
So far, you’ve produced streams by transforming an existing collection with functions such as Stream.map or Stream.filter. Some functions from the Stream module allow you to create a stream from scratch.
One such function is Stream.iterate/2, which can be used to produce an infinite collection, where each element is calculated based on the previous one. For example, the following snippet builds an infinite stream of natural numbers:
iex(1)> natural_numbers = Stream.iterate(
1,
fn previous -> previous + 1 end
)
We can feed this infinite collection to other Enum and Stream functions to produce a finite sequence. For example, to take the first 10 natural numbers, you can use Enum.take/2:
iex(2)> Enum.take(natural_numbers, 10) [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
Another example is the function Stream.repeatedly/1, which repeatedly invokes the provided lambda to generate an element. In the following example, we’ll use it to repeatedly read the user’s input from the console, stopping when the user submits a blank input:
iex(3)> Stream.repeatedly(fn -> IO.gets("> ") end)
|> Stream.map(&String.trim_trailing(&1, "\n"))
|> Enum.take_while(&(&1 != ""))
> Hello
> World
>
["Hello", "World"]
The Stream module contains a few more functions that produce an infinite stream, such as Stream.unfold/2 or Stream.resource/3. Take a look at the official documentation, discussed earlier in the section, for details.
This style of coding takes some getting used to. You’ll use the techniques presented here throughout the book, but you should try to write a couple such iterations yourself. The following are some exercise ideas that may help you get into the swing of things.
Using large_lines!/1 as a model, write the following functions:
A lines_lengths!/1 that takes a file path and returns a list of numbers, with each number representing the length of the corresponding line from the file.
A longest_line_length!/1 that returns the length of the longest line in a file.
A longest_line!/1 that returns the contents of the longest line in a file.
A words_per_line!/1 that returns a list of numbers, with each number representing the word count in a file. Hint: To find the word count of a line, use length(String.split(line)).
Solutions are provided in the enum_streams_practice.ex file, but I strongly suggest you spend some time trying to crack these problems yourself.
Pattern matching is a mechanism that attempts to match a term on the right side to the pattern on the left side. In the process, variables from the pattern are bound to corresponding subterms from the term. If a term doesn’t match the pattern, an error is raised.
Function arguments are patterns. The aim of calling a function is to match the provided values to the patterns specified in the function definition.
Functions can have multiple clauses. The first clause that matches all the arguments is executed.
For conditional branching, you can use multiclause functions and expressions such as if, unless, cond, case, and with.
Recursion is the main tool for implementing loops. Tail recursion is used when you need to run an arbitrarily long loop.
Higher-order functions make writing loops much easier. There are many useful generic iteration functions in the Enum module. Additionally, the Stream module makes it possible to implement lazy and composable iterations.
Comprehensions can also be used to iterate, transform, filter, and join various enumerables.