Time complexity of ‹extend›
Introduction
Each year there is a lot of confusion regarding time complexity of the extend
operation on the lists in Python. I will introduce two specific examples from previous year and also will try to explain it on one of the possible implementations of extend
operation.
Technicalities
At the beginning we should clear some of the “myths” regarding extending of the lists. There is a common misunderstanding regarding differences between a += b
, a.extend(b)
and a + b
.
-
a.extend(b)
- adds all elements fromb
toa
.Time complexity: , where denotes the length of
b
. -
a += b
- equivalent toa.extend(b)
-
a + b
- constructs a new list that contains elements froma
followed by elements fromb
.Time complexity: , where denote the length of
a
andb
respectively.Space complexity: , where denote the length of
a
andb
respectively, since we construct new list.
Example #1
Let us assume function that uses divide & conquer strategy to return indices at which we can find specific element in any list.
def recursive_find_in_list(
values: List[Any], key: Any, lower: int, upper: int
) -> List[int]:
if lower == upper:
return [lower] if values[lower] == key else []
indices = []
mid = (lower + upper) // 2
indices.extend(recursive_find_in_list(values, key, lower, mid))
indices.extend(recursive_find_in_list(values, key, mid + 1, upper))
return indices
def find_in_list(values: List[Any], key: Any) -> List[int]:
return recursive_find_in_list(values, key, 0, len(values) - 1)
This implementation works nicely, extend
is linear (with the respect to the length of the list that is being appended).
Let us try to dissect the way this function works on some specific input (that will be pushed to the extreme, just in case ;)
find_in_list([1] * 5000, 1)
. What shall be the result of this? Since we have key = 1
and the list contains only 1
s, we should get list of all indices.
If we were to draw a tree of call hierarchy of recursive_find_in_list
, we would notice that in sum it is still linear to the length. However we use extend
!
In the leaves of the tree we return lists of length 1. In this case it means calling extend
5000-times at the second-to-last level of the tree on the 1-element long lists, next level 2500 calls on 2-elements long lists, next one 1250 on 4-elements long lists, etc. At the top-level we get 2 calls on 5000/2-element long lists.
A lot of extend
calls, right? And the lengths of the lists are growing (in this example, second call happens on 2500-elements long lists).
Because of the extend
in each level of the tree (call hierarchy) we traverse all of the elements. That means:
because we have levels in the tree and elements at each level.
Example #2
As we could observe in the example above, extend
iterates over all of the elements that it adds. In case of recursive calls, it results in iterating over the same elements multiple times.
Consider constructing of this list:
Let us assume that you extend the result with the list that you get from the recursive call.
-
B iterates through 1, 2 and 3; returns
[1, 2, 3]
-
C iterates through 4, 5 and 6; returns
[4, 5, 6]
-
D iterates through 7, 8 and 9; returns
[7, 8, 9]
-
now we return those lists to the calls from A), so each of the
extend
calls iterates through:- 1, 2, 3 that was returned from B
- 4, 5, 6 that was returned from C
- 7, 8, 9 that was returned from D
and returns
[1, 2, 3, 4, 5, 6, 7, 8, 9]
If the recursion had bigger depth and/or more elements, it would iterate through them more than twice, therefore it does not take constant time to do nor some constant multiple of the input, since it traverses all of the elements in each of the levels.
Implementation of extend
There is an example of dynamic array:
For the sake of Algorithms and Data Structures I we consider APPEND
operation, i.e. adding the element to the end of the list, to have time complexity (amortized; which is out of the scope of IB002).
If we have a look at the extend
implementation in this dynamic array example:
void dynamic_array_extend(struct dynamic_array_t *arr, struct dynamic_array_t *src)
{
if (arr == NULL || src == NULL)
{
return;
}
for (size_t i = 0; i < src->count; i++)
{
dynamic_array_push_back(arr, dynamic_array_at(src, i));
}
}
Apart from checking edge cases, we can notice that we run for
-loop over the elements from the other array and add them one-by-one to the arr
. Time complexity of this operation is time dependant on the src
array.
In this specific implementation, you could also resize the memory allocated for the array in one go and copy whole src
array in one go. However even if you did so, it would be still dependant on the size of the src
array. Cause you still need to copy bytes. From that we can assume that for specific instance of array the is fixed, therefore we consider it a constant. That way we are getting as a time complexity of our extend
operation.