<p>Each year there is a lot of confusion regarding time complexity of the <code>extend</code> operation on the lists in Python. I will introduce two specific examples from previous year and also will try to explain it on one of the possible implementations of <code>extend</code> operation.</p>
<h2class="anchor anchorWithStickyNavbar_LWe7"id="technicalities">Technicalities<ahref="#technicalities"class="hash-link"aria-label="Direct link to Technicalities"title="Direct link to Technicalities"></a></h2>
<p>At the beginning we should clear some of the “myths” regarding extending of the lists. There is a common misunderstanding regarding differences between <code>a += b</code>, <code>a.extend(b)</code> and <code>a + b</code>.</p>
<ul>
<li>
<p><code>a.extend(b)</code> - adds all elements from <code>b</code> to <code>a</code>.</p>
<p>Time complexity: <spanclass="katex"><spanclass="katex-mathml"><mathxmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mimathvariant="script">O</mi><mostretchy="false">(</mo><mi>n</mi><mostretchy="false">)</mo></mrow><annotationencoding="application/x-tex">\mathcal{O}(n)</annotation></semantics></math></span><spanclass="katex-html"aria-hidden="true"><spanclass="base"><spanclass="strut"style="height:1em;vertical-align:-0.25em"></span><spanclass="mord mathcal"style="margin-right:0.02778em">O</span><spanclass="mopen">(</span><spanclass="mord mathnormal">n</span><spanclass="mclose">)</span></span></span></span>, where <spanclass="katex"><spanclass="katex-mathml"><mathxmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>n</mi></mrow><annotationencoding="application/x-tex">n</annotation></semantics></math></span><spanclass="katex-html"aria-hidden="true"><spanclass="base"><spanclass="strut"style="height:0.4306em"></span><spanclass="mord mathnormal">n</span></span></span></span> denotes the length of <code>b</code>.</p>
</li>
<li>
<p><code>a += b</code> - equivalent to <code>a.extend(b)</code></p>
</li>
<li>
<p><code>a + b</code> - constructs a new list that contains elements from <code>a</code> followed by
elements from <code>b</code>.</p>
<p>Time complexity: <spanclass="katex"><spanclass="katex-mathml"><mathxmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mimathvariant="script">O</mi><mostretchy="false">(</mo><mi>m</mi><mo>+</mo><mi>n</mi><mostretchy="false">)</mo></mrow><annotationencoding="application/x-tex">\mathcal{O}(m + n)</annotation></semantics></math></span><spanclass="katex-html"aria-hidden="true"><spanclass="base"><spanclass="strut"style="height:1em;vertical-align:-0.25em"></span><spanclass="mord mathcal"style="margin-right:0.02778em">O</span><spanclass="mopen">(</span><spanclass="mord mathnormal">m</span><spanclass="mspace"style="margin-right:0.2222em"></span><spanclass="mbin">+</span><spanclass="mspace"style="margin-right:0.2222em"></span></span><spanclass="base"><spanclass="strut"style="height:1em;vertical-align:-0.25em"></span><spanclass="mord mathnormal">n</span><spanclass="mclose">)</span></span></span></span>, where <spanclass="katex"><spanclass="katex-mathml"><mathxmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>m</mi><moseparator="true">,</mo><mi>n</mi></mrow><annotationencoding="application/x-tex">m, n</annotation></semantics></math></span><spanclass="katex-html"aria-hidden="true"><spanclass="base"><spanclass="strut"style="height:0.625em;vertical-align:-0.1944em"></span><spanclass="mord mathnormal">m</span><spanclass="mpunct">,</span><spanclass="mspace"style="margin-right:0.1667em"></span><spanclass="mord mathnormal">n</span></span></span></span> denote the length of
<code>a</code> and <code>b</code> respectively.</p>
<p>Space complexity: <spanclass="katex"><spanclass="katex-mathml"><mathxmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mimathvariant="script">O</mi><mostretchy="false">(</mo><mi>m</mi><mo>+</mo><mi>n</mi><mostretchy="false">)</mo></mrow><annotationencoding="application/x-tex">\mathcal{O}(m + n)</annotation></semantics></math></span><spanclass="katex-html"aria-hidden="true"><spanclass="base"><spanclass="strut"style="height:1em;vertical-align:-0.25em"></span><spanclass="mord mathcal"style="margin-right:0.02778em">O</span><spanclass="mopen">(</span><spanclass="mord mathnormal">m</span><spanclass="mspace"style="margin-right:0.2222em"></span><spanclass="mbin">+</span><spanclass="mspace"style="margin-right:0.2222em"></span></span><spanclass="base"><spanclass="strut"style="height:1em;vertical-align:-0.25em"></span><spanclass="mord mathnormal">n</span><spanclass="mclose">)</span></span></span></span>, where <spanclass="katex"><spanclass="katex-mathml"><mathxmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>m</mi><moseparator="true">,</mo><mi>n</mi></mrow><annotationencoding="application/x-tex">m, n</annotation></semantics></math></span><spanclass="katex-html"aria-hidden="true"><spanclass="base"><spanclass="strut"style="height:0.625em;vertical-align:-0.1944em"></span><spanclass="mord mathnormal">m</span><spanclass="mpunct">,</span><spanclass="mspace"style="margin-right:0.1667em"></span><spanclass="mord mathnormal">n</span></span></span></span> denote the length of
<code>a</code> and <code>b</code> respectively, since we construct new list.</p>
</li>
</ul>
<h2class="anchor anchorWithStickyNavbar_LWe7"id="example-1">Example #1<ahref="#example-1"class="hash-link"aria-label="Direct link to Example #1"title="Direct link to Example #1"></a></h2>
<p>Let us assume function that uses divide & conquer strategy to return indices at which we can find specific element in any list.</p>
<p>This implementation works nicely, <code>extend</code> is linear (with the respect to the length of the list that is being appended).</p>
<p>Let us try to dissect the way this function works on some specific input (that will be pushed to the extreme, <em>just in case</em> ;)</p>
<p><code>find_in_list([1] * 5000, 1)</code>. What shall be the result of this? Since we have <code>key = 1</code> and the list contains only <code>1</code>s, we should get list of <strong>all</strong> indices.</p>
<p>If we were to draw a tree of call hierarchy of <code>recursive_find_in_list</code>, we would notice that in sum it is still linear to the length. <strong>However we use <code>extend</code>!</strong></p>
<p>In the leaves of the tree we return lists of length 1. In this case it means calling <code>extend</code> 5000-times at the second-to-last level of the tree on the 1-element long lists, next level 2500 calls on 2-elements long lists, next one 1250 on 4-elements long lists, etc. At the top-level we get 2 calls on 5000/2-element long lists.</p>
<p>A lot of <code>extend</code> calls, right? And the lengths of the lists are growing (in this example, second call happens on 2500-elements long lists).</p>
<p>Because of the <code>extend</code> in each level of the tree (call hierarchy) we traverse all of the elements. That means:</p>
<p>because we have <spanclass="katex"><spanclass="katex-mathml"><mathxmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>log</mi><mo></mo><mi>n</mi></mrow><annotationencoding="application/x-tex">\log n</annotation></semantics></math></span><spanclass="katex-html"aria-hidden="true"><spanclass="base"><spanclass="strut"style="height:0.8889em;vertical-align:-0.1944em"></span><spanclass="mop">lo<spanstyle="margin-right:0.01389em">g</span></span><spanclass="mspace"style="margin-right:0.1667em"></span><spanclass="mord mathnormal">n</span></span></span></span> levels in the tree and <spanclass="katex"><spanclass="katex-mathml"><mathxmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>n</mi></mrow><annotationencoding="application/x-tex">n</annotation></semantics></math></span><spanclass="katex-html"aria-hidden="true"><spanclass="base"><spanclass="strut"style="height:0.4306em"></span><spanclass="mord mathnormal">n</span></span></span></span> elements at each level.</p>
<h2class="anchor anchorWithStickyNavbar_LWe7"id="example-2">Example #2<ahref="#example-2"class="hash-link"aria-label="Direct link to Example #2"title="Direct link to Example #2"></a></h2>
<p>As we could observe in the example above, <code>extend</code> iterates over all of the elements that it adds. In case of recursive calls, it results in iterating over the same elements multiple times.</p>
<p><imgloading="lazy"alt="Rendered construction of the list"src="/assets/images/construction_light-02b0be76041a8b1379107378e8f8b64c.svg#gh-light-mode-only"width="851"height="276"class="img_ev3q">
<imgloading="lazy"alt="Rendered construction of the list"src="/assets/images/construction_dark-fac28e7cafcc43d7e2fb5f0b6c25504e.svg#gh-dark-mode-only"width="851"height="276"class="img_ev3q"></p>
<p>If the recursion had bigger depth and/or more elements, it would iterate through them more than twice, therefore it does not take constant time to do nor some constant multiple of the input, since it traverses all of the elements in each of the levels.</p>
<h2class="anchor anchorWithStickyNavbar_LWe7"id="implementation-of-extend">Implementation of <code>extend</code><ahref="#implementation-of-extend"class="hash-link"aria-label="Direct link to implementation-of-extend"title="Direct link to implementation-of-extend"></a></h2>
<p>For the sake of <em>Algorithms and Data Structures I</em> we consider <code>APPEND</code> operation, i.e. adding the element to the end of the list, to have time complexity <spanclass="katex"><spanclass="katex-mathml"><mathxmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mimathvariant="script">O</mi><mostretchy="false">(</mo><mn>1</mn><mostretchy="false">)</mo></mrow><annotationencoding="application/x-tex">\mathcal{O}(1)</annotation></semantics></math></span><spanclass="katex-html"aria-hidden="true"><spanclass="base"><spanclass="strut"style="height:1em;vertical-align:-0.25em"></span><spanclass="mord mathcal"style="margin-right:0.02778em">O</span><spanclass="mopen">(</span><spanclass="mord">1</span><spanclass="mclose">)</span></span></span></span> (<strong>amortized</strong>; which is out of the scope of IB002).</p>
<p>If we have a look at the <code>extend</code> implementation in this dynamic array example:</p>
<p>Apart from checking edge cases, we can notice that we run <code>for</code>-loop over the elements from the other array and add them one-by-one to the <code>arr</code>. Time complexity of this operation is time dependant on the <code>src</code> array.</p>
<p>In this specific implementation, you could also resize the memory allocated for the array in one go and copy <em>whole</em><code>src</code> array in one go. However even if you did so, it would be still dependant on the size of the <code>src</code> array. Cause you still need to copy <spanclass="katex"><spanclass="katex-mathml"><mathxmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mtextmathvariant="monospace">count</mtext><mostretchy="false">(</mo><mi>s</mi><mi>r</mi><mi>c</mi><mostretchy="false">)</mo><mo>⋅</mo><mtextmathvariant="monospace">elementSize</mtext><mostretchy="false">(</mo><mi>s</mi><mi>r</mi><mi>c</mi><mostretchy="false">)</mo></mrow><annotationencoding="application/x-tex">\texttt{count}(src) \cdot \texttt{elementSize}(src)</annotation></semantics></math></span><spanclass="katex-html"aria-hidden="true"><spanclass="base"><spanclass="strut"style="height:1em;vertical-align:-0.25em"></span><spanclass="mord text"><spanclass="mord texttt">count</span></span><spanclass="mopen">(</span><spanclass="mord mathnormal">src</span><spanclass="mclose">)</span><spanclass="mspace"style="margin-right:0.2222em"></span><spanclass="mbin">⋅</span><spanclass="mspace"style="margin-right:0.2222em"></span></span><spanclass="base"><spanclass="strut"style="height:1em;vertical-align:-0.25em"></span><spanclass="mord text"><spanclass="mord texttt">elementSize</span></span><spanclass="mopen">(</span><spanclass="mord mathnormal">src</span><spanclass="mclose">)</span></span></span></span> bytes. From that we can assume that for specific instance of array the <spanclass="katex"><spanclass="katex-mathml"><mathxmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mtextmathvariant="monospace">elementSize</mtext><mostretchy="false">(</mo><mi>s</mi><mi>r</mi><mi>c</mi><mostretchy="false">)</mo></mrow><annotationencoding="application/x-tex">\texttt{elementSize}(src)</annotation></semantics></math></span><spanclass="katex-html"aria-hidden="true"><spanclass="base"><spanclass="strut"style="height:1em;vertical-align:-0.25em"></span><spanclass="mord text"><spanclass="mord texttt">elementSize</span></span><spanclass="mopen">(</span><spanclass="mord mathnormal">src</span><spanclass="mclose">)</span></span></span></span> is fixed, therefore we consider it a constant. That way we are getting <spanclass="katex"><spanclass="katex-mathml"><mathxmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mimathvariant="script">O</mi><mostretchy="false">(</mo><mtextmathvariant="monospace">count</mtext><mostretchy="false">(</mo><mi>s</mi><mi>r</mi><mi>c</mi><mostretchy="false">)</mo><mostretchy="false">)</mo></mrow><annotationencoding="application/x-tex">\mathcal{O}(\texttt{count}(src))</annotation></semantics></math></span><spanclass="katex-html"aria-hidden="true"><spanclass="base"><spanclass="strut"style="height:1em;vertical-align:-0.25em"></span><spanclass="mord mathcal"style="margin-right:0.02778em">O</span><spanclass="mopen">(</span><spanclass="mord text"><spanclass="mord texttt">count</span></span><spanclass="mopen">(</span><spanclass="mord mathnormal">src</span><spanclass="mclose">))</span></span></span></span> as a time complexity of our <code>extend</code> operation.</p></div><footerclass="theme-doc-footer docusaurus-mt-lg"><divclass="theme-doc-footer-tags-row row margin-bottom--sm"><divclass="col"><b>Tags:</b><ulclass="tags_jXut padding--none margin-left--sm"><liclass="tag_QGVx"><aclass="tag_zVej tagRegular_sFm0"href="/algorithms/tags/c/">c</a></li><liclass="tag_QGVx"><aclass="tag_zVej tagRegular_sFm0"href="/algorithms/tags/python/">python</a></li><liclass="tag_QGVx"><aclass="tag_zVej tagRegular_sFm0"href="/algorithms/tags/dynamic-array/">dynamic array</a></li><liclass="tag_QGVx"><aclass="tag_zVej tagRegular_sFm0"href="/algorithms/tags/time-complexity/">time complexity</a></li><liclass="tag_QGVx"><aclass="tag_zVej tagRegular_sFm0"href="/algorithms/tags/recursion/">recursion</a></li></ul></div></div><divclass="theme-doc-footer-edit-met
