Python lists are one of the most versatile and widely-used data structures in Python. They're dynamic, flexible, and can hold a variety of object types. But have you ever wondered how Python manages the memory behind the scenes as you append more and more items to a list?

In this post, we’ll dive into the details of how Python dynamically allocates memory to lists, explore why the size of a list in memory isn’t directly proportional to the size of its contents, and how this behavior varies across Python versions. We’ll use some simple code snippets to illustrate how Python optimizes list memory allocation, helping you better understand and perhaps optimize your own code.

Let’s start by seeing how much memory an empty list takes up and how that changes as we add elements to it.

import sys


def show_memory_size():
    items = []
    print("Size of list:", sys.getsizeof(items), "bytes")


show_memory_size()

When this code is run on with Python 3.12.5 we get 'Size of list: 56 bytes'. So, the size of an empty list in Python 3.12 is 56 bytes. You will notice that I am emphasizing the python version here as the core python team has recently been working hard on python optimizations and part of that has been the reduction in python object sizes. As an example, if you run this same code with the Python 3.7.9 you will see 'Size of list: 64 bytes'. 

I believe the next logical question is how much does our list grow in size as we add elements to it.

import sys

def show_memory_size():
    items = []
    empty_list_size = sys.getsizeof(items)
    print(f"Size of empty list: {empty_list_size} bytes")
    for i in range(1000):
        items.append(i)
    total_list_size = sys.getsizeof(items)
    net_list_size_less_inital_size = total_list_size - empty_list_size
    print(f"Size of list w/ Items: {total_list_size} bytes")
    size_per_item = net_list_size_less_inital_size / len(items)
    print(f"Size of each item added to list: {size_per_item} bytes")


show_memory_size()

The output of this script will show that the list grows by 8.8 bytes per item appended into the list. In this case we are appending an int to the list so let us check the size of an int object in python to determine if those two values match. 

import sys

def show_memory_size():
    items = []
    empty_list_size = sys.getsizeof(items)
    print(f"Size of empty list: {empty_list_size} bytes")
    for i in range(1000):
        items.append(i)
    total_list_size = sys.getsizeof(items)
    net_list_size_less_inital_size = total_list_size - empty_list_size
    print(f"Size of list w/ Items: {total_list_size} bytes")
    size_per_item = net_list_size_less_inital_size / len(items)
    print(f"Size of each item added to list: {size_per_item} bytes")


def size_of_int(int_obj):
    sys_of_int = sys.getsizeof(int_obj)
    print(f"Size of int: {sys_of_int} bytes")


show_memory_size()
size_of_int(10)

You will see the size of the number ten an int object is 28 bytes which does not match the 8.8 bytes our list increased by when appending 1000 numbers to it. The reason for this is to due how python handles list when you append an element to a list such as our example of a int number. You are not actually appending that element or object to the list but a pointer to that object. Lets further demonstrate this idea with some python code. 

Below I am using the python function id() which returns an object's memory address. To show that when this int object (the number 10) is added to the list that int is not copied but just a pointer of that object is added to the list. You will see this python script will print out the same memory address for the number itself and in the item in the list. 

def show_int_id():
    new_int = 10
    print(id(new_int))
    list_of_ints = [new_int]
    for i in list_of_ints:
        print(id(i))

show_int_id()

Since pointers bytes sizes can vary with architecture and other factors there is not much guarantee that the size per item or pointer size in python list will always be the same. You may get a different byte size than my outcome on your machine (64 bit will be different than 32 bit). 

Another reason why calculating the list size per item is not accurate is due to how python dynamically allocates a list size as you append items to it. Python lists are over-allocated to optimize for appends. When a list requires more space extra memory is allocated than is needed for the item being appended. This reduces the number of reallocations as the list grows. This means the list doesn't grow by exactly the size of each new item/pointer when you append.

See the below python code.

import sys

def show_python_list_allocation():
    items = []
    initial_size = sys.getsizeof(items)
    print(f"Initial List Size: {initial_size} bytes")

    for i in range(1000):
        items.append(i)
        if i in [0, 10, 100, 500, 999]: 
            current_size = sys.getsizeof(items)
            print(f"List size after {i+1} items: {current_size} bytes, increased by {current_size - initial_size} bytes")
            initial_size = current_size

show_python_list_allocation()

While the allocations on your machine maybe different than mine. I believe you will find that the allocations are as described where python is allocating more memory than necessary to add items to the list to optimize for less reallocating.

This demonstrates how Python dynamically manages memory for lists. As we append items to a list, Python does not merely increase the size by the exact memory footprint of each object. Instead, the list holds pointers to the objects and uses an over-allocation strategy to minimize the performance hit of frequent resizing. This results in memory usage that may seem surprising at first, like the 8.8 bytes per item in our example, rather than the 28 bytes an integer actually takes.