Search for Courses, Topics

Python Decorators | Decorators in Python

Learn about python decorators.

23 Feb 2022-19 mins read

Overview

A decorator in Python is a function that takes another function as an argument and extends its behavior without explicitly modifying it. It is one of the most powerful features of Python. It has several usages in the real world like logging, debugging, authentication, measuring execution time, and many more.

Scope of Article

  • This article defines decorators in Python.
  • We first discuss the prerequisites and then learn more about decorators and how to use them properly with functions and classes.
  • We also discuss the real-world usages of decorators in Python Programming.

Introduction

Suppose, you have a set of functions and you only want authenticated users to access them.
Therefore, you need to check whether a user is authenticated or not before proceeding with the rest of the code in the function.

One way to do this is by calling a separate function inside all the functions and using conditional statements. But this will require us to change the code for each function.
A better solution here would be to use a Decorator.

A Decorator is just a function that takes another function as an argument and extends its behavior without explicitly modifying it.

This means that a decorator adds new functionality to a function.

By the end of this article, you will understand what does "extending a function without actually modifying it" means.

Prerequisites for Learning Decorators

To understand decorators in Python, you must have an understanding of the following concepts:

  • How functions work.
  • First-Class Objects.
  • Inner Functions.
  • Higher-Order Functions.

Don't worry! We will go through these things in the next section. If you are already familiar, feel free to skip.

Takeaway:

  • A Decorator is a function that takes another function and extends the behavior of the latter function without explicitly modifying it.

Functions in Python

A function returns a value based on the given arguments.

For instance, the following function returns twice of a number:

def twice(number):
    return number * 2

A function may also have side effects like the print function. The print function returns None while having the side effect of outputting the given string to the console.

First-Class Objects

In Python, a function is treated as a first-class object. This means that a function has all the rights as any other variable in the language.

That's why, we can assign a function to a variable, pass it to a function or return it from a function. Just like any other variable.

Assigning a function to a variable

You can assign a function to a variable as follows:

def foo():
  print("I am foo")

also_foo = foo

foo()
also_foo()

Output:

I am foo
I am foo

Explanation:

As everything is an object in Python, the names we define are simply identifiers referencing these objects.

Thus, both foo and also_foo points to the same function object as shown below in the diagram:

first class objects in python decorators

That's why we got the same output from both the function call.

Passing a function to another function

There are multiple use cases of passing a function as an argument to another function in Python. For instance, passing a key function to sort lists. Decorators also use this technique as we will see later.

A function can be passed to any other function just like a normal variable as follows:

def do_twice(func):
  func()
  func()

def say_hello():
  print("Hello!")

do_twice(say_hello)

Output:

Hello!
Hello!

Explanation:

The do_twice function accepts a function and calls it twice in its body. We defined another function say_hello and passed it to the do_twice function, thus getting Hello! two times in the output.

Returning a function from a function

Returning a function from a function is another technique used by decorators in Python.

We can return a function from a function as follows:

def return_to_upper():
  return str.upper

to_upper = return_to_upper()

print(to_upper("scaler topics"))

Output:

SCALER TOPICS

Explanation:

Just for illustration, we are returning the upper method of str from the function return_to_upper. We called the function and stored the reference to the returned function in to_upper. Then used it to print the upper case of "scaler topics".

Higher-order function is a function that takes a function as an argument or returns a function.

Inner Functions

We can define a function inside other functions. Such functions are called inner functions or nested functions. Decorators in Python also use inner functions.

For example, the following is a function with two inner functions:

def parent():
  print("I am the parent function")

  def first_child():
    print("I am the first child function")

  def second_child():
    print("I am the second child function")

  first_child()
  second_child()

parent()

Output:

I am the parent function
I am the first child function
I am the second child function

Explanation:

We created two functions inside the function parent and then called both of them in the parent function body. The order in which we define the inner functions does not matter. The output is only dependent on the order of calling the functions.

Note 📝:

The inner functions are locally scoped to the parent. They are not available outside of the parent function. If you try calling the first_child outside of the parent body, you will get a NameError.

Inner functions can access variables in the outer scope of the enclosing function. This pattern is known as a Closure.

Consider the following example:

def outer(message):
  def inner():
    print("Message:", message)

  return inner

hello_msg = outer("Hello!")
hello_msg()

bye_msg = outer("Bye!")
bye_msg()

Output:

Message: Hello!
Message: Bye!

Explanation:

The message is remembered by the inner function even after the outer function has finished executing. This technique by which some data gets attached to the code is called closure in Python.

Takeaway:

  • Functions are first-class objects in Python.

Introduction to Decorators

Now that we have the pre-requisite knowledge for understanding decorators, let's go on to learn about Python decorators.

As discussed before, a decorator in Python is used to modify the behavior of a function without actually changing it.

Syntax:

func = decorator(func)

where func is the function being decorated and decorator is the function used to decorate it.

Let's see an example to understand what does this mean:

def decorator(func):
  def wrapper():
    print("This is printed before the function is called")
    func()
    print("This is printed after the function is called")
  
  return wrapper

def say_hello():
  print("Hello! The function is executing")


say_hello = decorator(say_hello)

say_hello()

Output:

This is printed before the function is called
Hello! The function is executing
This is printed after the function is called

Explanation:

We have two functions here:

  • decorator: This is a decorator function, it accepts another function as an argument and "decorates it" which means that it modifies it in some way and returns the modified version.
    Inside the decorator function, we are defining another inner function called wrapper. This is the actual function that does the modification by wrapping the passed function func.
    decorator returns the wrapper function.
  • say_hello: This is an ordinary function that we need to decorate. Here, all it does is print a simple statement.

The most important line in the code is this:

say_hello = decorator(say_hello)

We passed the say_hello function to the decorator function. In effect, the say_hello now points to the wrapper function returned by the decorator.
However, the wrapper function has a reference to the original say_hello() as func, and calls that function between the two calls to print().

Takeaway:

  • A decorator function modifies a function by wrapping it in a wrapper function.

Syntactic Decorator

The above decorator pattern got popular in the Python community but it was a little inelegant. We have to write the function name thrice and the decoration gets a bit hidden below the function definition.

Therefore, Python introduced a new way to use decorators by providing syntactic sugar with the @ symbol.

Syntax:

@decorator
def func(arg1, arg2, ...):
    pass

Syntactic sugar is syntax within a programming language that is designed to make things easier to read or to express.

The following example does the same thing as the previous example:

def decorator(func):
  def wrapper():
    print("This is printed before the function is called")
    func()
    print("This is printed after the function is called")
  
  return wrapper

@decorator
def say_hello():
  print("Hello! The function is executing")


say_hello()

Output:

This is printed before the function is called
Hello! The function is executing
This is printed after the function is called

Explanation:

The output and working are the same as the previous example, the only thing that changed is that we are using @decorator instead of say_hello = decorator(say_hello).

Takeaway:

  • We can easily decorate a function using the @decorator syntax.

Preserving the Original Name and Docstring of the Decorated Function

In Python, functions have a name attribute and a docstring to help with debugging and documentation.
But, when we decorate a function its identity is changed to the wrapper function.

See the following example (we are using the same decorator created before):

@decorator
def say_hello():
  """This function says hello when called"""
  print("Hello! The function is executing")


print(say_hello.__name__)
help(say_hello)

Output:

wrapper
Help on function wrapper in module __main__:

wrapper()

Although technically true, this is not what we wanted. As the say_hello now points to the wrapper function, it is showing its information instead of the original function.

To fix this, we need to use another decorator called wraps on the wrapper function.

The wraps decorator is imported from the in-built functools modules.

This is how we do it:

import functools

def decorator(func):
  @functools.wraps(func)
  def wrapper():
    print("This is printed before the function is called")
    func()
    print("This is printed after the function is called")
  
  return wrapper


@decorator
def say_hello():
  """This function says hello when called"""
  print("Hello! The function is executing")


print(say_hello.__name__)
help(say_hello)

Output:

say_hello
Help on function say_hello in module __main__:

say_hello()
    This function says hello when called

This time, we got the correct docstring from the help function and the correct name from the __name__ attribute.

Takeaway:

  • Use the functools.wraps decorator to preserve the original name and docstring of the decorated function

Reusing Decorator

A decorator is just a regular Python function. Hence, we can reuse it to decorate multiple functions.

Let's create a file called decorators.py with the following code:

import functools

def do_twice(func):
  @functools.wraps(func)
  def wrapper():
    func()
    func()
  
  return wrapper

do_twice is a simple decorator that calls the decorated function two times.

Now, you can reuse the do_twice decorator any number of times by importing it.
Here's an example:

from decorators import do_twice

@do_twice
def say_hello():
  print("Hello!")

@do_twice
def say_bye():
  print("Bye!")

say_hello()
say_bye()

Output:

Hello!
Hello!
Bye!
Bye!

Explanation:

We imported any used the do_twice decorator on both the functions and called them. Therefore, we got two outputs for each function.

Takeaway:

  • A decorator can be reused just like any other function.

Decorators Functions with Parameters

What if the function we are decorating has some parameters?

Let's try it with an example:

import functools

def do_twice(func):
  @functools.wraps(func)
  def wrapper():
    func()
    func()
  
  return wrapper

@do_twice
def say_hello(name):
  print(f"Hello, {name}!")

say_hello("Kitty")

Output:

TypeError: wrapper() takes 0 positional arguments but 1 was given

Explanation:

We got an error because the wrapper function we defined inside the decorator does not accept any argument.

The straightforward way to solve this would be to let the wrapper accept one argument, but then we won't be able to use the do_twice decorator with a function with more than one argument.

So, a better solution is to accept a variable number of arguments in the wrapper function and then pass those arguments to the original function func.

Here is how you would do it:

import functools

def do_twice(func):
  @functools.wraps(func)
  def wrapper(*args, **kwargs):
    func(*args, **kwargs)
    func(*args, **kwargs)
  
  return wrapper

@do_twice
def say_hello(name):
  print(f"Hello, {name}!")

say_hello("Kitty")

Output:

Hello, Kitty!
Hello, Kitty!

Explanation:

*args and **kwargs allow us to pass multiple arguments or keyword arguments to a function.
Thus, we passed "Kitty" as name to say_hello which was received by the wrapper function and the wrapper function used it to call the actual func function. Outputting Hello, Kitty! twice.

Takeaway:

  • Use a variable number of parameters in the wrapper function to handle any number of arguments in the decorated function.

Returning Values from Decorated Functions

What happens to the returned value from the decorated function? Let's check out with an example.

Consider the following add function, it prints a statement then returns the sum of the two numbers, we are decorating it with the previously created do_twice decorator:

import functools

def do_twice(func):
  @functools.wraps(func)
  def wrapper(*args, **kwargs):
    func(*args, **kwargs)
    func(*args, **kwargs)
  
  return wrapper

@do_twice
def add(num1, num2):
  print(f"Adding {num1} and {num2}")
  return num1 + num2

print("The sum is:", add(1, 2))

Output:

Adding 1 and 2
Adding 1 and 2
The sum is: None

The add function was called twice as expected but we got None in the return value. This is because the wrapper function does not return any value.

To fix this, we need to make sure the wrapper function returns the return value of the decorated function.

Here is how you would do it:

import functools

def do_twice(func):
  @functools.wraps(func)
  def wrapper(*args, **kwargs):
    func(*args, **kwargs)
    return func(*args, **kwargs)
  
  return wrapper

@do_twice
def add(num1, num2):
  print(f"Adding {num1} and {num2}")
  return num1 + num2

print("The sum is:", add(1, 2))

Output:

Adding 1 and 2
Adding 1 and 2
The sum is: 3

Explanation:

We are calling the func twice in the wrapper function. But this time, we made sure to return the value of the second call back to the caller.

Now, we are getting the correct sum!

Takeaway:

  • The wrapper function should return the return value of the decorated function, otherwise, it would be lost.

Decorators with Arguments

You can pass arguments to the decorator itself!
All you need to do is define the decorator inside another function that accepts the arguments and then use those arguments inside the decorator. You also need to return the decorator from the enclosing function.

Let's see what does this means with code to better understand it.

Previously, we created a decorator called do_twice. Now, we will extend it to repeat any number of times. Let's call this new decorator repeat.

import functools

def repeat(num_times):
  def decorator_repeat(func):
    @functools.wraps(func)
    def wrapper(*args, **kwargs):
      for _ in range(num_times):
        value = func(*args, **kwargs)
      return value

    return wrapper
  
  return decorator_repeat

@repeat(num_times=3)
def say_hello(name):
  print(f"Hello, {name}!")

say_hello("Kitty")

Output:

Hello, Kitty!
Hello, Kitty!
Hello, Kitty!

Explanation:

Let's break down the code:

  • The most inner function wrapper is taking a variable number of arguments and then calling the decorated function num_times times. It finally returns the return value of the original decorated function.

    def wrapper(*args, **kwargs):
      for _ in range(num_times):
        value = func(*args, **kwargs)
      return value
    
  • One level above is the decorator_repeat function which does the work of a normal decorator, it returns the wrapper function.

    def decorator_repeat(func):
        @functools.wraps(func)
        def wrapper(*args, **kwargs):
          ...
    
        return wrapper
    
  • On the outermost level is the repeat decorator function that accepts an argument and provides it to the inner functions using the closure pattern.

    def repeat(num_times):
      def decorator_repeat(func):
        ...
    
      return decorator_repeat
    

Finally, we used the decorator with a parenthesis () unlike before to pass an argument.

In summary,

@repeat(num_times=3)
def say_hello(name):
  print(f"Hello, {name}!")

Is equivalent to:

say_hello = repeat(num_times=3)(say_hello)

That is, repeat is called with the given argument and then its return value (the actual decorator) is called with the say_hello function.

Decorators with arguments are used as a decorator factory to create new decorators.

Takeaway:

  • You can pass arguments to a decorator by wrapping them inside of another decorator function.

Chaining Decorators

Chaining the decorators means that we can apply multiple decorators to a single function. These are also called nesting decorators.

Consider the following two decorators:

import functools

def split_string(func):
  @functools.wraps(func)
  def wrapper(*args, **kwargs):
    return func(*args, **kwargs).split()
  
  return wrapper

def to_upper(func):
  @functools.wraps(func)
  def wrapper(*args, **kwargs):
    return func(*args, **kwargs).upper()
  
  return wrapper
  • The first one takes a function that returns a string and then splits it into a list of words.
  • The second one takes a function that returns a string and converts it into uppercase.

Now, we will use both the decorators on a single function by stacking them like this:

@split_string
@to_upper
def say_hello(name):
  return f"Hello, {name}!"

print(say_hello("Kitty"))

Output:

['HELLO,', 'KITTY!']

Explanation:

The order of the decorators' matters in this case. First to_upper is applied to the say_hello function. Then, split_string is applied.

@split_string
@to_upper
def say_hello(name):
  return f"Hello, {name}!"

is equivalent to:

say_hello = split_string(to_upper(say_hello))

Takeaway:

  • You can apply multiple decorators to a single function by stacking them on top of each other.

Fancy Decorators

You need a basic understanding of classes in Python for this section.

I recommend going through the Class in Python article on Scaler Topics if you are unfamiliar with classes.

Till now, you have seen how to use decorators on functions. You can also use decorators with classes, these are known as fancy decorators in Python. There are two possible ways for doing this:

  • Decorating the methods of a class.
  • Decorating a complete class.

Decorating the Methods of a Class

Python provides the following built-in decorators to use with the methods of a class:

  • @classmethod: It is used to create methods that are bound to the class and not the object of the class. It is shared among all the objects of that class. The class is passed as the first parameter to a class method. Class methods are often used as factory methods that can create specific instances of the class.
  • @staticmethod: Static methods can't modify object state or class state as they don't have access to cls or self. They are just a part of the class namespace.
  • @property: It is used to create getters and setters for class attributes.

Let's see an example of all the three decorators:

class Browser:
  __NO_OF_WINDOWS = 0  # private member

  def __init__(self, page):
    self._page = page
    self.is_incognito = False

    Browser.__NO_OF_WINDOWS += 1

  @property
  def page(self):   # Getter
    return self._page

  @page.setter
  def page(self, new_page):
    if type(new_page) is not str:
      raise TypeError("Page must be a string")
    
    self._page = new_page

  @classmethod
  def with_incognito(cls, new_page):  # factory method for incognito window
      instance = cls(new_page)
      instance.is_incognito = True

      return instance

  @staticmethod
  def get_browser_info():
    return {
        "name": "Google Chrome",
        "version": "96.0.4664.110",
        "OS": "Windows"
    }

Explanation:

We created a class called Browser. The class contains a getter and setter for the page attribute created with the @property decorator.

It contains a class method called with_incognito which acts as a factory method to create incognito window objects.

It also contains a static method to get the information for the browser which will be the same for all objects (windows).

Decorating a Complete Class

You can also use decorators on a whole class.

Writing a class decorator is very similar to writing a function decorator. The only difference is that the decorator will receive a class and not a function as an argument. Decorating a class does not decorate its methods. It's equivalent to the following:

className = decorator(className)

It just adds functionality to the instantiation process of the class.

One of the most common examples of using a decorator on a class is @dataclass from the dataclasses module:

from dataclasses import dataclass

@dataclass
class User:
  username: str
  password: str
  active: bool

sheldon = User("sheldon", "fakepassword", True)

print(sheldon.username)

A data class is a class mainly containing data. It comes with basic functionality already implemented. We can instantiate, print, and compare data class instances straight out of the box.

The username:str syntax is called type hints in Python. Type hints are a special syntax that allows declaring the type of a variable.
Editors and tools use these types of hints to provide better support like auto-completion and error checks.

In the example, we have created a class called User which saves the data related to a user. Then, we created a user and printed its username.

Takeaway:

  • Decorators can be used with the methods of a class or the whole class.

Classes as Decorators

We can also use a class as a decorator. Classes are the best option to store the state of some data, so let's understand how to implement a stateful decorator with a class that will record the number of calls made for a function.

There are two requirements to make a class as a decorator:

  • The __init__ function needs to take a function as an argument.
  • The class needs to implement the __call__ method. This is required because the class will be used as a decorator and a decorator must be a callable object.

Also note that we use functools.update_wrapper instead of functools.wraps in case of a class as a decorator.

Now, let's implement the class:

import functools

class CountCalls:
  def __init__(self, func):
    functools.update_wrapper(self, func)
    self.func = func
    self.num_calls = 0

  def __call__(self, *args, **kwargs):
    self.num_calls += 1
    print(f"Call {self.num_calls} of {self.func.__name__!r}")
    return self.func(*args, **kwargs)

Now, use the class as a decorator as follows:

@CountCalls
def say_hello():
  print("Hello!")

say_hello()
say_hello()

After decoration, the __call__ method of the class is called instead of the say_hello method.

Output:

Call 1 of 'say_hello'
Hello!
Call 2 of 'say_hello'
Hello!

Takeaway:

  • Classes can also be used as decorators by implementing the __call__ method and passing the function to __init__ as an argument.

Real World Usage of Decorators

One real-world usage of decorators in Python is to measure the execution time of a function.
Consider the following example:

from time import time, sleep
    
def measure(func):
    def wrapper(*args, **kwargs):
        t = time()
        func(*args, **kwargs)
        print(func.__name__, 'took', time() - t)
    return wrapper

@measure
def sleepy_function(sleep_time):
    sleep(sleep_time)

sleepy_function(0.3)
sleepy_function(0.5)

Output:

sleepy_function took 0.3003675937652588
sleepy_function took 0.5005733966827393

Explanation:

The wrapper function of the measure decorator uses the time function from the time module to calculate the time difference between the start and end of the function execution and then print that on the console.

The sleepy function is used just for illustration, it uses the sleep function from the time module to freeze the execution for a certain amount of time. We can measure the execution time of any other function in the same way.

Other Use Cases of Decorators

  • Authorization in Python frameworks like Flask and Django.
  • Logging and debugging code.
  • Caching return values of a function.
  • Validating JSON (JavaScript Object Notation).

Takeaway:

  • Decorators in Python have several real-world usages like measuring execution time, authentication, logging, etc.

Conclusion

  • A Decorator in Python is a function that takes another function and extends the behavior of the latter function without explicitly modifying it.
  • Functions are first-class objects in Python.
  • We can easily decorate a function using the @decorator syntax.
  • We can pass arguments to a decorator by wrapping them inside of another decorator function.
  • We can apply multiple decorators to a single function by stacking them.
  • Decorators can be used with the methods of a class or the whole class.
  • Classes can also be used as decorators.
  • Decorators in Python have several real-world usages like measuring execution time, authentication, logging, etc.
certificate icon
Certificates
Python Tutorial
Module Certificate

Criteria

You’ll be able to claim a certificate for any course you have access to only after you’ve spent enough time learning. The time required is determined by the length of the course.
Certificates are a fantastic way to showcase your hard-earned skills to employers, universities, and anyone else that may be interested. They provide tangible proof that you’ve completed a course on Scaler Topics. In many cases they can help you claim a training reimbursement or get university credit for a course.
certificate icon
Certificates
Python Tutorial
Module Certificate

Criteria

You’ll be able to claim a certificate for any course you have access to only after you’ve spent enough time learning. The time required is determined by the length of the course.
Certificates are a fantastic way to showcase your hard-earned skills to employers, universities, and anyone else that may be interested. They provide tangible proof that you’ve completed a course on Scaler Topics. In many cases they can help you claim a training reimbursement or get university credit for a course.