Requests Module in Python

Learn via video course
FREE
View all courses
Python Course for Beginners With Certification: Mastering the Essentials
Python Course for Beginners With Certification: Mastering the Essentials
by Rahul Janghu
1000
4.90
Start Learning
Python Course for Beginners With Certification: Mastering the Essentials
Python Course for Beginners With Certification: Mastering the Essentials
by Rahul Janghu
1000
4.90
Start Learning
Topics Covered

Overview

Python has a module called requests that helps us deal with creating HTTP requests and accepting HTTP responses. The requests module takes care of creating the request-response objects, establishing connection to the server, error handling, session handling, ssl verification, passing headers, and so on

Introduction to requests module in python

In client-server architecture, we use HTTP very frequently to communicate between the client and the server. Typically, a client makes a request and the server returns some response. The request could be a request to get/ insert/ update some data. Similarly, the response can either return some data or some error. In some cases, the client undergoes authentication to ensure the request is valid. In this article, we shall see how to implement these HTTP requests and responses in Python using the requests module.

. We need to import the following module to get started:

Why learn Python Requests Module?

Learning the requests module will enable you to start making HTTP requests within a matter of writing a few lines of code. You do not need to worry about how to create the HTTP client, how to create the request object, how to deserialize the response object. A lot of the low level details are abstracted out to you using sub modules, classes and simple methods. As a web developer who builds apps using client-server architecture requests module is a handy tool to have in your toolbox

Making a Request

Let us see how an HTTP request is made in Python. We will use an open weather API that gives the current weather details at a particular latitude and longitude

Output

All Methods in the Requests Module

Requests module supports the following methods:

MethodDescription
getSend an HTTP GET request
putSend an HTTP PUT request
postSend an HTTP POST request
deleteSend an HTTP DELETE request

GET Request

HTTP GET method is used to retrieve data from server. GET methods can be invoked using the get() method of requests module.

Syntax

  • url*: url of the API endpoint you want to send the request to

  • params*: A dictionary used to send query params to the API endpoint

  • args*: args can be used to specify some client level configs, such as Headers, authorization details, timeouts, etc

Let us say, you are building a cab booking app Rider. The very first requirement that you need to implement is fetching the users lat-long. Let us assume the following API endpoint exists to do the same:

We can convert the above cURL request to a Python requests module request. Assuming that the response is json, we can use the json() method on the response object provided by the Python requests module itself. This will help us parse the json

Output

POST Request

HTTP POST method is used to submit data to a server. A POST request can be executed by first creating a request object and using post() method of requests module

Syntax

  • url*: url of the API endpoint you want to send the request to

  • data*: This is an optional parameter that you can use in case you need to send a file object or a dictionary to the server

  • json*: This is the json request payload to the server

Let us take our cab booking app Rider example again. Now that we have the lat long of a user, let us say we have the following POST API to request for a cab ride given the lat long details

Output

Response object

Requests module has a response object that is used for the following:

  1. While fetching response from server
  2. Knowing the HTTP status code of the response
  3. If the response status is unsuccessful, getting the deserialized error message received from the server

All Response Methods

When you invoke a method of the request module, such as get() or post() the module returns a response object. The following summarizes the important methods of the response object

MethodDescription
headersTo get the dictionary of the headers
encodingTo know the encoding used to decode the content in the response
close()To close the connection to the server
contentGet the content in the response body
cookiesGet the cookies sent by the server to identify requests coming from the same browser
json()Get the response payload as a json string. If the response was written in json an error is raised
urlGet the response of the url
textGet the response body as a unicode text
status_codeGet the HTTP status code of the response
requestGet the request object that requested for the response
reasonReturns a text with the status code. This is useful for parsing error message in case of success code
okResponse.ok returns True if 200 OK type of response is returned
raise_for_status()Get an error object if a non successful response is returned. It is used for debugging the erroneous request

Authentication

Authentication means validating a client if they can access an API. This serves several purposes:

  1. Limiting requests to an API
  2. Filtering malicious requests

To implement authentication, we need to import the following module present in requests module:

The HTTPBasicAuth module is present in requests.auth and has classes and methods to detail with HTTP Authentication

An Authentication object needs to be created and passed to the get() body method using the 'Auth' key. We are trying to authenticate a user who is trying to login using a username and password. If the username and password entered by the client do not match the credentials present in the server's database, the server will throw a 403 forbidden error, else a 200 OK response

Syntax

Let us say we are not working on our cab booking app Rider's login page. Login API naturally, needs to be authenticated, the user's credentials will be passed to the server and the server will match them with the stored credentials in its database. We can check for authentication status using the status_code method

SSL Certificate Verification

Many websites are not on HTTPS which means the client needs to verify an SSL certificate for the client to access the resource. A website needs an SSL certificate to maintain user data security, verify ownership of the website, prevent attackers from creating a fake version of the site, and gain user trust

For this purpose the 'verify' key of the get method can be used

If our Rider APIs were hosted on an HTTPS secure server, clients would need to verify their identity using a certificate

The Session Object

HTTP is a stateless protocol. This means that there is no persistence of a request executed by a client, hence every request is independent of the other. However, there are few states a server-side application might need to store in a client-server architecture. For example:

  1. Login state of a client: Once a user has logged in the login state shall be preserved for some time
  2. Cookies generated by the server: Cookies are used by servers to identify requests coming from the same web browser, for faster rendering of pages
  3. Request Id for a request, for caching purposes where the server does not want to call a database but instead server the data from caches whenever a client hits a request to a server, the server-generated
  4. Advertising ID of a user to understand the browsing preferences of the user

For these use cases, we can use a Session object

For example, in our Rider app, once a user logs in, we would like to store the login state, so that when any user switches to some another app and comes back to Rider, they are not logged out

Syntax

All variables in the requests module

VariableDescription
paramsA dictionary of path params that need to be sent with the request url
dataA dictionary for request payload
timeoutTo configure lient side timeouts for request
cookiesTo specify cookies as part of request
headersTo specify headers as part of request in the form of a dictionary
verifyIt set to True, SSL certification is expected
streamTo convert the response from server to a raw stream of data

Conclusion

  1. In client-server architecture, we need to deal with HTTP requests and responses
  2. Python offers the requests module to handle HTTP calls
  3. We can trigger GET, PUT, POST, DELETE requests using requests module
  4. We can extract the response obtained in an object and if required parse it as a json
  5. If the server returns an error we can get the HTTP status code as well as the error message using response object methods
  6. For authenticating requests we can use an HTTPBasicAuth object
  7. For enabling SSL certification we can make use of verify variable as part of the request method
  8. For storing state across requests we can use Sesssion object