How to upload and process the Excel file in Django

In this article we will discuss how to upload an Excel file and then process the content without storing file on server. One approach could be uploading the file, storing it in upload directory and then reading the file. Another approach could be uploading file and reading it directly from post data without storing it in memory and displaying the data.

We will work with the later approach here.

You may create a new project or work on existing code.

If you are setting up a new project then create a new virtual environment and install Django 2.0 and openpyxl modules in virtual environment using pip.

 

Continue reading “How to upload and process the Excel file in Django”

Adding Robots.txt file to Django Application

Robots.txt is a standard used by websites to communicate with web crawlers and other web robots. The standard specifies how to inform the web robot about which areas of the website should not be processed or scanned.

Why robots.txt is important:

Before a search engine crawls your site, it will look at your robots.txt file as instructions on where they are allowed to crawl/visit and index on the search engine results. If you want search engines to ignore any  pages on your website, you mention it in your robots.txt file.

Basic Format:
Example:
Steps to add robots.txt in Your Django Project:

Lets say your project’s name is myproject.

Create a directory templates in root location of your project. Create another directory with the same name as your project inside templates directory.

Place a text file robots.txt in it.

Your project structure should look something like this.

Add user-agent and disallow URL in it.

 

Now go to your project’s urls.py file and add below import statement

Add below URL pattern.

Now restart the server and go to localhost:8000/robots.txt in your browser and you will be able to see the robots.txt file.

Serving robots.txt from web server:

You can serve robots.txt directly from your web server. Below is the sample configuration for apache.

Quick Tips:
  1. robots.txt is case sensitive. The file must be named robots.txt, not Robots.txt or robots.TXT.
  2. robots.txt file must be placed in a website’s top-level directory.
  3. Make sure you’re not blocking any content or sections of your website you want crawled as this will not be good for SEO.

 

Host your Django App for Free.

Displaying custom 404 error (page not found) page in Django 2.0

It happens very frequently that a visitor on your website typed a wrong URL or the page user is looking for no longer exists. What do you do to handle such cases?

You have three options.

  • Redirect the visitor to home page, silently.
  • Show a boring 404 error page and then ask them to click on a link to go to home page.
  • Create your own funny/awesome/informative custom 404 error page.

In this article we will discuss the third option i.e. How to show your own error page in Django 2.0 project when a URL is not found.

Code is available on Github.

Featured Image source:  https://www.pinterest.com/pin/101612535313085400/

Continue reading “Displaying custom 404 error (page not found) page in Django 2.0”

Generating and Returning PDF as response in Django

We might need to generate a receipt or a report in PDF format in Django app. In this article, we will see how to generate a dynamic PDF from html content and return it as a response.

Create a Django project. If you are not using virtual environment, we strongly recommend to do so.

Installing Dependencies:

Once virtual environment is ready and activated, install the below dependencies.

For pdfkit  to work, we need wkhtmltopdf  installed in our Linux system.

Continue reading “Generating and Returning PDF as response in Django”

Using signals in Django to log changes in models

Sometimes we need to know who made what changes to which table. This might be required for legal audit purpose or for simple organisational level logging.

There are multiple Django apps available online which can help you log the model changes but there is no fun in doing that. We will see how to do it without using ready-made app and hence will learn something in the process.

Signals:

Signals lets a sender notify another receiver that some event have occurred and some action needs to be performed.

For example, we have some data in cache as well in DB. We read data from cache and if not found then goes to DB as fallback. Now whenever a DB is updated, we need to update the cache as well. But we might update the model from multiple views. Hence it is tough and not clean to write cache update logic in every such view. Signals comes into picture now.

Continue reading “Using signals in Django to log changes in models”

py_instagram_dl – The Python Package to Download All pictures of an Instagram User

I created a small script to download all pictures of an Instagram user without using APIs as APIs poses few limitations like rate limit.

After few rounds of tweaking, optimisation and beautifying code, I though of creating a python package out of it. If you want to know how to create a distributable python package, this article will be extremely helpful as steps are discussed in great detail.

You can find the  py_instagram_dl  package listed on pypi.
link is –  https://pypi.python.org/pypi/py-instagram-dl.

How to download all pictures of an Instagram user:
  • Create a virtual environment. Optional but strongly recommended. You may follow this simple and step by step pocket guide on Python Virtual Environment.
  • Install dependencies. This package instead few other python packages to work.
  • Now install this package.
  • Use the installed package in your code.
    Parameter Options:
Download  method have one mandatory and two optional parameters as of now.

Mandatory Parameter:
Parameter 1: Valid username of Instagram user.

Optional Parameter:
verbose
: default value – True (boolean) : Decides whether information should be printed on screen. Recommended to have it set to True so that in case of large number of downloads you can make sure script is working and is not just freezed.

wait_between_requests : default value – 0 (integer) : This is the time in seconds for which scripts waits to send new hit to download the picture to Instagram. It is recommended to pass a positive value for this parameter. If you are getting rate limit exceptions after downloading few pictures, pass 1 in this parameter, i.e. wait for 1 second between each request.

Exceptions:

InvalidUsernameException: When a non existent username is provided.
RateLimitException: When rate limit is reached. Use parameter wait_between_requests  to avoid this.

 

Source code.

Server Access Logging in Django using middleware

Some application admins need to know which user performed what action in the application. We also felt the need of such tracking hence we started developing the access log system for our application.

In this article we will see how to develop the server access logging app for Django project.

We will be storing below information:

  • URL/link visited.
  • Request method, get or post.
  • Get or Post Data
  • Referrer
  • IP address of visitor
  • Session ID

Continue reading “Server Access Logging in Django using middleware”

Get latest Bitcoin and other crypto-currencies rates using python Django

Everybody is investing in bitcoins. James Howells is trying to dig a landfill site to get 7500 bitcoins that were dumped there in 2013.

To be a good investor, it is necessary that you keep track of ups and downs in the market. There are multiple platforms where you can track the price of bitcoin. But for a python programmer that is no fun. Being a python programmer we will develop our own project where we can get latest bitcoin and other crypto-currency prices.

Let’s start.

Continue reading “Get latest Bitcoin and other crypto-currencies rates using python Django”

How to develop a distributable Django app to block the crawling IP addresses

In software engineering, don’t repeat yourself (DRY) is a principle of software development aimed at reducing repetition of code. All the installed packages and apps we use in our Python/Django projects are great example of DRY concept.

In this article we will learn:
– How to create custom middleware in Django project.
– How to create reusable Django App / Python Package to block the IPs.
– How to upload the python package on pypi and djangopackages.org.

Code of the developed Django app in article is hosted on Github. Package is available on pypi and djangopackages.

Continue reading “How to develop a distributable Django app to block the crawling IP addresses”

How to backup database periodically on PythonAnyWhere server

You can host your Django app effortlessly on PythonAnyWhere server. If you are using the database in your app then it is strongly recommended to take backup of database to avoid loss of data.

This PythonAnyWhere article explain the process to take sql dump. We will extend the same article to take database backup periodically and delete the old files.

Continue reading “How to backup database periodically on PythonAnyWhere server”