How to upload and process the Excel file in Django

In this article we will discuss how to upload an Excel file and then process the content without storing file on server. One approach could be uploading the file, storing it in upload directory and then reading the file. Another approach could be uploading file and reading it directly from post data without storing it in memory and displaying the data.

We will work with the later approach here.

You may create a new project or work on existing code.

If you are setting up a new project then create a new virtual environment and install Django 2.0 and openpyxl modules in virtual environment using pip.

 

Continue reading “How to upload and process the Excel file in Django”

Creating sitemap of Dynamic URLs in your Django Application

A site map is a list of a website’s content designed to help both users and search engines navigate the site. A site map can be a hierarchical list of pages, an organization chart, or an XML document that provides instructions to search engine crawl bots.

Why sitemaps is required:

XML Sitemaps are important for SEO because they make it easier for Google to find your site’s pages—this is important because Google ranks web PAGES not just websites. There is no downside of having an XML Sitemap and having one can improve your SEO, so we highly recommend them.

Example:

Sitemap for this blog can be found at http://thepythondjango.com/sitemap_index.xml .

example sitemap
Example sitemap

Steps to add Sitemaps to your Django Application:

Create a file sitemap.py  in your app.

Create two different classes in sitemap.py file, one for static pages and another for Dynamic URLs.

Lets assume your website sell some product where product details are stored in database. Once a new product is added to database, you want that product page to be searchable by search engines. We need to add all such product pages/urls to sitemaps.

Static Sitemap:

Define a class StaticSitemap  in your sitemap.py  file. Define the mandatory function  items  in it which will return the list of objects. These objects will be passed to location method which will create URL from these objects.

Here in items function, we are returning appname:url_name  which will be used by location method to convert into absolute URL. Refer you app’s urls.py file for url names.

Dynamic Sitemap:

Similarly we will create Dynamic sitemap by fetching values from DB.

Here we are getting all products from database and generating URLs like  http:example.com/product/12 .

Adding sitemaps in URLconf:

Now add these sitemap class in URLconf. Edit the project’s urls.py  file and add below code in it.

 

Now reload your server and go to localhost:8000/sitemap.xml  and you will be able to see your sitemap there.

 

Reference : https://docs.djangoproject.com/en/2.0/ref/contrib/sitemaps/

 

Host your Django App for Free.

Displaying custom 404 error (page not found) page in Django 2.0

It happens very frequently that a visitor on your website typed a wrong URL or the page user is looking for no longer exists. What do you do to handle such cases?

You have three options.

  • Redirect the visitor to home page, silently.
  • Show a boring 404 error page and then ask them to click on a link to go to home page.
  • Create your own funny/awesome/informative custom 404 error page.

In this article we will discuss the third option i.e. How to show your own error page in Django 2.0 project when a URL is not found.

Code is available on Github.

Featured Image source:  https://www.pinterest.com/pin/101612535313085400/

Continue reading “Displaying custom 404 error (page not found) page in Django 2.0”

Server Access Logging in Django using middleware

Some application admins need to know which user performed what action in the application. We also felt the need of such tracking hence we started developing the access log system for our application.

In this article we will see how to develop the server access logging app for Django project.

We will be storing below information:

  • URL/link visited.
  • Request method, get or post.
  • Get or Post Data
  • Referrer
  • IP address of visitor
  • Session ID

Continue reading “Server Access Logging in Django using middleware”

Solving Django error ‘NoReverseMatch at’ URL with arguments ‘()’ and keyword arguments ‘{}’ not found.

Every Django developer encounters below error at least once in their life for sure.

Beginners spend many hours debugging the issue, jumping from question to question on Stackoverflow and posting in multiple groups on Facebook.

In this article we have tried to list all the common mistakes developer makes which leads them to above error.

Continue reading “Solving Django error ‘NoReverseMatch at’ URL with arguments ‘()’ and keyword arguments ‘{}’ not found.”

How to develop a distributable Django app to block the crawling IP addresses

In software engineering, don’t repeat yourself (DRY) is a principle of software development aimed at reducing repetition of code. All the installed packages and apps we use in our Python/Django projects are great example of DRY concept.

In this article we will learn:
– How to create custom middleware in Django project.
– How to create reusable Django App / Python Package to block the IPs.
– How to upload the python package on pypi and djangopackages.org.

Code of the developed Django app in article is hosted on Github. Package is available on pypi and djangopackages.

Continue reading “How to develop a distributable Django app to block the crawling IP addresses”

Using MongoDB in Django Application

In previous article we learned how to install Elasticsearch and Kibana and connect them together. In second part of the same article we used Elasticsearch in Django project.

Elasticsearch can be used with relational database or non-relational database. In those articles we used relational database MySQL.

In this article we will see how to use MongoDB in Django application.

I use PythonAnyWhere to host Django application quickly. Read here how to host your Django app on PythonAnyWhere for free.

Continue reading “Using MongoDB in Django Application”

Elastic Search with Kibana and Django

In this article we will see how to implement fast text search using elastic search instead of using MySQL or PostgreSQL.

System configurations:

– Ubuntu 16.04
– Python 3.5, Django 1.10
– Elastic Search v6.0.0
– Kibana v6.0.0

Elastic Search:

As per Wikipedia:

Elasticsearch is a search engine based on Lucene. It provides a distributed, multitenant-capable full-text search engine with an HTTP web interface and schema-free JSON documents.

ElasticSearch indexes documents for your data instead of using data tables like a regular relational database does.

There are two client libraries to interact with ElasticSearch with Python.
1. elasticsearch-py
2. elasticsearch-dsl

Continue reading “Elastic Search with Kibana and Django”

Elastic Search with Kibana and Django – part 2

In previous part, we installed Elasticsearch and Kibana. Now we will create a Django project and will link the elastic search with the project.

Django Project:

You can follow the official Django tutorials to create and start a Django project.  Or you follow below steps. We will be following the official documentation, but only the necessary steps.

– Make sure python 3 is installed on your machine. Although not mandatory but it is recommended to use virtual environment.

Install Django. Setup a database. We will be using MySQL.

– Create a project.
$ django-admin startproject elasticsearchdjango

– Change directory to elasticsearch. Create a new app ‘poll’.
$ python manage.py startapp polls

– If you are using virtual environment than install the dependencies using pip install -r req.txt  file after saving below lines in req.txt  file.

– Create a model in polls/models.py file.

Continue reading “Elastic Search with Kibana and Django – part 2”

Automatically updating Django website hosted on PythonAnyWhere server with every git push

Until now this is how I use to develop and deploy (update) code on PythonAnyWhere server.

  1. Make changes in code on my local machine.
  2. Commit and push the code to remote repository.
  3. Login to PythonAnyWhere server and start bash terminal.
  4. Pull the code from remote repository.
  5. Reload the web app from web tab.

Steps 3 to 5 are time consuming, repetitive and boring. So I thought of eliminating these steps.

In this article we will see how can you get rid of these steps and your web site is automatically updated with the code as soon as you push it to remote repository.

Continue reading “Automatically updating Django website hosted on PythonAnyWhere server with every git push”