Django: Create a Publish/Subscribe Service

You ever heard of publish/subscribe? Well if you know the basics and how cool it is, And you are using Django as your Backend but need a pub/sub service beside Django, to publish data on a message broker or subscribe to topics or messages received in a message broker, Well I guess this article could help you.

As you know, Django is an synchronous framework, which means that tasks will be executed in a blocking manner.

For example if a request to an url of django is made, the response will be generated after all lines of code in that block (function or class in views mapped to that url) is executed. So in first glance, It’s not suitable to create a publish-subscribe system with it. But in this article will show you how to publish your data to your message broker (will be using redis in this example) and how to subscribe to them in Django.

Publish

Imagine you have a micro-service and have lot’s of services that share data in an async manner with a publish/subscribe method. For example you have a shared message broker (redis, rabbitMQ, kafka, …) for all your services, and each app will publish it’s data to that message broker and all other services will subscribe to their needed topics.

To achieve this and be able to publish data in your django apps, first install needed packages. As i mentioned in our examples we are using redis as message broker. So install packages django-redis and redis . Then create a module for example named pub.py and add code below in it:

import json
import redis
redis_client = redis.StrictRedis(host='localhost', port=6379, db=1)
def publish_data_on_redis(json_data, channel_name):
    redis_client.publish(channel_name, json.dumps(json_data))

This function, simply will connect o redis (notice that we are connecting to database 1 of that redis) and will publish your data (will dump it to string from json before sending it) to specific channel you want. And whenever you want to share any data with other services, you can simply just use this method.

For example you have a web socket service and all users are connected to it. So if any event is received from other services, they will be notified via web socket connection. So in your django project when something is created and you want web socket users be notified, you can publish your data like below to the channel (topic) web socket service is subscribing to:

json_data = {'message': 'Hello to all connected clients', 'date': '2019-02-02', 'title': 'welcome'}
publish_data_on_redis(json_data, 'notification.new')

This will publish a data to redis on notification.new channel. To be sure that data is published, you can check it in redis:

$ redis-cli
> psubscribe notification.*

this will subscribe to topic notification, and whenever a data from djagno is published to that channel, will get it.

Subscribe

What if you have a micro-service, and you want to be notified in django from other services. For example You have another service, which is handling web socket connections of online users. So whenever a new user is connected via web-socket, you want django be notified with async communication as mentioned earlier. (I know you can use django-channels to create a web-socket and not use external service for that. I’m just saying for example; Also in micro-service design, You shouldn’t handle all the good things with just one service..)

To achieve this, As I said in top of this article, Django isn’t the best framework to do it; But if you insist on doing it, Then create a manage.py command for example subscriber.py and add code below in it:

class Command(BaseCommand):
    def handle(self, *args, **options):

        r = redis.StrictRedis(host='localhost', port=6379, db=1)
        p = r.pubsub()
        p.psubscribe('user.*')
        for message in p.listen():
            if message:
                m_type = message.get('type', '')
                m_pattern = message.get('pattern', '')
                m_channel = message.get('channel', '')
                data = message.get('data', '')
                if m_channel == 'user.connected':
                    print('A new user is connected')
                    # Published data of that user => `data`

So if you run this command, It will subscribe to channel user forever and will receive all topics in that channel. For example if a new user is connected and data is published to user.connected , Django will receive it like this and will handle that data.

And for deployment, Just simply put this code in a service of Linux (systemd), or Use supervisor or any deploy tools of your interest.

Hope this article was useful. peace ;)