Campaign Management System Design

Fake Company is going to build a campaign management system. And a part of it is sending campaign messages to users via email. In the Customer model of Fake Company, they have the user details (such as email, phone number…etc) of all their customers. The campaign will be launched on June 24, 2020. Therefore we’ll have to design a Campaign Management System through which we will launch various types of campaigns targeting many customers and let them know about the ongoing campaigns through email.

In this system, we will fetch the desired user’s information from the user database and take the email id and start sending them an email.

This whole process is triggered by a Cron Job which is executed at the 00:00 am. So we will write a routine in the cronwhich will trigger our fetch-item-from-db process. We can put the fetch-item-from-db and the cron routine in the same server to lessen the delay.

In this version, we have two separate processes P1 and P2 to handle the Email work and SMS work. And we are also fetching only the email and phone number of the user.

Looks good?

Well, if you want to send 100~300 emails, well then it seems good. But what if you want to send 20,000 emails? Then we have a problem.

If the email process (the triangle in the upper portion of the image) is an HTTP/HTTPS process, then we can send the email id one by one from our db-fetching-process to our email process. So what is the problem?

So, seems like this connection between the db-fetching-process and email process is a bottleneck of our system. So how can we mitigate this?

We can use a Message Oriented Middleware (MoM) here. Let’s use a FIFO Queue between these two connections.

Let’s look at the diagram first.

So there’s a Queue in between db-fetching-process and email process. What we are achieving with this?

We first collect all the desired users from the database and push all their email id one by one to the Queue. And our email process will pick them and send the email.

Here, our db-fetching-process is the publisher which is publishing messages to the queue and our email process is the consumer, consuming messages from the Queue.

So, with this how do we mitigate the above bottleneck?

Say, we have to send 20,000 emails. We get all the 20,000 email id from the database and start publishing them to the Queue. Now say after publishing 10,000 email id, our db fetching process is down. But now, we are not stuck. Because we have email id already inside the Queue, the email process won’t have to pause. It will execute its task. It will consume email id from the Queue and will send the mail. And like this, it just solved our issue.

If you look at our version: V3 image carefully, you can see that we have a new component here which is a Content Management System(CMS). System V1 and System V2 did not contain them. What’re its usages?

We have to send an email, right? So we have an email message body. And it can vary from campaign to campaign. So certainly we can’t hardcoded them in our email process. So that’s why we need a service where we can create, update, delete content. And our other services can get the content’s from this CMS.

So in our email process we will have the campaign id(inside our message that we are consuming from Queue) and with the campaign id, we can fetch our desired content which is the email message body from our CMS.

Looks good, right? Ok, so you developed this system and deployed in the production and you are now chilling. The very next day you are getting complaints from your customers that you are flooding their inbox with lots of email advertising a certain campaign. And now, you are sweating. What went wrong? The system looked good.

Let’s find out.

So identified 4 problems. Let’s see how can we fix this.

Let’s look at the diagram below:

Related posts:

Welcome to the Experience Invest blog. Experience Invest is a leading provider of high yielding, buy-to-let property investments. Are you thinking of investing in student accommodation for 2020? Then…

Sometimes you find yourself feeling very new Those days tend to seem a little blue But that’s magick screaming at you To embrace the humility behind your rues Inspect your mind for the colourful…

Imagine being 5 again sitting in front of the giant box tv in the living room and inserting the VHS tape of The Little Mermaid. Your younger self is enamoured with her fiery red hair and pretty shiny…