About educcess

Posts by educcess:

Platform Architecture in Computer Science: where do API Gateway & GraphQL fit

In our previous article we have seen why and how to design the platform as multiple apps running in independent server with their own database.

Read in Previous Article

API Gateway and Need of API Gateway in Platform Architecture

But for any platform the most important Managing the API Calls, what I mean here is there should be something in between clients and various services which will receive incoming requests, invokes various apis and return the appropriate results.

Also any backend platform should keep track of Api calls, rate limiting – it is defining the rate at which consumer can access apis, limit the number of api calls per second per user or per app, throttling – is controlling usage of apis for consumers for a given period of time, securing apis through api keys and user authentication.

in platform architecture API Gateway is like front facing for incoming client requests. Apigee is one very popular api gateway, however it is suitable for enterprise and big applications, for small apps and startups where one can’t afford Apigee, they can go for other API Gateways like Google Cloud End Points but it has very limited features OR Kong API gateway.

Boosting your Client Apps Performance and Stability by Adopting GraphQL

GraphQL is a query language for APIs, it is like giving control to clients/apps to fetch what exactly they want rather than decided by server.

Ask for What you need and Get exactly that 🙂

Send a GraphQL query to your API and get exactly what you want

<code>//GraphQL Query to API
{
  product: {
    name,
    images,
    price
  }
}

//This will be input to your API and you will get only that in //response quickly

//Response
{
  product: {
    name: 'Micro Max G6',
    images: &#91;],
    price: '6599 Rs'
  }
}</code>

Get Multiple Information in Single Request

GraphQL will make your apps lightning speed which even out perform greatly in slow network speed as well, because GraphQL is powerful and returns multiple information in single request, which apps typically get it from different api calls.

There are multiple bunch of benefits of GraphQL and its worth to consider using GraphQL while architecting your Platform. Well where GraphQL fits in your platform architecture ? See the updated architecture diagram

Continue Reading About Micro-Service Platform Architecture

Platform Architecture in Computer Science: With Multiple Apps Approach

There are many ways to design the platform architecture, however every platform has its own unique requirements. The architecture of every platform can vary totally to meet the needs of respective platform.

There are many case studies to look into platform architecture which I will cover in different article, we can look into linked-in, ebay, Amazon and netflix architectures to get better understanding on Platform architecture.

For now in this article we will understand first approach that Having Multiple Apps with its own independent Database and Backend. For our understanding purpose we will consider designing a platform for an Amazon Kind of Online Shopping Platform

Read Basics of Platform Architecture before further reading

Platform Architecture for Amazon Kind of Online Shopping Platform with Multiple Apps Approach

If you consider Amazon kind of Online Shopping Platform, we first have to identify which are the different huge modules which are candidates for making it as a separate app altogether

  1. Sellers or Vendors who sell the products
    1. Seller is the one who has his shop and sell products in amazon
    2. He should be able to add his products
    3. He should be able to manage discounts, offers for his products
    4. He should be able to dispatch his product
    5. He should be able to manage his revenue/profit and other finance aspects
  2. Buyers
    1. The one who buy products in Amazon
    2. He should be able to manage his Cart
    3. He should be able to manage his delivery addresses
    4. He should be able to do payment using his choice of payment mode
  3. Product Dispatch Management
    1. Tracking the product while dispatching from one place to another
    2. Every city has a collection centers and products has to be managed for timely delivery
  4. Products Delivery Management
    1. Once product is reached the destination city it has to be picked by delivery person
    2. Assign delivery person
    3. Deliver to customer and update the status
    4. receive cash in case is Cash-On-Delivery
  5. Amazon Finance Management
    1. This is kind of Super Admin Functionality
    2. Managing refunds
    3. managing failed transactions
    4. Transferring money to Seller

I listed only few functionalities however there will be many more under each of these main features.

Why Can’t be its one super big Monolithic Application ? Rather than multiple Apps ?

To answer this question we have to think of following important aspects which we covered in our first Article Which are as follows

  1. Scalability : For Amazon Kind of Applications, millions of requests will be coming every second. There are millions of users buying and millions of sellers selling and millions of products listed, millions of transactions happening. It is highly impossible to handle everything in one huge monolithic server, because every server has some upper limits with respect to hardware and capacity, one huge database to scale it to that level we have to divide it into multiple pieces and manage it independently.
  2. Maintainability: It is very hard to maintain one huge monolithic app for such a huge volume.
  3. Resilient: Any major update to particular part of the platform should not affect to other parts of the platform
  4. Development: For amazon kind of huge platforms, they have huge team spread across the world and each teach is enhancing their own module by doing lot of research within their small team. It is not possible to manage development activity effectively if its one huge monolithic app

Now let us see how the Architecture Diagram looks: You can use various apps for designing Architecture Diagram

In the above diagram, each app is one server and it has its own database. There are third party services used commonly across many apps, like

  1. Elastic Search Service : For faster product search, Elastic Database is must for better searching capability.
  2. Cloudinary is another service used for multi media file handling
  3. Authentication service is like firebase, auth0 for user authentication
  4. Many more such services will be used

Advantages of Multi Application Platform Architecture

Scalability: In this architecture the platform is scalable to handle high traffic, because during peak time like Amazon big billion day, more and more people will buy products, the traffic will be too high for Consumer App. So in that case if its independent server by itself, we can increase the capacity of that server alone to handle millions of requests, however other apps may not need so much so at this stage.

Continue Reading : Where API Gateway and GraphQL Fit in Platform Architecture

Platform Architecture in Computer Science: Different approaches and Benefits

What is Platform Architecture in Computer Science

A platform architecture is an abstract high level design of the platform that answers to some of the key important factors of any platform like scalability, resilience, maintainability. It should cover almost all important big modules, multiple apps, all the servers, databases, caching, third party solutions and etc.

A platform should be ever evolvable, should be able to adopt to the new changes and needs of the platform and it helps to meet all technical and operational requirements with focus on optimising performance and security.

Desirable Properties of Platform Architecture

  1. High level of Abstraction – It should be simple enough and comprehensive enough to get high level of understanding
  2. Resilient – Each major module (app) should not be get affected so much due to one defective app
  3. Scalable – Architecture should consider the scalability of the platform “Scalability” refers to the number of users, sessions, transactions, and operations that can be accommodated by the entire system
  4. Maintainable – Should be able to make changes to any part of the system without breaking or damaging too much to other parts
  5. Ever Enhancing – it should be able to evolve every-time when there is a need of change

Important Aspects to focus on while designing platform architecture.

  1. How many servers we need to maintain, is it per app level or every major functionality
  2. How caching is handled in platform
  3. API Gateways for API security, metering, throttling
  4. Searching functionality within the platform
  5. Users and Identity Management
  6. Security
  7. Databases
  8. File Servers
  9. Third Party Services if any
  10. Data query and Manipulation for faster data access like using GraphQL
  11. Communication between each apps or each servers using event queue, event bus or intermediate app whichever way it is
  12. Handling batch processes using cronjobs or cloud functions
  13. Data mining, AI System

For Different Approaches for designing Platform Architectures click here

Postgres Sequelize Invalid value Error: while executing Find Query

I have a Model Defined as

<code>const sequelizeHelper = require("../helpers/sequelizeHelper");
const db = sequelizeHelper.db;
const sqlOperator = sequelizeHelper.sqlOperator;

const NoticesModel = db.define(
  "notices",
  {
    noticeId: {
      field: "noticeId",
      type: sqlOperator.UUID,
      primaryKey: true,
      defaultValue: sqlOperator.UUIDV4
    },
    title : {
      field: "title",
      type: sqlOperator.STRING(250),
    },
    notice : {
      field: "notice",
      type: sqlOperator.TEXT,
    },
    isBroadcast : {
      field : "isBroadcast",
      type: sqlOperator.BOOLEAN,
      defaultValue: false
    },
    isDeleted : {
      field : "isDeleted",
      type: sqlOperator.BOOLEAN,
      defaultValue: false
    },
    isForAllStaff : {
      field : "isForAllStaff",
      type: sqlOperator.BOOLEAN,
      defaultValue: false
    } 
  },
  {
    tableName: "notices",
    timestamps: true
  }
);

NoticesModel.associate=()=>{
  const UserModel = require("./UserModel");
  
  NoticesModel.belongsTo(UserModel, {
    foreignKey : {
      name : "noticeBy",
      allowNull: false
    },
    // as : "noticeBy", //This is needed as there are multiple user mapping so sequelize will not get to know which one to use
    targetKey: "userId"
  });

  NoticesModel.belongsTo(UserModel, {
    foreignKey: "noticeTo",
    // as: "noticeTo", //This is needed as there are multiple user mapping so sequelize will not get to know which one to use
    targetKey: "userId"
  });

  const MasterNoticesStateModel = require("./MasterNoticesStateModel");

  NoticesModel.belongsTo(MasterNoticesStateModel,{
    foreignKey : {
      name : "noticeState",
      allowNull: false
    },
    targetKey : "stateId"
  });

  const MasterClassModel = require("./MasterClassModel");

  NoticesModel.belongsTo(MasterClassModel,{
    foreignKey: "classs",
    targetKey: "classId"
  });

  //Associating Notice to school
  const SchoolModel = require("./SchoolModel");
  NoticesModel.belongsTo(SchoolModel,{
    foreignKey: "schoolId",
    targetKey: "schoolId"
  });

  const MasterUserRolesModel = require("./MasterUserRolesModel");
  NoticesModel.belongsTo(MasterUserRolesModel,{
    foreignKey: "roleId",
    targetKey: "roleId"
  });
  NoticesModel.sync({ alter: true });
};

module.exports = NoticesModel;</code>

And a Sequelize $or conditioned Query I have is as follows

<code>const query&#91;"where"] = {};
const orConditions = &#91;];
if (noticeParams.noticeTo) {
      orConditions.push({
        noticeTo: {
          $eq: noticeParams.noticeTo
        }
      });
    }
    if (noticeParams.classId) {
      orConditions.push({
        classs: {
          $eq: noticeParams.classId
        }
      });
    }
    if (noticeParams.roleId) {
      orConditions.push({
        roleId: {
          $eq: noticeParams.roleId
        }
      });
      orConditions.push({
        isForAllStaff: {
          $eq: true
        }
      });
    } else if (noticeParams.isForAllStaff) {
      orConditions.push({
        isForAllStaff: {
          $eq: noticeParams.isForAllStaff
        }
      });
    }
    query&#91;"where"] = {
      $or: orConditions
    }</code>

Postgres Sequelize Exception while executing Find Query

Error: Invalid value { noticeTo: { ‘$eq’: ’02faac43-a777-4bb8-902b-da0eebbcd70e’ } }
at Object.escape (/Users/rama/Data/Development/Social/vk_proj/source/server/node_modules/sequelize/lib/sql-string.js:65:11)
at PostgresQueryGenerator.escape (/Users/rama/Data/Development/Social/vk_proj/source/server/node_modules/sequelize/lib/dialects/abstract/query-generator.js:986:22)
at _joinKeyValue.value.map.item (/Users/rama/Data/Development/Social/vk_proj/source/server/node_modules/sequelize/lib/dialects/abstract/query-generator.js:2486:69)
at Array.map ()
at PostgresQueryGenerator._whereParseSingleValueObject (/Users/rama/Data/Development/Social/vk_proj/source/server/node_modules/sequelize/lib/dialects/abstract/query-generator.js:2486:52)
at PostgresQueryGenerator.whereItemQuery (/Users/rama/Data/Development/Social/vk_proj/source/server/node_modules/sequelize/lib/dialects/abstract/query-generator.js:2268:19)
at Utils.getComplexKeys.forEach.prop (/Users/rama/Data/Development/Social/vk_proj/source/server/node_modules/sequelize/lib/dialects/abstract/query-generator.js:2173:25)
at Array.forEach ()
at PostgresQueryGenerator.whereItemsQuery (/Users/rama/Data/Development/Social/vk_proj/source/server/node_modules/sequelize/lib/dialects/abstract/query-generator.js:2171:35)
at PostgresQueryGenerator.getWhereConditions (/Users/rama/Data/Development/Social/vk_proj/source/server/node_modules/sequelize/lib/dialects/abstract/query-generator.js:2583:19)
at PostgresQueryGenerator.selectQuery (/Users/rama/Data/Development/Social/vk_proj/source/server/node_modules/sequelize/lib/dialects/abstract/query-generator.js:1315:28)
at QueryInterface.select (/Users/rama/Data/Development/Social/vk_proj/source/server/node_modules/sequelize/lib/query-interface.js:1122:27)
at Promise.try.then.then.then (/Users/rama/Data/Development/Social/vk_proj/source/server/node_modules/sequelize/lib/model.js:1759:34)
at tryCatcher (/Users/rama/Data/Development/Social/vk_proj/source/server/node_modules/bluebird/js/release/util.js:16:23)
at Promise._settlePromiseFromHandler (/Users/rama/Data/Development/Social/vk_proj/source/server/node_modules/bluebird/js/release/promise.js:547:31)
at Promise._settlePromise (/Users/rama/Data/Development/Social/vk_proj/source/server/node_modules/bluebird/js/release/promise.js:604:18)
at Promise._settlePromise0 (/Users/rama/Data/Development/Social/vk_proj/source/server/node_modules/bluebird/js/release/promise.js:649:10)
at Promise._settlePromises (/Users/rama/Data/Development/Social/vk_proj/source/server/node_modules/bluebird/js/release/promise.js:729:18)
at _drainQueueStep (/Users/rama/Data/Development/Social/vk_proj/source/server/node_modules/bluebird/js/release/async.js:93:12)
at _drainQueue (/Users/rama/Data/Development/Social/vk_proj/source/server/node_modules/bluebird/js/release/async.js:86:9)
at Async._drainQueues (/Users/rama/Data/Development/Social/vk_proj/source/server/node_modules/bluebird/js/release/async.js:102:5)
at Immediate.Async.drainQueues as _onImmediate
at runCallback (timers.js:705:18)
at tryOnImmediate (timers.js:676:5)
at processImmediate (timers.js:658:5)
at process.topLevelDomainCallback (domain.js:126:23)

Solution for Postgres Sequelize Invalid value Error:

Very strange is the query written above is as per the standards of sequelize library, however its throwing a strange exception. I fixed by changing the query as follows

NOTE: instead of using $or and $eq, I am using sequelize op ( operator )

Update: As per latest sequelize library v4.x, For better security Sequelize recommends dropping alias operators $ (e.g $and, $or …) – Reference

<code>const Op = require('sequelize').Op

const query&#91;"where"] = {};
const orConditions = &#91;];
if (noticeParams.noticeTo) {
      orConditions.push({
        noticeTo: {
          &#91;Op.eq]: noticeParams.noticeTo
        }
      });
    }
    if (noticeParams.classId) {
      orConditions.push({
        classs: {
          &#91;Op.eq]: noticeParams.classId
        }
      });
    }
    if (noticeParams.roleId) {
      orConditions.push({
        roleId: {
          &#91;Op.eq]: noticeParams.roleId
        }
      });
      orConditions.push({
        isForAllStaff: {
          &#91;Op.eq]: true
        }
      });
    } else if (noticeParams.isForAllStaff) {
      orConditions.push({
        isForAllStaff: {
          &#91;Op.eq]: noticeParams.isForAllStaff
        }
      });
    }
    query&#91;'where'] = {
      &#91;Op.or]: orConditions
    }</code>

PostgreSQL: ERROR: SequelizeDatabaseError operator does not exist character varying = bigint1

Getting this Error for find query using sequelize library in nodejs:

Error: SequelizeDatabaseError: operator does not exist: character varying = bigint1

My Sequelize Database Model looks as follows

<code>const sequelizeHelper = require("../helpers/sequelizeHelper");
const db = sequelizeHelper.db;
const sqlOperator = sequelizeHelper.sqlOperator;
const UserRolesModel = require("./UserRolesModel");

const UserModel = db.define(
  "users",
  {
    userId: {
      field: "userId",
      type: sqlOperator.UUID,
      primaryKey: true,
      defaultValue: sqlOperator.UUIDV4
    },
    firstName: {
      field: "firstName",
      type: sqlOperator.STRING(100),
      require: true
    },
    lastName: {
      field: "lastName",
      type: sqlOperator.STRING(100),
    },
    email: {
      field: "email",
      type: sqlOperator.STRING(1025),
      validate: {
        isEmail: true
      }
    },
    userName: {
      field: "userName",
      type: sqlOperator.STRING(1025),
    },
    phone: {
      field: "phone",
      type: sqlOperator.STRING(13),
    },
    photo: {
      field: "photo",
      type: sqlOperator.STRING(1024),
      defaultValue: null,
      validate: {
        isUrl: true
      }
    },
    studentDetails: {
      field: "studentDetails",
      type: sqlOperator.JSON,
      defaultValue: null,
    },
    profession: {
      field: "profession",
      type: sqlOperator.STRING(13),
      defaultValue: ""
    },
  },
  {
    tableName: "users",
    timestamps: true,
    indexes: &#91;
      {
        unique: true,
        fields: &#91;"email"],
        name: "user_email_unique_index"
      },
      {
        unique: true,
        fields: &#91;"userName"],
        name: "user_user_name_unique_index"
      },
    ]
  }
);

UserModel.associate = () => {
  UserModel.hasMany(UserRolesModel, {
    as: "roles"
  });
  UserModel.sync({ alter: true });  
}
module.exports = UserModel;
</code>

Solution:

This kind of issue usually occur when we try to compare mismatched data types. You cannot compare an integer with a varchar. PostgreSQL is strict and does not do any magic typecasting for you. I’m guessing SQLServer does typecasting automagically (which is a bad thing).

<code>const phone = 9998886660;
UserModel.count({
      where: {
        phone: phone
      }
    });</code>

The above code throws operator does not exist: integer = character varying exception, because we are trying to compare integer with string. To fix this issue change as follows

<code>const phone = "9998886660"; //this will fix the issue
UserModel.count({
      where: {
        phone: phone
      }
    });</code>

For other sequelize errors: Read More Here

Server Side Rendering using React16, material-ui and redux Video Tutorial

Complete Source Code for server side rendering in ReactJS using material-ui and redux is explained step by step here

You can also see details about each code segment and download source code from github here

Life Beyond Structure, Analysis, program and computers

I found this wonderful video and found the meaning of life, you may be a programmer, a civil engineer, a car maker, a researcher, a teacher or a student whoever you may be, but one must listen to this talk by Prof. Devdas Menon, Department of Civil Engineering, IIT Madras

Some of the very interesting point taken out from this video are as follows

George Calvin’s famouse quote in 80’s
The paradox of our time and history is that


We tall the buildings but short the temper,
wide our free ways but narrowed our view point
we spend more but we have less
we buy more but we enjoy less
we have bigger houses but smaller family
we have more knowledge but less judgement
we have more experts but yet more problems
we have more medicines but less wellness
we multiplied our positions but reduced our values
we talk too much , drive too fast , stay up too late, getup too tired, we have reached moon and back to earth but trouble crossing a road to meet a neighbour,
we have conquered outer space but not inner space
we have done larger things but not better
we have learnt to rush but not to wait
we have fast food but slow digestion
a big men but small character
too many young couple but more divorces
more entertainment but less happiness

MASLOW’s Hierarchy Of Needs

Matthieu Ricard’s Quote

A French Scientist, who turned as tibetan monk

We all tried consciously or unconsciously, competently or clumsily , Passionately or calmly, adventurously or routinely to be Happier, Yet we more often confuse the genuine happiness with merely seeking enjoyable sensation. Happiness is state of inner fulfilment but not the gratification of inexhaustible desire for outer things. This is traditional wisdom.

Kabir Das Said it beautifully in 15th century in one of his famous doha as

कस्तूरी कुंडल बसे,मृग ढूँढत बन माही। ज्योज्यो घट– घट राम है,दुनिया देखें नाही ।।

Listen to this nice video by Prof. Devdas Menon, Department of Civil Engineering, IIT Madras