Serialization and Deserialization for backend engineers

what is serialization and der serialization okay let's look at this we have already discussed this that usually we have a client let's say we have this client it's a browser all right it's a

browser let's say it's Chrome and this is called front end right in our typical technical language and usually we communicate with a server that's running in your Local Host

or it's running somewhere remote in cloud in AWS or gcp or Azure anywhere and the client communicates with the server through some means of network communication it can be through HTTP

right our traditional rest API end points or it can be through grpc or it can be through websocket we have different means of communications for clients and servers to communicate with

each other for this example and throughout the series since we are focusing on HTTP or our traditional rest API style communication let's assume this client is a Javascript app can be a

react or angular or view any JavaScript framework or Library it's a Javascript app usually most of the client side apps are built with JavaScript highly interactive app in order to send a

request to the server the client has to make an HTTP request which we have already discussed in the previous videos looks something like this get etc etc the URL the

HTTP right some domain.com slath something like this so either a get request or if it's a post request we have the URL and Below we have the

headers and below that we have the request body and this is how front end sends the data the request with all the information that is required or the client wants to send to the server this

is how client sends information to the server and the server responds with some response which the client is able to pass and make sense of so the thing to notice here and the thing to focus here

in this chapter or in this session is how clients and servers can make sense of the data that is being received or the data that is being sent for example in our case the client is a Javascript

app and the server can be a rust app right the web server is built with rust language so in JavaScript let's say we are sending this object right and we have

name is some string we are sending this and after we send this in the request body and it reaches the server which is a rust

server how does the server take this data and is able to make sense of it it is able to make sense of it in its own format in its own data types because JavaScript and rust have completely

different data types JavaScript is a completely Dynamic language it is not compiled on the other side rust is very strict when it comes to types and it is a compiled language the all the data

types are completely different from JavaScript so how does this happen how one data type that is transmitted over a network is able to reach another machine that is somewhere else in the internet

and it is able to make sense of that data and perform some business logic and send some response and the client is again able to make sense of that response and able to show

some UI or perform some business logic how does this all come together so before we talk about data transmission over the Internet it's very good to have some knowledge about OSI

model if you're a CS student you might have already come across it if you're not then you don't have to completely Master it or understand every aspect of it like IP packets or data frames etc

etc but having a good high level overview or a high level understanding of how these OSI layers different layers of network communicate with each other over

the internet to transmit some kind of data over Hardware right we have different types of layers called application layers and network layers right there

are a lot of good resources in YouTube or any other platform that you can look up and just get a high level understanding of OSI model and going too deep into this is out of scope for this

video and will probably take too much time and people who are way better than me have explained this in a way better format and now assuming you have a high level understanding of OSI models or

just are aware of what this model is and how it helps make sense of data transmission over the internet now the layer on top is called application layer and after all the layers the

bottom layer is called physical layer or you can call it top call it bottom does not matter right these two are on the in the opposite end of each other right and the same thing happens on the server

side here in the server we have application layer and in the bottom we have physical layer and there are layers in between now

imagine if you were asked to solve this problem the problem that we have two machines that are in different locations and they are connected over

the internet right let's say over the cloud and you are asked you have to come up with some way or some technique or some protocol to go with the tech technical terminology you have to come

up with a way so that clients or one machine is able to send data to another machine and they should be able to parse that data and make sense of it in a way that it is language agnostic and a very

obvious and common solution that you might come up with is figuring out a common standard which is just a fancy way of saying you will come up with some set of

rules and you will ask both the clients and the server to agree that this is the standard or this is the format that the client should send the data in or the server should send the response in so

that client does what it needs to do during the execution of the session of the user and when sending data to the server it transforms or converts whatever the logic is from that language

let's say from JavaScript to this standard this format and sends it over the network and when receiving the server takes that data from that standard and converts it into

its own standard let's say it's rust so it converts it into a struct in Rust and it understands the data format and whatever it needs to do and when sending the response it converts that data let's

say from rust truct into this standard this common format and sends it over the internet and again the client reads it and parses

it into JavaScript and it's it's able to do whatever it needs to do this part this is a solution what is the solution the solution is coming up with a serialization

standard common standard that the both client and the server agree to send and receive data and to summarize this whole thing realization and derealization is basically converting to and from data to

a common format so that during transmission or storage we'll get to the storage part but from this example we can say that the serialization and deserialization is basically converting

data from and to a common format while transmission so that it is domain agnos tic or in this case language agnos that machines and servers and clients from different domains and different

languages from different environments are able to make sense of that data and are able to respond to that data and before we dive deep into the serialization standard I just want to

make a note here that which I have already mentioned that backend is a huge domain and there are a lot of Technologies hundreds of technologies that is used in the industry so it's

impossible to understand the concepts behind all of those Technologies what we will do in this video and in this whole playlists we'll go with the most used Technologies for example for the data

transmission there are a lot of Technologies getting used for example HTTP and websockets and you have grpc etc etc so we'll go with HTTP since rest apis are still the most common

means of communication over the Internet between clients and servers and as a backend engineer if you are aware of and have good experience in HTTP then you're good to go you can learn all these other

Technologies along the way you just need something which is used all of the industry to get started and get some experience and you can learn Technologies as you progress over your

career in the same way for databases we have a lot of different types of databases we have relational right we have non- relational in relational we have

postgress we have MySQL we have sqlite right same way in non- relational we have mongodb we have Dynamo DB from

AWS a lot of different options for databases but in this playlist we'll go with post gra right it's one of the most popular relational databases and is used by a lot of companies and mostly the

first choice these days for a lot of startups and Enterprise servers in the same way we have different types of serialization Standards

serialization standards and we will go with a popular one the one that gets used I won't give a number but if I had to given number I'll say 80% of the time which is

Json is one of the most popular serialization and deserialization standards that gets used for client and server communication we have other standards

for serialization der serialization that does not necessarily gets used for HTTP communication but definitely gets used for serialization for example we have yl we have XML and inside serialization

standards these three including Json and XML yaml these things are called text based serialization right we have another type

of serialization standard which is called binary format binary format and in binary format we have all again different types and the most popular one is called

protuff we have other formats also like a etc etc for serialization standards we have text based and binary format and inside text base we have different types called Json yaml XML and for now we will

only focus on Json since that's the most popular choice for sttb based communication between clients and servers now that's clear let's jump into Json and this is what a typical

Json object looks like so to give you a little context Json stands for JavaScript object notation and you can read more about Jon's history and its Evolution from

other resources but just to give an idea of Json from the looks of it and from the behavior looks very similar to a JavaScript object and

and that's why you can understand why it's name like that JavaScript object notation but it is not limited to JavaScript and it is used everywhere you will find Json in configuration files

and of course uh during HTTP rest API transmission between clients and servers you will find Json in log file Json is a very popular choice for logging application data or server data during

runtime it has a lot of use cases for this use case we are going to focus on transmission of course this is what a typical Json looks like in the looks of it you can understand it's completely

human readable and that is one of the strong points of Json it was made to be human readable and the data types are also pretty fundamental if you are even a little familiar with different

programming concepts for example things to focus here is we have a starting CIS and then ending one and all the keys are inside double codes so this is one rule you have to follow the keys will be

inside double Cotes and they will be strings you cannot make any other data type as a key keys will be inside double codes and there will be strings and in the value it can be a string or it can

be a number it can be a bullion or it can be an array or it can be another object right Json can be a nested object also for example let's say you have here an address field inside the same Json

you have a column here and you can start another crais here you can write country you can say it's a string it's India and your phone number right phone number and it can be a number something

3 4 5 6 and you have to provide an ending CIS right this is the typical structure of Json that's all there is to it uh you would find a lot of Concepts in Json

these are the things you have to remember there has to been starting Braes and ending craes and the keys will have to be inside double quotes and there will there will be strings and in

the value you can provide an string a number a bullion or an array or a nested object which will again have the same characteristics of the initial object right there'll be Keys which will be

inside double quotes and there'll be strings and they will have values again in the same characteristics so that's all you need to know about Json this is one of the most popular

serialization standard that we use during client and server communication so let's just see a simple demo in fact let's just see our earlier demo that we saw in the previous video and this time

we'll focus on the request and response format the serialization and deserialization format and see how it looks like pretty much looks like the same thing that we saw here but let's

just see in a real environment here we have our BP Su interface and this is the earlier demo that we saw in our previous video but this time we'll hit an API and focus on the data format that gets

transmitted and the data format that gets received from the client and from the server so let's just hit the post API that was here post books and let's go to http history and

open this one and let's expand this and go to the request and L this is a post request and we are calling this path API SL books this is the data that we are sending so

let's just focus on the request first here we can see our earlier discussed Json format we have a crais and we have all the keys that are double CED and those are strings and we have a number

here and two strings like ID title and author this thing gets transmitted over the network so now one thing that I want to mention here is if you went deep into the OSI model so you must have come

across that from application layer to it gets converted into Data frames and etc etc it gets converted into IV packets and there is transmission layer and in the end it gets converted into bits with

0 one 01 stuff stuff Network related stuff right so as a backend engineer the thing you have to focus here is the mental model you should have is you can assume that you have this client and you

have this server right the client has to adhere to a particular standard let's say this is Json and the server has to understand this standard called Json we transmit

this the server understands this and returns some data again in Json format with respect to your knowledge of OSI model which has different different layers which the data gets converted

into different different formats from data frames to IP packets to physical bits that gets transmitted D with voltage signals over Optical F etc etc the mental model to have is in

application layer gets converted into Json as a backend engineer you don't that's all your responsibility is does not matter what format it gets converted into after this point that is not of

your concern so it gets converted into different different formats and then in the end 0 1 01 and then again it reaches the server reaches 0 1 01 that's it reads all the bits and does all the

network related stuff then it again gets converted into Json only in the server side also does not matter what the intermediary steps convert the data into that does not matter before receiving

before in the receiving end of the server the data again gets converted into this Json format only and the server only gets to read the Json so you don't have to worry about all the

intermediary steps now that's clear this is the request Json that the client sends and the server understands this and it takes this ID and all these information and adds it here and

responds with another Json right it's another Json so again we have the opening calias and ending calias we have one key which is an array and each array has an object

again it is also in Json format you can see all the keys are in double codes and it has values in number and string formats and this is an array and it's received by the client and the client

understands it and if you see here we are rendering it here so the client is able to receive this data from the server and it is able to make sense of it it is able to pass it and it is able

to render it in its own way right in this case it is Javascript but it can be PHP also etc etc right this is how Json flow looks like in client server model in real life scenarios it does not have

a lot of Concepts realizing in dis realization do not have a lot of Concepts it is a phenomena in the backend world that that exists you have to know that it exists and Json getting

transmitted by the client to the server the server passing it understanding it making sense of it and responding with appropriate data to the client and again the client passing it understanding it

making sense of of it this whole flow is called serialization and der serialization so again to summarize

serialization and deserialization are the techniques using which data is converted into a common format a standard format so that clients can send data to the server in that format and

the servers can receive that data make sense of pass it and perform the business logic and send the data again in that common format so that the client can again do the same thing realization

and dation are basically dealing with standard format so that data is understandable across domains and language that's all there is to it in serialization and deserialization