0:00 Transitioning from a mid-level developer 0:01 to a senior engineer requires shifting 0:04 your focus from simply writing code to 0:07 mastering high-level architectural 0:09 design. This video breaks down the 0:11 essential roadmap for building scalable 0:15 production-ready systems from the ground 0:17 up, covering everything from API 0:20 protocols to database selection. You 0:24 will learn the core differences between 0:26 REST, GraphQL, and gRPC, as well as the 0:30 strategies for scaling through load 0:32 balancing and robust security practices. 0:35 Hayek developed this course. Most 0:37 developers cannot design systems or 0:39 features from scratch. They can add to 0:42 someone else's architecture with tasks 0:44 with clear requirements and already on 0:47 mature systems. But, if you ask them to 0:49 design something from the ground up, 0:51 most of them usually will freeze. And 0:54 actually, that is the exact skill that 0:56 separates mid-level developers from 0:58 seniors, because seniors are also able 1:00 to make decisions, design tradeoffs, 1:03 design the architecture from scratch, 1:06 and make decisions with rough 1:08 requirements. So, companies are not 1:10 paying six-figures for people who can 1:13 just code or follow instructions, but 1:15 they are paying for architectural 1:17 decisions, for making the system 1:19 performant, for optimizing the data 1:21 storage, and making the decisions that 1:24 also affect the customers and the 1:27 software that they are building. 1:29 So, in this video, I'm going to teach 1:30 you the exact concepts that I mastered 1:33 to be able to design such systems from 1:35 scratch and also get to senior roles. 1:38 This is how I passed the system design 1:40 interviews without any problems, and 1:42 these are the skills that I learned to 1:44 get to senior level within the second 1:47 year of my career. So, I'm not teaching 1:49 you some theory from books or from 1:51 newsletters. This is what actually works 1:53 in real jobs and in real interviews. 1:56 So, let's jump into my computer to see 1:58 what we are going to cover in this 2:00 course. First of all, we will start from 2:02 the foundations, the core concepts that 2:04 you need to understand before anything 2:06 else in system design. 2:08 Then, we'll get to API design, which is 2:10 a big part of designing systems, like 2:13 how to actually design APIs that scale 2:15 and also make sense for other developers 2:18 who'll be using it. 2:19 Then, we'll get into databases, how to 2:21 choose the right database for different 2:23 scenarios, and design your data layer 2:26 properly. 2:27 Next, we'll get into caching, how to use 2:30 caching, how to use CDNs, load balancing 2:32 to make your systems fast and reliable. 2:36 Then, we'll get into big data 2:37 processing, because that's a big topic 2:39 in itself, how to handle large-scale 2:42 data the right way. 2:43 Then, we'll get to designing for 2:45 productions, like how to build systems 2:48 that actually work in the real world, 2:50 not just on your laptop or on a single 2:52 machine. 2:53 And lastly, you can see how I'm 2:55 designing the systems for interviews and 2:58 handling this step, so that you can nail 3:00 these interviews and get the offers that 3:02 you need for getting to senior roles. 3:05 Designing a system to support millions 3:07 of users is challenging, but every 3:09 complex system starts with something 3:11 simple. That's why in this lesson, we'll 3:13 build a basic setup that supports just 3:15 one single user, and then we'll 3:17 gradually expand it as we go, because 3:19 starting small allows us to understand 3:21 each core component before adding more 3:23 complexity. So, let's start with the 3:25 first step and build a single server 3:28 setup. Imagine that we're setting up a 3:30 system for a small user base. This means 3:32 that everything runs on one single 3:34 server, the web application, the 3:36 database, the cache, and also the other 3:39 components. And this setup allows us to 3:41 visualize the core workings without 3:43 added complexity. Now, let's break down 3:46 how this single server setup handles the 3:48 user requests. We have some users who 3:51 are trying to access our website or our 3:53 API on the server. They can be either 3:56 using the web browser or a mobile app to 3:58 access our server. And on the other 4:00 hand, we have our server, which has the 4:02 necessary files to serve to the web 4:04 browsers and also the necessary API 4:07 endpoints to serve to the mobile app. 4:09 And it is hosted on this example IP 4:11 address. Initially, our users don't have 4:14 this IP address, they have the domain 4:16 which they are trying to access. Let's 4:17 say it's app.demo.com. 4:20 So, if they just type this domain name 4:21 and hit enter, their web browser, for 4:23 example, will contact the DNS, which 4:26 stands for domain name system. This is a 4:28 provider which maps the domains to the 4:31 IP addresses. And in our case, let's say 4:33 our domain name is mapped to the IP 4:36 address, which is the server's IP 4:37 address that we have. So, now this DNS 4:40 provider will send the IP address back 4:42 to the web browser or to the mobile app, 4:45 to our clients. And this IP address is 4:47 our server's IP address, so now they 4:49 have the location where they are trying 4:51 to send requests. So, with this IP 4:54 address in hand, the user's device sends 4:56 an HTTP request to our server asking for 4:59 specific data. And then our server 5:01 processes this request and sends back 5:04 the requested data. This might be an 5:06 HTML page for a browser or a JSON 5:08 response for the app, depending on the 5:11 request type. In this setup, traffic 5:13 usually originates from two main 5:15 sources. The first one is the web 5:17 applications, and the second one is the 5:19 mobile applications that are trying to 5:22 access our server. For our web users, 5:24 the server handles the business logic, 5:26 data storage, and also presentation 5:29 using HTML, CSS, and JavaScript. And for 5:32 mobile users, communication typically 5:34 happens over HTTP. These mobile apps 5:37 request data from the server using API 5:39 calls, and JSON is often used for 5:41 responses because it's lightweight and 5:43 easy for mobile devices to interpret. 5:46 Here is an example API request that we 5:48 can receive for our server. It can be a 5:51 get request to our domain/products/the 5:54 ID of that product. And for this 5:56 endpoint, we need to retrieve the 5:57 details of a product. And here is an 6:00 example response that we might send back 6:02 to the client. This is a JSON response, 6:04 which contains the product ID. It 6:06 contains the name of this product, some 6:09 description, the price of the product, 6:11 and some other metadata that is useful 6:13 for the client. And then this will be 6:15 used by the mobile app or by the web 6:18 browser to display this product on the 6:20 screen. And as we continue, our goal 6:22 will be to identify areas where a single 6:25 server might not be enough for the user 6:27 demand. For now, this setup is ideal for 6:30 small user bases, but it may struggle 6:32 under heavy traffic. So, next we'll 6:35 explore ways to scale each part of the 6:37 system to support more users 6:39 effectively. Some key takeaways that we 6:41 can have from this is that we need to 6:43 start small. We need to begin with a 6:45 straightforward single server setup to 6:47 understand the essential components of 6:50 system architecture. Now, we also 6:52 understand how these requests flow 6:54 through your system, which is 6:55 fundamental for building more scalable 6:57 systems. And we also recognize the 7:00 unique demands for web and mobile 7:02 applications and how they interact with 7:04 your server. And in the next lessons, 7:07 we'll start looking at strategies for 7:09 optimizing and scaling this setup. As 7:11 our user base grows, a single server 7:14 isn't enough to handle the increased 7:15 demand. And to accommodate more users, 7:18 we can separate our web tier, which is 7:20 handling the web and mobile traffic, and 7:22 the data tier, which is managing the 7:24 database. This setup enables us to scale 7:27 each server based on its specific load. 7:30 But when it comes to choosing the right 7:31 database, how do we know which specific 7:34 database is the best for our specific 7:36 application? When it comes to database 7:38 selection, there are two main options. 7:41 The first option is relational databases 7:43 or RDBMS, which are structured in tables 7:46 and rows. Some popular examples are 7:48 PostgreSQL, MySQL, Oracle database, or 7:51 SQLite. On the other hand, we have 7:53 non-relational or NoSQL databases. These 7:57 are suited for applications that require 7:59 flexibility and fast access to large 8:02 volumes of unstructured data. Some 8:04 examples are Cassandra, MongoDB, Redis, 8:07 or Neo4j. Let's start by exploring the 8:10 relational databases. These databases 8:12 use structured query language or SQL for 8:15 finding and manipulating data. The data 8:18 here is structured in tables, which are 8:20 the fundamental building blocks of SQL 8:22 databases, and these are similar to 8:25 spreadsheets. Each table consists of 8:27 columns, which can be thought as the 8:29 fields or attributes of the table. And 8:31 it also consists of rows, which are 8:33 single records within this table. For 8:35 example, if you imagine a customer's 8:37 table, within this table, we can have 8:39 columns like ID, name, age, and email. 8:43 And for each rows, we can have specific 8:45 customers, like the ID of 123, and the 8:48 name will be John, and the age will be 8:50 40, and so on. But, what are the 8:52 advantages of using an SQL database? 8:55 First of all, they support complex join 8:57 operations across multiple tables. For 8:59 example, if you imagine we have a 9:01 customer's table and also a product's 9:03 table. And now we want to create a 9:05 separate table that will connect the 9:07 customers and the products that they 9:10 have ordered. With SQL, you can join 9:12 these two tables together into an orders 9:15 table, and this will hold the 9:16 information about the customer IDs who 9:19 have this order, and also the product 9:21 IDs which this customer has ordered. And 9:24 this process of combining two or more 9:26 tables into one table are called join 9:29 operations in SQL. And the other big 9:31 advantage is they provide robust data 9:33 consistency and integrity, especially 9:36 for transactions. Transactions in SQL 9:39 are a sequence of one or more SQL 9:41 operations that are performed as a 9:43 single atomic unit, and each transaction 9:45 in SQL follows the ACID acronym. You can 9:48 think of a transaction example like a 9:51 bank transfer. So, first of all, all of 9:53 the transactions are atomic, which Which 9:55 that the entire transaction is treated 9:57 as a single unit, which either 9:59 completely succeeds or completely fails. 10:01 Each transaction is also consistent, 10:03 which means that it transforms the 10:05 database from one valid state to another 10:07 valid state. And they also come with 10:10 isolation, which means that 10:11 modifications made by concurrent 10:13 transactions are isolated from one 10:15 another, and they don't interfere with 10:17 each other. And lastly, they come with 10:19 durability, which means even if the 10:21 system fails or the database server 10:24 fails, the data will still remain there. 10:26 And now let's have a look at 10:28 non-relational databases. Non-relational 10:30 databases can be in different forms. For 10:33 example, we have document stores like 10:34 MongoDB, or you can use wide column 10:37 stores like Cassandra, key value stores 10:39 like Redis, and graph stores like Neo4j. 10:43 Let's have a look at each of these types 10:44 separately, and let's start with the 10:46 document stores. MongoDB is the most 10:49 popular example of a document store, and 10:51 the data here is stored in JSON-like 10:53 documents, which allows us to have 10:55 complex data structures within a single 10:57 record. Next, we have wide column 10:59 stores, where data is stored in tables, 11:02 rows, and dynamic columns. Some examples 11:05 here are Cassandra or Cosmos DB. The 11:07 main advantage of these databases is 11:09 they can handle massive scales and are 11:12 very good for many write operations. The 11:14 other option is graph databases, which 11:16 focus on storing the entities and their 11:19 relationships as graphs. An example of a 11:21 graph database is Neo4j. For example, in 11:24 Amazon, they used the Neptune graph 11:26 database, which helps them to make you 11:28 product recommendations based on your 11:31 previous orders. And the other popular 11:33 type is key value stores. Here, data is 11:36 stored in key value pairs. The biggest 11:38 advantage of key value stores is their 11:40 simplicity and speed. Since they are 11:42 primarily stored in RAM, reading and 11:44 writing to these databases is extremely 11:47 fast compared to other databases. Some 11:49 examples of key value stores are 11:51 Memcached or Redis. So, that's the main 11:54 four types of NoSQL databases. Now let's 11:57 have a look at the advantages of these 11:58 NoSQL databases. If you have a look at 12:01 the same example that we had for the SQL 12:03 databases, where we have customers and 12:06 products, and we want to join them in 12:08 orders. For example, in MongoDB, you 12:10 could have this as a single document, so 12:12 you could store all of the user data, 12:14 also the orders and products in a single 12:17 document. And because of this structure, 12:19 the NoSQL databases can handle highly 12:22 dynamic and large data sets without the 12:24 structure imposed by relational 12:26 databases. And also, they are optimized 12:29 for low latency and scalability. So, 12:31 when should you use relational versus 12:33 non-relational databases? Here is a 12:36 quick comparison of both. If your 12:38 application data is well structured with 12:40 clear relationships, then you should use 12:42 SQL databases. For example, if you have 12:45 an e-commerce application tracking 12:47 customers and orders, that's a good use 12:49 case of using an SQL database. Next, if 12:52 you need strong consistency and 12:54 transactional integrity. For example, if 12:56 you have a financial application or 12:58 banking system, then you should use the 13:00 SQL databases. However, if your app 13:03 demands super low latency for quick 13:05 responses, then you should go with 13:07 non-relational databases. Or if the data 13:10 is unstructured or semi-structured, like 13:12 JSON objects, and the relationships 13:15 aren't that crucial, then you should 13:16 also go with NoSQL databases. And 13:19 lastly, if your application requires 13:20 flexible and scalable storage for 13:23 massive data volumes. For example, a 13:25 recommendation engine storing user 13:27 activity data and key value format, then 13:30 you should also go with NoSQL databases. 13:32 Let's explore the two primary approaches 13:35 to scaling, which are vertical and 13:37 horizontal ways of scaling. And we'll 13:39 also see why horizontal scaling is 13:42 generally more suitable for high traffic 13:44 applications. First, we have the 13:46 vertical scaling, or sometimes it's also 13:49 called scale up. This just means that we 13:51 are adding more resources to our 13:53 existing server, meaning RAM, CPU, or 13:56 any other resources that might help us 13:58 to handle more traffic. And this 14:00 approach is simple and works well for 14:02 applications that have low or moderate 14:05 traffic. However, it comes with its 14:07 limitations, which are firstly, resource 14:10 limits. There is a hard cap on how much 14:13 you can add to a single server, and 14:15 eventually you will reach a limit on how 14:17 much you can upgrade your new server. 14:20 And the second reason is lack of 14:22 redundancy, meaning if this server goes 14:24 down, you don't have any other servers 14:26 to serve your users, which means that 14:29 your whole application goes down with 14:31 your single server. 14:32 On the other hand, we have horizontal 14:34 scaling, which is also sometimes called 14:36 scale out. In case of horizontal 14:39 scaling, we are just adding more servers 14:41 to share the load. So, instead of having 14:43 the single server, we might replicate 14:45 and have three of that same server. And 14:48 now we can share that load between these 14:50 servers instead of handling all of them 14:52 in a single server. Generally, this is 14:55 more suitable for large-scale 14:57 applications, as it comes with higher 14:59 fault tolerance. And higher fault 15:01 tolerance means if one of our servers 15:03 goes down, we still have two servers 15:05 available. So, these two servers can 15:07 continue serving our users while the 15:10 second server recovers from the failure. 15:12 And it also comes with better 15:14 scalability, because you can just add 15:16 more servers as needed. Instead of 15:18 having three, you might introduce a 15:20 fourth one, which will handle the new 15:22 incoming traffic. But how do we 15:24 implement the horizontal scaling? In 15:27 case of a single server, we know that 15:29 all of our client requests went to the 15:31 single server, whether it's from mobile 15:33 app or from the desktop. But what if now 15:36 we have three servers to handle all the 15:38 load? How do we distribute the client 15:40 requests? Let's say our mobile app makes 15:42 a request. How do we know where this 15:44 request should go? Whether it should go 15:46 to the server one or server two or to 15:49 server three? And it seems like we need 15:51 to have something in the middle, which 15:53 will direct the traffic to the 15:55 appropriate servers. And that part in 15:58 the middle is called a load balancer. We 16:01 use load balancers to distribute the 16:03 traffic across multiple servers. For 16:05 example, here we have three servers, 16:07 server one, two, and three. Whenever we 16:10 have a new request from the clients, the 16:12 load balancer decides where we have the 16:14 least load, and then it redirects the 16:17 traffic to that server. And it also 16:19 controls the fault tolerance, meaning if 16:22 one of our server goes down, like the 16:24 server three, it will stop sending 16:26 traffic to the third server since it's 16:28 not available anymore, and it will send 16:30 all of the traffic to server two and one 16:33 until the server three is available 16:35 again. And it also can make our app more 16:38 scalable, because we can introduce a new 16:40 fourth server and any other servers that 16:43 we want, and this load balancer will 16:45 ensure that all of the traffic is 16:47 distributed evenly. So, that's the two 16:49 main approaches of scaling, which are 16:51 vertical and horizontal ways of scaling. 16:54 In case of vertical scaling, we are just 16:56 adding more resources to our same 16:58 server, but in case of horizontal 17:01 scaling, we are adding more users to our 17:03 server base, and then we use a load 17:05 balancer, which distributes the traffic 17:07 across multiple servers. But right now, 17:10 this load balancer is kind of a black 17:12 box for us, because we don't understand 17:15 how does it work? How does it take the 17:17 requests? And how does it distribute the 17:19 traffic? So, let's explore that in the 17:21 next lesson, and let's see how this 17:23 exactly works and what are the 17:25 strategies that we use in load 17:27 balancing. Load balancers distribute the 17:29 incoming traffic across multiple 17:31 servers, while also ensuring that no 17:34 single server bears too much load. But 17:36 how does it actually happen? And how 17:38 does the logic work of distributing the 17:41 incoming traffic? To understand load 17:43 balancers better, let's explore seven 17:45 strategies and algorithms that are 17:47 commonly used in load balancing. 17:50 Let's start with round robin, which is 17:52 one of the most popular algorithms. 17:55 That's mainly because it's the simplest 17:57 form of load balancing, where each 17:59 server seen the pool gets a request in 18:01 sequential rotating order, which 18:04 basically means that the first request 18:06 that it receives, it directs it to the 18:08 first server, and the next request will 18:10 go to the second server, and the third 18:13 one will go to the third server. 18:15 And once the last server is reached, in 18:18 this case it's the server three, it 18:20 redirects it back to the first server, 18:22 and then again to the second server, and 18:24 so on. 18:26 This works well for servers with similar 18:28 specifications, meaning if all of our 18:30 three servers have the same capability, 18:33 then round robin will be a good choice 18:35 here. 18:36 Next option is the least connections 18:38 algorithm. It directs traffic to the 18:41 server with the fewest active 18:42 connections. For example, if we have 10 18:45 active connections on the server one, we 18:48 have nine active connections on the 18:49 server two, and we have 40 active 18:52 connections on the server three. 18:54 If it receives a new request from the 18:56 client, it will direct it to the server 18:59 two, because it has the least active 19:01 connections at the moment. So, now it 19:03 will have one more connection. And this 19:05 is particularly useful for applications 19:08 where you have sessions of variable 19:10 lengths, meaning that one of your 19:11 sessions might last 10 minutes, the 19:14 other one might last 1 minute, and so 19:16 on. And in this case, the load balancer 19:18 will take that into account, and it will 19:20 send the traffic to the least connection 19:22 server. The third option is least 19:25 response time. 19:26 This algorithm is more focused on 19:28 responsiveness of the servers. 19:31 Let's say our first server is highly 19:32 responsive, the second one is low 19:35 responsiveness, and the third one is 19:37 medium responsiveness. 19:39 In that case, the load balancer chooses 19:41 the lowest response time and with the 19:43 fewest active connections, meaning first 19:46 it will try to send as many connections 19:48 to the high responsive server as as 19:50 but it also takes into account the 19:52 active connections. Let's say this 19:54 server reaches 40 active connections, 19:57 then it will switch to the third server 19:59 because this is the medium 20:00 responsiveness server, and it will send 20:03 some traffic, let's say 20 other 20:04 requests to the medium responsiveness 20:07 server. And after that, it will switch 20:09 to the second server, and it might send 20:11 another 10 requests to this third server 20:14 until it redirects them back to the 20:16 first server. 20:17 This is effective when the goal is to 20:19 provide the fastest response time to 20:21 requests, and you also have different 20:23 servers with different capabilities. 20:26 The fourth option is the IP hash 20:28 algorithm, which determines which server 20:30 receives the request based on the hash 20:33 of the client's IP address. This is 20:35 useful when you want your clients to 20:37 consistently connect to the same server. 20:40 Let's say client one makes a request to 20:42 your load balancer. 20:43 The load balancer will use the client's 20:45 IP address, and based on this, it will 20:48 hash it and send it to a proper create 20:50 server, let's say server two, and all of 20:52 the future requests of the client one 20:54 will go to the load balancer, and it 20:57 will use the same IP hashing algorithm, 20:59 and based on this IP address, it will 21:01 again redirect the user one requests to 21:04 the server two. This is useful if it's 21:06 important for a client to consistently 21:09 connect to the same application. 21:11 If every of your server has some 21:13 information about the clients that are 21:15 connected to it, in that case, the IP 21:17 hashing is a good choice. 21:19 Then, there are also weighted 21:21 algorithms. These are variants of the 21:23 above methods that can be also weighted. 21:26 For example, you can have a weighted 21:28 round robin or weighted least 21:29 connections. 21:31 In this case, servers are assigned to 21:33 weights, typically based on their 21:34 capacity and performance metrics. 21:37 For example, if the first server has 16 21:39 gigs of RAM, the second one has 32, and 21:43 the third one has 64, 21:45 based on the server RAM and other 21:47 metrics, they are assigned to weights, 21:49 and the load balancer takes that into 21:51 account when redirecting the traffic. 21:54 First, it will try to send as many 21:56 connections to the first server as 21:58 possible because it's more weighted, 22:00 meaning it has more performance, and 22:02 then it will try to send the other 22:04 traffic to server two, and then the last 22:06 and small portion will go to server one. 22:09 There are also geographical algorithms, 22:12 which are location-based algorithms that 22:14 direct requests to the server 22:16 geographically closest to the user. 22:19 Let's say this application is for US 22:21 users, so mostly users are connecting to 22:23 this application from US, but we also 22:26 have some part of the users who are 22:28 connecting from Europe. And in our pool 22:30 of servers, we can have one server that 22:33 is located in US East, another server 22:35 that is located in US West, and the last 22:38 server can be located somewhere in 22:40 Europe for this small base of users who 22:43 are located in Europe. So, if a user 22:45 comes from Europe and makes a request to 22:47 this load balancer, it will redirect 22:50 this user to the server in Europe, or if 22:52 a user comes from your US and makes a 22:54 request to this load balancer, it will 22:57 check the location of this US user based 22:59 on its IP address, and then it will 23:01 redirect either to the US East or US 23:03 West. This type of load balancing is 23:06 useful for global services where latency 23:09 reduction is important. And the last 23:12 most popular type is consistent hashing. 23:14 In this case, we use a hash function to 23:17 distribute data across various nodes. We 23:20 have a hash function inside of a load 23:21 balancer, and we usually imagine a hash 23:24 space along with this that forms a hash 23:27 ring, like a circle. This hash function 23:30 forms a circle where we have the 23:31 servers, for example, the server one, 23:34 two, and three, which are located in 23:36 front of this load balancer. So, 23:38 whenever a new request comes from a 23:40 user, this hash function takes the IP 23:43 address of that user, and then based on 23:45 that, it locates this user on this hash 23:47 ring. Let's say it locates it somewhere 23:49 here, and then depending to which server 23:52 this point is closest to, for example, 23:54 in this case, this is closer to server 23:56 two, it redirects the traffic to that 23:59 server. 24:00 This is a bit more complicated way of 24:02 load balancing, but it also ensures that 24:05 the same client consistently connects to 24:07 the same server, like in case of IP 24:09 hashing. 24:10 We also talked about that whenever a 24:12 server goes down, this load balancer 24:14 ensures that traffic is not redirected 24:17 to that server. 24:18 But how does it know in the first place 24:20 that this server is not available? For 24:23 that, most load balancers come with 24:25 health check features, which means that 24:27 they are consistently monitoring the 24:29 servers by sending a health check 24:31 requests to all of these servers, and 24:34 they have the information about which 24:35 servers are online, let's say the first 24:37 three servers are available, and which 24:40 ones are offline, which means the fourth 24:42 server, which is offline. 24:44 So, whenever it detects a failure in the 24:46 health check, it knows that this fourth 24:48 server is not available anymore, and 24:51 based on that information, if the next 24:53 request comes from the client, it won't 24:55 redirect them to the fourth server until 24:58 the health check again succeeds and it 25:00 knows that the fourth server is back 25:02 online. 25:03 And now, let's see some load balancer 25:05 examples, and what are these actually? 25:07 How do we implement them? First, we have 25:10 software load balancers. For example, 25:12 Nginx is probably the most common type 25:15 of the software load balancer. 25:17 It has other features, and it's also 25:19 used as a web server, but it also offers 25:21 the functionality of a load balancer. 25:24 Typically, you install this Nginx on 25:26 your server, and then configure the 25:28 servers that should be load balanced, 25:31 and also the algorithm. And as you can 25:33 see, it also comes with health checks, 25:35 which I mentioned. So, you can set up 25:37 health checks among your servers, and 25:39 then this will consistently monitor your 25:41 servers, and whenever one of your server 25:43 goes down, it won't redirect traffic to 25:46 that server. 25:47 Another example of a software load 25:49 balancer is HAProxy, which is an 25:51 open-source software that again you can 25:53 install on your server and configure as 25:56 you want. But apart from software load 25:58 balancers, we also have hardware load 26:00 balancers. For example, we have the F5 26:03 load balancer, which is a widely used 26:06 hardware load balancer known for its 26:08 high performance and feature set. 26:10 Next, we have Citrix, which also comes 26:13 with load balancing functionality, and 26:15 again, this is a hardware type of load 26:17 balancer. 26:18 But if you don't want to configure all 26:20 of that yourself on your server or as a 26:22 hardware, then the easier solutions are 26:25 cloud-based load balancers. For example, 26:27 AWS comes with elastic load balancing, 26:30 and if you have your servers also set up 26:32 in AWS, then it's pretty easy to 26:34 configure this with your servers. And 26:37 you can also see it in the benefits that 26:39 it automatically comes with security, 26:41 automatic scaling, meaning that it will 26:43 automatically add new servers to the 26:46 pool if the demand increases of your 26:48 application, and it also comes with 26:50 monitoring, which is the same as health 26:52 checks. So, you don't have to set it up 26:54 yourself. And other examples similar to 26:57 AWS are Azure's load balancer and Google 27:00 Cloud's load balancing. Now, let's talk 27:02 about the concept which is called a 27:04 single point of failure in system 27:06 design. This is one part of your whole 27:09 system that whenever it fails, it will 27:11 bring the entire system down with it. 27:14 So, to put it simply, it is any 27:16 component that could cause the whole 27:18 system to fail whenever it stops 27:21 working. For example, if you imagine 27:23 this setup when the clients connect to 27:26 our load balancer, and then load 27:28 balancer distributes them to the APIs, 27:30 and then we have a single database which 27:33 is used for all API servers, database 27:36 here is one example of a single point of 27:39 failure. Whenever this database goes 27:41 down, all of these APIs won't be able to 27:44 connect to the database, and because of 27:46 that, all of these also won't function 27:48 properly, and our clients won't be able 27:51 to receive any response from the 27:53 servers. So, having single points of 27:56 failures in your system is problematic 27:58 because they can create vulnerabilities. 28:01 The first obvious downside is the 28:03 reliability because a single failure, 28:05 like the failure of this database, can 28:08 take the entire system down, which could 28:10 mean business losses because users are 28:12 not able to access our platform. Maybe 28:15 they are also not able to access the 28:17 checkout page or any other parts of the 28:19 system, which can bring losses in the 28:22 business. 28:23 It is also an issue for scalability 28:26 because systems that have single point 28:28 of failures like this can often struggle 28:30 to scale as each component will add a 28:33 risk of failing this single part. And 28:36 the last part, it also brings a security 28:38 issue because if you have a single point 28:40 of failure in your system, like the load 28:42 balancer, attackers can compromise this 28:45 point by sending huge traffic to it, and 28:48 if this fails, the whole system will go 28:50 down. 28:51 We will talk about how to avoid the 28:53 database single points of failure in the 28:55 databases section, but in this section, 28:58 we can have a look at how to avoid the 29:00 load balancers to become a single point 29:03 of failure because right now, we have 29:05 only one load balancer setup, and if 29:07 this load balancer goes down, then all 29:09 of our users won't be able to access 29:11 this point, and they will also not be 29:13 able to access to our APIs. 29:16 The first strategy is adding redundancy 29:19 to our system. This means that we can 29:21 use more than one load balancer, and for 29:24 example, if the second load balancer 29:26 goes down, users won't be able to 29:28 connect to this load balancer, but in 29:30 that case, we can redirect all of the 29:32 traffic to the first one, and then this 29:34 first load balancer will balance the 29:37 load between those servers, and we will 29:39 monitor the health of this second load 29:41 balancer, and whenever it's back online 29:44 and it's again available, we will also 29:46 redirect 50% of the traffic to the 29:49 second load balancer. 29:51 Another strategy is to use health checks 29:53 and monitoring for load balancers 29:56 themselves. As we saw, load balancer can 29:58 do health checks for the servers and 30:00 check whenever our servers are online or 30:03 offline. We can do the same strategy for 30:06 load balancers and we can check their 30:08 health continuously and whenever one of 30:10 our load balancer goes down, we will 30:12 know that we shouldn't redirect any 30:14 traffic to this load balancer until it 30:17 is back online. And the third common 30:19 type is self-healing systems, which 30:22 means that we again monitor the health 30:24 of our load balancer and if at any point 30:27 we detect that it goes down, we will 30:29 replace this with a new load balancer, 30:32 which is basically an instance of the 30:34 same load balancer and this way we won't 30:36 cause any interruptions and our clients 30:39 will be able to connect to this new load 30:41 balancer. 30:42 If you're finding this useful so far, I 30:44 have a lot more system design content on 30:47 my YouTube channel, including deep dives 30:50 into specific components and also case 30:52 studies designing systems from scratch. 30:55 Just search for Alex Simonyan on YouTube 30:57 or check out the link in the 30:59 description. 31:00 Welcome to this section where you will 31:02 learn the fundamental principles of API 31:05 design, which will enable you to create 31:07 efficient, scalable and also 31:09 maintainable interfaces between software 31:12 systems. Here is what we're going to 31:14 cover in this lesson. We'll start from 31:16 what APIs are and what is their role in 31:19 system architecture. Then we'll cover 31:21 the three most commonly used API styles, 31:24 which are REST, GraphQL and gRPC. 31:28 We'll discuss the four essential design 31:30 principles that make great APIs and also 31:33 how application protocols influence the 31:36 API design decisions. We'll also cover 31:39 the API design process, so starting from 31:41 the design phase to development phase to 31:43 deployment, so we'll see how that 31:45 process looks like. 31:47 So let's start by understanding what is 31:49 an API. API stands for application 31:52 programming interface, which defines how 31:54 software components should interact with 31:56 each other. 31:57 Let's say on one side you have the 31:58 client, which is either the mobile phone 32:01 or the browser of this user and on the 32:03 other side you have the server, which 32:05 will be responding to the requests. 32:08 So API here is just a contract that 32:10 defines these terms, which are what 32:13 requests can be made. So it provides us 32:15 with an interface on how to make these 32:17 requests, meaning what endpoints do we 32:19 have, what methods can we use and so on. 32:22 Also, what responses can we expect from 32:25 the server for a specific endpoint. 32:28 So first of all, it is an abstraction 32:30 mechanism because it hides the 32:32 implementation details while exposing 32:35 the functionality. For example, we can 32:37 make a request to save a user data in 32:40 this server, but we don't care at all 32:42 about how the logic applies behind the 32:45 scenes inside of this server, so we only 32:47 care about the interface that is 32:49 provided through this API and we only 32:51 use that endpoint and we store the user 32:54 without even knowing about the 32:56 implementation details. And it also sets 32:59 the service boundaries because it 33:01 defines clear interfaces between systems 33:04 and components. So this allows us to 33:06 have multiple servers. We can have one 33:08 server that is responsible for managing 33:10 the users. We can have another one that 33:13 is responsible for some other records, 33:15 let's say for managing the posts and so 33:17 on. 33:18 So this allows different systems to 33:20 communicate regardless of their 33:22 underlying implementation, like client 33:25 browsers with servers or servers with 33:27 another servers and so on. 33:30 Now let's focus on the most important 33:32 API styles you will encounter during the 33:34 design phase. These are RESTful, GraphQL 33:37 and gRPC. The most common one out of 33:40 these is REST, which stands for 33:42 representational state transfer. These 33:45 type of APIs use resource-based approach 33:48 by using the HTTP methods as a protocol. 33:52 One of the advantages of REST APIs is 33:54 that they are stateless, meaning that 33:56 each request contains all of the 33:57 information needed to process it and we 34:00 don't need any prior requests to be able 34:02 to process the current request. And it 34:05 uses the standard methods on HTTP 34:08 protocol, which are GET for fetching 34:10 data, POST for storing data, PUT or 34:13 PATCH for updating data and DELETE for 34:16 deleting data. 34:17 So based on its characteristics, the 34:20 REST is most commonly used in web and 34:22 mobile applications. Next, we have 34:25 GraphQL, which is the second most common 34:27 API style after the REST APIs. GraphQL 34:30 is a query language that allows clients 34:33 to request exactly what they need. This 34:36 means that it comes with a single 34:38 endpoint for all of the operations and 34:40 we can choose what we are expecting to 34:42 receive from this API by providing the 34:45 payload in the request. 34:47 And the operations here are called query 34:50 whenever we are retrieving data or 34:52 mutation whenever we are updating data, 34:54 so this is the equivalent in PUT or 34:57 PATCH or POST in the RESTful APIs. And 35:00 there is also a subscription in 35:02 operations, which is for real-time 35:04 communication. The advantage of GraphQL 35:07 APIs is that it allows us to have 35:09 minimal round trips. Let's say we need 35:11 some data that in RESTful APIs we will 35:14 need to make three requests to get all 35:16 of this data. 35:17 In GraphQL case, we can make a single 35:20 request and get all of this data, 35:22 avoiding the unnecessary two requests 35:24 that we will otherwise have to make in 35:26 RESTful. 35:28 And because of that, this is the 35:29 recommended option for complex UIs, so 35:31 wherever you have some complex UIs where 35:34 on one page you might need different 35:36 data, on another page you might need 35:37 some other complex nested data. In these 35:40 cases, GraphQL is the better choice over 35:43 RESTful APIs. 35:44 And the last option is gRPC. I would say 35:47 this is the least common one out of 35:49 these three. 35:50 gRPC is a high-performance RPC 35:53 framework, which is using protocol 35:55 buffers for communication. 35:58 The methods in gRPC are defined as RPCs 36:01 in the proto files and it supports 36:04 streaming and bidirectional 36:06 communication. This is an excellent 36:08 approach for microservices especially 36:11 and internal system communication, as it 36:13 is more efficient when you're working 36:15 between servers compared to GraphQL or 36:18 compared to RESTful APIs. 36:20 So the difference between REST, GraphQL 36:23 and gRPC APIs is kind of clear, but 36:26 let's also clarify the real difference 36:28 between REST and GraphQL APIs on 36:30 examples. 36:31 So as you saw, REST comes with 36:33 resource-based endpoints. For example, 36:35 here if we take a look at these 36:37 requests, you can see that the resource 36:39 here is users, so you always expect to 36:42 see some users endpoint or some 36:44 followers endpoint or let's say posts 36:46 endpoint, so it is resource-based. 36:49 And sometimes we might need to make 36:51 multiple requests for getting the 36:53 related data. As you can see here, we 36:55 need let's say the user details, but we 36:58 also need the user posts and followers. 37:01 So in this case, we need to make three 37:02 requests to get all of this data. 37:05 And it uses HTTP methods to define 37:07 operations. As you can see, these are 37:09 HTTP endpoints and we are using the GET 37:12 method specifically. And the response 37:15 structures are fixed, meaning if you got 37:17 one response for this specific user, 37:19 next time you can expect to have exactly 37:21 the same response structure. Maybe some 37:24 data will be modified, but the structure 37:26 always remains the same. And it also 37:29 provides explicit versioning, so as you 37:31 can see, it comes with V1 for the V1 37:33 API, then later if it got a major 37:36 upgrade, then this will become V2 and so 37:38 on. And you can use the headers on the 37:41 requests to leverage the HTTP caching on 37:44 RESTful APIs. Now if we compare that to 37:47 GraphQL APIs, it comes with a single 37:50 endpoint for all operations, so mostly 37:52 it is /graphql or /some API endpoint 37:56 that is commonly used for all operations 37:59 and in this case we will use a single 38:01 request to get the precise data that we 38:04 need and we will use the query language 38:06 of GraphQL. 38:08 This is what the query language looks 38:09 like. As you can see, we start with a 38:11 query and then we define what we need. 38:14 For example, we need the user with ID 38:16 123. Then we need the name of the user, 38:19 the posts and then we define whatever we 38:22 need from the posts. Maybe we need only 38:24 title and content and nothing more. 38:26 And also the followers and what we need 38:29 from followers, maybe only names. So 38:31 this allows us to be more efficient in 38:33 our requests compared to RESTful APIs 38:36 where we will need to make three 38:38 requests for this same data. 38:40 This means that client needs to specify 38:43 the response structure and in this case, 38:45 the schema evolution is without 38:47 versioning. So here as you saw, it is 38:50 with V1, V2 and so on. In this case, the 38:53 schema usually evolves without 38:55 versioning, but there is also a common 38:57 pattern to start versioning the fields. 39:00 For example, you can have followers V2 39:03 and that will be the second type of 39:05 followers schema. But you can also go 39:08 without versioning, so you can just 39:10 start modifying the followers or posts 39:13 if you are sure that there are no other 39:15 clients using your old API. 39:18 And in this case, you can leverage the 39:20 application level caching instead of the 39:22 HTTP caching. 39:24 Now let's discuss the major design 39:26 principles that will allow us to create 39:28 consistent, simple, secure and also 39:31 performant APIs. 39:33 Ultimately, the best API is the one that 39:36 we can use without even reading the 39:38 documentation. For example, if you saw 39:40 the previous endpoints in the users, 39:42 you'll see that we have {slash} users 39:45 {slash} 123. And obviously, we are 39:47 expecting to get the user details of 39:50 this specific user. And if you make a 39:52 request, for example, to that endpoint 39:54 to fetch user details, but then you find 39:56 out that it also updates some followers 39:59 or something while making this request, 40:01 then obviously that is a very bad type 40:03 of API as we didn't expect it to do such 40:06 operations. 40:08 So, first of all, the good API should be 40:10 consistent, meaning it should use the 40:12 consistent naming, casing, and patterns. 40:16 For example, if you use camel case in 40:18 one of the endpoints, let's say you have 40:20 user details and you do this in camel 40:23 case, but in another case you do it with 40:25 a snake case like user {slash} details, 40:29 then this is not common and this is not 40:31 consistent. 40:32 The second key principle is to keep it 40:35 very simple and focus on core use cases 40:38 and intuitive design. 40:40 So, you should minimize complexity and 40:42 aim for designs that developers can 40:45 understand quickly without even maybe 40:47 reading the documentation. And 40:49 simplicity again comes down to this, 40:51 which is the best API is one that 40:53 developers can use without even reading 40:55 the documentation. 40:57 Next, obviously it has to be secure, so 41:00 you have to have some sort of 41:01 authentication and authorization between 41:04 users. Also, if you have inputs, then 41:06 you need to make sure that these are 41:07 validated and you should also apply rate 41:10 limiting, so these are the most basic 41:12 things that you have to do to keep your 41:15 APIs secure. 41:16 And the last pillar is performance, so 41:19 you should design for efficiency with 41:21 appropriate caching strategies, with 41:24 pagination. If you have a large amount 41:26 of data, let's say thousands of posts, 41:29 you don't want to retrieve all of these 41:31 whenever they make a request to get the 41:33 post, so you should always have 41:34 pagination with some limit and offset. 41:37 Also, the payloads, meaning the data 41:40 that you will send back, should be 41:41 minimized. And also, whenever possible, 41:44 you should reduce the round trips. So, 41:47 if you have the opportunity to send some 41:49 small data along with the request of one 41:51 of the endpoints, then it's better to do 41:54 this if you know that you're going to 41:55 use it instead of making another 41:57 endpoint for making a request to get the 42:00 same data. 42:02 Now, each of these APIs use different 42:04 protocols and we will learn more about 42:06 these in the next lesson, but basically, 42:09 your protocol choice will fundamentally 42:11 shape your API design options. For 42:14 example, the features of HTTP protocol 42:17 directly enable RESTful capabilities, so 42:19 it makes more sense to use HTTP along 42:22 with RESTful APIs because it also 42:25 provides you with status codes and these 42:27 are great to be used with crowd 42:29 operations that you will have in RESTful 42:31 APIs. 42:32 On the other hand, WebSockets, which is 42:35 another type of protocol, enable 42:37 real-time data and also enable 42:39 bidirectional 42:41 APIs. So, these can be used along with 42:43 real-time APIs wherever you need some 42:45 chat application or some video 42:47 streaming, this is a good use case of 42:50 WebSocket APIs. 42:51 In case of GraphQL APIs, you again will 42:54 use the HTTP protocol instead of 42:56 WebSockets or gRPC. 42:59 gRPC, on the other hand, can be used 43:01 among with microservices in your 43:03 architecture to make it faster compared 43:06 to HTTP. So, your protocol choice will 43:09 affect the API structure and also the 43:12 performance and capabilities. 43:14 Therefore, you should choose it based on 43:16 its limitations and strengths and the 43:19 one that makes more sense in the type of 43:21 API that you'll be developing. 43:24 Now, let's discuss the API design 43:26 process. It all starts with 43:28 understanding the requirements, which is 43:30 identifying core use cases and user 43:33 stories that you will need to develop. 43:35 Also, defining the scope and boundaries 43:38 because if it's a huge API, then you 43:41 probably won't develop all of the 43:42 features at once, so you should scope it 43:45 to some specific features that you'll be 43:47 developing and also what are out of 43:49 scope for now. Then you should determine 43:52 the performance requirements and 43:54 specifically in your API case, what will 43:56 be the bottlenecks and where you need to 43:59 make sure that it's performant. 44:01 And you should also not overlook the 44:03 security constraints, so you should 44:05 implement all of the basic features like 44:07 authentication, authorization, the rate 44:10 limiting, but maybe some more stuff 44:12 depending on the API that you'll 44:14 develop. When it comes to design 44:16 approaches, there are couple of ways to 44:18 go about it. The first one is top-down 44:21 approach, which is you start with 44:23 high-level requirements and workflows. 44:26 This is more common in interviews where 44:28 they give you the requirements on what 44:30 the API will be about and then you start 44:33 defining what the endpoints will be, 44:35 what the operations will be, and so on. 44:38 But there is also the bottom-up 44:40 approach, which is if you have existing 44:42 data models and capabilities, then you 44:44 should design the API based on this. So, 44:47 this is more common when you're working 44:49 in a company and they already have their 44:52 data models and capabilities of their 44:54 APIs, so you should take that into 44:56 account when designing the API. 44:59 And we also have contract-first 45:01 approach, which is you define the API 45:03 contract before implementation, meaning 45:06 what the requests should look like and 45:08 what the responses should look like. And 45:10 this is more similar to top-down 45:12 approach and this is also commonly used 45:14 in interviews. 45:16 When it comes to lifecycle management of 45:18 APIs, it starts with the design phase 45:21 where you design the API, discuss the 45:24 requirements and the expected outcomes 45:27 of the API. And only after that you can 45:30 start the development and maybe local 45:32 testing of your API. 45:34 After that, you usually deploy and 45:36 monitor it, so you do some more testing, 45:39 but now on staging or on production. 45:42 But then it also comes the maintenance 45:44 phase and this is why it's important to 45:46 develop it with keeping the simplicity 45:49 in place, so it will be easier for you 45:51 to maintain or for other developers to 45:53 maintain in the future. 45:55 And lastly, APIs also go through 45:57 deprecation and retirement phase. So, 46:00 some APIs eventually get deprecated 46:03 because there might come up a new 46:05 version of the API that you should use 46:07 or let's say you are transitioning from 46:10 V1 to V2 API. So, that's also the 46:12 deprecation phase of the V1 API. 46:16 So, developing APIs is not only in the 46:18 development phase as you might assume, 46:20 it's not just coding. So, the big part 46:23 of it is designing it and also keeping 46:26 it maintainable and also eventually you 46:28 might need to retire it at the end. 46:31 So, let's recap and see what our next 46:33 steps are. We learned what APIs are and 46:36 about the most dominant three type of 46:39 API styles, which are RESTful, GraphQL, 46:42 and gRPC. 46:43 We've covered the four key principles 46:46 that will guide us when creating API 46:48 designs effectively. And you now also 46:51 understand how the design choice of your 46:53 protocol will influence the design of 46:56 your API and also the whole API design 46:59 process from start to finish. 47:01 But we didn't discuss the limitations 47:03 and strengths of these API protocols, so 47:07 that's why in the next lesson, we will 47:08 learn all about the API protocols that 47:11 we can use with API design and which one 47:14 we should choose based on the 47:15 requirements of our API. Choosing the 47:18 wrong protocol for our API can lead to 47:21 performance bottlenecks and also 47:22 limitations in functionality. That's why 47:25 we need to first understand these 47:26 protocols, which will allow us to build 47:29 APIs that meet our specific user 47:31 requirements for latency, throughput, 47:34 and also interaction patterns. That's 47:36 why in this lesson, we'll cover the role 47:38 of API protocols in the network stack, 47:42 the two fundamental protocols, which are 47:44 HTTP and HTTPS, and also their 47:47 relationship to APIs. 47:49 Also, another common type of protocol, 47:51 which is WebSocket for real-time 47:53 communication. We'll also cover advanced 47:56 message queuing protocol, which is 47:58 commonly used for asynchronous 47:59 communication. And lastly, we'll cover 48:01 the gRPC, which is Google's remote 48:04 procedure call, and it is also another 48:06 common type of protocol used commonly 48:09 within servers. Let's start by 48:11 understanding the application protocols 48:13 in network stack. Application layer 48:16 protocols sit at the top of network 48:18 stack, building on top of protocols like 48:21 TCP and UDP, which are at the transport 48:24 layer. These protocols at application 48:27 layer define the message formats and 48:29 structures, also the request-response 48:32 patterns, and management of the 48:34 connections and error handling. 48:37 Now, below that, we have many other 48:38 layers like the network layer or data 48:41 link layer or even physical layers, but 48:44 when building APIs, we are mostly 48:46 concerned with the API layer protocols, 48:49 which are HTTP, HTTPS, WebSockets, and 48:52 so on. The most common type of protocol 48:55 and also the foundation of web APIs is 48:57 HTTP, which stands for Hypertext 49:00 Transfer Protocol. This is the typical 49:02 interaction between client and server 49:04 when they are interacting over HTTP. As 49:07 you can see, client always sends a 49:08 request and they define the method, 49:11 which can be get, post, or other 49:12 methods, and they define the resource 49:15 URL, which can be at {slash} API {slash} 49:18 products. Let's say they are requesting 49:19 data for this specific ID of the 49:22 product, and they also define the 49:24 version of the HTTP protocol that they 49:27 are using. They also define the host, 49:29 which is the domain of your server where 49:32 the information is accessed, and usually 49:34 they also authenticate before accessing 49:37 any resources, so it can be either a 49:39 bearer token or a basic authentication, 49:42 OAuth, and so on. So, once the request 49:45 is authenticated in the server, it 49:47 receives the response, which is in 49:49 similar format, and it's in HTTP 49:51 response. So, you get the HTTP version, 49:54 which is again the same as you requested 49:56 with, and the status code, which can be 49:58 200 if it was successful, or it can be 50:01 400 if the client was error, or 500 if 50:04 the error happened in server, and so on. 50:07 You receive the content type, which can 50:09 be usually application JSON, but it can 50:12 also be a static web page or something 50:14 else. And there are many other headers 50:17 that you can control, like controlling 50:19 cache, you can use the cache control 50:20 header or some other properties, but 50:23 these are the main things that you would 50:25 notice in HTTP request response cycles. 50:28 Now, when it comes to methods, you have 50:30 get for retrieving data, post for 50:32 creating data in the server, put or 50:35 patch for updating data partially or 50:38 fully, and delete for removing data from 50:41 the server. And when it comes to status 50:43 codes, which are received by the server, 50:46 so you have 200 series, which are 50:47 successful cases. You have 300 for 50:50 redirection, 400 means that client made 50:53 an error in the request, so this is an 50:55 issue from client side, or 500, which 50:58 means that server made an error, or like 51:00 some error happened in the server, which 51:02 means that this is the issue in the 51:04 server. 51:05 And these are the common headers, like 51:07 content type, which is defined by the 51:09 server usually, but also from the 51:10 client, authorization for making a 51:13 request and authorizing to the server, 51:16 accept headers, cache control, user 51:18 agent, and there are more headers, but 51:20 these are the common ones. 51:22 Then we also have HTTPS, which is 51:24 basically the same HTTP protocol, but 51:26 with some sort of TLS or SSL encryption, 51:29 which means that our data is now 51:31 protected in transit when we are making 51:34 requests. So, it adds a security layer 51:37 through these TLS or SSL certificates 51:39 and encryption, and it protects data in 51:42 the transit. And benefits of HTTPS is 51:45 obviously your data is encrypted in the 51:47 transit, it comes with data integrity, 51:50 and you also authenticate users before 51:52 providing any data, and it also adds SEO 51:55 benefits. And you have many risks when 51:57 you are using HTTP only without any 52:00 encryption, so the golden standard is to 52:02 always use HTTPS in servers. 52:05 The next type of protocols are web 52:07 sockets. While we have HTTP, which is 52:10 very good at request response patterns, 52:12 sometimes HTTP has limitations. For 52:15 example, let's say you're polling some 52:17 data. Let's say this is a user chat, so 52:19 you have the client and server. On the 52:21 client side, you have the user chat, and 52:23 on the server, you have the messages 52:25 between two users. 52:27 When one of the users messages the 52:29 other, it sends a request to the server 52:31 to notify that a message has been sent, 52:34 and it receives a response from the 52:36 server, maybe the messages from the 52:38 other users, if there are any. And then 52:41 next time, if you need to know if you 52:43 have new messages, you need to make 52:45 again another request to the server, and 52:48 maybe you don't have any new messages, 52:49 so you will receive an empty response 52:51 with no new data. So, this was basically 52:54 a non-necessary request response cycle. 52:57 And you might request from some other 52:59 time, let's say from 1 minute, and 53:01 receive a response. Now you have some 53:02 messages, but it can be also empty 53:05 again. So, this way is not ideal for 53:08 real-time communication. As you can see, 53:10 you get increased latency, you waste 53:12 some bandwidth with making requests that 53:15 are empty, and you also use the server 53:17 resources without the need of making 53:20 requests to the server. And for such 53:22 cases, we have web sockets, which solve 53:25 this issue. So, in web socket, you have 53:27 usually a handshake that is happening 53:29 within the first request, and now you 53:31 have both like two-side communication 53:34 between client and the server, which 53:36 means that once the handshake is being 53:38 made, the server can independently 53:40 decide to push data to the client. Let's 53:43 say now you have two new messages on the 53:45 server. So, server can decide to send 53:48 these messages to the client without 53:50 even client requesting for it. But 53:52 client can still request data, so if 53:55 client needs some external data or more 53:57 data from the server, it can still make 53:59 requests, but server is now also able to 54:02 independently push data to the client. 54:05 So, this is what unlocks the real-time 54:07 data with minimal latency. As soon as 54:10 you have some new data in the server, it 54:12 pushes the new data to the client, and 54:15 it also reduces the bandwidth usage by 54:17 allowing bidirectional communication. In 54:20 client server model with HTTP, you would 54:23 make, let's say, new requests per 5 54:25 seconds or 10 seconds to see if there 54:28 are any new data in the server, but in 54:30 this scenario, you don't make any more 54:32 requests other than the first one. And 54:35 now, whenever there are new data, the 54:37 server will push it, and whenever there 54:39 are no data to be requested, then you 54:41 don't need to make unnecessary requests 54:43 to the server. 54:45 The next very common type of protocol is 54:47 Advanced Message Queuing Protocol, which 54:50 is an enterprise messaging protocol used 54:52 for message queuing and guaranteeing 54:55 delivery. In this setup, you usually 54:57 have the producer, which can be either a 55:00 web service or payment system or 55:02 something like that. And on the other 55:04 side, you have the consumer, which can 55:06 be the processor of the payments or 55:09 notification systems and stuff like 55:11 that. So, producer publishes messages to 55:14 the message broker, and here is where 55:17 you have the Advanced Message Queuing 55:18 Protocol. You have queues in the middle. 55:21 Let's say one of these queues is for 55:23 order processing. So, whenever a new 55:25 order has been placed, producer 55:27 publishes a message to this queue. And 55:30 then whenever this consumer is free, it 55:32 can pull messages from this queue and 55:34 start updating the inventory and data in 55:37 the database. This allows the consumer 55:40 to only pull data from here whenever it 55:43 has capacity. And whenever this consumer 55:45 is busy with some other tasks, it leaves 55:48 the message in the queue, and then later 55:50 on, whenever it has some free capacity, 55:52 it will pull the message and start 55:54 updating the data. And when it comes to 55:57 exchange types, you have direct 55:59 one-on-one exchange or fan out or 56:02 topic-based communication, and we will 56:04 explore these more when we come to the 56:06 message queuing section. 56:08 The other common type of protocol is 56:10 gRPC, which works with protocol buffers. 56:14 This is a high-performance RPC framework 56:17 invented by Google, and it uses HTTP/2 56:20 for transport, meaning the second 56:22 version of the HTTP. This means that 56:24 clients should support HTTP/2, otherwise 56:28 this can't be used between client and 56:30 server. But that's why this is most 56:32 commonly used between servers, so 56:34 usually the client is another server, 56:36 and we have some other microservices 56:39 communicating with each other with this 56:41 gRPC framework. It mainly uses protocol 56:44 buffers, and it also comes with built-in 56:47 streaming capacities because it uses 56:49 HTTP/2. 56:50 So, these are the most common types of 56:53 API protocols. There are many more, but 56:55 usually in 90% of cases, you would see 56:58 only these protocols. And when choosing 57:00 the right one, you should mainly 57:02 consider the interaction patterns. 57:04 Usually, by default, you go with HTTP if 57:07 it's just a request response cycle, but 57:09 if you're building something like 57:10 real-time chat or some real-time 57:12 communication, then you would need to go 57:14 with web sockets. 57:16 The choice also depends from the 57:17 performance requirements. So, if you 57:19 have multiple servers, microservices 57:21 communicating with each other, and there 57:23 is an opportunity to use gRPC, for 57:26 example, then you can go with it to 57:28 increase the performance and speed of 57:30 the communication, but it also comes 57:32 down to client compatibility. For 57:35 example, most browsers don't support the 57:37 latest version of the HTTP, that's why 57:39 gRPC isn't that very common for 57:42 browser-server communication. 57:44 It also comes down to the payload size, 57:46 meaning the volume of the data and 57:49 encoding, security needs based on the 57:51 authentication, encryption, and so on, 57:53 and also the developer experience, so 57:56 the tooling and documentation. And it 57:58 also comes down to the developer 58:00 experience because you're mostly going 58:02 to work with this API, and it needs to 58:04 have good documentation and tooling for 58:07 you to fully work with this type of API 58:09 protocol. So, to recap, we have explored 58:12 the role of application protocols in 58:14 network stack, the HTTP and HTTPS, which 58:18 are the most fundamental types of 58:20 protocols, web sockets for real-time 58:23 communication, AMQP, which stands for 58:25 Advanced Message Queuing Protocol, which 58:28 allows us to have asynchronous 58:29 communication and adding message queues 58:32 between the consumer and producer, and 58:35 also gRPC, which stands for Google 58:37 Remote Procedure Call, and the main 58:39 advantage of this is that it's 58:40 high-performance RPC framework which 58:43 uses HTTP/2 for transport. 58:46 So, we discussed the application layer, 58:48 which includes these protocols that we 58:50 usually use for building APIs, but we 58:53 don't know yet about this transport 58:55 layer, which includes the TCP and UDP. 58:58 So, in the next lesson, we are going to 59:00 discuss this layer and understand which 59:02 of these transport layers, whether TCP 59:05 or UDP, are the best choice depending on 59:08 the API that we are building. Most 59:10 developers work with APIs, but never 59:13 think about what's actually delivering 59:15 those packets, like how does it happen 59:17 that the request is being made from 59:19 client to server, and how does this 59:21 request go through the internet? 59:24 That's where the second layer comes in 59:26 in the OSI model, which is the transport 59:28 layer that has the TCP and UDP inside of 59:32 it. 59:33 These are both transport layer 59:34 protocols, meaning they handle how data 59:37 moves from one machine to another over 59:39 the network, but both are doing it very 59:42 differently. In this lesson, we'll learn 59:45 about these transport layer protocols. 59:47 We'll start with TCP, which is the 59:49 reliable but slower version. Then we'll 59:52 learn about the UDP, which is in short, 59:54 it's faster and unreliable version of 59:57 TCP. And we'll compare both of them and 1:00:00 decide which one we need to choose based 1:00:02 on the API requirements. 1:00:04 Let's start with TCP, which stands for 1:00:06 transmission control protocol. Think of 1:00:09 it like sending a packet with a receipt, 1:00:12 tracking, and also signature that is 1:00:14 required. So, when you send some packets 1:00:17 over the internet, you usually don't 1:00:19 send all of it at once. Sometimes the 1:00:21 data is larger, let's say it's divided 1:00:24 in three chunks, so you need to send 1:00:26 them separately. The first chunk, the 1:00:28 second chunk, and also the third chunk. 1:00:30 So, in this case, TCP guarantees 1:00:33 delivery of all of these three chunks. 1:00:36 If one of these packets is lost or 1:00:38 arrives out of order, TCP will resend or 1:00:41 reorder it. 1:00:43 It's also connection-based, which means 1:00:45 that before sending any data, it 1:00:47 performs a three-way handshake, which is 1:00:50 establishing the connection between 1:00:52 client and server. 1:00:54 It also orders these packets. Let's say 1:00:56 the client receives the first packet 1:00:58 first, then the third packet, then the 1:01:01 second packet. It makes sure that it's 1:01:03 reordered to first, second, and third. 1:01:06 This, of course, adds overhead, but it 1:01:08 ensures that it's accurate and reliable. 1:01:11 That's why APIs that involve payments, 1:01:13 authentication, or user data always use 1:01:16 TCP. On the other hand, we have UDP, 1:01:19 which stands for user datagram protocol. 1:01:22 It's fast and efficient, but the 1:01:24 downside of this is that it doesn't 1:01:26 guarantee that all of the packets will 1:01:28 arrive. For example, if you're sending 1:01:30 four packets from the server to the 1:01:33 client, one of these packets might be 1:01:35 lost, and it's won't be pushed to the 1:01:37 client, and UDP won't make sure that 1:01:39 this eventually gets delivered. So, 1:01:42 there is no delivery guarantee. There is 1:01:45 also no handshake or connection or any 1:01:47 sort of tracking. But because of these 1:01:50 trade-offs, it is faster transmission, 1:01:53 and it comes with less overhead as it 1:01:55 doesn't need to make sure that all of 1:01:57 the packets are delivered or in the 1:01:59 correct order. For example, in video 1:02:02 calls, UDP can be the best protocol 1:02:04 because if some information was cut in 1:02:07 the middle, or let's say you're in a 1:02:09 call with someone and their internet 1:02:11 connection lags, you don't need to 1:02:13 receive that old connection or the old 1:02:15 data on what they said because you are 1:02:17 in the call right now. So, UDP is the 1:02:20 go-to for video calls, online games, or 1:02:23 live streams because if one of these 1:02:25 packets drops, it's still fine, and you 1:02:27 don't need to go back and resend this 1:02:30 packet. You can just move on and send 1:02:32 the next packets. 1:02:34 This is what the three-step handshake 1:02:36 looks like in TCP. As you can see, the 1:02:38 first step is that client sends a 1:02:40 request to the server. In the second 1:02:42 step, server syncs and acknowledges the 1:02:45 request. And in the first step, the 1:02:47 client acknowledges the server. And this 1:02:50 is where the connection is established 1:02:52 between the client and server. And now 1:02:54 they can start sending data back and 1:02:56 forth on top of this TCP protocol. 1:02:59 So, in short, TCP is the safer and 1:03:02 reliable version of UDP, but it is 1:03:05 slower. And on the other hand, UDP is 1:03:08 faster and lightweight, but it is risky. 1:03:10 For example, if one of the packets in 1:03:12 between the source and destination is 1:03:14 lost, it doesn't resend it, so there is 1:03:17 no guaranteed delivery. But on the other 1:03:20 hand, if in TCP one of the packets is 1:03:22 lost, after some timeout, it still 1:03:24 resends the fourth packets, and this way 1:03:27 it guarantees that all data will be 1:03:29 delivered compared to UDP, where some 1:03:31 data might be lost, but it will still 1:03:33 keep going. And when choosing between 1:03:36 those two, these are the main things 1:03:38 that you need to look for. If you need 1:03:40 the connection to be safe and reliable, 1:03:42 then you need to go with TCP. Or if you 1:03:45 need it to be fast, lightweight, but 1:03:47 some data loss might be acceptable, then 1:03:49 you will need to go with UDP. For 1:03:52 example, it is best for using TCP in 1:03:54 bankings, emails, payments, and so on. 1:03:57 And on the other hand, UDP is mostly 1:04:00 used in video streaming, streaming, 1:04:02 gaming, and so on. 1:04:04 These are the main things that you need 1:04:05 to know about the application and 1:04:07 transport layers, and these are the only 1:04:10 layers that will need to be used to 1:04:12 building APIs. And in the next lesson, 1:04:15 we will learn about RESTful APIs and how 1:04:18 we usually design APIs in RESTful 1:04:20 format. RESTful APIs let different parts 1:04:23 of a system talk to each other using the 1:04:26 standard HTTP methods. They are the most 1:04:29 common way developers build and consume 1:04:31 APIs today. And in this video, you'll 1:04:33 learn how to design clean REST APIs by 1:04:36 following the proven best practices so 1:04:39 that you avoid creating messy and 1:04:41 inconsistent patterns that make the APIs 1:04:44 hard to use and maintain. We'll start by 1:04:47 learning about the architectural 1:04:49 principles and constraints of RESTful 1:04:51 APIs, about the resource modeling and 1:04:54 URL design, also the status codes and 1:04:58 the error handling, as well as 1:05:00 filtering, sorting, and so on. 1:05:02 And we'll learn the best practices when 1:05:04 using and developing RESTful APIs. 1:05:07 Let's start from the resource modeling. 1:05:09 Resources are the core concepts in REST. 1:05:13 Let's say you have the business domain, 1:05:14 which consists of the products, orders, 1:05:17 and reviews. When modeling these to a 1:05:20 RESTful API, you usually convert these 1:05:22 into nouns and not verbs, meaning that 1:05:25 the product becomes products, order 1:05:27 becomes orders, and same for order 1:05:30 reviews. These can be collections or 1:05:33 individual items. For example, this 1:05:35 first request, which is to {slash} API 1:05:37 {slash} products, will return you the 1:05:39 collection of products, not a single 1:05:42 product. But on the other hand, you 1:05:44 could have {slash} products and {slash} 1:05:46 specific ID of a product, which will 1:05:48 return you the individual item. And 1:05:51 notice that we are using {slash} 1:05:53 products when retrieving the collection 1:05:55 of products, and we are not using 1:05:57 something like get products, which will 1:06:00 be not a best practice in RESTful APIs. 1:06:03 As I mentioned, we are using nouns here 1:06:05 and not verbs. So, to fetch orders, for 1:06:08 example, you don't define the URL as get 1:06:11 orders. You just define it as {slash} 1:06:14 orders, and depending on the method that 1:06:16 we'll use, let's say it's a get method, 1:06:18 then you will retrieve the orders. If 1:06:20 it's a post method, then you will create 1:06:22 an order, and so on. 1:06:24 So, all the resources should be clearly 1:06:26 identifiable through the URLs. For 1:06:29 instance, this is an example of getting 1:06:32 a collection. This is an example of 1:06:34 getting a specific item. And also, 1:06:36 nested resources should be clearly 1:06:39 defined. For example, if you want to 1:06:41 retrieve reviews for some specific 1:06:43 product, then we would assume that if 1:06:45 you make a request to {slash} products 1:06:47 {slash} ID of that product, and then 1:06:49 {slash} reviews, you would get the 1:06:51 reviews for that specific product. But 1:06:54 in real-world APIs, you rarely want to 1:06:57 return all the results at once. That's 1:06:59 why we usually incorporate filtering, 1:07:01 sorting, and pagination in APIs. So, 1:07:04 let's start from the filtering. For 1:07:06 example, if you make a request to get 1:07:09 all the products, you usually add some 1:07:11 query parameter, which in this case you 1:07:13 can see it's category, so you're first 1:07:15 of all filtering them by category. And 1:07:18 then also, with the and sign, you add 1:07:20 that they should be in stock, so the in 1:07:23 stock should be true. And this way you 1:07:26 are only returning the items that you're 1:07:28 going to display on the UI, and you're 1:07:30 not making some requests that will waste 1:07:33 the bandwidth of this API, and also it 1:07:36 will be a huge response for you in the 1:07:38 front-end side. Next, we also have 1:07:40 sorting. In this case, again, it's 1:07:42 controlled through the query parameters, 1:07:44 and query parameters are anything that 1:07:46 start after the question mark in the 1:07:48 URL. So, in this case, you usually pass 1:07:51 the sort attribute, and this can be, for 1:07:54 example, ascending by price or ascending 1:07:57 by reviews, or it can be also the 1:08:00 sending order. 1:08:01 So, based on this, you will get the 1:08:03 response from the API in a sorted order 1:08:06 because if you, for example, have 1,000 1:08:08 items in the back end, in the database, 1:08:11 you don't want to retrieve all of these 1:08:14 in unsorted order to the front end 1:08:16 because, let's say, the front end now 1:08:18 needs to sort them by the price 1:08:20 ascending. This means that it needs to 1:08:22 make request to get all of the products, 1:08:25 which are these 1,000 items that you 1:08:27 have in the database. So, that will be 1:08:30 very inefficient. That's why we do the 1:08:32 sorting in the back end instead. So, 1:08:34 your back end should support sorting 1:08:36 functionality. This way, the front end 1:08:38 can just make a request to your back end 1:08:41 and pass this sort query parameter, and 1:08:44 then that way, it will get the sorted 1:08:46 products to be displayed on the screen. 1:08:49 And next, we also have pagination. 1:08:51 Again, with the query parameter, you 1:08:53 usually pass the page which you want to 1:08:55 retrieve, and also the limit because if 1:08:58 you don't pass the limit, then again, it 1:09:00 will give you all of the products 1:09:02 starting from the page two till the end, 1:09:04 which can be a lot of items. So, you 1:09:07 also pass some sort of limit, and that 1:09:09 limit is whatever you're going to 1:09:11 display on the front end. And then based 1:09:13 on that, you will get the response. And 1:09:15 here, let's say you fetched 10 items, so 1:09:17 you're going to display those 10 on the 1:09:20 UI. And then once they click on the next 1:09:22 page, you will make another request to 1:09:24 the page three this time, and you will 1:09:26 get the next items from the server. Now, 1:09:29 usually we use page for pagination, but 1:09:32 there is another common attribute that 1:09:34 is offset. So, some APIs use offset 1:09:37 instead of the page, and they use this 1:09:40 in combination with limit, which 1:09:42 basically means if you have 1,000 items, 1:09:44 so offset will tell the API from where 1:09:47 to start counting these 1,000 items, and 1:09:50 the limit is the same as you have it 1:09:52 here. So, it's basically limiting the 1:09:54 number of items that you are getting 1:09:56 from this offset to retrieve to the 1:09:59 front end. And the last option, you can 1:10:01 also have this cursor based. So, instead 1:10:03 of page and limit, you would pass a 1:10:06 cursor, which will be the hash of the 1:10:08 page you want to retrieve. So, this 1:10:10 approach of adding filtering, sorting, 1:10:13 and pagination comes with benefits. So, 1:10:15 first of all, it saves the bandwidth of 1:10:17 your server. It also improves the 1:10:19 performance both in the server side and 1:10:22 on the front end side, and it also gives 1:10:24 the front end more flexibility, because 1:10:26 now you can fetch only the things that 1:10:28 you need, and not some unnecessary data 1:10:31 from the database. Now, let's come to 1:10:33 the HTTP methods that REST APIs use, 1:10:36 because they rely on HTTP protocol, and 1:10:39 hence they are using the HTTP methods, 1:10:42 especially for CRUD operations. So, 1:10:45 these are the most common types of CRUD 1:10:47 operations you would see in REST APIs. 1:10:50 First of all, we have the get method, 1:10:52 which is used for reading data from the 1:10:55 API. So, this is for retrieving 1:10:57 resources, as you saw, like retrieving 1:10:59 the products, retrieving the reviews, 1:11:01 and so on. And the URL usually looks 1:11:04 like this. You make a get request to the 1:11:07 {slash} API {slash} version of the API 1:11:09 {slash} the resource name. And these 1:11:12 type of requests are both safe and item 1:11:15 potent, which basically means if you 1:11:17 make a request to {slash} products two 1:11:20 or three times, you expect to receive 1:11:22 the exact same output every time, unless 1:11:25 some new products obviously have been 1:11:27 added to the database. Next, we have the 1:11:30 post method. This is usually when you're 1:11:32 creating a resource in your server. The 1:11:35 common example is again, you will make 1:11:37 the request to exact same endpoint as 1:11:39 you have it for the get to create a 1:11:41 collection, but in this case, instead of 1:11:44 get, you are using post method, and this 1:11:47 tells the API that you need to create a 1:11:49 resource in the products and not 1:11:51 retrieve them. These type of requests 1:11:54 change the state of the server. They are 1:11:56 adding a new item, and also they are not 1:11:59 item potent, which means that they are 1:12:01 creating a resource. So, the first time 1:12:03 you create a resource, you will get the 1:12:05 ID of the first item that you created. 1:12:08 The second time you create it, you will 1:12:10 get the ID of the second one, and so on. 1:12:13 Next, we have the put and patch methods, 1:12:15 which are very similar, and they are 1:12:18 updating resources in your API, but they 1:12:21 do it a bit differently. The put method 1:12:23 replaces the whole resource, whereas the 1:12:26 patch method partially updates the 1:12:28 resource in your API. Now, you can see 1:12:31 that the request URL is exactly the same 1:12:33 in both of their cases. So, it's to 1:12:35 {slash} products {slash} ID of a product 1:12:38 you want to modify. Just in case of the 1:12:41 put request, it will take this whole 1:12:43 product with the ID of 123, and it will 1:12:46 basically replace it with the new one 1:12:48 that is coming from the front end. 1:12:50 Whereas in case of the patch, it will 1:12:52 again take this item from the database 1:12:55 with ID 123, but it will update it 1:12:57 partially. Let's say you just updated 1:13:00 the title from the front end, and you 1:13:02 made the request with patch method. So, 1:13:05 this will only update the title of this 1:13:07 product, and it will leave the other 1:13:09 parts, other properties unchanged. And 1:13:12 the last CRUD operation is delete, and 1:13:15 we use delete method in this case. And 1:13:18 obviously, as the name tells, it deletes 1:13:20 the resource from the database. So, 1:13:22 again, the URL is exactly the same as 1:13:25 you have for modifying items. It's to 1:13:27 {slash} products {slash} ID of the 1:13:29 resource. And in this case, you are not 1:13:31 passing anything in the request body. 1:13:34 So, you are just making a delete request 1:13:36 to this item, and you are removing this 1:13:38 from the database. And each of these 1:13:41 operations return you different status 1:13:43 codes depending on how the request went, 1:13:46 whether it was successful or not. For 1:13:49 that, we have status codes and error 1:13:51 handling in RESTful APIs. 1:13:53 So, you should use the appropriate 1:13:55 status codes when working with REST 1:13:57 APIs. For example, the 200 series are 1:14:00 for successful requests. For example, 1:14:02 200 is okay, 201 is resource has been 1:14:06 created, 204 is there is no content 1:14:08 here. 1:14:09 Let's say you made a request, the 1:14:11 previous request we were talking about, 1:14:13 to {slash} products {slash} some ID of a 1:14:16 product, and you successfully retrieved 1:14:18 this item. This means that you also need 1:14:21 to set the status code to 200, because 1:14:24 the request has been successful. In the 1:14:26 other case, where you're creating a 1:14:28 product, and you're making a post 1:14:29 request to {slash} products, this time 1:14:32 you shouldn't response with the same 200 1:14:34 code, because 200 generally means that 1:14:37 the status was okay. But in 201 case, it 1:14:40 means that the resource has been 1:14:42 created. And in this case, since you're 1:14:44 creating a new product, you should 1:14:46 obviously response with the 201 status 1:14:48 code, meaning resource has been created. 1:14:51 You also have 300 series, which are for 1:14:53 redirection. Let's say you make a 1:14:55 request to a URL, and now this URL has 1:14:58 been moved to somewhere else. So, it 1:15:00 will respond with a 300 series, and it 1:15:03 will redirect you to the new URL. In 400 1:15:06 series, we have the client errors. So, 1:15:08 this is whenever your front end made a 1:15:11 bad request, or the user made a bad 1:15:13 request. For example, 400 is a generic 1:15:16 bad request. In 401, we have 1:15:18 unauthorized requests, meaning the user 1:15:21 is not authenticated to make this 1:15:23 request. For 404, we have not found. So, 1:15:26 generally when you visit some URL, or 1:15:28 you make a request for some specific 1:15:30 resource that doesn't exist, you would 1:15:33 get this 404 status code. 1:15:35 So, for 400 case, let's say you made a 1:15:38 request with invalid parameters, or some 1:15:41 wrong JSON format. In this case, you 1:15:43 would get a generic 400 bad request. But 1:15:46 if a user makes a request to to get some 1:15:49 product, which is let's say the product 1:15:51 with this ID, and it doesn't exist in 1:15:54 the database after querying it, then you 1:15:56 should respond with the 404 status code, 1:15:59 meaning that the resource has not been 1:16:01 found. 1:16:02 And lastly, we have 500 series. These 1:16:04 are things when error happens in your 1:16:07 server. So, you don't know the exact 1:16:09 reason, and it's also not a client 1:16:11 error, meaning client requested 1:16:13 everything properly. And in this case, 1:16:16 we throw unexpected server-side errors. 1:16:18 You generally respond with a server 1:16:21 error message, and you return the 500 1:16:24 status code along with it. 1:16:26 When it comes to best practices of 1:16:28 RESTful APIs, first of all, notice that 1:16:30 we are using plural nouns for all of the 1:16:33 resources. So, instead of {slash} 1:16:35 product, we are using {slash} products 1:16:38 for retrieving the products collection. 1:16:40 So, you should always use the plural in 1:16:42 this case. Also, in the CRUD operations, 1:16:45 we use the proper HTTP methods. For 1:16:48 example, when making a request to delete 1:16:50 users, we expect to make a request to 1:16:53 users {slash} ID of a user, and not some 1:16:56 post request to {slash} users {slash} 1:16:58 ID. So, first of all, the HTTP methods 1:17:01 needs to be properly set up, and also 1:17:04 the URL. We don't expect some random 1:17:06 things like {slash} delete to delete a 1:17:09 resource from the database. 1:17:11 As you saw, we also support filtering, 1:17:13 sorting, and pagination in good REST 1:17:15 APIs. Not only pagination. For example, 1:17:19 in this case, we only have the page 1:17:21 three, but we cannot limit the amount of 1:17:23 products that we want to retrieve. 1:17:25 Whereas in this case, we can fully 1:17:27 control what we want to get from the 1:17:29 API. We want to get the items from page 1:17:32 three. We want this number of limit to 1:17:34 be applied on the products, and we also 1:17:36 want to apply some sort, like sorting, 1:17:39 to sort the price, or sort by ratings, 1:17:42 and so on. And also versionings in the 1:17:45 RESTful APIs. As you noticed in all of 1:17:47 these requests, they all come with a 1:17:50 prefix, which is {slash} API, and then 1:17:53 {slash} the ID of the API, which is 1:17:55 either V1, V2, V3, and so on. So, let's 1:17:59 say in the future, you migrate your API, 1:18:02 and you start using bunch of new 1:18:03 features, but you also break something 1:18:05 in the previous version one. Then, if 1:18:08 you use the versioning, you won't break 1:18:10 it on the front end, because they can 1:18:12 use the old version of your API, and 1:18:14 still use the old features and 1:18:16 functionalities, while you continue to 1:18:18 develop the new version, let's say 1:18:20 version three, and you support new 1:18:21 features here, and you might have broken 1:18:24 something here, but they are still using 1:18:26 the old API. So, it is doesn't impact 1:18:29 the end users. 1:18:30 So, to recap, we learned about the REST 1:18:33 architectural principles and 1:18:35 constraints, also about the resource 1:18:37 modeling and URL design, and how we 1:18:39 model the business domain into the 1:18:42 RESTful API domain. Also, the status 1:18:45 codes, error handling, and the proper 1:18:47 methods to be used with the basic CRUD 1:18:50 operations. 1:18:52 And lastly, we covered the best 1:18:54 practices for RESTful APIs that you 1:18:56 should use to keep your APIs consistent 1:18:59 and also predictable for other 1:19:01 developers who are using it. Traditional 1:19:04 RESTful APIs often return too much or 1:19:07 too little data, which requires us to do 1:19:09 multiple requests for a single view to 1:19:11 get all the data that we need. GraphQL 1:19:14 solves this issue by giving clients 1:19:16 exactly what they requested for, but 1:19:18 designing GraphQL APIs is different from 1:19:21 designing RESTful APIs. That's why in 1:19:23 this video we'll cover the core concepts 1:19:25 of GraphQL and why it exists, the schema 1:19:28 design and type system of GraphQL, 1:19:31 queries and mutations, error handling, 1:19:33 and also best practices for designing 1:19:36 GraphQL APIs. 1:19:37 Let's start by understanding why GraphQL 1:19:39 exists in the first place. It was 1:19:42 created by Facebook to solve a very 1:19:44 specific pain, which is clients needing 1:19:46 to make multiple API calls and still not 1:19:49 getting the exact data that they needed. 1:19:51 For example, if we imagine we have the 1:19:53 Facebook APIs like user API, posts API, 1:19:57 comments and likes for the Facebook 1:19:59 page. Most of the times client can make 1:20:01 requests to all of these APIs separately 1:20:04 and still not get all the data that it 1:20:06 needs, which will require it to do 1:20:08 multiple requests to the same API. This 1:20:11 of course adds up to the overall latency 1:20:14 of the page because the page is still 1:20:16 not loaded until all of these requests 1:20:19 are made and the data is fetched. But in 1:20:21 case of GraphQL APIs, you have a single 1:20:24 GraphQL endpoint. So the client 1:20:26 specifies the shape of the response and 1:20:29 this one endpoint handles all of the 1:20:30 data interactions. 1:20:32 It is still an HTTP request, but as you 1:20:34 can see we can specify the exact data 1:20:37 that we need. For example, we need the 1:20:38 user with ID 123 and we need only the 1:20:41 name of the user, also posts and from 1:20:44 the posts we can specify only title, so 1:20:46 we don't need the images for this view. 1:20:49 And again with the comments you can 1:20:50 specify the exact data that you need 1:20:52 within the object so that you are not 1:20:54 doing over fetching of the data. 1:20:56 Now let's see the schema design and type 1:20:58 system of GraphQL and how it's different 1:21:01 from restful APIs. 1:21:03 The schema in this case is a contract 1:21:05 between the client and server. In schema 1:21:08 first of all you have types, which can 1:21:09 be for example user type that you 1:21:12 specify and you specify all the fields 1:21:14 that exist on this user type, which are 1:21:16 ID, name, posts and so on. And as you 1:21:19 can see if the type is not a primitive 1:21:21 type like posts, then you can specify 1:21:23 another type of post array and then this 1:21:26 post type can be defined separately. 1:21:29 Next we have queries to read data. So 1:21:31 this is the equivalent of doing get 1:21:33 requests in restful API. You specify the 1:21:37 query and the function of this query. 1:21:39 This can be the user query which fetches 1:21:41 the user with specific ID and also the 1:21:45 return type of this query, which in this 1:21:47 case is the user type that we defined 1:21:49 above. 1:21:50 And GraphQLs also come with mutations. 1:21:53 You can think of these as the equivalent 1:21:55 to post, put, patch and delete methods 1:21:57 in restful APIs. So anytime you are 1:22:00 mutating a data in the database, you are 1:22:03 making a mutation query. 1:22:05 Here as you can see we have an example 1:22:07 of create user method, which accepts 1:22:09 name and of course many things in real 1:22:11 world and then it returns the user type 1:22:13 that we have defined above. 1:22:15 So if you have good schema design in 1:22:17 GraphQL, it should mirror your domain 1:22:20 model and it should be intuitive and 1:22:22 flexible. 1:22:23 Next, once you define the schema design 1:22:25 and type system, you can start querying 1:22:28 and mutating data with this GraphQL API. 1:22:31 For that we have queries for fetching 1:22:33 data. Again, this is like the get 1:22:35 requests in restful APIs and here you 1:22:37 can specify exactly what you need from 1:22:39 the user. This is the same user method 1:22:42 that we defined there in the schema. So 1:22:44 here you can also specify the exact 1:22:47 attributes like the name, posts and from 1:22:49 posts you need the title only. 1:22:51 And this will make a request to your 1:22:52 GraphQL API and return the exact data 1:22:55 that you requested. 1:22:57 Similarly, you can also use the 1:22:58 mutations that you defined. For example, 1:23:00 if you have a create post method defined 1:23:03 as a mutation, you can use this to 1:23:05 mutate the post. For example, setting 1:23:08 the title and body of the post and then 1:23:10 you also specify what data you need to 1:23:12 retrieve after this post is created, 1:23:14 which is ID and title. When it comes to 1:23:17 error handling in GraphQL APIs, this is 1:23:20 a bit different than in restful APIs 1:23:22 since GraphQL always returns 200 OK 1:23:25 status for all responses, even if there 1:23:27 was an error. In this case we have to 1:23:30 return errors field in the response, 1:23:32 which will indicate that there was an 1:23:34 error. So partial data can still be 1:23:36 returned with errors. Like in this case 1:23:38 we have the user, which is null, and 1:23:40 then we have the errors field, which 1:23:42 indicates that you have the status code 1:23:44 404, message not found and path, which 1:23:47 is the user in your schema. 1:23:49 As you can see in this case you can 1:23:50 specify the status code in the errors 1:23:53 array. Since we are returning 200 status 1:23:55 codes for all GraphQL requests, that's 1:23:57 why we have the status code specifically 1:24:00 mentioned in the errors so that we know 1:24:02 what kind of error this is, which is 1:24:04 user not found. 1:24:05 There are also best practices that we 1:24:07 normally follow when designing GraphQL 1:24:09 APIs. 1:24:10 First of all, the schemas that we saw, 1:24:12 it's a good practice to keep them small 1:24:14 and modular. Also, we should avoid 1:24:17 deeply nested queries. For example, you 1:24:19 can have a user and then nested post and 1:24:21 then within the post you can have a 1:24:23 comment, so this can be infinitely 1:24:25 nested. And to avoid that we usually 1:24:27 implement query limit depths, which is 1:24:30 how deep you can go, like how many 1:24:32 layers nested you can have in your data. 1:24:35 So you specify something like six or 1:24:37 seven layers deep. We also use 1:24:39 meaningful naming for types and fields 1:24:42 so that it also makes from the client 1:24:44 side because they both are going to use 1:24:46 the same schema. And when mutating data, 1:24:49 we always use the input types for 1:24:51 mutations. 1:24:52 Before a system can authorize or 1:24:55 restrict anything, it first needs to 1:24:57 know the identity of the requester, 1:25:00 whether it's a user accessing our 1:25:02 service through a browser or through 1:25:04 mobile app or it's a third-party service 1:25:08 trying to access our system. That's what 1:25:10 authentication does. It verifies that 1:25:13 the user or service trying to access our 1:25:16 system is who they claim to be. here is 1:25:19 where most software engineers confuse or 1:25:21 mix up concepts. They mix up 1:25:23 authentication methods with 1:25:25 authorization frameworks. They treat JWT 1:25:28 as an authentication method when in 1:25:31 reality it's just a token format. They 1:25:33 also confuse the bearer authentication 1:25:36 with JWT. They sometimes call OAuth2 an 1:25:39 authentication method when in reality 1:25:42 it's actually an authorization 1:25:43 framework. And they mix up single 1:25:46 sign-on with authentication methods when 1:25:48 it's really a user experience pattern. 1:25:51 In this video we're going to fix all of 1:25:53 that by covering first of all what 1:25:55 authentication is and then all the major 1:25:58 types of authentication starting from 1:26:00 basic to digest authentication to API 1:26:03 keys, sessions and cookies, bearer 1:26:06 authentication and JWT tokens, what are 1:26:09 access and refresh tokens. Also we'll 1:26:11 cover OAuth2, OpenID Connect, also 1:26:15 single sign-on and identity protocols 1:26:17 and understand what each one actually is 1:26:20 and where this all fits. 1:26:22 Let's first understand what is 1:26:23 authentication and then we'll get into 1:26:25 the different authentication methods. So 1:26:28 authentication really answers one simple 1:26:31 question, which is who the user is, 1:26:33 whoever is trying to access our system. 1:26:36 Let's say you have your system like your 1:26:38 API gateway, the layer of APIs, then 1:26:41 your service layer and also the data 1:26:43 storage. Before anyone can make requests 1:26:46 to your API gateway and start accessing 1:26:49 services and data, they first need to be 1:26:52 authenticated. That is where they send a 1:26:55 login request. This comes either from a 1:26:58 user or another service. This is where 1:27:01 we confirm their identity if it's valid 1:27:03 and grant access to our system, to our 1:27:06 API gateway and all the other services. 1:27:09 Or if the identity is not confirmed, 1:27:11 then we reject it with a 401 1:27:14 unauthorized response. This is the first 1:27:16 step before they will get into the 1:27:18 authorization, which is what they can 1:27:21 access and what they can do once they 1:27:23 can sign in to your system. But that's a 1:27:26 separate discussion in itself. So in 1:27:29 this one we're primarily focusing on the 1:27:31 authentication and different 1:27:33 authentication methods that we can use 1:27:35 to verify the user's identity. Now let's 1:27:38 see the different authentication methods 1:27:40 that we have to verify the identity of 1:27:43 the requester and let's start with the 1:27:45 basic authentication methods. These are 1:27:48 the basic of digest of authentication, 1:27:50 API keys and session-based 1:27:52 authentication. 1:27:54 Let's start with the very first one on 1:27:56 the list, which is the basic 1:27:58 authentication flow. This is the 1:28:00 simplest form of authentication. Let's 1:28:02 say you're making a request to the 1:28:04 server to access some resource like 1:28:06 API/users to retrieve the user data. You 1:28:10 will first receive an unauthorized 1:28:12 response because you didn't provide the 1:28:14 credentials. So we prompt the user or 1:28:17 the service to provide credentials 1:28:19 before accessing any resource in the 1:28:21 server. So in the upcoming request to 1:28:24 the same resource, they also provide the 1:28:27 authorization header and this header 1:28:29 contains the base64 encoded version of 1:28:33 the username and password for this user. 1:28:36 This is where we verify it on the server 1:28:38 side. If the credentials are valid, then 1:28:41 we respond with 200 OK status with the 1:28:44 user data returned in the body or we 1:28:47 unauthorized it again marking this as 1:28:50 credentials invalid. The problem with 1:28:53 this method is that base64 is easily 1:28:56 reversible, so this is an insecure 1:28:59 method unless it is wrapped with HTTPS 1:29:02 protocol. And even then it's rarely used 1:29:05 nowadays in production outside of the 1:29:08 internal tools because you're sending 1:29:10 the credentials with every request and 1:29:12 you're sending the base64 encoded 1:29:15 version, which is not that secure. 1:29:17 That's why we also have a digest 1:29:20 authentication, which is slightly better 1:29:22 and it uses the MD5 hashing. So, this 1:29:25 method works similar to the 1:29:28 authentication with basic version. So, 1:29:30 you are, let's say, trying to access the 1:29:32 same resource like the users. It will 1:29:35 first respond with 401 unauthorized, 1:29:38 prompting you to include the credentials 1:29:41 and then you'll make the same request 1:29:43 but with the hashed response and that 1:29:46 contain the MD5 hash version instead of 1:29:49 the plain password and username. And 1:29:53 same process as the previous one. If the 1:29:56 credentials are invalid, you will 1:29:57 receive 401 unauthorized. Otherwise, you 1:30:00 will receive the successful response 1:30:02 with the user data in the request body. 1:30:05 This is slightly better than the basic 1:30:07 off as it uses the MD5 hashing, but it's 1:30:10 still outdated and rarely used today 1:30:13 because we have better options as you 1:30:15 will see soon. And if you're wondering 1:30:17 how do we set these options in the 1:30:19 authorization? For instance, if you're 1:30:21 making the request from Postman or if 1:30:24 you're doing this from the code, then 1:30:25 you'll include it as the header in the 1:30:28 request. This is where you can set the 1:30:30 authentication type and you will notice 1:30:32 the things that we're discussing here 1:30:34 like the basic authentication, which was 1:30:36 the first version, or digest 1:30:38 authentication, which is the second 1:30:40 version, and you will see the other 1:30:42 methods available here, also the API key 1:30:45 option. And Postman calls all of these 1:30:47 authentication types to just keep it 1:30:50 simple on the interface, but that's also 1:30:52 one of the reasons why developers get 1:30:54 confused and they think that all of 1:30:56 these are authentication types when some 1:30:59 of them are authentication methods, some 1:31:01 of them are authorization frameworks. 1:31:04 Next, we have API key authentication. 1:31:06 This is where you generate a unique key 1:31:09 for each client and then they send it 1:31:11 with each request to access the 1:31:14 resources. So, for the same request as 1:31:16 we discussed, it comes to your API 1:31:19 server first and it will include either 1:31:22 authorization header or X-API-Key and 1:31:25 that will include the API key that you 1:31:28 generated for the user. These API keys 1:31:31 are typically stored in a database with 1:31:33 the key hash and also the scopes for the 1:31:36 API key. And for instance, if you ever 1:31:39 tried to access APIs by generating a key 1:31:43 on the dashboard and then it gives you 1:31:45 the key back, which you can attach to 1:31:47 the requests, that is where you already 1:31:50 used the API key of that service to 1:31:52 access the data. So, if you included 1:31:55 that key in the request, then the server 1:31:58 will first do an API key lookup in the 1:32:01 permissions or users table and if it's 1:32:03 able to verify that the API key is 1:32:06 valid, then we will authorize the 1:32:08 request and send the successful response 1:32:10 with the data in the response body. 1:32:13 Otherwise, the user will get a 401 1:32:15 unauthorized response. 1:32:17 And if the key is missing overall like 1:32:20 the authorization header or X-API-Key, 1:32:23 then we just return a 400 bad request 1:32:26 because the API key is required to 1:32:28 access this type of system. 1:32:30 One issue with API keys is that if the 1:32:33 key ever leaks, then anyone can use it 1:32:36 and start accessing the resources on 1:32:39 your behalf with your API key and there 1:32:42 is no built-in expiration unless you 1:32:44 implement it yourself. 1:32:46 Another thing is that this might seem 1:32:48 similar to JSON Web Tokens, but API keys 1:32:51 are just random strings with no embedded 1:32:54 information while in JWT, we can store 1:32:57 also information as you will see 1:32:59 shortly. So, the server here has no way 1:33:02 to know who owns the key or what 1:33:04 permissions they have without looking it 1:33:06 up in the database. 1:33:08 Next, we have the traditional web 1:33:10 approach, which is the session-based 1:33:13 authentication. This is where a user 1:33:15 logs in with their credentials and then 1:33:18 we create a session in some sort of 1:33:20 session storage. This session storage 1:33:23 can be as simple as just in memory like 1:33:26 just a variable, but the problem here is 1:33:28 that we will lose it once the server 1:33:30 restarts or crashes. The other option is 1:33:33 we can use tools like Redis, which is 1:33:36 one of the most common ones in 1:33:37 production because it's fast and it 1:33:40 supports expiration for the sessions. 1:33:42 Or we can use a dedicated database here 1:33:45 like SQL type of database. Another 1:33:48 option, which is very rare, is to use 1:33:50 the file system of the server that 1:33:53 you're using. The problem with this one 1:33:55 is that it's not scalable and overall, 1:33:58 Redis is usually the go-to for 1:34:00 production because it's fast and has 1:34:02 built-in key expiration. So, with the 1:34:04 first request, we are fetching the 1:34:06 session ID and then we set the session 1:34:09 cookie on the client side. Then for any 1:34:12 other upcoming requests that contain 1:34:14 this cookie, we look up the session in 1:34:17 the session storage here and then if the 1:34:19 session is valid, we will get back the 1:34:21 user data and we will send it with 1:34:24 authorized response. Otherwise, if it's 1:34:26 not found, if we can't find the session, 1:34:28 then this user is not authenticated, so 1:34:30 we send them an unauthorized response. 1:34:33 One challenge with session-based 1:34:35 authentications is that it is stateful, 1:34:37 which means that the server must 1:34:39 remember the sessions. We need to have 1:34:41 some sort of session storage here. And 1:34:44 it works great for traditional web apps, 1:34:46 but cannot scale as easily for APIs or 1:34:49 distributed systems. Now, let's look at 1:34:52 token-based authentication. We'll cover 1:34:54 bearer authentication, JWT tokens, 1:34:57 access and refresh tokens, and how this 1:35:00 compares to the session-based 1:35:01 authentication. Instead of sessions, 1:35:04 modern applications usually use tokens. 1:35:07 So, the client sends a token with each 1:35:09 requests. For example, we have a login 1:35:12 with credentials where user will include 1:35:15 their credentials in the authorization 1:35:17 header, which will include the type of 1:35:20 authentication, which is bearer, and 1:35:22 also the token, which we will validate 1:35:24 on the server side. One thing developers 1:35:27 confuse here is the bearer token and 1:35:29 JSON Web Tokens. Bearer token just means 1:35:32 whoever has this token gets access, so 1:35:35 it's a pattern but not a specific 1:35:37 method. And the most common type of 1:35:39 bearer token is JWT, JSON Web Token. 1:35:43 It's basically a signed JSON object that 1:35:46 contains the user ID or email for us to 1:35:50 validate the user, also expiration time 1:35:53 and other claims as we need to store 1:35:56 them like roles, permissions, and so on. 1:35:58 So, what we do on the authentication 1:36:00 server is we validate the credentials 1:36:03 once we receive that authorization 1:36:05 header and it is stateless, meaning that 1:36:08 we don't need a database here to look up 1:36:10 and that is why it's also scalable 1:36:13 compared to the session-based 1:36:14 authentication. Before the JWT, let's 1:36:18 say, revolution, a token was just a 1:36:20 string with no information and that 1:36:23 token was sent and then this was looked 1:36:26 up in some sort of database and only 1:36:28 then we could verify that the user has 1:36:31 access. The downside of that was that, 1:36:33 of course, it's still stateful because 1:36:36 we need the database access or cache, 1:36:38 which is required every time the token 1:36:40 is used. 1:36:42 With JSON Web Tokens, now we can encode 1:36:44 and verify via signing their own claims 1:36:48 and this is what now allows us to issue 1:36:50 a short-lived JWT tokens that are 1:36:53 stateless, meaning they are 1:36:55 self-contained and they don't depend on 1:36:57 anybody else. They do not need to hit 1:37:00 the database and this reduces the 1:37:03 databases load and it also simplifies 1:37:06 the authentication process for the 1:37:08 server. So, the first time we will 1:37:10 receive the credentials and validate the 1:37:12 user and if it is valid, we will 1:37:14 generate the JSON Web Token and send it 1:37:17 to the client. From this point forward, 1:37:20 the client can make requests and include 1:37:23 this bearer token, which is this 1:37:25 authorization header that contains the 1:37:27 bearer authentication with the token. 1:37:30 And that token is, most cases, it is a 1:37:33 JSON Web Token. We verify that signature 1:37:36 locally without needing to hit the 1:37:38 database. And if the token is valid, we 1:37:41 return the requested data. Otherwise, we 1:37:44 return an unauthorized response. 1:37:46 Modern systems also use two types of 1:37:49 tokens. One of them is the access token 1:37:52 and the other one is the refresh tokens. 1:37:55 The reason we need two tokens here is 1:37:57 that access tokens are short-lived and 1:38:00 they are used for API calls to the 1:38:02 server, while the refresh tokens are 1:38:04 long-lived and they are used to get new 1:38:07 access tokens, basically to renew the 1:38:10 access token. 1:38:12 Whenever user sends a login request and 1:38:15 signs in, they get both of these tokens. 1:38:17 We generate an access token that's valid 1:38:20 for 15 minutes to 1 hour and we generate 1:38:23 a refresh token that can last for days 1:38:26 or even weeks. 1:38:28 Client now will use the access token to 1:38:31 access the API and make the requests and 1:38:34 it also stores the refresh tokens. One 1:38:37 important note here is that we never 1:38:39 store it in local storage, but we store 1:38:41 it in HTTP-only cookies. This prevents 1:38:45 us from XSS attacks on the client side. 1:38:49 And after this, user will stay logged in 1:38:51 without re-entering credentials. If 1:38:54 their access token expires, they will 1:38:56 get an unauthorized response and this is 1:38:58 where we will use that refresh token, 1:39:00 which we stored, to generate a new 1:39:03 access token on the off server side. We 1:39:06 can make a request with that new token 1:39:08 and this will successfully return us the 1:39:10 data since we renewed the access token. 1:39:14 Next, let's get into OAuth 2 and OpenID 1:39:17 Connect, which are some of the 1:39:20 misunderstood concepts, and let's 1:39:22 clarify whether these are authentication 1:39:25 methods or authorization frameworks and 1:39:27 how they work. OAuth 2 is one of the 1:39:30 concepts that is often misunderstood. 1:39:32 It's an authorization framework and not 1:39:35 an authentication. So, it answers what 1:39:38 can this app access on behalf of the 1:39:40 user? 1:39:41 For instance, if you want to grant an 1:39:43 application access to your Google Drive 1:39:46 to be able to read your files from 1:39:48 there, you would typically connect your 1:39:51 Google Drive for this external 1:39:53 application. 1:39:54 And you're giving the app permission to 1:39:57 access your data. The way it works is it 1:39:59 first will redirect you to consent 1:40:02 screen from the Google OAuth 1:40:05 authentication, and it will show you the 1:40:07 permission request. And if you allow 1:40:09 access for this application to be able 1:40:13 to read the Drive files on your behalf, 1:40:16 then it will return the authorization 1:40:18 code to this external application, or it 1:40:21 can also be your application. 1:40:23 And the way it works after that is that 1:40:25 you exchange the code for token, and you 1:40:29 return the access token from Google 1:40:31 OAuth to be able to read the data. 1:40:34 That is the confusing part because 1:40:36 you're getting back an access token for 1:40:38 the Google Drive API, and you might 1:40:41 think that this is an authentication 1:40:43 method, but the access token just proves 1:40:46 that the app can access your resources. 1:40:49 But it doesn't tell the app who you are. 1:40:52 It just proves that the app is allowed 1:40:54 to access certain resources from your 1:40:56 Google Drive. So, after this point, the 1:40:59 application will be able to request 1:41:01 files with that token and return the 1:41:03 user files from Google Drive API. 1:41:07 Next, we have OpenID Connect, which adds 1:41:09 authentication on top of OAuth 2. 1:41:13 So, when you click on sign in to Google, 1:41:15 let's say, via your app, it will 1:41:17 redirect you to the authorization 1:41:20 endpoint. And this will show you the 1:41:22 login screen where you grant access to 1:41:25 sign in to Google through your 1:41:26 application. If you enter your 1:41:29 credentials and consent, then the 1:41:31 provider will return the authorization 1:41:33 code. And after this step, your 1:41:36 application will exchange the code for 1:41:38 tokens and return the access token in 1:41:41 combination with the ID token. 1:41:44 From here, the access token is for OAuth 1:41:47 2 authorization, but the ID token is a 1:41:50 JSON Web Token that contains your 1:41:52 identity, which includes your email or 1:41:54 username, user ID. 1:41:56 Which means that after this point, your 1:41:58 application is able to verify the 1:42:02 signature and extract the user's 1:42:04 identity to send the ID token for 1:42:07 verification to your backend. 1:42:09 And by having this ID token, your 1:42:12 application can now create its own 1:42:14 session and grant the access token for 1:42:17 that user. 1:42:18 This is a modern solution. It's secure 1:42:21 and also scales well. And that's also 1:42:23 why most applications nowadays use that 1:42:26 type of authentication like sign in with 1:42:28 Google, GitHub, Microsoft, and so on. 1:42:31 And lastly, let's cover single sign-on 1:42:33 and identity protocols. 1:42:36 Single sign-on is a user experience, not 1:42:39 an authentication method, which means 1:42:41 that you're able to log in once, but 1:42:43 access multiple services. For example, 1:42:46 when you want to log in to Google or 1:42:48 Okta, let's say you want to get access 1:42:51 to your Gmail, to your Google Drive, to 1:42:53 YouTube, to Google Calendar, you can do 1:42:56 this by logging in once to the identity 1:43:00 provider. Let's say it can be Google in 1:43:03 this case if you want to access these 1:43:05 services. 1:43:06 And single sign-on uses identity 1:43:08 protocols underneath to validate these 1:43:12 sessions. So, once you sign in with the 1:43:14 identity provider, let's say it's Google 1:43:17 in this case, your global session is 1:43:19 stored in a session storage. And then 1:43:21 you get back a single sign-on cookie to 1:43:24 your client to be able to access other 1:43:26 resources. So, let's say you want to 1:43:29 access Gmail for the first time, then 1:43:32 once you log in, you verify also the 1:43:34 session, and now you're able to access 1:43:36 Gmail. And for the next request, if you 1:43:39 need to access Google Drive for the next 1:43:42 one, you don't need to log in again 1:43:44 because you have these cookie and the 1:43:46 session stored in the session storage. 1:43:48 So, we just verify your session, and if 1:43:50 it's valid, then you get access to 1:43:52 Google Drive as well. And similarly to 1:43:54 YouTube, to Google Calendar, and other 1:43:56 services. 1:43:58 As I mentioned, single sign-on uses 1:44:00 identity protocols underneath. And these 1:44:03 protocols are SAML, which is Security 1:44:06 Assertion Markup Language, or OpenID 1:44:09 Connect. 1:44:10 Both of these are identity protocols, 1:44:13 which are used in combination with 1:44:15 single sign-on. 1:44:16 In case of SAML, to be able to access 1:44:19 the app, you're redirected to login, and 1:44:22 this is where we use SAML for 1:44:23 authentication. 1:44:25 This is a common solution in enterprise 1:44:27 and legacy systems like Salesforce, 1:44:30 corporate dashboards, and so on. It is 1:44:33 an XML-based protocol. So, once you want 1:44:36 to sign in, you are redirected to login, 1:44:39 and then you get back the SAML assertion 1:44:42 in XML format. And after that, your 1:44:45 identity is confirmed for the user, and 1:44:48 now you're able to access the 1:44:50 third-party application. 1:44:52 SAML is still widely used, but it's an 1:44:55 older version compared to OpenID 1:44:57 Connect. So, the next option is the 1:45:00 OpenID Connect as an identity protocol. 1:45:02 Let's say you want to access an app, and 1:45:04 in this case it's Gmail. 1:45:06 You will be redirected to login to 1:45:09 provide your credentials. And once you 1:45:11 provide your credentials, the user is 1:45:14 authenticated, and now you get back the 1:45:17 ID token in JSON Web Token format. And 1:45:20 this is what you will use for confirming 1:45:23 your identity with Gmail. 1:45:25 This is, for instance, what Google uses 1:45:27 under the hood, and it's a more modern 1:45:30 approach compared to SAML, but both of 1:45:32 them are still very secure and relevant. 1:45:36 These are the most common types of 1:45:38 authentication, and that is just the 1:45:40 first step for accessing our system. 1:45:43 After you know who the user is with 1:45:45 authentication, you need to also know 1:45:47 what they can do and what permissions 1:45:49 they have. The authentication is just 1:45:51 the first step before users can access 1:45:54 your service. So, this tells you who the 1:45:56 user is and if they are allowed to 1:45:58 access your service. That is when they 1:46:01 send a login request, and you confirm or 1:46:03 deny their identity. But after that, you 1:46:06 also have the authorization step, which 1:46:09 tells you what resources exactly this 1:46:11 user can access to. Basically, it tells 1:46:13 you what they can do, what the user can 1:46:15 do in your system. And that is what we 1:46:18 will cover next in the next video. 1:46:21 Authorization is the step that happens 1:46:23 after authentication, once someone is 1:46:25 logging in into our system. So, once 1:46:28 their login request is approved, which 1:46:30 means that the system now knows who the 1:46:32 user is, the next step is deciding what 1:46:34 they can do, which is the step of 1:46:36 authorization. It needs to check what 1:46:38 resources or actions that user has 1:46:40 permissions to access, and also what are 1:46:43 the denied actions for this user. This 1:46:45 is how we control security and privacy 1:46:48 in the systems, and in this video you'll 1:46:50 learn how applications and systems 1:46:52 manage permissions using the three main 1:46:54 authorization models. The first one is 1:46:57 role-based access control. Next, we have 1:46:59 attribute-based access control. Also, 1:47:02 access control list, which is another 1:47:04 way of managing authorization. Plus, 1:47:06 you'll learn how technologies like OAuth 1:47:08 2 and JWTs help us to enforce those 1:47:11 rules in practice. So, authentication 1:47:14 happens first, which tells us who the 1:47:16 user is and if they are allowed to 1:47:18 access our system. But on the next step, 1:47:20 we have authorization, which determines 1:47:22 what you can actually do as a user in 1:47:25 this system. If we take a look at GitHub 1:47:27 as an example and accessing repositories 1:47:30 on GitHub, there you have different 1:47:32 permissions for different users. For 1:47:34 example, user A can have write access 1:47:36 only, which means they can only push 1:47:38 code to this repo. But on the other 1:47:41 hand, we can have user B, and here you 1:47:43 can grant only read access, which means 1:47:45 they can only read this repository, but 1:47:47 they cannot push code to it, or they 1:47:49 cannot create pull requests, and so on. 1:47:52 And on the other side, we can have also 1:47:54 admin users, which have full control, so 1:47:56 they can manage all the settings for the 1:47:58 repository. They can even decide to 1:48:00 delete this repository, and so on. So, 1:48:03 you can see that different users can 1:48:05 have different access controls on 1:48:07 systems. To manage these access 1:48:09 controls, we have common authorization 1:48:12 models. So, the one that we just looked 1:48:14 at is the role-based authentication 1:48:16 model, which assigns roles to users, 1:48:18 something like admin, editor, or 1:48:21 read-only access, write-only access. And 1:48:23 this is the most common approach among 1:48:26 these authorization models. But we also 1:48:28 have attribute-based access control, 1:48:31 which is based on the user or resource 1:48:33 attributes. So, this is more flexible 1:48:36 and more complex compared to the 1:48:38 role-based authentication. 1:48:40 And the other common approach is to have 1:48:42 access control lists, ACL, and each 1:48:45 resource here has its own permissions 1:48:47 list. So, you can assign permission 1:48:49 lists to a resource, and this is what 1:48:51 will determine what resources you can 1:48:53 access. For example, this is a common 1:48:55 way of managing Google Docs, and we will 1:48:58 look at this in more detail now. And 1:49:00 each of these models has its trade-offs, 1:49:03 pros and cons. So, this depends on the 1:49:05 specific system requirements, but real 1:49:08 systems often combine also multiple 1:49:10 models together to have more complex and 1:49:13 more secure setup. So, first off we have 1:49:16 role-based access control or RBAC as an 1:49:19 acronym. Here users are assigned to 1:49:22 roles and each role has a defined set of 1:49:24 permissions. For example, as you saw 1:49:26 with the GitHub, you can have admins and 1:49:29 admins usually have full access to all 1:49:31 resources. So, they can create, they can 1:49:34 read or update resources. They can even 1:49:36 delete resources and also manage other 1:49:38 users in the roles. And next you have 1:49:41 editor, which is usually a bit less than 1:49:44 admins. So, they can edit content like 1:49:47 creating or reading content or updating 1:49:49 resources, but they cannot delete 1:49:51 resources and they cannot also manage 1:49:54 other users. 1:49:55 And next you can have viewer users, 1:49:57 which can only read data. So, they can 1:50:00 read the resources and content, but they 1:50:02 cannot update anything or they cannot 1:50:05 create anything in your system. 1:50:07 This is the most common way in 1:50:09 authorization models and this is used in 1:50:11 apps that you use daily like you saw 1:50:13 with GitHub or Stripe dashboards or CMS 1:50:16 tools, team management tools and so on. 1:50:20 The next model is attribute-based access 1:50:22 control or ABAC in short. This access 1:50:25 control goes beyond the roles. So, it 1:50:28 uses the user attributes or resource 1:50:31 attributes and environment conditions to 1:50:33 define the access. Some example policy 1:50:36 you can see here. Let's say you want to 1:50:38 only allow access if some conditions are 1:50:41 met. In this case, whenever the user 1:50:43 department is set to HR and you can 1:50:46 combine this with multiple conditions 1:50:48 like whenever the resource attribute 1:50:50 equals to internal and so on. And only 1:50:53 in this case you allow them access and 1:50:55 you either allow them read access or 1:50:57 write access. So, this can also be 1:50:59 combined with the role-based 1:51:01 authorization. 1:51:03 But in this case you are checking the 1:51:04 user model or resource model in your 1:51:07 database and based on the attributes, 1:51:09 you either allow or deny the access. So, 1:51:13 here as you can see we are checking user 1:51:15 attributes like the department, the age 1:51:17 or whatever you want to check here. Next 1:51:20 you can also combine it with resource 1:51:22 attributes like confidentiality or the 1:51:25 owner of the resource or classification. 1:51:28 And this can also be combined with 1:51:30 environment like time of the day, 1:51:32 location, device type and so on. Since 1:51:35 you're combining these attributes to 1:51:37 either grant or restrict access, this is 1:51:40 more flexible than the role-based 1:51:42 authorization, but it requires good 1:51:44 policy management and generally it's 1:51:46 more complex and you can encounter 1:51:48 conflicts here with the attribute-based 1:51:51 access control. The third common type is 1:51:53 the access control lists. Instead of 1:51:56 providing role-based access or 1:51:58 attribute-based access, you can have 1:52:00 access control list for the specific 1:52:02 resource. Let's say you have a resource 1:52:04 like a document or a JSON file. And here 1:52:07 you can have a permission list on which 1:52:09 users can access this document. Like 1:52:13 user Alice has only read access or user 1:52:16 Bob has both read and write access and 1:52:19 another user has no access to this 1:52:21 document. So, as you can see we're 1:52:23 managing two things here. First of all, 1:52:25 which users are allowed to access this 1:52:27 document and second, what are their 1:52:30 permissions. So, each of the users has 1:52:32 different permissions on this document. 1:52:35 ACLs are highly specific and also 1:52:38 user-centric, which means it's hard to 1:52:40 scale them well in systems with millions 1:52:43 of users or objects unless you manage 1:52:46 them carefully. But for example, Google 1:52:48 Drive is one example of this where you 1:52:51 have documents like a Google Doc and 1:52:54 then you share this Google Doc with your 1:52:55 colleagues, right? So, you share someone 1:52:58 with read access only and then you share 1:53:00 this Doc with someone else, but now they 1:53:02 can also edit and add comments to this 1:53:05 document. So, this is a example of ACL 1:53:09 access control list, which is used in 1:53:11 Google Drive and Google Documents. This 1:53:14 gives you more control over resources 1:53:16 and documents, but it's also harder to 1:53:19 scale with millions of users, but it's 1:53:21 possible as you can see because Google 1:53:23 Drive is using this for their documents, 1:53:25 Excel sheets and so on. 1:53:27 So, these were the access control 1:53:29 models, but how do systems enforce those 1:53:32 authorizations? This are where OAuth 2 1:53:35 and JWT or access tokens come into play. 1:53:38 So, first we have OAuth 2, which is 1:53:41 delegated authorization, which is a 1:53:43 protocol used when service wants to 1:53:45 access another service's resources on a 1:53:48 behalf of a user. For example, if you 1:53:51 want to let a third-party app read your 1:53:54 GitHub repositories. Let's say you're 1:53:56 deploying your app to Vercel, so you 1:53:58 need to give Vercel control over your 1:54:01 repository on GitHub. Instead of giving 1:54:04 your username and password to the 1:54:06 third-party application, which won't be 1:54:08 secure at all because you don't know 1:54:10 what they can do with your username and 1:54:12 password. This way you are giving them 1:54:14 full control. Instead, GitHub gives them 1:54:16 the token that represents the 1:54:19 permissions which you approved to use. 1:54:21 So, you as a user send a request with 1:54:24 the third-party app to request access to 1:54:27 your repositories and then GitHub gives 1:54:30 you the access token, which you should 1:54:32 create. So, you should also provide what 1:54:34 resources, what repositories this 1:54:37 third-party app can access and also what 1:54:39 they can do. Can they create, read, 1:54:41 update or can they delete or whatever 1:54:43 the permissions you set. And then GitHub 1:54:46 sends them the token, which contains the 1:54:48 permissions which this third-party app 1:54:50 is allowed to use. And OAuth 2 defines 1:54:53 the flow for securely issuing and 1:54:55 validating those tokens. So, you give 1:54:58 them the access token and not your 1:55:00 password, which represents the 1:55:02 permissions that you approved 1:55:03 personally. So, it can be reading 1:55:05 specific repos or also creating, pushing 1:55:08 to those repositories, but not deleting 1:55:11 those repositories. And next we have 1:55:13 also token-based authorization using JWT 1:55:16 or bearer tokens and permission logic. 1:55:19 Once a user is authenticated, most 1:55:21 systems use a token, typically a JWT 1:55:24 token or this can be also bearer token 1:55:27 that carries this information like user 1:55:30 ID, the roles like admin or editor and 1:55:33 also scopes, which is what scopes they 1:55:35 are allowed to access and whenever this 1:55:38 token is expiring and who is the issuer 1:55:41 of this token. So, whenever a user makes 1:55:44 a request, it always carries this token 1:55:46 information and reaches to the backend 1:55:48 server. This is where the server will 1:55:51 check your token and validity and it 1:55:53 will apply the appropriate permission 1:55:55 logic. So, to not confuse this with 1:55:58 authorization models, there is a key 1:56:00 distinction. The token usually carries 1:56:02 the identity and claims of your user as 1:56:05 you see it here. But authorization 1:56:07 models like role-based or 1:56:09 attribute-based, this is what defines 1:56:11 what is allowed to access as a user. So, 1:56:14 tokens are just mechanisms while these 1:56:17 are authorization models. So, in 1:56:19 summary, authorization isn't just 1:56:21 letting users in like authentication, 1:56:24 but it also controls what they can 1:56:25 access once they are in. 1:56:27 We learned what authorization is, what 1:56:29 are the three most common authorization 1:56:32 models, which are role-based, 1:56:33 attribute-based and access control list. 1:56:36 And also you saw couple of real-world 1:56:38 examples like how GitHub manages your 1:56:40 authorization tokens. And this should 1:56:43 give you an idea on when to use each 1:56:45 model based on the system that you're 1:56:47 building. And you also saw some 1:56:49 implementation patterns with OAuth 2 or 1:56:51 JWT tokens. Each of these models has 1:56:54 their own tradeoffs, their own pros and 1:56:57 cons and real systems often combine 1:56:59 multiple models to stay flexible and 1:57:01 secure. APIs are like doors into your 1:57:04 system. If you leave them unprotected, 1:57:06 then attackers and anyone can walk right 1:57:09 in and do whatever they want with your 1:57:11 user data and overall the system. That's 1:57:14 why in today's video we'll look at seven 1:57:16 proven techniques, which will help you 1:57:18 to protect your APIs from unwanted 1:57:20 attacks. 1:57:21 The first one we have in the list is 1:57:23 rate limiting, which controls how many 1:57:25 requests a client can make in a given 1:57:28 time. For example, you can set a limit 1:57:30 for user A to make, let's say 100 1:57:33 requests per period of time to your API. 1:57:37 And if they cross that limit and let's 1:57:39 say make 101 requests, then you block 1:57:42 the next request and allow some time to 1:57:45 pass before they can send their next 1:57:47 request. If you don't set this to your 1:57:49 API, then attackers can overwhelm your 1:57:51 system. They can send like thousands of 1:57:54 requests per minute and then overwhelm 1:57:56 your API, which will take your system 1:57:58 down or it can also brute force your 1:58:00 data. And these rate limits can be set 1:58:03 per endpoint. For instance, let's say 1:58:05 you have some {slash} comments endpoint 1:58:08 and here they can send a request to 1:58:09 either create a comment or fetch 1:58:11 comments. You can set that limit for 1:58:14 endpoint level. So, these comments 1:58:16 endpoint will be set to some strict 1:58:19 number of requests per minute. You can 1:58:21 also set it per user or IP address. 1:58:24 Let's say in A we have the IP address of 1:58:27 first user and then B for the second, C 1:58:29 for this one and your attacker has some 1:58:32 IP address, which corresponds to D. If 1:58:35 you get the 101st 1:58:37 request from the D IP address, then you 1:58:40 will know that this user overused the 1:58:42 API. So, you will block it at the user 1:58:45 IP level. 1:58:46 And there is also overall rate limiting 1:58:49 to protect from DDoS attacks. Since you 1:58:52 can set the rate limit to work per user 1:58:54 or per IP address, that means that this 1:58:57 attacker alone cannot send that many 1:58:59 requests. You will block it with your 1:59:01 rate limiting in the API. But what they 1:59:04 can do is they can spin up some bots and 1:59:06 each bot will have their own limit, 1:59:09 right? Let's say you've set it to 100 1:59:11 per IP address. So each of these bots 1:59:13 has 100 and overall they have more than 1:59:16 you would allow or your system could 1:59:19 handle. That's why you have also overall 1:59:21 rate limiting which can be some bigger 1:59:24 number. So whenever all the traffic 1:59:26 coming into your server reaches or 1:59:29 passes this number then you will 1:59:31 temporarily block all requests until you 1:59:33 find out the root cause. And of course 1:59:36 these numbers are just examples. So in 1:59:38 reality it's much more than 1,000, but 1:59:40 that's just an example. The second one 1:59:43 on the list is course which stands for 1:59:45 cross-origin resource sharing. This 1:59:48 controls which domain can call your API 1:59:50 from a browser and without proper course 1:59:53 malicious websites could trick users' 1:59:55 browsers into making requests on their 1:59:58 behalf. For instance, if your API is 2:00:01 only meant to serve your front-end app 2:00:03 which is at app.yourdomain.com, 2:00:06 then only requests from this source 2:00:09 should be allowed. If anyone else sends 2:00:11 you a request like 2:00:12 app.anotherdomain.com, 2:00:14 then you should block this request and 2:00:16 not allow them to use your API for 2:00:19 authenticating or using any of its data. 2:00:22 The third one is also a common one which 2:00:24 is SQL and no SQL injections. Injection 2:00:28 attacks can happen when the user input 2:00:30 is directly included in the database 2:00:32 query. For instance, attacker can modify 2:00:35 it and send some queries to read or 2:00:38 delete your data. 2:00:39 Here, for example, this part bypasses 2:00:41 the checks entirely and then attacker 2:00:44 can use this query to start reading data 2:00:47 from your database or modify anything or 2:00:50 they can also delete all the data, all 2:00:52 the user data and any other tables that 2:00:55 you have in this database. 2:00:57 So to fix this we always use 2:00:59 parameterized queries or ORM safeguards. 2:01:02 The next technique to use is firewalls. 2:01:05 A firewall acts as a gatekeeper 2:01:08 filtering the malicious traffic from the 2:01:10 other normal traffic. So typically you 2:01:13 have it between your API and the 2:01:15 incoming traffic. For example, if you 2:01:18 use the AWS's web application firewall, 2:01:21 this can block requests with unknown 2:01:23 attack patterns such as suspicious SQL 2:01:26 keywords or strange HTTP methods which 2:01:29 means it will block any suspicious 2:01:30 requests from attackers, but it will 2:01:33 allow others to bypass the request and 2:01:36 reach to your API. 2:01:38 Some APIs are also private and should 2:01:40 only be accessed from specific networks. 2:01:43 That's why we have also VPNs which 2:01:45 stands for virtual private networks. The 2:01:48 APIs that are within the VPN network can 2:01:51 only be accessed by someone who is also 2:01:53 within that same network which means 2:01:55 that some APIs are public-facing meaning 2:01:58 these APIs will allow any requests from 2:02:00 the internet from your users. But these, 2:02:04 for example, can be within the VPN 2:02:06 network which means if a user from web 2:02:08 tries to reach your API, then this 2:02:11 request will be blocked because the user 2:02:13 is not within the same network. But on 2:02:16 the other hand, if you have another user 2:02:18 here which is within the VPN network, 2:02:21 they can make a request to these APIs 2:02:23 and in this case they will bypass the 2:02:25 checks and their request will reach to 2:02:27 your APIs. 2:02:29 This is useful where you have internal 2:02:31 tools. Let's say you have internal admin 2:02:33 dashboard and the API for this admin 2:02:35 panel will only be reachable by 2:02:38 employees connected to the company VPN. 2:02:41 Next we have CSRF which stands for 2:02:43 cross-site request forgery. This tricks 2:02:46 a logged-in user's browser into making 2:02:48 unwanted requests to the API. Let's say 2:02:51 you as a user are logged in into your 2:02:54 bank system and your bank system uses 2:02:56 cookies for authentication. If the bank 2:02:59 system is not secure and they only use 2:03:02 session cookies, another malicious site 2:03:04 might use your cookie and submit a 2:03:06 hidden transferring money request 2:03:08 through your cookie. So to prevent such 2:03:11 attacks companies also use CSRF tokens 2:03:14 in combination with session cookie. So 2:03:17 the banking system will check if the 2:03:18 session cookie is present, but it will 2:03:21 also check if the CSRF token matches 2:03:23 with the one that they have. And if it 2:03:25 doesn't, then it will block this request 2:03:28 from the other unknown source while it 2:03:30 will allow request from your behalf. 2:03:33 And the last one we have is XSS or it's 2:03:35 also called cross-site scripting. This 2:03:38 lets attackers to inject scripts into 2:03:41 web pages served to other users. For 2:03:44 example, if you have a comment section 2:03:46 and this comment gets submitted to your 2:03:49 API, next your API will also store it in 2:03:52 a database. You can get normal requests 2:03:54 like nice picture or something like that 2:03:57 and this will get to your API. Your API 2:03:59 will store it in the database. So 2:04:01 everything is fine there. But what if an 2:04:03 attacker places a script in this comment 2:04:06 section and within this script they can 2:04:09 try to do many different things. For 2:04:11 example, they can try to fetch the 2:04:13 cookie for another user or they can try 2:04:16 to inject something into your database. 2:04:19 And if you allow this, then it will 2:04:21 reach to your server and the information 2:04:23 will be written into the database. Later 2:04:26 when the other users load this comment 2:04:29 section on their screen, they will get 2:04:31 also the injected comment directly into 2:04:34 their web page and the browser will 2:04:36 execute this malicious JavaScript code 2:04:39 into the other users' browser. What you 2:04:41 just watched were the first two parts of 2:04:44 my system design mastery course. I also 2:04:46 have deep dives into databases, caching, 2:04:49 CDNs and production infrastructure on my 2:04:52 YouTube channel. Just search Haik 2:04:54 Simonian on YouTube or check out the 2:04:56 first link in the description. I also 2:04:59 cover full case studies where I take 2:05:01 real systems like WhatsApp, Spotify, 2:05:04 TinyURL and more and design them from 2:05:07 scratch using the exact components you 2:05:09 learned about here. 2:05:11 And if you also want access to the full 2:05:13 system design mastery course where I go 2:05:15 deeper into all of these pillars, then 2:05:18 you'll find more information on that on 2:05:20 my channel.