🔌 Protocols & Communication Patterns in System Design

Complete guide to HTTP, TCP, UDP, RPC, gRPC, REST, and GraphQL with practical examples, diagrams, and architecture discussions.

---

📋 Table of Contents

• Transport Layer (TCP/UDP)
• HTTP Protocol
• REST Architecture
• GraphQL
• RPC (Remote Procedure Call)
• gRPC
• Comparison & When to Use
• Architecture Decision Guide

---

🔷 Transport Layer (TCP/UDP)

What Are Transport Protocols?

Transport protocols are Layer 4 (Transport Layer) protocols that handle how data physically moves across networks. Think of postal services: TCP is registered mail (guaranteed delivery, slower), UDP is regular mail (may get lost, but fast).

TCP - Transmission Control Protocol

TCP ensures reliable, ordered delivery of data.

TCP Characteristics:

Aspect	Details
Connection	Connection-oriented (Three-way handshake)
Reliability	Guaranteed delivery, packet ordering
Speed	Slower due to acknowledgment overhead
Flow Control	Built-in flow and congestion control
Error Detection	Checksums, retransmissions
Port Ranges	Well-known (0-1023), registered (1024-49151)

TCP Use Cases:

✅ Web applications (HTTP/HTTPS)
✅ Email (SMTP, POP3, IMAP)
✅ File transfer (FTP, SFTP)
✅ Databases (MySQL, PostgreSQL over TCP)
✅ SSH remote access
✅ Any scenario where data loss is unacceptable

---

UDP - User Datagram Protocol

UDP prioritizes speed over reliability - "fire and forget."

UDP Characteristics:

Aspect	Details
Connection	Connectionless (No handshake)
Reliability	No guaranteed delivery
Speed	Very fast (minimal overhead)
Ordering	Packets may arrive out of order
Error Detection	Optional checksums only
Latency	Lowest latency communication

UDP Use Cases:

✅ Live video/audio streaming (YouTube, Twitch)
✅ Online gaming (multiplayer games, real-time updates)
✅ VoIP (Skype, Zoom)
✅ DNS queries (fast, stateless lookups)
✅ DHCP (network configuration)
✅ Any scenario where speed > reliability

---

TCP vs UDP Comparison

Decision Rule:

• When accuracy matters more than speed → Use TCP
• When speed matters more than accuracy → Use UDP

---

🌐 HTTP Protocol

What is HTTP?

HTTP (HyperText Transfer Protocol) is an Application Layer (Layer 7) protocol built ON TOP of TCP. It defines how clients and servers communicate on the web.

HTTP Request/Response Model

HTTP Methods (Verbs)

Method	Purpose	Idempotent	Safe	Example
GET	Retrieve data	✅ Yes	✅ Yes	GET /users/123
POST	Create resource	❌ No	❌ No	POST /users with body
PUT	Replace entire resource	✅ Yes	❌ No	PUT /users/123 with body
PATCH	Partial update	❌ No	❌ No	PATCH /users/123 field
DELETE	Remove resource	✅ Yes	❌ No	DELETE /users/123
HEAD	Like GET, no body	✅ Yes	✅ Yes	Check if resource exists
OPTIONS	Describe communication options	✅ Yes	✅ Yes	CORS preflight

HTTP Status Codes

1xx - Informational (100-199)
  100 Continue: Server ready for request body
  101 Switching Protocols: Upgrading connection (WebSocket)

2xx - Success (200-299)
  200 OK: Request succeeded
  201 Created: Resource created
  204 No Content: Success, no body

3xx - Redirection (300-399)
  301 Moved Permanently: Resource moved
  302 Found: Temporary redirect
  304 Not Modified: Use cached version

4xx - Client Error (400-499)
  400 Bad Request: Malformed request
  401 Unauthorized: Missing credentials
  403 Forbidden: Access denied (authenticated, not authorized)
  404 Not Found: Resource doesn't exist
  429 Too Many Requests: Rate limit exceeded

5xx - Server Error (500-599)
  500 Internal Server Error: Unexpected error
  502 Bad Gateway: Invalid response from backend
  503 Service Unavailable: Server overloaded
  504 Gateway Timeout: Slow backend

HTTP Versions Evolution

Version	Year	Key Features
HTTP/1.0	1996	Separate connection per request
HTTP/1.1	1997	Keep-alive connections, pipelining
HTTP/2	2015	Multiplexing, binary framing, server push
HTTP/3	2022	QUIC protocol, improved latency

---

🏗️ REST Architecture

What is REST?

REST (Representational State Transfer) is an architectural style for building APIs using HTTP. It's not a protocol, but a set of design principles.

REST Constraints

1. Client-Server Architecture: Separation of concerns
2. Stateless: Each request contains all information needed
3. Uniform Interface: Consistent, predictable API design
4. Cacheable: Responses can be cached
5. Layered: Multiple layers in the architecture
6. Code on Demand (optional): Server can extend client

REST Design Principles

REST Example Architecture

REST API Code Example (Node.js + Express)

const express = require("express");
const app = express();

app.use(express.json());

// ===========================
// REST API - User Management
// ===========================

// In-memory data store (replace with real database)
const users = [
  { id: 1, name: "Alice", email: "alice@example.com" },
  { id: 2, name: "Bob", email: "bob@example.com" },
];

let nextId = 3;

// GET /api/users - Retrieve all users
app.get("/api/users", (req, res) => {
  console.log("[REST] GET /api/users - Retrieve all users");
  res.status(200).json({
    status: "success",
    data: users,
  });
});

// GET /api/users/:id - Retrieve specific user
app.get("/api/users/:id", (req, res) => {
  const userId = parseInt(req.params.id);
  const user = users.find((u) => u.id === userId);

  if (!user) {
    return res.status(404).json({
      status: "error",
      message: "User not found",
    });
  }

  console.log(`[REST] GET /api/users/${userId} - Retrieved user`);
  res.status(200).json({
    status: "success",
    data: user,
  });
});

// POST /api/users - Create new user
app.post("/api/users", (req, res) => {
  const { name, email } = req.body;

  // Validation
  if (!name || !email) {
    return res.status(400).json({
      status: "error",
      message: "Name and email are required",
    });
  }

  const newUser = {
    id: nextId++,
    name,
    email,
  };

  users.push(newUser);

  console.log(`[REST] POST /api/users - Created user ${newUser.id}`);
  res.status(201).json({
    status: "created",
    data: newUser,
  });
});

// PUT /api/users/:id - Update entire user
app.put("/api/users/:id", (req, res) => {
  const userId = parseInt(req.params.id);
  const user = users.find((u) => u.id === userId);

  if (!user) {
    return res.status(404).json({
      status: "error",
      message: "User not found",
    });
  }

  const { name, email } = req.body;
  user.name = name || user.name;
  user.email = email || user.email;

  console.log(`[REST] PUT /api/users/${userId} - Updated user`);
  res.status(200).json({
    status: "success",
    data: user,
  });
});

// PATCH /api/users/:id - Partial update
app.patch("/api/users/:id", (req, res) => {
  const userId = parseInt(req.params.id);
  const user = users.find((u) => u.id === userId);

  if (!user) {
    return res.status(404).json({
      status: "error",
      message: "User not found",
    });
  }

  // Only update provided fields
  if (req.body.name) user.name = req.body.name;
  if (req.body.email) user.email = req.body.email;

  console.log(`[REST] PATCH /api/users/${userId} - Partially updated user`);
  res.status(200).json({
    status: "success",
    data: user,
  });
});

// DELETE /api/users/:id - Delete user
app.delete("/api/users/:id", (req, res) => {
  const userId = parseInt(req.params.id);
  const index = users.findIndex((u) => u.id === userId);

  if (index === -1) {
    return res.status(404).json({
      status: "error",
      message: "User not found",
    });
  }

  const deletedUser = users.splice(index, 1);

  console.log(`[REST] DELETE /api/users/${userId} - Deleted user`);
  res.status(200).json({
    status: "success",
    message: "User deleted",
    data: deletedUser,
  });
});

app.listen(3000, () => {
  console.log("🚀 REST API Server listening on http://localhost:3000");
  console.log("\nREST Endpoints:");
  console.log("  GET    /api/users         - Get all users");
  console.log("  GET    /api/users/:id     - Get user by ID");
  console.log("  POST   /api/users         - Create user");
  console.log("  PUT    /api/users/:id     - Update entire user");
  console.log("  PATCH  /api/users/:id     - Partially update user");
  console.log("  DELETE /api/users/:id     - Delete user");
});

REST API Request Examples

# 1. Get all users
curl -X GET http://localhost:3000/api/users

# 2. Get specific user
curl -X GET http://localhost:3000/api/users/1

# 3. Create user
curl -X POST http://localhost:3000/api/users \
  -H "Content-Type: application/json" \
  -d '{"name":"Charlie","email":"charlie@example.com"}'

# 4. Update user (full replacement)
curl -X PUT http://localhost:3000/api/users/1 \
  -H "Content-Type: application/json" \
  -d '{"name":"Alice Updated","email":"alice.new@example.com"}'

# 5. Partially update user
curl -X PATCH http://localhost:3000/api/users/1 \
  -H "Content-Type: application/json" \
  -d '{"name":"Alice Modified"}'

# 6. Delete user
curl -X DELETE http://localhost:3000/api/users/1

---

📊 GraphQL

What is GraphQL?

GraphQL is a query language and runtime for APIs. Unlike REST which has fixed endpoints, GraphQL has ONE endpoint where clients request exactly what data they need.

GraphQL vs REST Comparison

REST Over-fetching vs GraphQL

REST Problem - Over-fetching:

Request: GET /api/users/1
Response: {
  "id": 1,
  "name": "Alice",
  "email": "alice@example.com",
  "age": 30,
  "address": "123 Main St",
  "phone": "555-1234",
  "birthDate": "1994-05-15",
  ... 20 more fields I don't need!
}

If I only need id, name, email → I'm wasting bandwidth & processing!

GraphQL Solution - Request Exactly What You Need:

{
  user(id: 1) {
    id
    name
    email
  }
}

Response:

{
  "data": {
    "user": {
      "id": 1,
      "name": "Alice",
      "email": "alice@example.com"
    }
  }
}

GraphQL Query Examples

1. Simple Query

query GetUser {
  user(id: 1) {
    id
    name
    email
  }
}

2. Nested Query

query GetUserWithOrders {
  user(id: 1) {
    id
    name
    orders {
      id
      total
      status
      items {
        productName
        price
        quantity
      }
    }
  }
}

3. Mutation (Create/Update)

mutation CreateUser {
  createUser(input: { name: "John", email: "john@example.com" }) {
    id
    name
    email
    createdAt
  }
}

4. Multiple Queries (Batch)

{
  currentUser: user(id: 1) {
    name
  }

  topUsers: users(limit: 5) {
    id
    name
  }

  ordersCount: orders {
    total
  }
}

GraphQL Implementation (Node.js)

const express = require("express");
const { graphqlHTTP } = require("express-graphql");
const {
  GraphQLSchema,
  GraphQLObjectType,
  GraphQLField,
  GraphQLString,
  GraphQLInt,
  GraphQLList,
  GraphQLNonNull,
} = require("graphql");

const app = express();

// ========================
// GraphQL Schema Definition
// ========================

// Mock database
const users = [
  { id: "1", name: "Alice", email: "alice@example.com" },
  { id: "2", name: "Bob", email: "bob@example.com" },
  { id: "3", name: "Charlie", email: "charlie@example.com" },
];

// User Type
const UserType = new GraphQLObjectType({
  name: "User",
  description: "A user in the system",
  fields: () => ({
    id: { type: new GraphQLNonNull(GraphQLString) },
    name: { type: new GraphQLNonNull(GraphQLString) },
    email: { type: new GraphQLNonNull(GraphQLString) },
  }),
});

// Root Query Type
const RootQueryType = new GraphQLObjectType({
  name: "Query",
  description: "Root Query",
  fields: () => ({
    // Query single user by ID
    user: {
      type: UserType,
      description: "Get a single user by ID",
      args: {
        id: { type: new GraphQLNonNull(GraphQLString) },
      },
      resolve: (parent, args) => {
        return users.find((user) => user.id === args.id);
      },
    },

    // Query all users
    users: {
      type: new GraphQLList(UserType),
      description: "Get all users",
      resolve: () => users,
    },

    // Query user count
    userCount: {
      type: GraphQLInt,
      description: "Total number of users",
      resolve: () => users.length,
    },
  }),
});

// GraphQL Schema
const schema = new GraphQLSchema({
  query: RootQueryType,
});

// ========================
// Express Setup
// ========================

app.use(
  "/graphql",
  graphqlHTTP({
    schema,
    rootValue: {},
    graphiql: true, // Enable GraphQL IDE at /graphql
  })
);

// Simple route to show examples
app.get("/", (req, res) => {
  res.send(`
    <h1>GraphQL API Examples</h1>
    <p>Visit <a href="http://localhost:4000/graphql">/graphql</a> to use GraphQL IDE</p>
    
    <h2>Example Queries:</h2>
    
    <h3>1. Get Single User:</h3>
    <pre>
    {
      user(id: "1") {
        id
        name
        email
      }
    }
    </pre>
    
    <h3>2. Get All Users:</h3>
    <pre>
    {
      users {
        id
        name
        email
      }
    }
    </pre>
    
    <h3>3. Get User Count:</h3>
    <pre>
    {
      userCount
    }
    </pre>
  `);
});

app.listen(4000, () => {
  console.log("🚀 GraphQL Server listening on http://localhost:4000");
  console.log("📊 GraphQL IDE available at http://localhost:4000/graphql");
});

GraphQL Architecture

---

🔄 RPC (Remote Procedure Call)

What is RPC?

RPC makes calling a function on a remote server look like calling a local function. It abstracts away network complexity.

RPC Implementation Example

// ===========================
// Server (Remote Service)
// ===========================
const http = require("http");

const server = http.createServer((req, res) => {
  res.setHeader("Content-Type", "application/json");

  if (req.method === "POST" && req.url === "/rpc") {
    let body = "";

    req.on("data", (chunk) => {
      body += chunk;
    });

    req.on("end", () => {
      const request = JSON.parse(body);
      const { method, params } = request;

      let result;

      // Remote Procedures
      if (method === "getUserData") {
        const userId = params[0];
        result = getUserData(userId);
      } else if (method === "createOrder") {
        const [userId, items] = params;
        result = createOrder(userId, items);
      } else {
        result = { error: "Method not found" };
      }

      res.end(JSON.stringify({ result }));
    });
  }
});

// Remote functions
function getUserData(userId) {
  console.log(`[RPC] Executing: getUserData(${userId})`);
  return { id: userId, name: "Alice", email: "alice@example.com" };
}

function createOrder(userId, items) {
  console.log(`[RPC] Executing: createOrder(${userId}, ${items.length} items)`);
  return { orderId: 999, userId, items, status: "created" };
}

server.listen(5000, () => {
  console.log("🚀 RPC Server listening on port 5000");
});

// ===========================
// Client (Making RPC Calls)
// ===========================
const http = require("http");

function callRPC(method, params) {
  return new Promise((resolve, reject) => {
    const data = JSON.stringify({ method, params });

    const options = {
      hostname: "localhost",
      port: 5000,
      path: "/rpc",
      method: "POST",
      headers: {
        "Content-Type": "application/json",
        "Content-Length": data.length,
      },
    };

    const req = http.request(options, (res) => {
      let responseData = "";

      res.on("data", (chunk) => {
        responseData += chunk;
      });

      res.on("end", () => {
        resolve(JSON.parse(responseData));
      });
    });

    req.on("error", reject);
    req.write(data);
    req.end();
  });
}

// Using RPC - Looks like local function call!
async function main() {
  console.log("[CLIENT] Calling remote procedures...\n");

  // Call 1: Get user data
  const user = await callRPC("getUserData", [1]);
  console.log("User RPC Result:", user);

  // Call 2: Create order
  const order = await callRPC("createOrder", [1, ["item1", "item2"]]);
  console.log("Order RPC Result:", order);
}

main();

---

⚡ gRPC

What is gRPC?

gRPC (gRPC Remote Procedure Call) is a modern RPC framework using HTTP/2 and Protocol Buffers. It's extremely fast and efficient for service-to-service communication.

gRPC vs REST

Aspect	REST	gRPC
Protocol	HTTP/1.1	HTTP/2
Data Format	JSON	Protocol Buffers (binary)
Performance	Slower	10x faster
Payload Size	Large	Small (binary)
Streaming	Limited	Bidirectional streaming
Browser	Easy	Requires gRPC-Web
Use Case	Public APIs	Internal services

gRPC Communication Models

1. Unary RPC (simple request-response)
   Client  →  [Request]  →  Server
   Client  ←  [Response] ←  Server

2. Server Streaming (client gets stream)
   Client  →  [Request]   →  Server
   Client  ←  [Stream]    ←  Server
   Client  ←  [Stream]    ←  Server
   Client  ←  [End]       ←  Server

3. Client Streaming (server gets stream)
   Client  →  [Stream]  →  Server
   Client  →  [Stream]  →  Server
   Client  →  [End]     →  Server
   Client  ←  [Response] ← Server

4. Bidirectional Streaming
   Client  ↔  [Stream]  ↔  Server
   (Both send and receive simultaneously)

Protocol Buffer Definition

// users.proto
syntax = "proto3";

package users;

service UserService {
  // Unary RPC
  rpc GetUser (GetUserRequest) returns (User);

  // Server Streaming
  rpc ListUsers (ListUsersRequest) returns (stream User);

  // Client Streaming
  rpc BatchCreateUsers (stream User) returns (BatchCreateResponse);

  // Bidirectional
  rpc Chat (stream Message) returns (stream Message);
}

message GetUserRequest {
  string user_id = 1;
}

message ListUsersRequest {
  int32 limit = 1;
}

message User {
  string id = 1;
  string name = 2;
  string email = 3;
}

message Message {
  string user_id = 1;
  string text = 2;
  int64 timestamp = 3;
}

message BatchCreateResponse {
  int32 created_count = 1;
  string status = 2;
}

gRPC Server Implementation (Node.js)

const grpc = require("@grpc/grpc-js");
const protoLoader = require("@grpc/proto-loader");

// Load proto file
const packageDefinition = protoLoader.loadSync("users.proto", {
  keepCase: true,
  longs: String,
  enums: String,
  defaults: true,
  oneofs: true,
});

const usersProto = grpc.loadPackageDefinition(packageDefinition);

// ========================
// gRPC Service Implementation
// ========================

const userServiceImpl = {
  // Unary RPC
  getUser: (call, callback) => {
    const userId = call.request.user_id;
    console.log(`[gRPC] GetUser called with ID: ${userId}`);

    const user = {
      id: userId,
      name: "Alice",
      email: "alice@example.com",
    };

    callback(null, user);
  },

  // Server Streaming
  listUsers: (call) => {
    console.log("[gRPC] ListUsers called");

    const users = [
      { id: "1", name: "Alice", email: "alice@example.com" },
      { id: "2", name: "Bob", email: "bob@example.com" },
      { id: "3", name: "Charlie", email: "charlie@example.com" },
    ];

    users.forEach((user) => {
      call.write(user);
    });

    call.end();
  },

  // Bidirectional Streaming
  chat: (call) => {
    console.log("[gRPC] Chat stream started");

    call.on("data", (message) => {
      console.log(`[gRPC] Received from ${message.user_id}: ${message.text}`);

      // Echo back with server prefix
      call.write({
        user_id: "server",
        text: `You said: ${message.text}`,
        timestamp: Date.now(),
      });
    });

    call.on("end", () => {
      console.log("[gRPC] Chat stream ended");
      call.end();
    });
  },
};

// ========================
// Start gRPC Server
// ========================

const server = new grpc.Server();
server.addService(usersProto.users.UserService.service, userServiceImpl);

server.bindAsync(
  "127.0.0.1:50051",
  grpc.ServerCredentials.createInsecure(),
  (err, port) => {
    if (err) throw err;
    console.log("🚀 gRPC Server listening on port 50051");
    server.start();
  }
);

gRPC Client Implementation

const grpc = require("@grpc/grpc-js");
const protoLoader = require("@grpc/proto-loader");

const packageDefinition = protoLoader.loadSync("users.proto", {
  keepCase: true,
  longs: String,
  enums: String,
  defaults: true,
  oneofs: true,
});

const usersProto = grpc.loadPackageDefinition(packageDefinition);

const client = new usersProto.users.UserService(
  "localhost:50051",
  grpc.credentials.createInsecure()
);

// ========================
// gRPC Client Calls
// ========================

// 1. Unary RPC
console.log("=== Unary RPC ===");
client.getUser({ user_id: "1" }, (err, response) => {
  if (err) throw err;
  console.log("User:", response);
});

// 2. Server Streaming
console.log("\n=== Server Streaming ===");
const stream = client.listUsers({});

stream.on("data", (user) => {
  console.log("Received user:", user);
});

stream.on("end", () => {
  console.log("Stream ended");
});

// 3. Bidirectional Streaming
console.log("\n=== Bidirectional Streaming ===");
const chatStream = client.chat();

chatStream.write({
  user_id: "client",
  text: "Hello from client!",
  timestamp: Date.now(),
});

chatStream.on("data", (message) => {
  console.log("Server:", message.text);
});

chatStream.end();

gRPC Architecture Diagram

Why gRPC for Microservices?

✅ 10x faster than REST (binary vs JSON)
✅ Smaller payload sizes (Protocol Buffers)
✅ Bidirectional streaming (chat, real-time updates)
✅ HTTP/2 multiplexing (multiple calls on one connection)
✅ Strong typing (proto definitions)
✅ Load balancing support

❌ Not browser-friendly (need gRPC-Web wrapper)
❌ Steeper learning curve
❌ Not human-readable (binary format)

---

📊 Comparison & When to Use

Protocol Selection Matrix

Quick Reference Table

Protocol	Speed	Ease	Browser	Real-time	Use Case
REST	⭐⭐	⭐⭐⭐⭐⭐	✅	❌	Public APIs
GraphQL	⭐⭐⭐	⭐⭐⭐	✅	✅ (subscriptions)	Flexible queries
gRPC	⭐⭐⭐⭐⭐	⭐⭐	❌	✅ (streaming)	Microservices
WebSocket	⭐⭐⭐⭐	⭐⭐⭐	✅	✅	Chat, live data
UDP	⭐⭐⭐⭐⭐	⭐	❌	✅	Gaming, video

---

🏛️ Architecture Decision Guide

Small Project (MVP)

Growing Company (Scaling)

Enterprise Scale

---

📝 Best Practices

API Design Tips

✅ REST API Versioning

/api/v1/users          ← Version in URL
/api/v2/users          ← Not backward compatible

✅ Consistent Response Format

{
  "status": "success",
  "data": {...},
  "error": null,
  "timestamp": "2024-01-15T10:30:00Z"
}

✅ gRPC Service Naming

// Good
service UserService {
  rpc GetUser(GetUserRequest) returns (User);
  rpc ListUsers(ListUsersRequest) returns (stream User);
}

// Avoid
service UserOps {
  rpc fetch_user_info() returns (UserData);
}

✅ GraphQL Performance

• Implement query complexity analysis
• Add rate limiting by query cost
• Cache at the resolver level
• Use dataloader for batch queries

---

🎯 Conclusion

When to Use	Why
REST	Public APIs, simplicity, understandability
GraphQL	Complex queries, mobile apps, reduce over-fetching
gRPC	Microservices, speed, internal communication
WebSocket	Real-time chat, live notifications, bidirectional
UDP	Video streaming, gaming, speed-critical systems

Your decision should depend on:

1. Who are the API consumers?
2. What performance requirements do you have?
3. What's your team's expertise?
4. How complex are your data relationships?
5. Do you need real-time capabilities?