Distributed systems are fundamentally communication systems.
Whenever two components are running on different machines, they need a network:
Application A
|
| Network
v
Application BA modern system may contain:
Browser
↓
CDN
↓
Load Balancer
↓
API Server
↓
Cache
↓
Database
↓
Message Broker
↓
WorkerMany of these components communicate over a network.
Therefore, if you understand networking, you can reason about:
A large part of System Design is understanding what happens between two machines.
Start with the simplest possible model:
Computer A
|
| Network
|
v
Computer BComputer A wants to send:
"Hello"to Computer B.
The operating system and network stack break this communication into layers.
A simplified model:
+-------------------------+
| Application |
| HTTP, DNS, SSH, etc. |
+-------------------------+
| Transport |
| TCP, UDP |
+-------------------------+
| Internet |
| IP |
+-------------------------+
| Link |
| Ethernet, Wi-Fi |
+-------------------------+
| Physical |
| Cable, radio, fiber |
+-------------------------+Each layer solves a different problem.
A network is a collection of connected devices that can exchange data.
Examples:
A very small network:
Laptop
|
Wi-Fi
|
RouterA larger network:
Laptop
|
Router
|
ISP
|
Internet
|
Cloud
|
ServerThe Internet is essentially a huge network of interconnected networks.
An IP address identifies a network interface/address used for communication.
Two major versions are:
IPv4
IPv6IPv4 uses 32 bits.
Example:
192.168.1.10It contains four decimal numbers called octets.
Each octet ranges from:
0 → 255because each octet contains 8 bits.
8 bits + 8 bits + 8 bits + 8 bits
= 32 bitsIPv6 uses 128 bits.
Example:
2001:db8:85a3::8a2e:370:7334IPv6 provides a vastly larger address space.
For System Design, the important distinction is:
IPv4 → 32-bit addresses
IPv6 → 128-bit addressesNot every IP address is directly reachable from the public Internet.
Private IPv4 address ranges include:
10.0.0.0/8
172.16.0.0/12
192.168.0.0/16Example:
Laptop
192.168.1.20may communicate through a home router that has a public Internet address.
Conceptually:
Laptop
192.168.1.20
|
v
Router
|
v
Public InternetThis leads to an important networking concept:
NAT — Network Address Translation
We will cover it later in this chapter.
An IP address identifies the network destination.
A port identifies a service endpoint on that host.
Think:
IP address = Building
Port = Apartment/officeFor example:
192.168.1.10:443means:
Host = 192.168.1.10
Port = 443Common ports include:
| Port | Typical use |
|---|---|
| 22 | SSH |
| 53 | DNS |
| 80 | HTTP |
| 443 | HTTPS |
| 25 | SMTP |
| 3306 | MySQL |
| 5432 | PostgreSQL |
| 6379 | Redis |
| 27017 | MongoDB |
The actual service configuration can vary, so port numbers are conventions rather than guarantees.
A socket is an operating-system abstraction used by applications for network communication.
A simplified view:
Application
|
v
Socket
|
v
Network stackFor TCP, a connection can be identified by a combination involving:
Source IP
Source Port
Destination IP
Destination Port
ProtocolExample:
Client:
192.168.1.20:52314
Server:
203.0.113.10:443
Protocol:
TCPThis allows one machine to maintain many simultaneous connections to the same server.
Applications don't generally send an arbitrarily large message as one giant physical transmission.
Data is processed through networking layers and transmitted in units such as packets.
Simplified:
Large message
|
v
+----+----+----+----+
| P1 | P2 | P3 | P4 |
+----+----+----+----+Each packet carries information needed to move the data through the network.
Packets can experience:
Reliable protocols such as TCP handle many of these issues at the transport layer.
One of the most important networking concepts is encapsulation.
Suppose an application sends:
HelloThe data moves down the protocol stack.
Conceptually:
Application data
↓
+------------------+
| TCP header |
| Application data |
+------------------+
↓
+------------------+
| IP header |
| TCP segment |
+------------------+
↓
+------------------+
| Link header |
| IP packet |
+------------------+Each layer adds information required by that layer.
At the destination, the reverse process happens.
Link
↓
IP
↓
TCP
↓
ApplicationThis is called decapsulation.
TCP stands for:
Transmission Control Protocol
TCP provides a reliable, ordered byte stream between endpoints.
It provides mechanisms for:
Mental model:
Application
↓
TCP
↓
Reliable ordered communicationTCP is commonly used by applications such as:
Note that modern protocols such as HTTP/3 use QUIC over UDP instead of TCP. We will discuss this later.
Before normal TCP data exchange, endpoints establish a connection.
Simplified:
Client Server
SYN ------------------------>
<------------------- SYN-ACK
ACK ------------------------>Then application data can flow.
Why does this matter in System Design?
Because creating connections has cost.
If an application repeatedly creates new connections, it can introduce:
This is one reason connection reuse / keep-alive matters.
Suppose packets are lost:
Packet 1 → received
Packet 2 → LOST
Packet 3 → receivedTCP can detect missing data and retransmit it.
Conceptually:
Sender
|
| P1
| P2 X
| P3
|
v
Receiver
Receiver indicates missing data
|
v
Sender retransmitsThis makes TCP useful when reliable ordered delivery is important.
Suppose packets arrive in a different order:
Sent:
P1 → P2 → P3
Received:
P1 → P3 → P2TCP can reorder the byte stream so the application sees the correct order.
The application normally does not have to implement packet ordering itself.
A sender can potentially produce data faster than the receiver can consume it.
Example:
Sender
10 GB/s
|
v
Network
|
v
Receiver
1 GB/sIf the sender continuously sends at 10 GB/s, the receiver can become overwhelmed.
TCP uses flow-control mechanisms to prevent a fast sender from overwhelming a receiver.
Mental model:
Fast sender
|
| controlled rate
v
Receiver capacityFlow control protects the receiver.
Congestion control deals with the network itself.
Imagine:
Server A
\
Server B ----> Shared network ----> Destination
/
Server CIf everyone sends huge amounts of traffic simultaneously, network links can become congested.
TCP uses congestion-control algorithms to adjust sending behavior based on network conditions.
This distinction is important:
Flow control
→ Protect receiver
Congestion control
→ React to network congestionUDP stands for:
User Datagram Protocol
UDP is connectionless and provides a simpler datagram transport.
Compared with TCP, UDP does not provide TCP's built-in guarantees for:
Mental model:
Application
↓
UDP datagrams
↓
NetworkUDP can be useful when:
Examples include certain real-time and streaming protocols.
| Feature | TCP | UDP |
|---|---|---|
| Connection-oriented | Yes | No |
| Reliable delivery built in | Yes | No |
| Ordered byte stream | Yes | No |
| Retransmission | Yes | No |
| Flow control | Yes | No TCP-style mechanism |
| Congestion control | Yes | Application/protocol dependent |
| Overhead | Higher | Lower |
| Typical use | Web, databases, SSH | DNS, real-time protocols, QUIC transport |
Important:
"UDP is faster than TCP" is an oversimplification.
Actual performance depends on the protocol, network, workload and application behavior.
Modern protocols can build sophisticated reliability and congestion-control mechanisms on top of UDP.
Modern web systems also use:
HTTP/3
↓
QUIC
↓
UDPInstead of:
HTTP/1.1 or HTTP/2
↓
TCPQUIC implements transport functionality over UDP, including:
You do not need to master QUIC yet.
For now, remember:
Traditional HTTPS:
HTTP → TLS → TCP → IP
HTTP/3:
HTTP/3 → QUIC → UDP → IPThis will become relevant when discussing modern web performance.
Suppose your computer wants to communicate with a server thousands of kilometers away.
The packet doesn't travel directly from your computer to the server.
It passes through network devices.
Simplified:
Laptop
↓
Home Router
↓
ISP
↓
Internet Router
↓
Internet Router
↓
Cloud Network
↓
ServerRouters make forwarding decisions based on destination network information.
A router connects networks and forwards packets between them.
Example:
Network A
|
Router
|
Network BA home router may connect:
Home LAN
|
Router
|
Internet / ISPCloud networks also use routing extensively.
LAN stands for:
Local Area Network
Examples:
Example:
Laptop
|
+--- Phone
|
+--- Printer
|
+--- ServerAll may communicate within the local network.
WAN stands for:
Wide Area Network
It connects geographically separated networks.
Example:
Delhi Data Center
|
|
Internet
|
|
Mumbai Data CenterWAN communication generally introduces more latency than local communication because of:
At the local network/link layer, devices use hardware/network-interface identifiers such as MAC addresses.
Example:
3C:52:82:AB:12:EFThe important distinction:
MAC address
→ Link/local network identity
IP address
→ Network-layer addressingYou do not normally design application architecture around MAC addresses, but understanding the distinction helps explain networking layers.
In IPv4 local networks, a host may need to discover the MAC address associated with a local IP address.
ARP stands for:
Address Resolution Protocol
Simplified:
Who has 192.168.1.1?
↓
Router:
"I have it."
↓
MAC address returnedARP is mainly a local-network concept.
IPv6 uses Neighbor Discovery rather than ARP.
NAT stands for:
Network Address Translation
A common home-network scenario:
Laptop
192.168.1.20
|
v
Home Router
Public IP
|
v
InternetPrivate addresses are not globally unique.
The router can translate private network traffic to a public address.
Conceptually:
192.168.1.20:52314
↓
NAT
↓
203.0.113.5:41001The external server sees the public-side address.
One major reason is IPv4 address conservation.
Private networks can reuse addresses such as:
192.168.1.10in many different locations.
For example:
Home A → 192.168.1.10
Home B → 192.168.1.10
Office C → 192.168.1.10NAT allows these private networks to communicate through shared public IPv4 addresses.
Bandwidth is the maximum data-transfer capacity of a network link under specified conditions.
Example:
1 Gbpsmeans approximately one billion bits per second of link capacity.
Important distinction:
Bandwidth ≠ LatencyA network can have:
High bandwidth
High latencyor:
Low bandwidth
Low latencyThese are different characteristics.
Latency is the time required for data or a request to travel through a system.
Network latency can be influenced by:
Example:
Client
|
| 80 ms
v
ServerA request may experience multiple network segments:
Client
↓ 10 ms
Router
↓ 15 ms
ISP
↓ 20 ms
Cloud
↓ 15 ms
ServerTotal network contribution is affected by the complete path.
Signals take physical time to travel.
A packet cannot travel from India to the United States instantaneously.
The physical medium matters:
Therefore, geographic distribution has unavoidable physical latency.
This is one reason global systems deploy services closer to users.
RTT means:
Round-Trip Time
It measures the time for communication to go from one endpoint to another and back.
Conceptually:
Client
|
| ------>
|
Server
|
| <------
|
ClientIf:
Request = 50 ms
Response path = 50 msRTT is approximately:
100 msReal measurements depend on the network and protocol.
RTT is extremely important in distributed systems.
Suppose a request performs five sequential remote calls:
Client
↓
Service A
↓
Service B
↓
Service C
↓
Service D
↓
DatabaseIf each remote interaction adds significant latency, total response time can grow rapidly.
This is one reason system designers try to:
These are often confused.
How long does one operation take?
Request → 100 msHow much work can the system process?
100,000 requests/secondA system can have:
High throughput + high latencyor:
Low latency + low throughputThey are separate dimensions.
Packets can be lost due to:
Example:
P1 → received
P2 → LOST
P3 → receivedFor TCP, loss can trigger retransmission.
For protocols that do not automatically retransmit, the application/protocol may decide what to do.
Jitter is variation in packet delay.
Suppose packets arrive with delays:
20 ms
22 ms
21 ms
80 ms
25 msThe variation is significant.
Jitter matters particularly for real-time applications such as:
Imagine a road:
Few cars
→ smooth traffic
Too many cars
→ traffic jamNetworking can behave similarly.
When traffic exceeds available capacity:
Packets
↓
Queue
↓
Delay increases
↓
Potential packet lossCongestion can therefore increase both:
Opening a new connection for every request can be expensive.
Instead:
Application
|
+--- Connection 1
+--- Connection 2
+--- Connection 3
+--- Connection 4The application maintains a pool of reusable connections.
This is common with:
Benefits:
But too many connections can also overwhelm a service.
Instead of:
Request 1
→ open connection
→ close
Request 2
→ open connection
→ close
Request 3
→ open connection
→ closeconnections can be reused:
Open connection
|
+--> Request 1
|
+--> Request 2
|
+--> Request 3
|
+--> Request 4Connection reuse reduces repeated setup overhead.
HTTP versions and transport protocols implement connection management differently, but the general optimization idea is important.
Every network request can potentially wait indefinitely if there is no timeout.
Example:
Service A
|
| Request
v
Service B
|
X No responseWithout a timeout:
Service A waits...
Service A waits...
Service A waits...With a timeout:
Request
|
| wait
|
X timeout
|
v
Fallback / retry / errorTimeouts are essential in distributed systems.
If a request fails temporarily, a client may retry.
Example:
Request
|
X failure
|
Retry
|
X failure
|
Retry
|
SuccessBut retries can be dangerous.
Suppose:
100 requestsall retry simultaneously.
The overloaded service can receive:
100 original requests
+
100 retries
+
100 second retriesThis can amplify the outage.
This is why production systems often use:
We will study these in the reliability section.
Instead of retrying immediately:
Retry 1 → 100 ms
Retry 2 → 200 ms
Retry 3 → 400 ms
Retry 4 → 800 msThe delay increases.
Random jitter can prevent many clients from retrying at exactly the same time.
Conceptually:
Failure
↓
Wait
↓
Retry
↓
Failure
↓
Wait longer
↓
RetryIn distributed systems, assume networks can fail.
Examples:
The destination may not be accepting connections.
The destination doesn't respond within the configured period.
An established connection is unexpectedly terminated.
Some packets fail to arrive.
The application cannot resolve the destination.
One service can reach another, but a third service cannot.
Partial failure is particularly important in distributed systems.
In a single-machine application:
Server crashes
→ Application stopsIn a distributed system:
Service A
|
X
Service B
|
v
Service CService A may still be healthy while communication with B fails.
This is called a partial failure.
Distributed systems are difficult partly because:
You cannot always tell whether the remote service is slow, dead, disconnected, or merely unreachable.
This idea becomes central later.
Suppose:
API
|
+--> User Service
|
+--> Product Service
|
+--> Recommendation ServiceRecommendation Service becomes slow.
If API waits indefinitely:
API
|
| waits
v
Recommendation ServiceThen API requests may pile up.
A better architecture may use:
API
|
+--> User Service
|
+--> Product Service
|
+--> Recommendation Service
|
X timeout
|
v
fallbackThe system can still return the core product response without waiting forever for recommendations.
This is graceful degradation.
A service may be slow because it is:
CPU usage = 100%or:
Waiting for remote serviceor:
Waiting for disk/databaseThese require different optimization strategies.
System design requires identifying the actual bottleneck instead of guessing.
Possible bottlenecks include:
Client
↓
Network bandwidth
↓
Load balancer
↓
Application
↓
Database connection pool
↓
DatabaseFor each layer, ask:
What happens when demand exceeds capacity?
Example:
Application capacity = 50,000 RPS
Database capacity = 10,000 QPSThe database may become the bottleneck.
Adding more application servers won't necessarily solve the problem.
A simplified example:
Request A → very slow
Request B → fast
Request C → fastIf the transport or application architecture forces B and C to wait behind A, overall latency can increase.
Modern protocols such as HTTP/2 and HTTP/3 provide mechanisms for multiplexing multiple streams, although each has different transport-level behavior.
This is an advanced topic, but remember:
How requests share connections can affect latency.
A system should not expose every component directly to the Internet.
A typical architecture might be:
Internet
|
v
Public Load Balancer
|
v
Private Application Network
|
+--> Cache
|
+--> DatabaseThe database may not have a public IP or public inbound access.
This reduces the attack surface.
Security architecture will be covered in detail later.
Suppose:
Browser
|
| HTTPS
v
api.example.comA simplified journey:
1. Browser resolves DNS
2. Browser obtains destination information
3. Network connection is established
4. TLS security is established
5. HTTP request is sent
6. Routers forward packets
7. Load balancer receives traffic
8. Backend server processes request
9. Response returns
10. Browser processes responseAt every stage there can be:
This is why networking is foundational to System Design.
Not exactly.
An IP address identifies a network endpoint/address.
A host can run multiple services using different ports.
IP: 10.0.0.10
10.0.0.10:80
10.0.0.10:443
10.0.0.10:5432TCP can detect and retransmit missing data, but the underlying network can still lose packets.
TCP provides a reliable abstraction to the application.
Not universally.
UDP has lower protocol overhead, but application behavior, congestion, retransmission, network conditions and protocol design all matter.
No.
Bandwidth and latency are different.
Bandwidth → how much data can move
Latency → how long communication takesRetries can improve recovery from transient failures, but excessive retries can make an outage worse.
Not necessarily.
A timeout means the caller did not receive an acceptable response within its configured time.
The server could be:
This uncertainty is fundamental to distributed systems.
TCP provides a reliable, ordered byte-stream abstraction with connection management, flow control and congestion control.
UDP provides connectionless datagrams without TCP's built-in reliable ordered delivery.
An address used by the network layer to identify a network destination/interface.
A logical endpoint used to identify a service/application endpoint on a host.
Network Address Translation translates addresses between network contexts, commonly allowing private IPv4 hosts to communicate through a public IPv4 address.
The time required for an operation or communication to complete.
The capacity of a communication link to transfer data over time.
Round-Trip Time: the time for communication to travel from one endpoint to another and back.
A condition where transmitted packets do not successfully reach their intended destination.
They prevent a caller from waiting indefinitely for a remote operation.
Because many clients retrying simultaneously can amplify load and make an outage worse.
Your service makes:
5 sequential API callsEach call has an average network latency of:
40 msWhat is the approximate network contribution if all five calls are strictly sequential?
Then consider what changes if the calls are independent and can execute concurrently.
A service can process:
20,000 requests/secondbut its database can process only:
5,000 queries/secondWhere is the bottleneck?
Would adding more application servers necessarily solve it?
Explain why.
Design a service that must communicate with another service that occasionally takes 10 seconds to respond.
Discuss:
Create a simple conceptual simulator that models:
Client
↓
Router
↓
ServerAdd configurable:
For example:
Latency: 100 ms
Packet loss: 5%
Timeout: 500 ms
Retries: 2Then reason about how these settings affect user-visible behavior.
The purpose is not to implement a real network stack.
The purpose is to understand network behavior.
We can now connect networking to architecture.
Suppose:
100,000 users
|
v
Load Balancer
|
+----> App Server A
|
+----> App Server B
|
+----> App Server CEach request involves network communication.
Now suppose each application server communicates with:
Redis
Database
User Service
Payment Service
Notification ServiceOne request may cause many network interactions.
Therefore:
More distributed components
↓
More network communication
↓
More latency + more failure modesThis is one of the central trade-offs of distributed architecture.
Remember these relationships:
IP
↓
Identifies network destination
Port
↓
Identifies service endpoint
TCP
↓
Reliable ordered byte stream
UDP
↓
Connectionless datagrams
Router
↓
Forwards packets between networks
NAT
↓
Translates network addresses
Bandwidth
↓
Data-transfer capacity
Latency
↓
Time required
RTT
↓
Time there + back
Packet loss
↓
Data fails to arrive
Timeout
↓
Stop waiting after a limit
Retry
↓
Try again, carefullyThe most important idea is:
Networks are not free, instantaneous or perfectly reliable.
A good system designer accounts for:
We covered:
These concepts form the networking foundation for the rest of the course.
We will go much deeper into DNS:
Domain Name
↓
DNS Resolver
↓
Root Server
↓
TLD Server
↓
Authoritative Server
↓
IP AddressWe will study:
Mark it as complete to track your mastery.