Skip to content

Network Diagnostics

The previous five articles in this module — addressing and routing, DHCP, DNS, persistent configuration, and time synchronization — each covered a separate mechanism operating at its own layer. In practice, when a problem shows up, there's no way to start without first knowing which layer it's in: the question "why can't I ping the server" alone can have at least four different causes. As the closing article of this module, this one applies the five-step methodology from The Troubleshooting Process specifically to network problems, using ping, traceroute/tracepath, ss, tcpdump, and a simple port check.

ping: The First and Fastest Check

ping -c 3 example.com

PING example.com (93.184.216.34) 56(84) bytes of data.
64 bytes from 93.184.216.34: icmp_seq=1 ttl=56 time=24.1 ms
64 bytes from 93.184.216.34: icmp_seq=2 ttl=56 time=23.8 ms
64 bytes from 93.184.216.34: icmp_seq=3 ttl=56 time=24.5 ms

--- example.com ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2003ms
rtt min/avg/max/mdev = 23.800/24.133/24.500/0.292 ms

-c 3 stops after three requests; without it, ping keeps running until Ctrl+C is pressed. The fields that matter:

  • the resolved IP (93.184.216.34) — this line alone already confirms DNS is working;
  • time — round-trip time (RTT), how long it took for a response to come back;
  • packet loss — the percentage of packets lost; 0% means every request got a response.

A Diagnostic Tree Built from ping Results

  • ping by name fails, but ping by IP works → the problem is in the DNS layer, not the network itself;
  • ping to a host on the local network (the gateway, for instance) fails → the problem is in the interface, the cable, or a local service (check with ip link/ip address from IP and Routing Tools);
  • the local network works, but a remote (internet) host doesn't respond → the problem is in routing or upstream (check the routing table);
  • one specific remote host (google.com, for instance) doesn't respond, but others do — this doesn't necessarily mean that host is "down": many organizations deliberately block ICMP as a matter of security policy. Check other, known-working hosts before drawing a conclusion.

Note

A single failed ping doesn't yet mean "the server is down." Always confirm with at least a second, independent check — a different host, or a different tool.

traceroute and tracepath: Seeing the Path

ping only says "it reaches" or "it doesn't"; traceroute shows which routers (hops) the packet passes through — useful for identifying exactly where along the path a response stops coming back.

traceroute -n example.com

traceroute to example.com (93.184.216.34), 30 hops max, 60 byte packets
 1  192.168.1.1  0.412 ms  0.389 ms  0.371 ms
 2  10.10.0.1  1.203 ms  1.198 ms  1.150 ms
 3  * * *
 4  203.0.113.5  8.921 ms  8.877 ms  8.812 ms
 ...
  • -n — don't resolve hop IPs to names via DNS; the result comes back faster, and this is useful when DNS itself is suspect;
  • each line — one hop, with the response time for three probe packets;
  • * * * — no response from that hop. This is often because an intermediate router deliberately deprioritizes traceroute traffic, or simply doesn't respond at all — if later hops do respond, this isn't necessarily a real problem.

traceroute requires root privileges or a special capability on some systems, depending on the packet type and port used. On systems without that permission, an alternative:

tracepath example.com

 1?: [LOCALHOST]                      pmtu 1500
 1:  192.168.1.1                                            0.417ms
 2:  10.10.0.1                                               1.212ms
 3:  no reply
 4:  203.0.113.5                                              8.903ms reached
     Resume: pmtu 1500 hops 4 back 4

tracepath runs without root privileges and also discovers PMTU (path MTU — the smallest packet size allowed along the whole path), but presents its results as one line per response received, rather than the same clearly numbered hop groups shown by traceroute.

mtr: traceroute and ping Combined, Continuously

traceroute takes a single snapshot of the path; intermittent loss on one hop is easy to miss in just three probes. mtr (my traceroute) runs the traceroute continuously and accumulates per-hop statistics, which makes it the standard tool for pinning down where on a path packets are being lost over time:

mtr -n --report --report-cycles 100 example.com

HOST: webprod01                   Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- 192.168.1.1                0.0%   100    0.4   0.5   0.3   1.2   0.1
  2.|-- 10.10.0.1                  0.0%   100    1.2   1.3   1.1   2.0   0.2
  3.|-- 203.0.113.5               12.0%   100    8.9   9.1   8.7  15.3   1.1

--report runs a fixed number of cycles (--report-cycles 100) and prints a summary instead of the live display; -n skips DNS resolution. The Loss% column is the one that matters, and reading it correctly is the whole skill: loss that first appears at one hop and then persists on every later hop points to that hop as the real culprit, while loss that shows up at one hop but clears on the hops after it is almost always just a router deprioritizing its own ICMP replies — not real traffic loss. That distinction is exactly what a single traceroute snapshot cannot show.

ss: Which Process Is Listening on Which Port

Modern systems use ss (socket statistics) in place of the older netstat — since it reads kernel data directly, it stays fast even with a large number of connections.

sudo ss -tlnp

State    Recv-Q  Send-Q  Local Address:Port   Peer Address:Port  Process
LISTEN   0       128     0.0.0.0:22            0.0.0.0:*          users:(("sshd",pid=812,fd=3))
LISTEN   0       511     127.0.0.1:5432         0.0.0.0:*          users:(("postgres",pid=1204,fd=6))
LISTEN   0       128     0.0.0.0:80             0.0.0.0:*          users:(("nginx",pid=1590,fd=8))

The flags:

  • -t — TCP sockets only (-u for UDP);
  • -l — listening sockets only;
  • -n — show ports and addresses as numbers rather than names (22 instead of ssh, for instance);
  • -p — show which process (name and PID) owns the port; this usually requires root.

127.0.0.1:5432 means this port only accepts connections from the local host — it's unreachable from the outside network; 0.0.0.0:80 accepts connections on every interface. This distinction is the classic key to explaining why a service isn't visible from outside.

Viewing all active (not just listening) TCP connections:

ss -tn state established

tcpdump: Looking at the Packet Itself

When ping and ss aren't enough — when it's necessary to confirm whether a request is even reaching the server, or in what format — tcpdump shows the raw traffic.

sudo tcpdump -n -i enp0s3 port 80

12:04:01.123456 IP 192.168.1.50.51422 > 192.168.1.10.80: Flags [S], seq 123456, win 64240
12:04:01.123789 IP 192.168.1.10.80 > 192.168.1.50.51422: Flags [S.], seq 987654, ack 123457, win 65160
12:04:01.124011 IP 192.168.1.50.51422 > 192.168.1.10.80: Flags [.], ack 987655, win 64240
  • -n — show numeric addresses/ports instead of resolving DNS names;
  • -i enp0s3 — only traffic on this interface;
  • port 80 — filter: only packets involving port 80;
  • Flags [S], [S.], [.] — the TCP handshake stages (SYN, SYN-ACK, ACK); if the handshake completes fully, there's no problem at the connection level — the problem may be at the application layer.

Viewing the HTTP request itself, in text form:

sudo tcpdump -n -i enp0s3 -A port 80

-A also prints packet content in ASCII (readable text); for text-based protocols like HTTP, the requested path (GET /health HTTP/1.1, for instance) becomes visible here.

Warning

Traffic captured with tcpdump can contain passwords, authentication tokens, or other sensitive data in plain text, especially over unencrypted protocols. Keep a captured file only as long as necessary and delete it once it's no longer needed; never leave it on an untrusted or shared system.

Checking Whether a Single Port Is Open or Closed

When a full packet analysis isn't necessary, a quick check of whether a specific TCP port is open:

timeout 3 bash -c 'echo > /dev/tcp/example.com/443' && echo "OPEN" || echo "CLOSED/NO RESPONSE"

This uses Bash's own built-in /dev/tcp/ pseudo-device — no extra package installation needed. Interpreting the results:

  • the connection succeeds immediately — the port is open, and a service is listening;
  • "Connection refused" — the host was reached, but nothing is listening on that port (or the firewall sent an explicit rejection);
  • no response at all, waiting until timeout triggers — the packet was lost somewhere: the firewall is silently dropping traffic, or the host is unreachable entirely.

A classic alternative is telnet (for interactive checking) or nc (netcat):

nc -zv example.com 443

-z only checks the connection without sending data; -v prints verbose output.

Practical Scenario: A Web Service Doesn't Open from the Outside

Problem: a newly deployed web application works from the server itself with curl localhost, but doesn't open in a browser from another computer.

Step 1 — confirm the problem on the server itself:

curl -sS -o /dev/null -w '%{http_code}\n' http://localhost:8080/

If the result is 200, the application itself is working — the problem is in the external connection layer.

Step 2 — check which interface the service is listening on:

sudo ss -tlnp | grep 8080

If the result shows 127.0.0.1:8080 rather than 0.0.0.0:8080, that's the cause: the service only accepts connections from the local host and is completely invisible from an external interface.

Step 3 — hypothesis: the application's configuration restricts its bind/listen address to 127.0.0.1.

Step 4 — test with one change: change the listen address in the application's configuration to 0.0.0.0 (or the specific external interface IP), reload the service, then re-check:

sudo ss -tlnp | grep 8080

Step 5 — confirm from outside: test the connection from another host (or with nc/tcpdump):

nc -zv <server_ip> 8080

If it still doesn't connect, the next hypothesis is a firewall rule — check this with the tools covered in Firewall and ufw.

Step 6 — documentation: record exactly which layer the problem was in (application configuration bound to 127.0.0.1) and how it was fixed.

Common Mistakes

Diagnosing an Entire Problem Based on a Single ping Result

ICMP is blocked by dedicated policy at many organizations; a failed ping doesn't automatically mean "the server is off." Confirm further with ss, tcpdump, or a port check.

Always Treating * * * in traceroute as a Problem

If later hops do respond, an intermediate router may simply be handling traceroute traffic at a lower priority — this is different from an actual break in the path.

Not Checking Which Interface a Service Is Listening On

Not knowing the difference between 127.0.0.1 and 0.0.0.0 is the single most common cause of "the application is running but not visible from outside." Always check this distinction first with ss -tlnp.

Expecting tcpdump Output to Be Fully Readable on Encrypted Traffic

On a TLS-encrypted connection, tcpdump only shows the handshake and metadata, not application-level content. For diagnosing a problem in an encrypted protocol, turn to the application's own logs or dedicated TLS-debugging tools instead.

Keeping a Captured Packet File Around Longer Than Necessary

A file written with tcpdump -w can contain sensitive data. Delete it once the problem is resolved, or handle it according to a secure storage policy.

Interview angle

A question worth preparing for: "a service works with curl from the server itself, but times out — not refused — from another host. What does that specific difference tell you?" A refused response means the packet reached the host and nothing was listening; a silent timeout usually means the packet never got a response at all, which points toward a firewall dropping the traffic outright, a routing problem, or the destination host being unreachable — not an application binding issue. Distinguishing these two failure modes early avoids wasting time investigating the application when the actual block is at the network or firewall layer.

Exercises

  1. Check several addresses with ping -c 3 — a local gateway, an internal server, and an external domain — and classify the results using the diagnostic tree above.
  2. Compare the output of traceroute -n and tracepath for the same address, and note the difference in output formatting.
  3. From the output of sudo ss -tlnp, find at least three services and determine whether each is listening on 127.0.0.1 or 0.0.0.0.
  4. On a test VM, start a simple HTTP server (python3 -m http.server, for instance) bound only to 127.0.0.1, then diagnose and fix the problem the same way as the practical scenario above.

Verification criterion: for exercise 4, after your fix, ss -tlnp must show the service listening on 0.0.0.0 (or the specific external interface address), and a connection test from a second host must succeed — a fix that only changes the server-local curl result is not sufficient.

References

  • man7.org: ping(8): https://man7.org/linux/man-pages/man8/ping.8.html
  • man7.org: traceroute(8): https://man7.org/linux/man-pages/man8/traceroute.8.html
  • man7.org: tracepath(8): https://man7.org/linux/man-pages/man8/tracepath.8.html
  • man7.org: ss(8): https://man7.org/linux/man-pages/man8/ss.8.html
  • TCPDUMP/LIBPCAP public repository — tcpdump(1): https://www.tcpdump.org/manpages/tcpdump.1.html
  • man7.org: nc(1) (netcat-openbsd): https://man7.org/linux/man-pages/man1/nc.1.html
  • GNU Bash Reference Manual — /dev/tcp redirection: https://www.gnu.org/software/bash/manual/html_node/Redirections.html