ss
The netstat article ended by naming ss as its replacement. This one is that replacement in full, because on a modern server ss is not a nicer netstat — it answers questions netstat can't ask at all, including the two that resolve most "the service is running but nobody can reach it" incidents.
ss stands for socket statistics, and it ships as part of iproute2 alongside ip. Where netstat walks text files in /proc/net and re-parses them, ss queries the kernel directly through a netlink socket, which is why it returns instantly on a host with tens of thousands of connections and why it can surface per-connection TCP internals that never appear in /proc/net/tcp in readable form.
The one invocation to memorise
Five flags, and it's worth being able to expand each one from memory because interviewers ask:
-t— TCP sockets-u— UDP sockets-l— listening sockets only-p— show the process that owns each socket-n— numeric; don't translate port 22 intosshor resolve addresses to names
sudo is what makes -p useful. Without root, the kernel won't tell you which process owns a socket you don't own, and the Process column comes back empty for everything but your own programs — which looks like a bug and isn't.
Netid State Recv-Q Send-Q Local Address:Port Peer Address:Port Process
udp UNCONN 0 0 127.0.0.53%lo:53 0.0.0.0:* users:(("systemd-resolve",pid=712,fd=14))
udp UNCONN 0 0 0.0.0.0:68 0.0.0.0:* users:(("dhclient",pid=845,fd=6))
tcp LISTEN 0 4096 127.0.0.53%lo:53 0.0.0.0:* users:(("systemd-resolve",pid=712,fd=15))
tcp LISTEN 0 128 0.0.0.0:22 0.0.0.0:* users:(("sshd",pid=982,fd=3))
tcp LISTEN 0 511 127.0.0.1:8080 0.0.0.0:* users:(("gunicorn",pid=1533,fd=5))
tcp LISTEN 0 128 [::]:22 [::]:* users:(("sshd",pid=982,fd=4))
Process IDs, ports, and daemon names will differ on your machine.
The Local Address column is where outages hide
Look at the two web-adjacent rows. sshd listens on 0.0.0.0:22; gunicorn listens on 127.0.0.1:8080. Those are not two ways of writing the same thing.
0.0.0.0 means "every IPv4 address this host has." A client on the LAN, a load balancer, a colleague — anyone who can route a packet to this machine can reach port 22.
127.0.0.1 means the loopback interface and nothing else. The socket is bound to an address that only exists inside this host. A request arriving on eth0 for port 8080 doesn't match that socket, so the kernel refuses it — and the application logs show nothing at all, because the connection never reached the application.
That single distinction explains a large share of "it works when I curl it on the server, it times out from anywhere else" reports. On the server, curl http://localhost:8080 goes over loopback and succeeds; from outside, it doesn't. The fix is a configuration change in the application (bind address 0.0.0.0, or better, the specific internal address), not a firewall rule — and reaching for the firewall first is the classic wasted hour.
[::] in the last row is the IPv6 equivalent of 0.0.0.0. On most Linux distributions a single [::] socket also accepts IPv4 connections through address mapping, which is why you sometimes see only the IPv6 row for a service that's demonstrably reachable over IPv4.
Recv-Q and Send-Q mean two different things depending on state
This is the part netstat users skim past, and it's genuinely useful.
For a socket in LISTEN state:
Send-Qis the maximum accept-queue depth — the backlog the application asked for.sshd's 128,gunicorn's 511,systemd-resolve's 4096 are all deliberate application settings.Recv-Qis the current number of connections that have completed the three-way handshake and are waiting for the application to callaccept().
A LISTEN row with Recv-Q sitting near Send-Q is a specific, actionable diagnosis: the network is fine, the handshake is completing, and the application is too slow or too busy to pick up the connections. Clients see this as long waits and eventual timeouts, and the application's own logs look clean because it never got the request. Once the queue is full, the kernel starts dropping new SYNs, and the client's TCP stack retries — which is why symptoms often appear as "some requests take exactly 1, 3, or 7 seconds."
For an ESTAB socket, the same columns mean something else: Recv-Q is bytes received and not yet read by the application, Send-Q is bytes written by the application and not yet acknowledged by the peer. Persistent non-zero Send-Q on many connections points at the network or the receiver; persistent non-zero Recv-Q points at your own application not reading fast enough.
Filtering, which is where ss pulls ahead
ss takes a filter expression after the flags, so you rarely need to pipe through grep:
Recv-Q Send-Q Local Address:Port Peer Address:Port
0 0 192.168.1.10:47122 142.250.185.78:443
0 0 192.168.1.10:47124 104.18.32.115:443
State names work as filters on their own, including the synthetic ones:
A large TIME-WAIT count on a busy proxy is normal and mostly harmless — it's the connection teardown mechanism doing its job. A large SYN-SENT count is not normal: it means this host keeps starting handshakes that never complete, which is what a blocked outbound port or a dead upstream looks like from the client side.
Watching one connection's TCP internals
State Recv-Q Send-Q Local Address:Port Peer Address:Port
ESTAB 0 0 192.168.1.10:47122 142.250.185.78:443
cubic wscale:8,7 rto:236 rtt:33.084/1.112 mss:1418 cwnd:10 bytes_sent:2841
bytes_acked:2841 bytes_received:14203 segs_out:12 segs_in:14 send 3.4Mbps
pacing_rate 6.8Mbps delivery_rate 2.9Mbps rcv_space:14480
-i prints the kernel's per-connection TCP state, and the fields tie directly back to the TCP deep dive: cubic is the congestion control algorithm in use, rtt:33.084/1.112 is the smoothed round-trip time and its variance in milliseconds, cwnd:10 is the current congestion window in segments, and mss:1418 is the negotiated segment size — notably below the usual 1460, which is what a tunnel with encapsulation overhead in the path looks like.
For a slow-transfer complaint, this is real evidence. High rtt with a healthy cwnd is distance or queuing. A cwnd that never grows past single digits, with retrans climbing, is loss.
Investigation: the port that "was already in use" for no reason
A deployment fails on startup:
The team's first instinct is that the old process didn't die. Check who actually holds the port:
State Recv-Q Send-Q Local Address:Port Peer Address:Port Process
LISTEN 0 511 127.0.0.1:8080 0.0.0.0:* users:(("gunicorn",pid=1533,fd=5))
There it is — a gunicorn from the previous release, still running under a supervisor that restarted it. pid=1533 is the answer, and no packet capture, log dive, or reboot was needed.
If instead that command returns nothing while the bind still fails, the cause is different and worth recognising: a socket lingering in TIME-WAIT on the same address and port can block a bind unless the program sets SO_REUSEADDR. Confirm with ss -tan state time-wait sport = :8080. The fix belongs in the application's socket setup, not in a kill command.
The two commands that answer 'is my service reachable?'
Run them in this order, on the server itself:
sudo ss -tlpn sport = :<port>— is anything listening, and on which address?ip route get <client-ip>— will replies to that client leave by the interface you expect?
If step 1 shows 127.0.0.1 when it should show 0.0.0.0, stop. Nothing downstream — firewall, load balancer, DNS — can fix a socket that isn't listening on a reachable address.
Practice
- Run
sudo ss -tulpnon a machine you administer and account for every listening socket: which process owns it, and does it need to be reachable from outside this host? Anything you can't justify is a finding, not a mystery. - Start a Python HTTP server twice — once with
python3 -m http.server 8000 --bind 127.0.0.1and once with--bind 0.0.0.0— and compare theLocal Addresscolumn each time. Then try to reach each from another machine. - Take a
LISTENrow from your own output and state, without looking back at this page, what itsRecv-QandSend-Qnumbers mean. Then do the same for anESTABrow, where the answer is different. - Using
ss -tin, find the connection on your machine with the highestrttand work out from the peer address whether the number is explained by distance.
Between ip and ss you can now describe a host's addresses, its routing decisions, and every socket attached to it. Both answer questions about the machine you're logged into. Neither answers the one the person filing the ticket actually asked — whether some other machine can reach any of it — and that takes a tool that opens a connection from the outside rather than inspecting one from within.