Linux in the Cloud
A cloud instance — an AWS EC2 instance, a Google Compute Engine VM, an Azure VM, a DigitalOcean droplet — is still, underneath the provider's dashboard and API, a Linux machine reachable over SSH, running systemd, writing to journald, and mounting block devices exactly as covered throughout this course. What genuinely differs is how that machine comes into existence and how it learns who it is: no one sits at a console and runs an installer. This article covers the two mechanisms that make that possible — the metadata service and cloud-init — and the small set of practices that follow from a server whose disk can vanish the moment the instance is terminated.
What You Will Learn
By the end of this article you will be able to:
- explain what the instance metadata service is and query it from inside a running instance;
- read and write a
cloud-inituser-data script that configures a fresh instance on first boot; - distinguish ephemeral instance storage from persistent block storage and choose correctly between them;
- reason about SSH key provisioning and IAM roles as cloud-specific extensions of authentication and authorization concepts already covered in this course;
- diagnose a cloud instance that fails to boot correctly using
cloud-init's own logs.
The Metadata Service: How an Instance Learns About Itself
Every major cloud provider exposes an HTTP endpoint, reachable only from inside a running instance, that answers the instance's own questions about itself: what is my public IP, what SSH keys should be trusted, what user-data script was I given at launch. On AWS, GCP, and most others, this lives at the well-known link-local address 169.254.169.254:
This is precisely how an SSH key you select in a cloud console ends up authorized on a brand-new instance without anyone manually editing ~/.ssh/authorized_keys — a first-boot script (covered next) fetches this exact value from the metadata service and writes it into the appropriate user's authorized_keys file, the same mechanism covered from the client side in SSH Key Authentication.
Warning
Because the metadata service is reachable at a fixed, well-known address with no authentication of its own by default, it has been the target of real, high-profile breaches — most notably a 2019 incident where a server-side request forgery (SSRF) vulnerability in a web application was used to make the application itself fetch temporary IAM credentials from its own instance's metadata service. AWS's IMDSv2 and equivalent hardened metadata service versions on other providers require a session token fetched with a PUT request first, specifically to make this class of SSRF attack much harder to exploit. Confirm IMDSv2 (or the equivalent) is enforced, not merely available, on any instance handling untrusted input.
cloud-init: Configuring a Fresh Instance on First Boot
cloud-init is the de facto standard tool — present by default on virtually every cloud-provided Ubuntu, Debian, and RHEL-family image — that runs once, on an instance's very first boot, to apply the configuration a user supplied at launch time. This user-supplied configuration is called user-data, and it is fetched from the metadata service described above.
#cloud-config
hostname: web-01
package_update: true
packages:
- nginx
- fail2ban
users:
- name: deploy
groups: sudo
shell: /bin/bash
ssh_authorized_keys:
- ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAI... deploy@laptop
runcmd:
- systemctl enable --now nginx
- systemctl enable --now fail2ban
Each field maps directly onto administration tasks already covered elsewhere in this course, just applied automatically instead of by hand:
hostnamesets the machine's name, the same value set manually elsewhere in this course withhostnamectl;usersandssh_authorized_keysprovision an account and its trusted key exactly as Users and Groups and SSH Key Authentication cover doing manually;packagesandruncmdinstall and start services the same wayapt installandsystemctl enable --nowdo interactively.
The practical value is not that any single step here is new — it is that an entire fleet of identically configured servers can be launched from one file, with zero manual intervention, which is the direct prerequisite for the Infrastructure as Code practices the next article builds on.
Reading cloud-init's Own Logs
When a freshly launched instance does not come up the way its user-data described, cloud-init's own logs are the first and most direct place to check — well before assuming the application or systemd itself is at fault:
/var/log/cloud-init-output.log captures the combined stdout/stderr of every step cloud-init ran, in order — a malformed YAML indentation in the user-data (an extremely common mistake), a package that failed to install, or a runcmd step that errored will all show up here, usually with the exact line number or command that failed.
There are two cloud-init logs, and knowing which is which saves real debugging time: /var/log/cloud-init-output.log (above) is the stdout/stderr of the commands cloud-init executed — what a package install or runcmd actually printed — while /var/log/cloud-init.log is cloud-init's own internal, timestamped debug log of every stage it moved through, at a much finer granularity. When cloud-init-output.log shows a step failed but not clearly why, /var/log/cloud-init.log is where the internal detail — which datasource was used, which module raised the error, and at exactly what stage — is recorded.
Validating the user-data's YAML schema before launching an instance from it catches most syntax errors immediately, rather than discovering them only after a launch fails silently partway through.
Ephemeral vs. Persistent Storage: A Decision With Real Consequences
Cloud instances typically offer two distinct categories of storage, and confusing them is one of the most consequential mistakes a new cloud administrator can make:
| Instance (ephemeral) storage | Persistent block storage | |
|---|---|---|
| Examples | AWS instance store, GCP local SSD | AWS EBS, GCP Persistent Disk, Azure Managed Disks |
| Survives a stop/start | No (instance store) | Yes |
| Survives instance termination | No | Yes, if not explicitly deleted with the instance |
| Typical use | Cache, scratch space, temporary build artifacts | Databases, application state, anything that must outlive the instance |
| Performance | Often faster (physically attached) | Slightly higher latency (network-attached) but resizable and snapshot-able |
Danger
Placing a production database's data directory on ephemeral instance storage is a real and repeated cause of catastrophic, unrecoverable data loss — the data does not merely risk corruption, it disappears entirely the moment the instance stops or terminates, with no recovery path. Confirm which storage category a mount point belongs to (lsblk, cross-referenced against the provider's documented device naming) before placing anything stateful on it, and prefer persistent block storage — combined with regular, tested snapshots — for any data that must survive an instance replacement.
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
xvda 202:0 0 8G 0 disk
└─xvda1 202:1 0 8G 0 part /
xvdf 202:80 0 50G 0 disk
On AWS, xvda/nvme0n1 (the root device) is typically the boot volume, usually EBS-backed and persistent by default, while additional volumes attached separately (xvdf here) need to be checked individually against what was actually requested at launch time — the provider's console or CLI (aws ec2 describe-volumes, gcloud compute disks list) is the authoritative source, since a device name alone does not reveal its persistence guarantees.
IAM Roles: Authorization Without Embedded Credentials
Cloud platforms extend the authorization concepts from Least Privilege to the instance level through IAM roles (AWS/GCP terminology; Azure calls the equivalent a "managed identity"): rather than embedding a long-lived API key inside an application's configuration file, an instance is assigned a role, and any process running on that instance can request short-lived, automatically rotating credentials from the metadata service — the same endpoint queried earlier for SSH keys.
{
"AccessKeyId": "ASIA...",
"SecretAccessKey": "...",
"Token": "...",
"Expiration": "2026-07-29T18:00:00Z"
}
The credentials returned here expire (the Expiration field), typically within hours — a direct improvement over a static, long-lived API key hardcoded into a configuration file, which never expires on its own and, if leaked, remains valid until someone notices and manually revokes it. This is the cloud-native equivalent of the least-privilege principle already covered in this course: scope the IAM role attached to an instance to exactly the permissions its workload needs, nothing broader, exactly as Least Privilege frames the same idea for a Linux user account's sudo rules.
Common Mistakes
- Hardcoding cloud credentials into an application instead of using an IAM role. This reintroduces exactly the long-lived, unrotated secret risk that IAM roles exist to eliminate, and it is a routine finding in real security audits of cloud-deployed applications.
- Placing stateful data on ephemeral storage "temporarily" and forgetting to migrate it before the first stop/terminate event. As covered above, this is not a recoverable mistake after the fact.
- Debugging a failed first-boot configuration by SSHing in and guessing, instead of reading
/var/log/cloud-init-output.logfirst. The log almost always names the exact failing step directly. - Treating IMDSv1 (or an unhardened equivalent) as acceptable on an instance running any application that accepts untrusted input. As covered above, this is a documented, exploited attack surface, not a theoretical concern.
Exercises
- On a lab instance (or a local VM with
cloud-initinstalled viacloud-localdsfor testing), query the metadata service for the instance's own hostname and public key, and explain how those two values got there. - Write a
cloud-inituser-data file that creates a non-root sudo user, installsnginx, and starts it — then intentionally introduce a YAML indentation error and usecloud-init-output.logto locate the resulting failure. - Run
lsblkon a real or lab cloud instance and, using the provider's documentation, determine which listed devices are ephemeral and which are persistent. Explain how you would find this out on a provider you had never used before. - Explain, in two or three sentences, why an IAM role's temporary credentials are a better security posture than a static API key stored in an environment variable, connecting your answer to the least-privilege principle from an earlier module.
Summary
A cloud instance is an ordinary Linux machine that discovers its own identity and configuration through a metadata service, and configures itself on first boot through cloud-init reading that service's user-data — the same administration tasks covered throughout this course (users, SSH keys, package installation, service startup), just automated and applied at launch instead of by hand. Ephemeral instance storage and persistent block storage carry very different durability guarantees that must be chosen deliberately, and IAM roles extend the least-privilege principle to cloud API access by replacing static, long-lived credentials with short-lived, automatically rotated ones. Infrastructure as Code Foundations takes this same cloud-init user-data concept and generalizes it into declaring an entire fleet of infrastructure as versioned, repeatable configuration.