DEV Community

DevOps Fundamentals profile image DevOps Fundamental
DevOps Fundamental for DevOps Fundamentals

Posted on

Ubuntu Fundamentals: LTS

#ubuntu #system #administration #lts

Long-Term Support: A Deep Dive for Production Ubuntu Systems

The recent CVE affecting glibc’s getaddrinfo function highlighted a critical operational challenge: maintaining security and stability across a large fleet of Ubuntu servers. Patching requires downtime, and frequent rolling upgrades are disruptive. This is where understanding and leveraging Long-Term Support (LTS) becomes paramount. Ignoring LTS considerations leads to technical debt, increased security risk, and ultimately, operational instability. This post assumes a production environment consisting of cloud VMs (AWS, Azure, GCP) and on-premise servers running Ubuntu Server.

What is "LTS" in Ubuntu/Linux Context?

LTS, or Long-Term Support, refers to a software release cycle where a specific version receives security updates and bug fixes for an extended period – typically 5 years for Ubuntu Server LTS releases. Unlike standard releases (supported for 9 months), LTS versions prioritize stability over bleeding-edge features.

Ubuntu LTS releases are designated with a date (e.g., 22.04 – Jammy Jellyfish). Debian, the foundation of Ubuntu, also offers LTS, but the support model differs. Debian LTS focuses on critical security updates only, and is maintained by the Debian LTS team, separate from the main Debian project.

Key system tools involved include:

  • APT (Advanced Package Tool): The package manager responsible for installing and updating software.
  • apt-get / apt: Command-line interfaces for APT.
  • unattended-upgrades: A daemon that automatically installs security updates. Configuration file: /etc/apt/apt.conf.d/50unattended-upgrades.
  • dpkg: The underlying package management system.
  • release-upgrade: Tool for upgrading between Ubuntu releases.
  • Kernel: The Linux kernel version is also a critical component of the LTS lifecycle.

Use Cases and Scenarios

  1. Production Web Servers: Maintaining a stable web server stack (Apache, Nginx, PHP, databases) is crucial. LTS provides a predictable environment for application deployment and minimizes unexpected breakages due to core system updates.
  2. Database Servers (PostgreSQL, MySQL): Database integrity is paramount. LTS ensures a consistent and well-tested base OS, reducing the risk of data corruption or performance regressions.
  3. Container Base Images: Building container images from LTS base images (e.g., ubuntu:22.04) provides a secure and stable foundation for microservices.
  4. Secure Bastion Hosts: Bastion hosts require maximum security. LTS, combined with regular security patching, minimizes the attack surface.
  5. CI/CD Runners: Stable CI/CD runners are essential for reliable builds. LTS provides a consistent environment for build processes.

Command-Line Deep Dive

  • Checking Ubuntu Version: 
lsb_release -a
Enter fullscreen mode Exit fullscreen mode

Output will show the Distributor ID, Description, Release, and Codename.

  • Checking Kernel Version: 
uname -r
Enter fullscreen mode Exit fullscreen mode

This is critical for understanding kernel security updates.

  • Listing Installed Packages: 
apt list --installed
Enter fullscreen mode Exit fullscreen mode

Useful for identifying packages requiring updates.

  • Configuring Unattended Upgrades: 
sudo nano /etc/apt/apt.conf.d/50unattended-upgrades
Enter fullscreen mode Exit fullscreen mode

Ensure Unattended-Upgrade::Allowed-Origins includes the correct LTS repository. Example:

Unattended-Upgrade::Allowed-Origins {
        "${distro_id}:${distro_codename}";
        "${distro_id}:${distro_codename}-security";
};
Enter fullscreen mode Exit fullscreen mode
  • Running Unattended Upgrades Manually: 
sudo unattended-upgrade -d -v
Enter fullscreen mode Exit fullscreen mode

-d enables debug mode, -v increases verbosity.

  • Checking APT Logs: /var/log/unattended-upgrades/unattended-upgrades.log provides a record of automatic updates.

System Architecture 

graph LR
    A[Application] --> B(Ubuntu Server LTS);
    B --> C{Systemd};
    C --> D[APT];
    C --> E[Kernel];
    D --> F[Ubuntu Repositories];
    E --> G[Hardware];
    B --> H[Unattended-Upgrades];
    H --> D;
    B --> I[Journald];
    I --> J[/var/log/syslog];
Enter fullscreen mode Exit fullscreen mode

This diagram illustrates how applications rely on the LTS Ubuntu base, managed by systemd. APT handles package updates from Ubuntu repositories, while the kernel provides the core OS functionality. Unattended-upgrades automates security patching. Journald collects system logs for debugging and monitoring.

Performance Considerations

LTS releases generally prioritize stability over performance. While not necessarily slower than standard releases, they may not include the latest kernel optimizations.

  • I/O Behavior: Monitor disk I/O during updates using iotop. Large updates can temporarily impact performance.
  • Memory Consumption: Use htop to monitor memory usage. Ensure sufficient RAM is available, especially during updates.
  • Sysctl Tuning: Adjust kernel parameters using sysctl to optimize performance. For example, increasing vm.swappiness can improve responsiveness on memory-constrained systems. 
sudo sysctl -w vm.swappiness=10
Enter fullscreen mode Exit fullscreen mode
  • Kernel Benchmarks: Use perf to profile kernel performance and identify bottlenecks.

Security and Hardening

LTS provides a longer window for security patching, but it doesn't eliminate the need for proactive security measures.

  • UFW (Uncomplicated Firewall): Enable and configure UFW to restrict network access. 
sudo ufw enable
sudo ufw default deny incoming
sudo ufw allow ssh
Enter fullscreen mode Exit fullscreen mode
  • AppArmor: Use AppArmor to confine application privileges. Check AppArmor status: sudo apparmor_status.
  • Fail2ban: Protect against brute-force attacks. Configuration files are located in /etc/fail2ban/.
  • Auditd: Enable auditd to track system events. Configuration file: /etc/audit/auditd.conf.
  • Regular Security Audits: Use tools like Lynis or OpenVAS to identify vulnerabilities.

Automation & Scripting

Ansible is ideal for automating LTS management.

---
- hosts: all
  become: true
  tasks:
    - name: Update APT cache
      apt:
        update_cache: yes
        cache_valid_time: 3600

    - name: Upgrade all packages
      apt:
        upgrade: dist
        autoremove: yes
        autoclean: yes
      register: upgrade_result
      until: upgrade_result is succeeded
      retries: 3
      delay: 60

    - name: Reboot if required
      reboot:
        msg: "Reboot initiated by Ansible for security updates"
        connect_timeout: 5
        reboot_timeout: 300
      when: upgrade_result.changed
Enter fullscreen mode Exit fullscreen mode

This Ansible playbook updates the APT cache, upgrades all packages, and reboots the server if necessary. Idempotency is ensured by the until clause.

Logs, Debugging, and Monitoring

  • Journald: Use journalctl to view system logs. Example: journalctl -u unattended-upgrades.
  • Dmesg: Check kernel messages with dmesg.
  • Netstat/ss: Monitor network connections with netstat -tulnp or ss -tulnp.
  • Strace/lsof: Troubleshoot application issues with strace and lsof.
  • System Health Indicators: Monitor CPU usage, memory usage, disk I/O, and network traffic using tools like sar or Prometheus.

Common Mistakes & Anti-Patterns

  1. Ignoring Security Updates: Failing to apply security updates promptly. Correct: Automate updates with unattended-upgrades.
  2. Upgrading to Non-LTS Releases: Upgrading to standard releases for perceived performance gains. Correct: Stick to LTS for stability.
  3. Modifying /etc/apt/sources.list Incorrectly: Adding incorrect or untrusted repositories. Correct: Use official Ubuntu repositories.
  4. Disabling Unattended Upgrades: Disabling automatic security updates. Correct: Enable and configure unattended-upgrades.
  5. Not Monitoring Logs: Failing to monitor system logs for errors or security events. Correct: Implement centralized logging and monitoring.

Best Practices Summary

  1. Always use LTS releases for production systems.
  2. Automate security updates with unattended-upgrades.
  3. Regularly audit systems for vulnerabilities.
  4. Implement a robust logging and monitoring solution.
  5. Use configuration management tools (Ansible, Puppet, Chef) for consistent configuration.
  6. Test updates in a staging environment before deploying to production.
  7. Document your LTS upgrade process.
  8. Monitor kernel versions and apply kernel security updates promptly.
  9. Harden systems with UFW, AppArmor, and Fail2ban.
  10. Maintain a rollback plan in case of upgrade failures.

Conclusion

Mastering LTS is not merely a matter of choosing a release; it’s a fundamental aspect of building reliable, secure, and maintainable Ubuntu-based infrastructure. Proactive management of LTS, coupled with robust automation and monitoring, is essential for minimizing operational risk and maximizing system uptime. Take the time to audit your current systems, build automated scripts, monitor LTS behavior, and document your standards. The investment will pay dividends in the long run.

Top comments (0)