A deep dive into Debian 13 /tmp: What's new, and what to do if you don't like it

This discussion centers on the decision by some Linux distributions, notably Debian, to make /tmp a tmpfs (RAM-based filesystem) by default. The debate highlights various technical considerations, potential benefits, and significant drawbacks, with many users expressing concern or outright disagreement with the change.

Performance Implications and Use Cases

A primary theme is the performance impact of using RAM for /tmp. While some see it as a positive optimization, others argue it's detrimental, especially when memory starts to run out.

• Potential Benefits: For low-end systems or devices with slow storage like SD cards (e.g., Raspberry Pi), using tmpfs can offer a speed advantage.

  • As GCUMstlyHarmls states, "Actually quite handy and practical to know about, specifically in the context of a 'low end box' where I personally would prefer that RAM exist for my applications and am totally fine with /tmp tasks being a bit slow (lets be real, the whole box is 'slow' anyway and slow here is some factor of 'vm block device on an ssd' rather than 1990s spinning rust)."
  • margalabargala highlights, "Computers like a Raspberry Pi, where the OS is on a sdcard, will hugely benefit."
  • jauntywundrkind agrees, mentioning that tmpfs is "definitely on the list" for extending the lifetime of SD cards.

• Performance Drawbacks and Swapping: The major concern is what happens when the system runs out of RAM. tmpfs files can be swapped out, leading to drastically slower access times compared to files on persistent storage.

  • ars argues strongly against it: "File is tmpfs will swap out if your system is under memory pressure. If that happens, reading the file back is DRAMATICALLY slower than if you had just stored the file on disk in the first place. This change is not going to speed things up for most users, it will slow things. Instead of caching important files, you waste memory on useless temporary files. Then the system swaps it out, so you can get cache back, and then it's really slow to read back. This change is a mistake."
  • The mechanism behind this slowdown is explained by cwillu, stating, "Because swapping back in happens 4kb at a time," a detail later confirmed by worthless-trash as being "treated like any other page."
  • nrdvana notes, "The third mitigating feature the article forgot to mention is that tmpfs can get paged out to the swap partition. If you drop a large file there and forget it, it will all end up in the swap partition if applications are demanding more memory."
  • Incipient raises a practical concern: "Using the example from the article, extracting an archive. Surely that use case is entity not possible using in-memory? What happens if you're dealing with a not-unreasonable 100gb archive?"

System Stability, Swap Usage, and Memory Management

The discussion delves into the broader implications for system memory management, particularly concerning swap space, thrashing, and the handling of Out-Of-Memory (OOM) situations.

• Server vs. Desktop Swap Philosophy: There's a pronounced difference in how swap is viewed for servers versus desktops. Many experienced users consider disabling swap on servers to be standard practice for stability and predictable performance.

  • k_bx states, "Disabling swap on servers is de-facto standard for serious deployments."
  • ravetcofx elaborates, "Swap always seemed more meant for desktop use. Servers you need to give the real memory expected of the application stack."
  • fh973 adds that swap "somewhat defeats the purpose of ECC memory: your program state is now subject to complex IO path that is not end-to-end checksum protected. Also you get unpredictable performance. So typically: swap off on servers."

• Thrashing and Unresponsiveness: The risk of system thrashing, where the system spends more time swapping than processing, is a significant concern.

  • gnyman warns, "Note though that if you don't have swap now, and enable it, you introduce the risk of thrashing... It's rare but when it happens the server usually becomes completely unresponsive, so you have to hard reset it."
  • edoceo prefers predictable process death over this: "Processes die. OOM. Linux can recover and not lose data. Just unavailable for a moment."

• OOM Killer and Memory Pressure: The behavior of the Out-Of-Memory (OOM) killer and how systems handle memory pressure is also discussed.

  • mhitza mentions improvements in systemd-oomd for killing processes more quickly, but acknowledges it's still too slow for critically available servers.
  • balinares notes that even without explicit swap, the kernel will "swap out code pages instead. So, running programs. The code pages then need to be loaded again from disk when the corresponding process is next scheduled and needs them. This is not very efficient and is why a bit of actual swap space is generally recommended."

• Swap Partition Configuration: Some users pointed out that many modern Linux installations don't have swap partitions configured by default.

  • m463 jokes, "what swap partition? I meant this sort of jokingly. I think have a few linux systems that were never configured with swap partitions or swapfiles."

Filesystem Behavior, Disk Wear, and Alternatives

The discussion touches on filesystem specifics, the impact on disk wear, and alternative solutions or configurations.

• Disk Wear: A minor benefit mentioned is the reduction in disk write cycles.

  • techjamie states, "Technically it'll have some impact on the number of write cycles your disk goes through, and marginally reduce the level of wear." However, they also note, "Most disks have a lot of write cycles available that you'll be fine anyway, but it's a tiny benefit."
  • imp0cat reiterates this, noting, "And this will reduce wear on your SSD."

• Filesystem Implementation: Subtle differences in how filesystems handle memory pressure are mentioned.

  • nolist_policy points out, "in ext4 the first write to a new file will hit the disk (the code mentions it is a workaround for something). Btrfs does it correctly."

• Alternatives and Workarounds: Several alternatives and methods to disable the new default are proposed.

  • Users suggested using /var/tmp for disk-backed temporary files. nolist_policy posits, "Use /var/tmp of you want a disk backed tmp. Not sure why the article omits that."
  • ComputerGuru feels the decision mixes agendas and that "if you’re under special circumstances where it matters (eg you actually need ram speeds or are running on an SD or eMMC) then you would know to use a tmpfs yourself."
  • For those who want to revert the change, imp0cat provides a link to Debian's release notes for directions, and rwmj offers a direct command: "'systemctl mask tmp.mount' - the most important command to run in these situations."
  • worthless-trash advocates for "poly instantiated /tmp and PrivateTmp (systemd)" as a way to close attack vectors.

/tmp Cleanup and Usage

The automatic cleanup of /tmp and the intended versus actual usage of /tmp also sparked debate.

• Automatic Cleanup Concerns: The automatic deletion of files in /tmp by default (a feature introduced in newer Debian versions) was met with strong disapproval.

  • greatgib finds the auto-cleanup "awful," stating, "It looks so arbitrary and source of random unexpected bugs to have files being automatically deleted there after random time." They suspect it's another "systemd 'I know best what is good for you' broken feature."
  • PhilipRoman confirms this negative experience, losing patches while using /tmp as a ramdisk for Yocto builds.

• /tmp Usage Specifications: The discussion also touched upon the FHS (Filesystem Hierarchy Standard) and POSIX specifications for /tmp, which state that files in /tmp should not be assumed to be preserved between reboots.

  • cycomanic reminds users, "Programs must not assume that any files or directories in /tmp are preserved between invocations of the program." and that "it is recommended that files and directories located in /tmp be deleted whenever the system is booted."
  • This led to a debate about user expectations versus standards, with perlgeek suggesting persistent user data might be better suited for a "scratch" folder rather than /tmp.

General Sentiment and Criticism

A common thread throughout the discussion is skepticism and criticism of the change, often attributing it to systemd's influence.

• "Dumb Idea" and Lack of Justification: Many viewed the move to a default tmpfs for /tmp as misguided.

  • rwmj calls it "a dumb idea that came from the systemd people. They've never explained properly why it's a good idea, but it's the systemd default and for some reason distros defer to that."

• Unnecessary Complexity: Some felt that if /tmp is needed for performance, users should configure it themselves.

  • ComputerGuru argues, "if you’re under special circumstances where it matters (eg you actually need ram speeds or are running on an SD or eMMC) then you would know to use a tmpfs yourself."

• "Footguns": A general concern about the introduction of potential problems for administrators.

  • someothherguyy warns of "plenty of footguns in that general advice, local in memory storage services with default config, etc."