Rust Server Keeps Getting Killed (Out of Memory) on Linux

A Rust server that "crashes" every few hours, or dies over and over right after you start it, is often not crashing at all. The Linux kernel is killing it because the machine ran out of memory. The fix is different from a plugin crash, so the first job is to tell the two apart.

This guide shows the exact log lines an out of memory kill leaves behind, taken from a real crash loop on our Ubuntu test host, how much RAM a Rust server actually used there, and what to change: sizing, servers per host, swap and daily restarts.

OOM kill or crash: read the journal

When the server runs as a systemd service, the reason it died is written to the journal by systemd itself. After an automatic restart, systemctl status already shows the new process and the old exit status is reset (checked on systemd 259), so the journal is the only place the reason survives.

Pull the systemd lines for your unit:

journalctl -u rust-server.service --no-pager | grep -E "Main process exited|Failed with result|restart counter|memory peak"

This is what an out of memory kill looked like on our test host when it had 4 GB of RAM:

rust-instance-...service: The kernel OOM killer killed some processes in this unit.
rust-instance-...service: Main process exited, code=killed, status=9/KILL
rust-instance-...service: Failed with result 'oom-kill'.
rust-instance-...service: Consumed 33.196s CPU time over 25.377s wall clock time, 3.4G memory peak.
rust-instance-...service: Scheduled restart job, restart counter is at 1.

Read the Failed with result value:

status=9/KILL on its own does not prove memory trouble: a force restart or an admin's kill -9 looks the same. The oom-kill result is what settles it. If you only have the KILL line, check the kernel log for the same PID.

The kernel side of the story

The kernel writes its own lines when the OOM killer fires. Read them with journalctl -k or dmesg:

journalctl -k --no-pager | grep -iE "invoked oom-killer|Out of memory"
sudo dmesg -T | grep -i "out of memory"

From the first kill of the same crash loop on our test host:

kernel: RustDedicated invoked oom-killer: gfp_mask=0x440dc0(...), order=0, oom_score_adj=0
kernel: oom-kill:constraint=CONSTRAINT_NONE,...,global_oom,task_memcg=/system.slice/rust-instance-...service,task=RustDedicated,pid=3307,uid=0
kernel: Out of memory: Killed process 3307 (RustDedicated) total-vm:9933596kB, anon-rss:3590936kB, ...

Three details matter:

• global_oom with CONSTRAINT_NONE means the whole machine was out of memory, not a limit you set on the unit.
• task_memcg names the service the victim belonged to. On a host with several servers, this tells you which one was killed.
• anon-rss is how much memory the process really held when it died, here about 3.6 GB. total-vm is reserved address space and is always much larger; ignore it.

Two traps. journalctl -k shows only the current boot by default, so after a reboot use journalctl -k -b -1. And on Ubuntu dmesg needs root, because kernel.dmesg_restrict is 1.

What an OOM crash loop looks like

A server that runs out of memory while it is still loading never gets to Server startup complete. systemd restarts it, it loads the same map, runs out of memory at the same point, and gets killed again. On our 4 GB test host, with a 4000 size map:

• The server started at 20:31:31 and was killed 25 seconds later, still generating the procedural map.
• Every restart ran for 25 to 34 seconds and died at about 3.5 GB resident memory.
• The kernel killed it 9 times in 4 and a half minutes, and the restart counter climbed to 9.
• systemd never gave up.

That last point comes from the unit. A service with Restart=always, RestartSec=5 and StartLimitIntervalSec=0 has no start limit, so a server that cannot fit in RAM is restarted every few seconds forever. That is the right choice for a server that crashes once a week, and it is also why a loop like this runs until someone notices. Watch it live:

journalctl -u rust-server.service -f | grep --line-buffered -E "Failed with result|restart counter"
systemctl show rust-server.service -p NRestarts

If you would rather have systemd stop after a burst of failures, replace StartLimitIntervalSec=0 (or the older StartLimitInterval=0 under [Service]) with a real limit in the [Unit] section, for example StartLimitIntervalSec=10min and StartLimitBurst=5. The unit then ends up failed with start-limit-hit instead of looping, and you run systemctl reset-failed before starting it again. The tradeoff: the server stays down until you act.

One more trap during a loop: the RCON port opens well before the map is loaded. On our host WebSocket RCON Started appeared about 14 seconds into each run, while the map was still generating. A script that treats "RCON port is open" as "server is up" will report a healthy server while it is being killed every 30 seconds.

How much RAM a Rust server needs

Plan for about 12 GB of RAM per server. That is the sizing Panelra asks for on every host, and our measurements show why it is not generous.

On the same test host, resized to 8 GB, the 4000 size map ran for two days with a restart every few hours. The memory peak of each run, from the journal:

The highest peak, 7.1 GB, came from a 50 minute run, and the 5 hour run peaked at 6.4 GB, so on this host peak memory did not simply follow uptime. That is a nearly empty server. What pushes the number up:

• Map size. Memory scales with the world, and area grows with the square of server.worldsize: a 4500 map is about 27% larger than a 4000 map.
• Entities. Every base, deployable, sleeper and dropped item lives in memory. Entity count climbs through a wipe, so memory does too.
• Players. More players means more entities, more network state and more of the map loaded.
• Plugins. Oxide or Carbon and every plugin with its data add their own share.

Our 4 GB machine could not even generate a 4000 map, as the loop above shows. 8 GB runs a nearly empty 4000 map with about 1 GB left available, which a busy wipe day will eat.

Check memory on your host

Four commands tell you where you stand:

free -h
systemctl show rust-server.service -p MemoryCurrent -p MemoryPeak
ps -o pid,rss,etime,cmd -C RustDedicated
journalctl -u rust-server.service --no-pager | grep "memory peak"

In free -h, read the available column, not free. Linux fills spare RAM with page cache, so free is always small; available is what a server can still claim. On our 8 GB host with the server running, free -h showed under 200 Mi free and 1.0 Gi available, with no swap.

MemoryCurrent and MemoryPeak are in bytes and cover the whole unit. rss from ps is in kilobytes. On recent systemd versions the memory peak figure is part of the Consumed ... line written each time a run ends, so it gives you a history per restart.

Too many servers on one host

Each server needs its own 12 GB. Two servers on a 16 GB box is a common way into this problem: both fit while the maps are fresh, then entity counts grow through the wipe and the host runs out.

When the whole machine runs out, the kernel kills the process with the highest OOM score, which is roughly the one using the most memory. On a Rust host that is almost always a Rust server, and not necessarily the one that grew. Server A can gain a thousand entities and server B pays for it. The task_memcg field in the kernel line tells you which one was killed.

You can contain the damage with a per unit limit in the [Service] section:

MemoryMax=10G

A server that hits its own limit is killed by the cgroup OOM killer instead, and its neighbours keep running. The unit result is still oom-kill. This does not make anything fit; it only decides who dies. The real fix is fewer servers per host or more RAM.

Swap: a safety net, not capacity

Many cloud images, our test host included, ship with no swap at all. Without swap, the first time RAM runs out something is killed immediately.

A small swap file gives the kernel room to push rarely used pages out during a short spike. As root:

fallocate -l 4G /swapfile
chmod 600 /swapfile
mkswap /swapfile
swapon /swapfile
echo '/swapfile none swap sw 0 0' >> /etc/fstab

The tradeoff is lag. If memory the game is actively using gets pushed to disk and has to be read back, server FPS drops and players rubber band long before anything is killed. A few gigabytes of swap turns a sudden kill into a slow server you can restart on your own terms. It will not let a 6 GB server run well on a 4 GB machine.

Daily restarts keep memory in check

A restart gives all of a server's memory back to the host and starts it again from a fresh baseline. A daily restart at a quiet hour is the standard way to keep growth over a long uptime from hitting the limit at your busiest time.

Warn players and save before you restart. The in-game restart command does both, then exits cleanly, and with Restart=always in the unit systemd starts the server again 5 seconds later:

restart 300

That is a 5 minute countdown in chat, then a save and a clean exit. Every finished countdown also adds one to NRestarts, so do not read that counter as a pure crash count. The restart schedule guide covers the command and a systemd timer alternative.

A restart does not fix a server that is simply too big for its host. If memory is close to the limit within an hour of starting, add RAM or move a server.

Checklist

• Journal says Failed with result 'oom-kill', or the kernel log has Out of memory: Killed process with the server's PID.
• journalctl -k -b -1 if the host rebooted since.
• Compare memory peak lines with total RAM from free -h.
• Budget about 12 GB per server, more for big maps, many players or heavy plugins.
• Count every server on the host; check task_memcg for which one was killed.
• Optional: MemoryMax= per unit so one server cannot take down the others.
• Add a small swap file as a safety net, not as capacity.
• Schedule a daily restart with a warning and a save.
• If a crash loop keeps going, stop the unit, fix memory, then start it again.

Let Panelra do this for you

The Panelra agent reads the same journal lines on your host. When a server dies it records the exit reason and whether the process ran out of memory, checking the kernel log for the server's PID when the journal only shows a SIGKILL, and sends a crash alert with the last lines of the server log. A server that dies 3 times in 15 minutes gets one crash loop alert instead of a stream of messages, and it only counts as running again once it answers RCON queries, not just when the port opens. Host RAM and per server memory charts show how close you are to the limit, and restart schedules restart your servers daily with a chat countdown and a save. See crash alerts, server monitoring and restart scheduling.

Free during the open beta. Pricing will be announced before the beta ends.