How do you diagnose memory failures and Deadlocks on production Linux environments with dotnet-dump and dotnet-trace?

Short answer

Collect a Memory Dump with dotnet-dump without stopping the server, then analyze it with SOS commands such as dumpheap -stat to find object leaks and thread locks.

Explanation

In cloud Linux containers, Visual Studio is not available. Microsoft provides cross-platform diagnostic tools over EventPipe. Use dotnet-dump collect -p <pid> to capture a complete memory snapshot of a container, then dotnet-dump analyze to inspect Managed Heap objects, threads blocked in sync-blocks, and root paths responsible for Memory Leaks.

Code example

dotnet tool install -g dotnet-dump
dotnet tool install -g dotnet-trace

dotnet-dump collect -p 1234 -o /tmp/app.dmp       # does not stop the process
dotnet-dump analyze /tmp/app.dmp
> dumpheap -stat                                   # which types dominate the heap? (memory leak)
> clrstack -all                                    # every thread's stack: look for threads waiting on the same lock
> syncblk                                          # which thread owns which monitor lock (deadlock)

dotnet-trace collect -p 1234 --duration 00:00:30   # CPU/event trace to open in PerfView or Speedscope

Common mistakes

• Taking Full Dumps on servers with huge amounts of RAM and limited disk space, filling the disk.

Follow-up questions

• What is the Large Object Heap (LOH) in .NET, and why is it prone to Memory Fragmentation?

Sources

• Microsoft Learn — Diagnostic tools in .NET