Platform Settings for Dell PowerEdge Servers

Operating System Tuning Parameters

kernel.randomize_va_space (ASLR)

This setting can be used to select the type of process address space randomization. Defaults differ based on whether the architecture supports ASLR, whether the kernel was built with the CONFIG_COMPAT_BRK option or not, or the kernel boot options used.
Possible settings:

• 0: Turn process address space randomization off.
• 1: Randomize addresses of mmap base, stack, and VDSO pages.
• 2: Additionally randomize the heap. (This is probably the default.)

Disabling ASLR can make process execution more deterministic and runtimes more consistent. For more information see the randomize_va_space entry in the Linux sysctl documentation.

Transparent Hugepages (THP)

THP is an abstraction layer that automates most aspects of creating, managing, and using huge pages. It is designed to hide much of the complexity in using huge pages from system administrators and developers. Huge pages increase the memory page size from 4 kilobytes to 2 megabytes. This provides significant performance advantages on systems with highly contended resources and large memory workloads. If memory utilization is too high or memory is badly fragmented which prevents hugepages being allocated, the kernel will assign smaller 4k pages instead. Most recent Linux OS releases have THP enabled by default.
THP usage is controlled by the sysfs setting /sys/kernel/mm/transparent_hugepage/enabled. Possible values:

• never: entirely disable THP usage.
• madvise: enable THP usage only inside regions marked MADV_HUGEPAGE using madvise(3).
• always: enable THP usage system-wide. This is the default.

THP creation is controlled by the sysfs setting /sys/kernel/mm/transparent_hugepage/defrag. Possible values:

• never: if no THP are available to satisfy a request, do not attempt to make any.
• defer: an allocation requesting THP when none are available get normal pages while requesting THP creation in the background.
• defer+madvise: acts like "always", but only for allocations in regions marked MADV_HUGEPAGE using madvise(3); for all other regions it's like "defer".
• madvise: acts like "always", but only for allocations in regions marked MADV_HUGEPAGE using madvise(3). This is the default.
• always: an allocation requesting THP when none are available will stall until some are made.

An application that "always" requests THP often can benefit from waiting for an allocation until those huge pages can be assembled.
For more information see the Linux transparent hugepage documentation.

drop_caches

sysctl is used to change kernel parameters at run-time.
-w vm.drop_caches=3 - clears filesystem caches

tuned-adm

This command line utility allows you to switch between user definable tuning profiles. Several predefined profiles are already included. You can even create your own profile, either based on one of the existing ones by copying it or make a completely new one.i The distribution provided profiles are stored in subdirectories below /usr/lib/tuned and the user defined profiles in subdirectories below /etc/tuned. If there are profiles with the same name in both places, user defined profiles have precedence.

Profiles Used:

• throughput-performance: Broadly applicable tuning that provides excellent performance across a variety of common server workloads.
• latency-performance: Optimize for deterministic performance at the cost of increased power consumption.

Firmware / BIOS / Microcode Settings

DRAM Refresh Delay

Default: Minimum

By enabling the CPU memory controller to delay running the REFRESH commands, you can improve the performance for some workloads. By minimizing the delay time, it is ensured that the memory controller runs the REFRESH command at regular intervals. For Intel-based servers, this setting only affects systems configured with DIMMs which use 8 Gb density DRAMs.

• Minimum: Memory controller runs REFRESH command at regular intervals.
• Performance: CPU memory controller delays running REFRESH commands for improved performance.

DIMM Self Healing (Post Package Repair) on Uncorrectable Memory Error

Default: Enabled

Enable/Disable Post Package Repair (PPR) on Uncorrectable Memory Error.

Logical Processor

Default: Enabled

Each processor core supports up to two logical processors. When set to Enabled, the BIOS reports all logical processors. When set to Disabled, the BIOS only reports one logical processor per core. Generally, higher processor count results in increased performance for most multi-threaded workloads and the recommendation is to keep this enabled. However, there are some floating point/scientific workloads, including HPC workloads, where disabling this feature may result in higher performance.

NUMA Nodes per Socket

Default: 2

Allows configuration of the memory NUMA domains per socket. The configuration can consist of one whole doman (NPS1), two domains (NPS2) or four domains (NPS4).
In the case of two-socket platforms, an additional NPS profile is available to have whole system memory be mapped as a single NUMA domain (NPS0).

L3 cache as NUMA Domain

Default: Disabled

This field specifies that each CCX within the processor will be declared as a NUMA Domain.

AMD CPU Speculative Store Mode

Default: Auto

Sets the mode that determines when store instructions send out their remote cache line copy invalidation.

• Auto: Sending might be delayed when the cache line exists but is not in a writable state in the local cache.
• Balanced: Sending might be delayed when the cache line exists but is not in a writable state in the local cache.
• More Speculative: Sending will occur as soon as possible.
• Less Speculative: Sending might be delayed when the cache line is not present or is not in a writable state in the local cache.

L1 Stream HW Prefetcher

Default: Enabled

When set to Enabled, the processor provides advanced performance tuning by controlling the L1 stream HW prefetcher setting. Use the recommended setting, and this option will allow for optimizing overall workloads.

L2 Stream HW Prefetcher

Default: Enabled

When set to Enabled, the processor provides advanced performance tuning by controlling the L2 stream HW prefetcher setting. Use the recommended setting, and this option will allow for optimizing overall workloads.

Secure Memory Encryption

Default: Enabled

Enables or disables AMD secure encryption features such as Secure Memory Encryption (SME) and Secure Encrypted Virtualization (SEV). In addition to enabling this option, SME must be supported and activated by the operating system. Similarly, SEV must be supported and activated by the hypervisor. This option also determines if other secure encryption feature such as TSME and SEV-SNP features can be enabled.

Memory Tracking Granularity

Default: Auto

• Auto: Finer-Grain
• Finer-Grain: Finer-Grain tracking enables higher memory efficiency and is expected to be the preferred performance option.
• Coarser-Grain: Coarser-grain tracking may improve memory performance for a narrow set of workloads.

C-States

Default: Enabled

C-States allow the processor to enter lower power states when idle.
When set to Enabled (OS controlled) or when set to Autonomous (if Hardware controlled is supported), the processor can operate in all available Power States to save power, but may increase memory latency and frequency jitter. Read-only unless System Profile is set to Custom.

• Enabled: OS controlled C-States.
• Disabled: Disables C-States.
• Autonomous: Hardware controlled C-States (if supported).

Core CC6

Default: Enabled

Control Core CC6 entry when ACPI C2 state is used or halt is issued.

CPPC

Default: Auto

Enabled/Auto: Allows the OS to make performance/power optimization requests using ACPI CPPC. Disabled: Prevents the OS from making performance/power optimization requests using ACPI CPPC.

• Auto: Same as Enabled - uses the default CPPC behavior.
• Enabled: Allows the OS to make performance/power optimization requests using ACPI CPPC.
• Disabled: Prevents the OS from making performance/power optimization requests using ACPI CPPC.
• SMU Configuration: SMU controls CPPC configuration.

CPU Power Management

Default: OS DBPM

Allows selection of CPU power management methodology. Read-only unless System Profile is set to Custom.

• Maximum Performance: Drives processor frequency to the maximum across all cores. This mode also offers the lowest latency of the CPU Power Management Mode options.
• OS DBPM: OS dynamically controls individual core frequency that can be utilized for power savings.

DF CState

Default: Enabled

This field enables/disables DF (Data Fabric) CState.

• Enabled (0xF): Enables DF CState.
• Disabled (0x0): Disables DF CState.
• SMU Configuration: SMU controls DF CState configuration.

Determinism Control

Default: Auto

Read-only unless System Profile is set to Custom.

• Auto: Use default performance determinism settings.
• Power Determinism: Maximizes workload performance to part-specific power limits, thereby tapping the additional performance headroom based on the silicon. If you are using Power determinism, then you can obtain maximum performance by setting the TDP and Package Power Limit (PPL) to the maximum TDP value supported by the CPU. Setting the TDP and PPL to Auto sets both parameters to the CPU default TDP value for energy-efficient operation.
• Performance Determinism: Makes the workload performance the same regardless of variations in the environment or silicon.

Memory Patrol Scrub

Default: Standard

Patrol Scrubbing searches the memory for errors and repairs correctable errors to prevent the accumulation of memory errors. Read-only unless System Profile is set to Custom.

• Disabled: No patrol scrubbing will occur.
• Standard: Entire memory array scrubbed once in a 24 hour period.
• Extended: Entire memory array scrubbed more frequently for increased reliability.

PCI ASPM L1 Link Power Management

Default: Enabled

When enabled, PCIe Advanced State Power Management (ASPM) can reduce overall system power a bit while slightly reducing system performance. NOTE: Some devices may not perform properly (they may hang or cause the system to hang) when ASPM is enabled; for this reason L1 will only be enabled for validated qualified cards.

• Enabled: Reduces system power with slight performance reduction.
• Disabled: No power reduction, maintains performance.

PCIE Idle Power Setting

Default: Optimize for Perf/Power

Modify PCIE Power Savings Features that Can Impact Lightly Loaded Latency.

TDP Control

Default: Auto

Allows the reconfiguration of the processor Thermal Design Power (TDP) levels based on the power and thermal delivery capabilities of the system. TDP refers to the maximum amount of power the cooling system is required to dissipate. Auto: Use platform and OPN default TDP. Manual: Set custom configuration TDP. NOTE: This option is only available on certain SKUs of the processors, and the number of alternative levels varies as well.

• Auto: Use platform and OPN default TDP.
• Manual: Set custom configuration TDP.

Customized cTDP

This option appears when TDP Control is set to Manual for customizing Thermal Design Power (TDP) in watts (decimal). The manual TDP should align with OPN design. The default value is the microprocessor's nominal TDP while this can be adjusted lower to minimize power consumption at the cost of peak performance or higher to increase peak performance by allowing the CPU to maintain higher dynamic clock speeds at the cost of power efficiency.

PPT Control

Default: Auto

Enables or disables the Package Power Tracking (PPT) control. Auto: Automatically set PPT in watts. Manual: Specify a custom PPT.

• Auto: Automatically set PPT in watts.
• Manual: Specify a custom PPT.

Customized PPT

This option appears when PPT Control is set to Manual for customizing Package Power Tracking (PPT) in watts (decimal). PPT should not exceed the configured TDP.

Power Profile Select

Default: Efficiency Mode

• High Performance Mode: In this mode, preference is given to core performance. All DF P-States are available in this mode and the default DF P-State and DLWM algorithms are active.
• Efficiency Mode (default): In this mode, the system is configured for power efficiency, which limits the frequency available to cores and restricts DF P-States available in the system.
• Maximum IO Performance Mode: In this mode, Data Fabric is configured to maximize IO sub-system performance.
• Balanced Memory Performance Mode: This mode uses High Performance Mode when the system is under heavy load, and Efficiency Mode when the system is under lighter loads.
• Balanced Core Performance Mode: This mode uses High Performance Mode when the system is under heavy load, and Efficiency Mode when the system is under lighter loads.
• Balanced Core Memory Performance Mode: This mode combines memory subsystem behavior from Balanced Memory Performance Mode and core behavior from Balanced Core Performance Mode.

System Profile

Default: Performance Per Watt Optimized (OS)

This field sets the System Profile to Performance Per Watt (OS), Performance, or Custom mode. When set to a mode other than Custom, BIOS will set each option accordingly. When set to Custom, you can change setting of each option.

• Performance Per Watt Optimized (OS): Optimizes for performance per watt with OS control.
• Performance: Sets all options for maximum performance.
• Custom: Allows manual configuration of individual options.

Optimizer Mode

Default: Disabled

Dell's Optimizer mode adds automatic tuning maximizing the processor's performance based on system configuration and thermal environment. For AMD platforms optimizer mode will only function when the determinism option is set to power determinism.

• Enabled: Enables automatic tuning for processor performance.
• Disabled: Turns off the feature.