SPECstorage™ Solution 2020_ai_image Result

Copyright © 2016-2024 Standard Performance Evaluation Corporation

WEKAIO, Inc. SPECstorage Solution 2020_ai_image = 700 AI_Jobs
WEKA - Public Cloud Reference Overall Response Time = 0.85 msec


Performance

Business
Metric
(AI_Jobs)
Average
Latency
(msec)
AI_Jobs
Ops/Sec
AI_Jobs
MB/Sec
500.625217524891
1000.635435059781
1500.6276525814669
2000.6348701119564
2500.63110876424452
3000.65313051729337
3500.66615226934226
4000.77417402239117
4500.83419577544006
5000.91421752848901
5501.03823928153794
6001.05226103458675
6501.32828278763571
7002.11630304268318
Performance Graph


Product and Test Information

WEKA - Public Cloud Reference
Tested byWEKAIO, Inc.
Hardware AvailableJanuary 2024
Software AvailableJanuary 2024
Date TestedJanuary 2024
License Number4553
Licensee LocationsCampbell, CA USA

The WEKA; Data Platform is purpose-built to seamlessly and sustainably deliver speed, simplicity, and scale that meets the needs of modern enterprises and research organizations without compromise. Its advanced, software-defined architecture supports next-generation workloads in virtually any location with cloud simplicity and on-premises performance.

Solution Under Test Bill of Materials

Item NoQtyTypeVendorModel/NameDescription
164Azure Virtual MachineAzureStandard L8sv3WEKA storage back ends - Azure Standard L8s v3 Virtual Machine (8 vCPUs, 64 GiB memory, 12.5 Gbps networking)
216Azure Virtual MachineAzureStandard D64sv3Ubuntu clients with WEKA POSIX-compliant filesystem driver - Azure Standard D64s v3 Virtual Machine (64 vCPUs, 256 GiB memory, 30 Gbps networking)

Configuration Diagrams

  1. WEKA in Azure

Component Software

Item NoComponentTypeName and VersionDescription
1WekaFSFile SystemVersion 4.2.7.64The WEKA filesystem (WekaFS) redefines storage solutions with its software-only approach and addresses common storage challenges by removing performance bottlenecks, making it suitable for environments requiring low latency, high performance, and cloud scalability.
2Weka Backend NodeStorage OSUbuntu 20.04Operating System (OS) for 50 storage virtual machines using Azure L8sv3 instances"
3ClientsClient OSUbuntu 20.04Operating System (OS) for 13 client virtual machines using Azure D64sv3 instances"

Hardware Configuration and Tuning - Virtual

Component Name
Parameter NameValueDescription
Accelerated NetworkingEnabledL8s instances have 12.5 Gbps and D64s instances have 30 Gbps connectivity

Hardware Configuration and Tuning Notes

None

Software Configuration and Tuning - Virtual

Networking
Parameter NameValueDescription
Jumbo Frames3900WekaFS recommended MTU for Azure.
num_cores3WekaFS mount option to designate the number of frontend cores to allocate for the client.

Software Configuration and Tuning Notes

The single filesystem was attached via a single mount per client. The mount string used was "mount -t wekafs -o net=enP19658s5np0/10.0.0.56/20/10.0.0.1 -o net=enP28661s2np0/10.0.0.64/20/10.0.0.1 -o net=enP31350s3np0/10.0.0.63/20/10.0.0.1 -o num_cores=3 -o mgmt_ip=10.0.0.76 10.0.0.60/default /mnt/weka" The mount specifies that 3 WEKA front ends (cores) were used on the clients.

Service SLA Notes

None

Storage and Filesystems

Item NoDescriptionData ProtectionStable StorageQty
1WEKA Filesystem with 88TB of usable storageWEKA 16+2 distributed data protectionLocal 1.92TB NVMe Disk64
Number of Filesystems1
Total Capacity88 TB
Filesystem TypeWekaFS

Filesystem Creation Notes

Each storage backend has one Azure SSD attached to it, which are dedicated to the WEKA filesystem created . The storage cluster consisted of 64 storage servers with a single WEKA filesystem created in the cluster. Each client mounted the WEKA filesystem with a single mount.

Storage and Filesystem Notes

None

Transport Configuration - Virtual

Item NoTransport TypeNumber of Ports UsedNotes
112.5 Gbps Virtual Nic64Used by WEKA Backend Server Nodes
230 Gbps Virtual Nic16Used by WEKA Frontend Client Nodes

Transport Configuration Notes

None

Switches - Virtual

Item NoSwitch NameSwitch TypeTotal Port CountUsed Port CountNotes
1Azure12.5 Gbps Ethernet with Accelerated Networking6464Used by WEKA Backend Server Nodes
2Azure30 Gbps Ethernet with Accelerated Networking1616Used by WEKA Frontend Client Nodes

Processing Elements - Virtual

Item NoQtyTypeLocationDescriptionProcessing Function
1512vCPUL8sv3 WEKA Backend2.8 GHz Intel(R) Xeon(R) Platinum 8370C processorsWekaFS, Network Communications, Storage Functions
21024vCPUD64sv3 WEKA Frontend2.8 GHz Intel(R) Xeon(R) Platinum 8370C processorsWekaFS, Network Communications, Storage Functions

Processing Element Notes

None

Memory - Virtual

DescriptionSize in GiBNumber of InstancesNonvolatileTotal GiB
Azure L8sv3 Virtual Machine Memory6464V4096
Azure D64sv3 Virtual Machine Memory25616V4096
Grand Total Memory Gibibytes8192

Memory Notes

None

Stable Storage

WekaFS does not use any internal memory to temporarily cache write data to the underlying storage system. All writes are commited directly to the storage disk, and protected via WekaFS Distributed Data Protection (16+2 in this case). There is no need for any RAM battery protection. In an event of a power failure a write in transit would not be acknowledged.

Solution Under Test Configuration Notes

The system under test consisted of 16 x D64sv3 Azure client virtual machines and 64 x L8sv3 backend virtual machines. Each client has a 30Gbps network connection and each WEKA backend has 12.5Gbps network connection.

Other Solution Notes

None

Dataflow

None

Other Notes

None

Other Report Notes

The solution under test was a standard WekaFS cluster in dedicated server mode. The solution will handle both large file I/O as well as small file random I/O and metadata intensive applications. No specialized tuning is required for different or mixed use workloads. None of the components used to perform the test were patched with Spectre or Meltdown patches (CVE-2017-5754, CVE-2017-5753, CVE-2017-5715).


Generated on Thu Feb 15 10:31:28 2024 by SpecReport
Copyright © 2016-2024 Standard Performance Evaluation Corporation