Network Storage Troubleshooting. How To Test the Transfer Speed

Network Storage Troubleshooting - How To Test the Transfer Speed

Product Line:  VERDE


ISSUE: In a cluster environment, all VERDE servers (nodes) are sharing the same storage. If not sized properly (IOPS, bandwidth) the storage device can quickly become a bottleneck for the entire system and induce performance issues.  This article describes how to quickly test the transfer rate between the VERDE server and the storage device to identify a possible I/O bottleneck.

Note that this information is not specific to VERDE and should be familiar to Linux administrators, additional information about the "dd" command is available from the Linux command reference.

SOLUTION:  The command below (dd) does a block copy between a source and a destination location. You can first do a test  copy to a local direct attached drive and then test the transfer rate to the remote storage, you will notice that the output is significantly different (it of course depends on your setup)

 

Copy to a local drive:

This example copies 10,000  times a 10KB file, for a total of 105MB.

      dd if=/dev/zero of=/tmp/test.img bs=10k count=10k

It will provide an output similar to this, note that the transfer rate (388MB/s in this example) will defer depending on the specifications of your drives.

     104857600 bytes (105 MB) copied, 0.0270529 s, 388 MB/s

Copy to a location on the shared storage:

On a VERDE cluster, the /home directory is located on the shared storage and contains the data shared by all servers.

      dd if=/dev/zero of=/home/test.img bs=10k count=10k

 This will return something like: 

      104857600 bytes (105 MB) copied, 1.26297 s, 83.0 MB/s


In this example the transfer rate is 83MB/s, as expected it is significantly lower that what was achieved on the local drive. But the question is: Is it good or bad? 

On a Gigabit Ethernet network, this can be considered a good transfer rate. Do not forget that there may be traffic other than VERDE on the network, and in this example we are almost using the entire bandwidth. You will have to correlate this result with how many users are concurrently accessing the storage to evaluate what is available per user and verify if it is sufficient.

This gives you a quick indication on the quality of the transfer, for example, if the result is a low number 10, 20MBps... On a Gigabit network you will need to investigate to isolate what his accessing the storage or heavily using the network at the same time and is impacting the performance of the VERDE system.

 

Repeat this process on each server where you suspect storage issue.

 

If additional troubleshooting is required you will need to involve the Linux administrator and install system monitoring tools, like Sysstat for example. 


    • Related Articles

    • How To Partition and Format Storage Devices in Linux

      Product Line:  VERDE ​ Introduction Preparing a new disk for use on a Linux system can be quick and easy. There are many tools, filesystem formats, and partitioning schemes that may complicate the process if you have specialized needs, but if you ...
    • External/Shared Storage - Minimal Requirements

      Product Line:  VERDE The shared storage is a key component of a VERDE cluster, the overall performance of the system will depend on its ability to deliver the required output. For this reason home built NFS servers are not supported as a viable ...
    • VERDE - Basic Troubleshooting Guide

      Product Line:  VERDE This is a basic troubleshooting guide put together at the end of 2012. This lists the most common "mistakes" or issues that we see on  new deployments. As the subject says, it is basic, but may still be useful for someone new to ...
    • Win2003 Network Prioritizing

      Product Line:  VERDE   ISSUE:   Working with a customer that was having issues with writing data to a database that was stored on a Win2003 server we discovered the following.  He was willing to install WireShark in his image/guest, and on an ...
    • Discussion About VERDE VDI and Required Network Bandwidth

      Product Line:  VERDE Before we build VERDE VDI at a customer site, we generally consider the environment for the network.  · It is necessary to predict how much bandwidth will be used and needed between the VM and the user device or end-to-end in ...