Network Diagnostic Tester (NDT)

The Network Diagnostic Tool (NDT) is a client/server program that provides network configuration and performance testing to a users desktop or laptop computer. The system is composed of a client program (command line or java applet) and a pair of server programs (a webserver and a testing/analysis engine). Both command line and web-based clients communicate with a Web100-enhanced server to perform these diagnostic functions. Multi-level results allow novice and expert users to view and understand the test results.

Several studies have shown that the majority of network performance problems occur in or near the users’ desktop/laptop computer. These problems include, but are not limited to, duplex mismatch conditions on Ethernet/FastEthernet links, incorrectly set TCP buffers in the user’s computer, or problems with the local network infrastructure. The NDT is designed to quickly and easily identify a specific set of conditions that are known to impact network performance. A multi-level series of plain language messages, suitable for novice users, and detailed test results, suitable for a network engineer, are generated and available to the user. Finally, the test results may be easily emailed to the appropriate administrator to assist in the problem resolution phase.

The NDT consists of several components. Both client and server processes are used to perform a specific set of tests. The server processes include a basic web browser (fakewww) to handle incoming web based client requests. The server also runs a second process (web100srv) that performs the specific tests needed to determine what problems, if any, exist. The web100srv process then analyzes the test results and returns these results to the client.

Both command line (web100clt) and Web based (java applet) based clients are included in the NDT package. The command line client (web100clt) can be compiled and manually downloaded onto numerous client computers. The web based client uses a java applet to automate the testing process. This applet is downloaded when the web server is accessed, removing the restriction to manually pre-load the software onto the client computer. This approach allows system administrators to pre-load the command line client onto their main server class computers while allowing the vast majority of users to automatically load the web based client on an as needed bases.

Features:
Simple web server process eliminates the need for a full blown web server
Throughput measurements reported in both directions
Web100 data extracted to analyze the test results
Web based java applet allows testing from any client
Command line tool allows testing from remote locations
Multi-level results tailored to novice and expert users

Requirements:
Linux Operation System
Web100 patched kernel
Web100 user library
pcap library

Supported Systems:
The NDT server process requires a Web100 enhanced kernel to gather the necessary details to analyze the tests. The Web100 kernel patches require specific version of the Linux 2.4.x kernel. See http://www.web100.org for more details.

The client processes, both command line and java applet, will operate on any *nix or Windows operating system. The command line client process requires that the user/administrator download, build, and install the web100clt program. See the below for details. The Java applet requires that the users desktop/laptop contain the Java Run Time Virtual Machine. This software can be downloaded and installed from the Sun Systems http://java.sun.com web site.

Building the Application:
The NDT software uses the gnu autoconf tools to configure the build process. NDT is distributed with a pre-built configure script so the actual autoconf tools should not be needed on the target system. (Although, gnumake may be required...) The configure script can be run with the --help option to determine the full set of configurable options.

A basic build procedure would be:

% tar -xvzf ndt-$VERS.tar.gz

% cd ndt-$VERS

  #  use the –prefix={path} option if you don't like the default

  #  installation directory (/usr/local/)

% ./configure

% make

% make install

Please report any build problems to rcarlson@internet2.edu

The command line client needs to be built for each Operating system it will be used on. At the present time this client has successfully been build on Linux 2.4.x, FreeBSD, and Windows XP (using cygwin). The build procedure is:

% tar -xvzf ndt-$VERS.tar.gz

% cd ndt-$VERS

  #  use the –prefix={path} option if you don't like the default

  #  installation directory (/usr/local/)

% ./configure

% cd src

% make web100clt

% cp web100clt /usr/local/bin

  #  copy the client program to your favorite location

Configuring the web100srv Program:
The web100srv program uses numerous options to control it’s behavior. The web100srv.1 man page describes each of these options. Options may be entered on the command line, or a configuration file may be used instead. The default configuration file is /etc/ndt.conf, but this may be changed by using the ‘-c fn’ option.

ndt.conf:
This file contains a set of keyword value pairs that describe each option. A sample version of the file is included in the NDT package conf directory. See the ndt-conf.1 man file for more details.

Running the Server Programs:
The NDT package comes with a simple start script (start.ndt) that can be used to start both server processes. In cases where this is not usable, each process may be manually started.

The fakewww process handles communications with the client’s web browser. It has a list of files that is can return to the client, all other request are ignored and a 404 error page is returned instead. When run in federated mode, the fakewww process also handles the server discovery process. See the fakewww.1 man page for more details.

To manually start the fakewww process:

% /usr/loca/sbin/fakewww –l /path/to/log/file >& /dev/null &

To see all the available options:

% /usr/loca/sbin/fakewww –h

The web100srv process handles testing between the client and server. It also analyzes the test results and returns these results to the client for display. The server uses three (3) TCP ports for these tests, one port is used to exchange control information between the client and server. The other two (2) ports are used to exchange test traffic both to and from the server. The web100srv program also supports a configuration file, allowing options to be pre-defined. See the web100srv.1 man page for more details.

To manually start the web100srv process:

% /usr/local/sbin/web100srv –a >& /dev/null &

To see all available options:

% /usr/local/sbin/web100srv –h

Caveats:
The NDT 3.3.11 version introduced support for the IPv6 protocol. This support results in the web100srv process using IPv4_mapped style addresses instead of native IPv4 addresses. This change tripped over a bug in the Web100 userland library code that processes IPv6 addresses. This bug causes the NDT client to report a "Protocol error" after starting the Middlebox test. Patches for the Web100 userland library (v1.5 and v1.6) are available to correct this bug. Contact the Web100 development team at web100.org for more details.

A workaround for this bug is to disable the IPv6 address processing by starting the web100srv process with the –4 or --ipv4 option. This will force the web100srv process to use native IPv4 addresses, avoiding the bug in the library's IPv6 code.

Running the Client Programs:
The Java Applet (Tcpbw100.jar) is downloaded automatically when you access a specific NDT server. Once the download is complete the applet is ready for testing. The user must press the ‘Start’ button to begin a test. Once a test is complete, the ‘Statistics’, ‘More Details’ and ‘Report Problem’ buttons are activated.

At the conclusion of the test, the main applet page shows the basic throughput results, both to and from the client, and a simple statement of what link technology may limit the throughput. Link types range from Dial-up to 10 Gbps. See the web100srv.1 man page for more details. Finally if warranted a ‘Warning’ or ‘Information’ messages may be printed.

The ‘Statistics’ page allows access to intermediate level details. These details include:

The results of 5 specific configuration tests performed by the NDT
The clients OS and Java information
Several performance details including round trip time (RTT), Packet size (MSS), and loss statistics (Retransmissions, duplicate ACKs, SACKs, and out-of-order delivery)
The percentage of time the server was receive buffer limited, send buffer limited, or limited by the congestion window
The various TCP options that were negotiated between the client and server (SACK, Nagle, ECN, Time stamps, Window Scaling)
The results of the middlebox tests (MSS modification and NAT detection)

The ‘More Details’ page allows access to expert level details including:

The raw web100 variables gathered on the server
The analysis results calculated by the server
The variables, and conditions, used by specific detection algorithms
The bandwidth*delay product for the send/receive/congestion window buffers
The theoretical throughput available to this client based on the Mathis et.al. formula
The results of the link detection algorithm

The web100clt program must be built for your particular Operating System. The simplest way to do this is to download and unpack the tar file, run the ‘configure’ program, and then ‘cd’ to the ‘src’ sub-directory and run the ‘make web100clt’ command. Then manually copy this file to your favorite location. Once this file is built you can connect to a server using the command:

% web100clt server.domain.name (for 3.1.4a and earlier)
% web100clt –n server.domain.name (for 3.3.11 and later)

To see all available options:

% web100clt –h

Mailing lists:
There are two email lists to support this software:

ndt-users

A general discussion list for users.

ndt-announce

This list will be used to announce new versions or significant developments.

Information about these lists, including links to subscribe, is at https://mail.internet2.edu/wws/lists/engineering.

Author:
Richard Carlson
Internet2
rcarlson@internet2.edu