( home )
 Last Modified:  24 May 2001
USER COST PER INTERRUPT


Notes after initial post:  The average cost per interrupt on aztec has been recalculated using a more accurate methods:  fixed time and fixed work; see CPU Overhead II and   CPU Overhead III , respectively.

NOTE (from initial post):  The information on this page is used to generally validate the graph shown concerning  CPU overhead .  See web page concerning constant cost .

PURPOSE: Determine the cost of an interrupt to the user application.  Under careful testing, cost in kernel time can be inferred from such user cost.
BACKGROUND: SSL already has  interrupt latency measurements  from the perspective of a network interface card.  SSL is interested in how much of this latency is due to actual kernel execution.  This test utilizes the methods for measuring  CPU overhead  plus a command line function "procinfo" to determine the cost to the user on a per interrupt basis.  This cost is related to kernel execution time per interrupt under the constraints of this test. 
UNIT MEASURE: The cost of an interrupt in nanoseconds.  Multiple tests have been run on Aztec, each with a different period of time between interrupts (i.e., different interrupt frequencies) and the results are plotted and discussed below.
USE: The timing program, GM driver, and Myrinet Control Program of the CPU overhead test were used.  First, for each data point, the command line function "procinfo" was used to record the initial number of IRQ 10 interrupts; the IRQ 10 interrupt is used to send the interrupt stream from the Myrinet card and this is the only use of that interrupt on the system.  Second, the MCP is loaded which, after a ~20 second delay, starts the stream of interrupts.  Third, during the delay the timing program is initiated.  Fourth, when the timing program completes, the command line function "procinfo" is used to determine the number of IRQ 10 interrupts after the test.  Note:  This assumes that the interrupt stream terminates before the timing period of the timing program terminates.  The tester must ensure (e.g., via dry runs) that this is the case.  For the data presented below and for each data point, the tester ran such a dry run for each graph point on the plot in order to determine the proper number of interrupts in the interrupt stream (e.g., larger RTC Plus Units mean a lesser number of interrupts for the same stream duration  time).
INTERNALS: A small change was made to the driver and MCP so that there was a ~20 second delay from the time that the MCP was loaded to the time that the interrupt stream began.  The "dry runs" were accomplished over a two minute period in ten second intervals.  The actual test for each point was run over a two minute period with just one interval (i.e., a two minute interval). 

Raw data  is available.  The "irqs" files contain the data from the calls to "procinfo" before and after eaching timing.  The "time" files contain the two minute timing results.  In the names of both of these sets of files, the first set of digits is the number of RTC Plus units and the second set is the number of targe interrupts for that timing run.  The "grain" files contain the previously mentioned dry runs and are similarly named.

PLATFORMS: While the test is extensible x86 platforms running Linux and having a Myrinet card, the setup for the test is relatively time consuming if the card was not previously in use on the host.  Setup involves installing a cross-compiler in order to compile the code for the Myrinet Control Program; installation takes into account specific host parameters such as the size of various types.
SAMPLE DATA: See the figues below.   The abscissa units represent a portion of the time period between interrupts.  The other portion ("Plus") is the actually experienced interrupt latency periods over the test (which is not explicilty indicated below); this is the same kind of units as were used in the CPU overhead test  The ordinate units are nanoseconds per interrupt and are  based on all CPU time not used by the user divided by the number of interrupts handled by the kernel.  In other words, the units are calculated by adding the system time used by the timing process to all time used outside of the process, and then dividing by the number of interrupts registed by the host. All testing was performed on Aztec.  See specific host and card information at the bottom of this page.
POSSIBLE FUTURE
MODIFICATIONS:
If one or two more test runs are to be made of the magnitude of that reported here, then the process should be automated.  The timing application should be able to call for the stream of interrupts as well as for the number of interrupts handled by the host during the timing.  Currently, the tester must coordinate as noted above for each data point.  However, it is expected that this data will be used to validate a more portable test to be developed in the future, rather than repeat the test in this report on other hosts.

CARD AND HOST INFORMATION:   The tests were run on an FIC VA-503+ motherboard, 500 MHz AMD-K6(tm)-2 processor, and Red Hat 6.2 with Linux Kernel 2.2.14-5.0.  The tests used a single Myrinet card having the below information.
 
ON CHIPS
ON BOARD
ON STICKER
LANai 7.2 9947
PCIDMA 1.0 9947
Myrinet M2L-PCI64/2-3.0
1999 Myricom Inc.
357957
M2L-PCI64A-2
40681
A-3935  3952
00:60:dd:7f:cd:21



Bill Lawry