• Overview
• Original S-LINK to PCI Driver and Application Source Code
• Revision History and Current Progress
• Derek's summer work is summarized in this talk

Overview

Pulsar board has two S-LINK ports (bi-directional) via P3 backplane (through a transition module in the back similar to the simple transition module made at CERN). This allows Pulsar to communicate directly with a PC or VME processor (both directions) via high bandwidth commercially available Simple S-LINK to PCI/PMC interface cards, or SSPCI (speed up to 120MB/s) . There are many device drivers (and related application software) developed for these PCI cards and they can be found at Linux , NT , LynxOS, Vxworks. Note that there is a new (commmercially available) high speed follow up of the SSPCI, namely the S32PCI64 (32-bit S-LINK to 64-bit PCI interface card with speed up to 260MB/s), The new board should be much faster and easier to use (see recent PCI performance tests for ATLAS readout). However, it is relatively new and here we will only foucs on the software for the older SSPCI (SLINK to PCI) and SPCIS (PCI to SLINK). These boards are good enough for teststand application. The following page describes initial work with SLINK-PCI software for Pulsar application (work done by Derek Kingrey, summer student from Cornell University). This page is maintained by Derek. Just like the firmware VHDL code for Pulsar, all SLINK related software discussed here is in CVS (managed by Peter Wittich).

Initially we would like to have software to do the following (some will be used for initial Pulsar prototype testing, some will be more advanced features for general purpose applications):

1. Provide a reliable driver to control both an LDC and an LSC via the PCI bus using a Simple S-Link to PCI (SSPCI) interface card and a Simple PCI to S-LINK (SPCIS) interface card.
2. Provide an application program utilizing the above driver to send and receive S-LINK data (to and from Pulsar). The application should allocate a memory buffer for incoming and outgoing S-LINK data, and should be able to write and read S-LINK data to and from this buffer.
3. For maximum speed, DMA should be used for all data transfers to and from the above memory buffer.
4. The driver should be interrupt driven.
5. The driver should use the LCTRL line from the LDC to control the beginning and ending of an event. (This allows for the event length to vary and saves on CPU overhead).
6. All incoming S-LINK data should be saved to disk to be read later.
7. will try Real-Time Linux for better performance.

Original S-LINK to PCI Driver and Application Source Code

Since the 32-bit S-LINK to PCI and PCI to S-LINK interface boards have been around for some time now, many drivers and applications have already been developed to satisfy some of the goals listed in the Overview. As a base, we choose to start with code developed by Sidik Isani of the CFHT Telescope project in Hawaii. This code was chosen as a base for its simplicity, kernel compatibility, robustness, and use of RTAI Linux for real time interrupt handling. The original source code includes a SSPCI driver and an application program for debugging and testing S-LINK performance. Also included are a number of supporting programs including one that allocates the himem memory area reserved at boot time. The driver initializes the PCI interface and sets up interrupt handling. The driver uses ioctl commands from the application program to control the LDC and DMA engine on the PCI card. The driver also uses these commands to initialize and reset the LDC interface on the PCI card as well as set up the memory ring buffer. An example application, designed to work with a SLIDAS board, has been included with the driver. We will simply modify the example code for initial Pulsar prototype testing needs (which does not need high speed).

The original code does not accomplish all of the goals listed in the Overview as it was designed for different application. For example, the driver is designed for SSPCI (SLINK to PCI) but not for SPCIS (PCI to SLINK).

Revision History and Current Progress:

6/10/02

• With SLIDAS as an LDC on the S-LINK to PCI interface card the original code was compiled and run successfully (with #undef COMPLETE_TESTS).
  
  

6/12/02

• Put LDC directly on S-LINK to PCI interface card. SLIDAS put on SLITEST as FEMB connected to LSC. Fiber run between LSC and LDC. Original code runs successfully.
  
  

6/14/02

• Removed CRC checking and use of slidas_random.c since we can verify the data in other ways.
• Removed all of COMPLETE_TESTS since we are only interested in DMA transfers and are not interested in testing the S-LINK itself.
• Added file saving so that the data sent to the buffer by the DMA engine could be read and saved to file.
• Changed the buffer to a circular split buffer (one buffer split into two buffers). Later, the program will be made to alternate the CPU and the DMA engine between these two buffers.
• Program no longer continually loops but instead transfers only one event (block of data).
  
  

6/26/02

• Reduced CPU overhead by postponing time consuming commands from the time critical part of the code until later in the program. The table below shows the improvement in speed for some small byte counts per event. The first graph below shows the effect of CPU overhead on program performance (data transfer rate) versus the number of bytes transferred(Click on the graph below for the full image.)
  
  

	Data Rate vs. Bytes Transferred

The number of microseconds was calculated programatically by reading the time just before the data transfer and again just after and taking the difference. The data rate was calculated by dividing the byte count (transferred) by the number of microseconds. (Compare to the study done by other people here...)

7/1/02

• Eliminated use of the URL lines to trigger an event on SLIDAS. An event can now be triggered by pressing the run/step button on the SLIDAS or by a rising edge on the LEMO connector. (This was done so that a function generator could later be used to drive the SLIDAS using the LEMO connector to test event transfer rate.)
• Included signal.h to use Ctrl-Z as a signal to stop the program and save the last event to disk. The program will now sleep until an event is received. Furthermore, it continues to loop and collect data (assuming it is being triggered) until Ctrl-Z is pressed or until we reach a maximum allowable number of events.
  
  

7/9/02

• The program now successfully alternates ("flip-flops") the CPU and the DMA engine between the two buffers. When the CPU is reading one buffer, the DMA engine is transferring S-LINK data to the other, and when a new event arrives they switch. ("Events", at this point, are blocks of 256 SLIDAS random data words.)
  
  

7/11/02

• The program now accepts both control and data words from the LDC and can switch between the two. This is done by generating an interrupt and waking up the driver when the LCTRL line changes states.
  
  

7/19/02

• the SLINK to PCI device driver has been modified to also work with the PCI to SLINK.
  
  

7/22/02

• Today we set up a SLINK teststand at U of Chicago as well.
  
  

Today

...more to come...

Written by Derek Kingrey
Cornell University
Email: dkingrey@fnal.gov
Fermilab Internship for Physics Majors, Summer 2002

Last modified: July 22, 2002