[Ftk_hardware] VME access for the AUX card

[Francesco Crescioli]

Dear Paola and Mel,

I'll add few informations:

Il giorno Sun, 26 Dec 2010 11:12:27 +0100
Paola Giannetti <paola.giannetti at pi.infn.it> ha scritto:

> Regarding VME I suppose we have the possibility to use 2 VME slaves,
> one in the AUX board and one in the AMboard, reducing to zero the
> extra signals we have to pass between the 2 and decoupling the 2

This is the best solution, in my opinion. We just have to plan well the
VME address space partition.

> designs. The AMB VME is under discussion and 2 proposals are going to
> be tried in the next prototype:
> 
> (1) try to use 64 bit VME transfers.  This implies slave firmware
> will deeply change.

I would like to point that VME has dedicated signals to comunicate to
slaves which kind of address/data type is going to use in the cycle.
D64 cycles will be used only for few registers for pattern bank
transfer, there's no need for the other VME slaves to implement
the D64 protocol.

> If we find an agreement about using the 64 or 32 bit transfer, we
> could develop a single  VME slave and use it every where, changing
> only the VME internal addressing. 
> Again we should  choose a single responsible that study and propose
> the basic f9irmware for every body, like  we should do it for
> functions that are shared between the AUX board and the final board.

We could try to write a modular VME code, common to every board, with
registers for D32 and D64 data types. The D64 data transfer require
a little more logic with respect to the D32 cycle and access to all
VME64x signals.
Anyhow it's not yet clear (to me) that the CPU is able to do D64
transfers correctly (the onboard chip is able, but the drivers
probably no, I don't know how hard is to add the support).

> 
> Coming to other details:
> we have a 5 bit geographical address to which both the boards have to 
> answer (the slot number), however we can choose an extra bit that
> will distinguish between the 2 boards belonging to the same slot.  So
> we have to decide how to share the board addresses.
> Unfortunately from what Francesco is learning on the new VME CPU
> (Atlas like), we don't have the possibility to use the entire field
> of VME addresses (32 bits). This is a general problem that will
> affect also Illinois and Fermilab.

As far as I have understood the VME on the ATLAS CPU works in this way:
there's a chip (Universe Tundra series) with 8 independent programmable
decoders that maps VME space into PCI space. Each decoder has a base
address and a window size.
If you are using A32 cycles (32-bit addresses, like in CDF) it's not
possible to map all VME space in the PCI space at once. You should
reprogram the a decoder every time you want to access a VME portion not
mapped.

For some reason ATLAS software discourages this kind of approach: the
configuration scripts programs the 8 decoders to map some VME space and
then you should use just that without reprogram the Universe Chip
during run. I don't see a software limitation for that, so I'm not yet
sure why this is the standard way of operation.

> It is not totally clear yet, but the use of a large number of address 
> bits is strongly discouraged. And more powerful is the VME CPU (more 
> memory it has) less VME space is available.

That's because you need a free contiguous memory space to map portions
of VME into PCI. The more real memory you have the less contiguous free
space is available at boot time. Or at least this is what I have
understood. Anyhow I currently have no problems with 24 bit window
size (one hardware decoder programmed) on a 1 Gb RAM machine.

> So let's consider the possibility that we have only roughly 20 bits
> for internal addressing.  We will need an internal register to choose 
> different memory location fields (Fifos, Spy buffers, AMchips, SS and
> AM Maps, Fit Constant memories.....).

If we are going to page our internal memory space, which is probably
necessary, we can think about using A24 cycles. As I said we can map
the whole 24 bit address space in PCI. Using A24 + 5 bits of
geographical address + 1 bit for AMB/AUX selection leave 18 bits of
internal addressing. Is it enough? The advantage of this is that a
single process can open all the VME space used, map into PCI and the
use it freely and fast.

Without reprogram the Universe chip during the run we can map up to 27
bit of VME address space (24 bit * 8 decoders), but probably not use it
all at the same time for the memory limitation I said before. I mean
that no single process can open a 27 bit window, but this isn't a
problem if we develop software partitioned for each board (at least 5
bits of addresses are for the GA and fixed within the single process).

Francesco