This website is no longer maintained. Its content may be obsolete. Please visit http://home.cern for current CERN information.
Meeting held on Thursday 11-10-2001 - reported by Mike Lamont
Present : Andy Butterworth, Mike Lamont, Jean-Jacques Gras, Alastair Bland, Kris Kostro, Quentin King, Stephen Page, Thijs Wijnands, Pedro Ribeiro, Marc Vanden Eynden
The LHC will require a real-time channel to equipment and instrumentation in addition to a middleware that provides command/response and publish and subscribe facilities. The foreseen requirements that justified this additional access mechanism are documented in the series of meetings on real-time for the LHC which may be found here. The required latencies, delays have been established, with work on progress on understanding the dynamics and closed loop response of LHC feedback system such as global orbit control. The demands are modest with something like a 10 Hz iteration rate being foreseen.
The consensus was that, in the first instance at least, the real-time channel should be a fairly simple protocol based on UDP. Quentin explained that he had implemented a similar system at JET. (Packet header containing sequence number, timestamp, version number etc. enabling packet loss, transmission times etc. to be monitored, data via C structure complied in server and client code.) Description of UDP based library + zip file here
The essential point is speed over the inherent reliability of TCP and acknowledgment of receipt. Occasional missing UDP packets should not degrade the performance of any client system. Clearly a well-defined API to the UDP solution is required. A open, generic solution is not required, we know what data we need to transmit by this channel.
Kris suggested the use of a commercial system such as NDDS which he felt would be able to meet the requirements. It was felt, however, that we should first try a simple solution and see if it will do.
Network performance - we'll have to take what's given by IT, test it and then start making more extravagant demands if it doesn't meet our requirements. Two Ethernet network interface cards will be required on the front-ends.
Dedicated RT machines will be required at the top level. Applications will be have to be developed for display etc. The data will have to be somehow pushed to the middleware. (Alastair mentioned the possibility of multicasting.)
Steven Page presented the results of recent tests using CMW/corba to perform asynchronous sends to multiple power converters using threads (enhancements - Kris Kostro). Very encouraging results have been obtained. Stephen's presentation may be found here. CMW is evolving into a realistic option.
Other points:
SPS prototype
As presented to the SLTC (please see appropriate minutes - May & September) will provide the opportunity to perform a realistic test of RT acquisition from 6 pickups equipped with LHC electronics. The pickups have been designed and go ahead given for their production. WBTN (analog part of acquisition system) on order. Data acquisition boards to be built at Triumph - on order (resp. BI Rhodri Jones)
Controls issues for SPS prototype:
Anticipate putting this in VME64 (Pedro) with appropriate diagnostics & CPU. WorldFIP cards are on order (31.25 kbits/s). PMC card/mezzanine to CPU to be finished. Will need a PMC extender with slots - 3 used. Fibres in place. Pedro comments that this is all more-or-less standard stuff.
TW: All BI & PO front ends concerned with real time control will be VME based PPCs with the current version of LynxOS and these will have a second 100 bT interface only for RT traffic
QK: As far as I know, the decision about the front end platform is not yet taken. By default it is VME/PPC, but I believe CompactPCI/Intel are being considered as potentially cheaper and more powerful. I understand that LynxOS 3.x is fairly settled as the RTOS of choice for LHC. I am happy to hear from the meeting that twin network interface frontends will be considered - this will certainly improve the control over the transmission/reception of UDP packets ahead of TCP packets.
JJG: I don't see the necessity to have the second 100bT interface in the transfer lines. I don't see the necessity to use the fast UDP channel for any loop there neither. Unless you explain me why this is necessary, we will not put it there and use the standard CMW way instead since the maximum acquisition frequency rate will always be lower than 1 Hz (1.2 s min between injections).
TW: That the second interface will be using a special driver that does not use the LynxOS HBTCPIP thread
QK: I didn't hear anything about this and I don't know what the HBTCPIP thread is. Could you elaborate?
TW: That all equipment concerned will have access to UTC time so we can monitor the time delays and take appropriate action in the feedback loops (gain scheduled control)
QK: Monitor time delays... and packet loss, yes, so we can say if the network infrastructure is delivering the necessary performance. Knowing the *overall* latency is needed to tune the feedback loops (acquistion to activation), but gain scheduled control? I don't remember this being mentioned - though perhaps it's a good idea?
I would summarise my understanding of the meeting as follows (Kris, is this right?): RT comms for the BI tests in SPS will be attempted using UDP with a simple API provided by SL/CO (see below for the library I wrote to do this at JET). Kris has already identified a commercial product which might be interesting if we find that simple UDP is not adequate.
For command/response, the CMW provided by Kris + Vito is being developed to support non-blocking sets/gets in the device adapter using ORBacus/E (the light weight embedded version of CORBA from ORBacus http://www.orbacus.com/obe/ORBacusE.pdf ). Non-blocking operation is essential for power converter control, but is not required (at least initially ;-) for BI.
For publish/subscribe, Kris is working on an improved device adapter (using ORBacus/E) which will be much more performant than the present version. This should be available for initial tests within a week or so. Anyway, as promised, you can look at the UDP based library I wrote at JET for RT communications.
Anyway, as promised, you can look at the UDP based library I wrote at JET for RT communications.