CERN Accelerating science

Client API

Based on discussions held on 6th and 21st June

Input: Greg Kruk, Steve Page, Lionel Mestre, Robin Lauckner, Herv� Milcent, Franck Di Maio, Adriaan Rijllart, Kris Kostro, Sergio Pasinelli, Mike Lamont.

Main input: Greg 
Reported by:  Mike

Client Interface attributes

• set system ID   e.g. QPS  - identifies sender, power converter, BCT, maybe not necessary, not available except by hard coding in FESA.  TBD.
• set device class e.g.  DQAMC
• device ID [] (optional). Keeping the "device ID" the same as the official LHC device name would make things more consistent on the database side of things. Could resolve this at upper level. e.g. FGC/gateway ID to power converter name  if a front-end didn't have access to the device names. (which ever is more convenient for client)
• Event timestamp or trigger timestamp TT - when trigger happened .[same timestamp for chunked data]
• Segment number  For use if the client chunks up the data. The sequence number with the event timestamp would form part of a primary key allowing the data to be pieced together and ordered. How does the client signal end of transaction? (Reverse sequence, reasonable timeout, server will not glue together, done later.) Client sends in order. It is predefined  how the break down of the data will be done. Client knows how long the sequence, thus can send the expected number of segments - see below. Server has timeout (time between block sends), raises alarm if it doesn't get what it expects. Could be a signal to go and look at local storage. (Browser will still keep and show incomplete data)
• What happens if you send the same thing twice? PM server discards it.
• Number of segments (or max sequence number.) PM server has to handle the chunks and uses this as a check that it has got everything.
• Alternative storage address if available in event of emergency cache. Optional. PC GW path to RAM disk (useful if network failure rather than power outage). Weak. What format? Could have standard utility to try and pick the pieces up. File path... Ethernet segments , nine PM servers instead of one
• Data - binary block, first bytes of data should specify the number of records in the data block [only for variable length arrays]. Not needed if we know before hand e.g. power converters.

Meta data already cached in PM server - sent by separate channel well before PM event [BinX XML description]

Example:

• System QPS   Class:  DQAMC   Device: MB.A12.L1 [4 buffers]
• System: PC or FGC  Class: 51  Device: RB.QS12.L1

Functionality

• Re-tries on fail
• Raise alarm in case of problem - debated. Steve would be happy to report an alarm himself in the case of an error being returned from the PM API. if an alarm is required this should not be left to the client. Yes.  Alarm ID
• Error  returned to client  No stderr/stdout.
• Sending in chunks - handled transparently by PM server
• [UTC, Naming] - data content issue
• Writes to local storage (if available) in case of irresolvable connection problem to PM server, clearly check local storage access before trying to write to it.
• scalability (high level issue) e.g. 100 PM buffers in 30 seconds from 3-6 GWs
• [heartbeat to server - not necessary in first implementation] Steve comments: I'm not completely convinced by the 'heartbeat' idea (ICMP ping / periodic CMW get of arbitrary property from server?). This seems to be an implementation of a network alarm within the PM API. In principle, there should already be alarms from IT in the following cases:

  - A front-end is not reachable on the network.
  - The PM server is not reachable on the network.
  - There is some sort of general fault with the network.

  If none of those cases is true, then the network link between the PM client and server should be okay. Would we really want PM alarms from all of the front-ends in the case of a network problem?
  RL I am also not sure if this should be offered. If it is then it would be intended so that clients could include post mortem ready in their �permit� conditions

  Team decided that it was unnecessary, however API should implement PMcheck call. Periodic checks if  felt needed. Raise alarm if problem (if it can it see the server). On re-arm of buffer check. etc.

Implementation

• C
• CMW - can we do a complex data type encompassing all above information? Check this. Multiple calls would be messy.
• Code should be re-entrant to support multiple calls to PM system

Data

• Data in binary
• BinX meta-data 

Platforms

• FESA
• Lynx/OS
• Linux (e.g. PIC)
• Others? Talk to interlock system guys. PVSS to CMW. (HP/Proliant)

Comments on  Above from Robin

The first 3 attributes: system ID, device class, device ID, are a triplet serving to identify the data object. As objects may vary from single values to complex structures (as in the case of the QPS) they will not generally map to �official� LHC device classes. Objects may even come from more than one �system� where I assume the definition of �system� to be linked to the accelerator hardware systems. While the main ambition is that the triplet is unique I would not try and map it to an existing entity within the control system.

The Event timestamp attribute identifies the instance of the data object. It should be in UTC.

The server should not be involved with re-assembling segmented data. The sequence number identifies the segments.

The first 5 attributes will be used by the server to generate a unique file path and name where the associated instance of the data object will be stored.

The Number of segments is not correct here. Assembly of segments should be done when the data is converted into the SDDS file. This quantity will be stored in the meta data. Consider two transactions with the same data object triplet but different Number of segments.

The Alternative storage address is not correct here. This safety mechanism should be hidden from the client.

Interface c/o Greg

CMWStructure {
char* systemId;  ???????
char* deviceClass;
char* deviceId;
64bits eventTimestamp;
longlong firstRecordTimestamp; [standardised ns]
(...)

byte[] pmData;
}

This is just an example how it could look like. As you see the additional meta information is not included in the binary data but as separate fields in CMW structure.

If you remember on the TC "Software for LHC HWC" on the presentation I made - I mentioned that it seems a good idea to have a kind of client API that could be used to send the data. I would see such API like:

//Pseudo-code in Java

PMClient client = new PMClient("MySystemID", "MyDeviceClass", "MyDeviceID");

PMData data = client.createData(<timestamp>, <binaryblock>);

data.setAdditionalProperty(...); ...

client.sendData(data);

//Pseudo-code in C++ !!

PMClient* client = new PMClient("MySystemID", "MyDeviceClass", "MyDeviceID");

PMData* data = client->createData(<timestamp>, <binaryblock>);

data->setAdditionalProperty(...); ...

client->sendData(data);

// C interface would implement the above as a function call

Present situation - c/o Sergio & Steve

// Thread to submit post-mortem data

void run(void)
{
unsigned int pm_index;
unsigned int pub_index;

while(pm_globals.run)
{
// Wait for a post-mortem event

sem_wait(&pm_globals.sem);

// Do nothing if data is not frozen for post-mortem

if(!pub.frozen) continue;

logPrintf("PM dump started\n");

// Copy frozen published data to post-mortem buffer

for(pm_index = 0, pub_index = (pub.published_data_index + 1) % (PM_BUF_LEN + 1) ; pub_index != pub.published_data_index ; pub_index = (pub_index + 1) % (PM_BUF_LEN + 1))
{
pm_globals.pm_data[pm_index++] = pub.published_data[pub_index];
}

// Un-freeze published data

pub.frozen = 0;

// Send post-mortem data

pmclient->simple_PO_test_set((const signed char *)pm_globals.pm_data, (unsigned long)sizeof(pm_globals.pm_data));

pmclient->simple_PO_test_send();

logPrintf("PM dump completed\n");
}
}