CERN Accelerating science

I shall comment mostly on the effects I would expect to see on the FGCs in the LHC in the event of a timing failure. Stephen may have more to say on the effects on the FGC gateways.

The timing cards in the gateways are designed with a local oscillator that will maintain autonomous operation if the GMT signal is interrupted. This means that the WorldFIP field buses should remain synchronised and the FGCs will not be aware of the interruption. As you write, the consequence will be that timing events will not get to the affected gateways, so FGCs controlled by those gateways will not start a ramp/trim when required and the gateway will not receive post-mortem triggers. In theory, the FGCs could be sent a command containing the absolute time of an event, but in practice there would be no point to implement this alternative route for timing events. If for some reason the timing interface in the gateway fails completely, then the WorldFIP segment will stop operating. It is possible to imagine failure modes that could cause the WorldFIP segment to operate in an unsynchronised manner. This was seen until the latest release of the WorldFIP interface software even without a failure of the timing; however it appears to have been corrected now. In principle, the same effect could result if the timing interface emitted timing pulses erratically. At the moment the FGC phase-locked loop software is not tolerant to unsynchronised timing pulses and power converter trips were sometimes seen. I have a new design of PLL algorithm that will overcome this vulnerability and which will be implemented before commissioning with beam in 2007. If traffic ceases on a WorldFIP segment for any reason (e.g. rebooting a gateway), each FGC will continue doing whatever it was doing with its timing assured by its local oscillator. If it was ramping then the ramp will continue. However, there will be a number of consequences at the global level:

1. No commands can be sent It will not be possible to send commands to the FGC so state changes for the converter cannot be requested, nor can reference changes be set up.

2. No timing events can be sent If events can still be received by the gateway (assuming a failure mode that affects the WorldFIP synchronisation but not the reception of timing events), then the events could not be distributed to the FGCs.

. No status data can be published The 50Hz publication of FGC status will not occur, so the gateways will not receive faults and warnings from the FGCs and therefore alarms will not be generated and continuous logging will be interrupted. Furthermore, the gateway post-mortem log buffers will not operate. If there is a beam dump, there may be no information about the state of the circuits affected by the loss of communication. I say "may" because there are circular log buffers in each FGC, however, they have a different triggering method: -The gateway post-mortem logging buffers will be stopped if a post-mortem event is received via the timing system -The FGC logging buffers will be stopped when the power converter stops (either voluntarily or involuntarily) So if the beam loss that caused the post-mortem event also results in tripping of the power converters controlled by the affected FGCs, then they will freeze their local log buffers and data will be available for the post-mortem once communication is re-established. If, however, the converters do not trip, then the data from the time of the beam loss will not be recoverable.

4. No real-time control will be possible If circuits affected by the communications loss are included in feedback on orbit, tune or chromaticity, then the feedback will be jeopardised. The FGCs will simply hold the last real-time value they received until the next arrives. If the value is a correction on pre-loaded ramp function, then the ramp will continue but with the last correction value. If, however, local function generation is not used and the real-time value is directly used as the current reference, then the current will simply freeze at the last received value. The effect on the controlled beam parameter will clearly depend on how much work the feedback loop had to do, which maybe a great deal during snapback and very little during physics. All this leads naturally to the question which I’ve asked in the past – should an FGC continue running a power converter indefinitely if communications are lost with the gateway?

Would it be safer at some point for the FGC to shutdown the power converter or would it be better to keep this decision at the top level and to dump the beam if FGCs have gone offline for longer than a certain time? I would certainly favour a grace period that would be long enough to enable the reboot of a gateway (~ 1 minute). For the moment the FGCs have no time out and will continue to run autonomously indefinitely. Finally, note that the FGCs that will be used to generate functions for the LHC RF system will receive timing signals on individual wires from timing receiver cards housed in nearby RF VME systems. If these signals are not generated, then the RF system will not respond correctly to LHC injections from the SPS and beam loss could be expected. Equally the RF system will not participate correctly in the LHC ramp.