25 May 2008

In the control room

As we haven't had a post for a while, I thought I would write something. As I am sitting in the control room of the D Zero experiment, waiting for the beam to start up again, control room work seems like a good topic.

I will give a brief explanation of what the D Zero experiment is to begin with: I will try to keep the physics quite simple. D Zero is a colliding beam experiment at the Tevatron accelerator on the outskirts of Chicago. The Tevatron accelerates protons (hydrogen nuclei) and anti-protons to a high energy before colliding them. The collisions produce a large amount of energy, which takes the form of subatomic particles. These particles are then detected and analysed by the D Zero detector.

So far, so good. However, you have probably realised that the experiment doesn't work on its own: it needs a team of people to keep it running smoothly. A bit like the Marquess of Bute's wine, the D Zero experiment requires at least four people to be present at all times. These four are the CalMuo shifter (who looks after the calorimetry and the muon system), the Tracking shifter (who looks after the tracker), the DAQ shifter (who looks after data acquisition) and finally the captain, who co-ordinates everyone's efforts. I am trained as a DAQ shifter, so it's my job to make sure that the data is flowing properly.

The D Zero control room is a narrow rectangular room in the basement of the main D Zero building. There are long desks running along each side of the room, with numerous telephones, keyboards, computer mice, pens, wires and bells cluttering their surfaces: this is par for the course in most physics labs. My station is about halfway along the room, and above my desk loom various loudspeakers, whiteboards, soft toys, ringbinders, lists of telephone numbers and ancient post-it notes covered with faded handwriting. It is nine large computer monitors, however, which dominate the space above me. It is my job to watch these, in order to make sure that the experiment is collecting data as expected.

This isn't as difficult as it sounds; essentially I only really care about two screens: one which shows the rate of data flow (I begin to get worried if it shows zero), and one which shows any errors that are affecting the system. The rest will help me to identify any problems the system might have. Usually things can be fixed quite simply, but occasionally the problem is quite complicated. More often than not I will find that I need the help of another shifter to solve things (for example, if the calorimeter has an error, I need to work with the CalMuo shifter to fix it). If it is beyond us, it is time to call in the experts, but I have only had to do that once.

Things get a lot more complicated if someone needs to make repairs (usually to systems close to the detector, but not within it). When the accelerator is inactive, it is safe enough for teams to go into the collision hall to make repairs (there are stringent safety measures in place, of course). Because it is very rare for the accelerator to be off, everyone will come down to the control room at once to work on their particular system. These times require some co-ordination, as everyone needs the system changed in a particular way. Sometimes it can be hard to tell someone far more experienced than you that they have to wait for someone else, but usually someone more authoritative will help me out.

That is the busiest it gets though. Most of the time, the DAQ system is running well and doesn't need much attention. I have been on night shift (midnight to 8am) this week. This has the disadvantage of making me become nocturnal, but it is good in a way because I can chat with my friend in Hong Kong. I have also been playing "Limericks" with a friend in London to pass the time: we write a Limerick one line at a time, taking it turns to come up with the next line. The end results, to be frank, range from poor to absolutely terrible, but it helps pass the time.

Having worked in the control room, I now realise how much work it takes to run a major particle physics experiment. Before I started my PhD, I was under the impression that you turned the thing on, and it just ran, churning out great physics 24/7. I now realise how naive that was. But it makes me wonder about the LHC experiments: D Zero is tiny in comparison with them. They are planning to have hundreds of people operating these experiments at a time: how will they communicate with each other effectively? How can they organise themselves when a problem needs fixing? Do they have procedures to follow for any eventuality worked out in advance? And then there is the sheer size of the detector: when something breaks down, how do they intend to get to it to fix it? So, not only is the LHC a huge technical challenge, it is an enormous logistic and organisational challenge, and it will be interesting to see the solutions they come up with.

Whoops, that was a very long article... congratulations if you got this far! It is hard to say something exciting about something that is essentially quite boring, but at least it gives you an idea of some of the work that we have to do in particle physics.

3 comments:

Alex said...

It takes about a hundred people in the control room (well there are around a hundred people there anyway, some may be superfluous) to start cosmic-ray muon runs with CMS at the moment. After initial problems are fixed and all the detector components are up and running it drops to perhaps twenty on night shift to monitor the detectors and run the DAQ and trigger. It's difficult to finding a parking space and we get through a lot of coffee.

Alex said...
This comment has been removed by the author.
Ian said...

In comparison, SciBooNE shifts are very lonely, it takes just a single person to watch both the SciBooNE and MiniBooNE experiments. On the plus, the control room we share is on the 10th floor of Wilson Hall, so you get a nice view and some natural light...