Electron cloud buildup driving spontaneous vertical instabilities of stored beams in the Large Hadron Collider

At the beginning of the 2016 run, an anomalous beam instability was systematically observed at the CERN Large Hadron Collider (LHC). Its main characteristic was that it spontaneously appeared after beams had been stored for several hours in collision at 6.5 TeV to provide data for the experiments, despite large chromaticity values and high strength of the Landau-damping octupole magnet. The instability exhibited several features characteristic of those induced by the electron cloud (EC). Indeed, when LHC operates with 25 ns bunch spacing, an EC builds up in a large fraction of the beam chambers, as revealed by several independent indicators. Numerical simulations have been carried out in order to investigate the role of the EC in the observed instabilities. It has been found that the beam intensity decay is unfavorable for the beam stability when LHC operates in a strong EC regime.


I. INTRODUCTION
At the CERN Large Hadron Collider (LHC), an anomalous beam instability was systematically observed in 2016 while the beams with 25 ns bunch spacing (hereinafter referred to as 25 ns beams) were kept stably in collision at 6.5 TeV.This instability exhibited the following characteristic features: (1) It affected only a few bunches near the end of each train of 72 bunches; (2) It appeared only in the vertical plane; (3) It could be mitigated by increasing the vertical chromaticity; (4) It disappeared when operating with shorter bunch trains (48 bunches); (5) It was observed only for bunch intensity less than a certain value.
The first four of the enumerated features suggest that this instability could be driven by an electron cloud (EC).
It has been known for many years that in high energy accelerators operating with positively charged particles, photoemission and secondary electron emission can give rise to an exponential electron multiplication within the beam chamber, which leads to the formation of a so called EC.At the LHC, several independent observations indicate that EC develops in a large fraction of the machine, especially when operating with closely spaced bunches [1,2].One of the most prominent undesired effects of the EC observed in these machines is the development of coherent instabilities caused by the coupled motion between particle beam and electrons [3].During the bunch passage, the electrons are attracted towards the center of the bunch resulting in an increase of the electron density within the bunch itself, called pinch effect.The distortion of the electron distribution is the mechanism that couples the motion within the bunches and gives rise to coherent instabilities and emittance growth [4,5].These instabilities mainly develop in the vertical plane when the EC builds up in the dipole magnets, where the motion of the characteristically low-energy electrons in the cloud is constrained to spiraling around vertical magnetic field lines.Throughout the years these effects have been observed in several proton and positron rings, such as the CERN Proton Synchrotron (PS), the Super Proton Synchrotron (SPS), as well as the KEKB [6].
In this paper, we show how numerical simulations can be employed to strengthen the hypothesis that the EC is the culprit of the observed instability and explain the observation 5 listed above.The paper is organized as follows.A detailed description of beam instability observations and of the deployed mitigation strategies is given in Sec.II, while Sec.III is focused on the EC observations at the LHC.The simulation tools used for the modeling of the cloud-bunch interaction are described in Sec.IV.In Sec.V, we first investigate the impact of the bunch intensity on the instability threshold for different EC densities in the LHC dipole magnets.Subsequently, we compare the identified thresholds with the estimated electron density at the beam location.The comparison between simulation results and experimental data is also discussed.Finally, we summarize the results and draw some conclusions in Sec.VI.

II. EXPERIMENTAL OBSERVATIONS
At the beginning of the 2016 run, the LHC operation was affected by a spontaneous instability which was observed systematically in most fills with more than 600 bunches, in spite of the large chromaticity value, the high strength of the Landau-damping octupole magnets and the presence of the beam-beam head-on tune spread.The characteristic feature of this instability lies in the fact that it was observed in both beams during the so-called stable beams (SB) mode, i.e., when the beams are kept stably in collision at 6.5 TeV to provide data for the experiments.Figure 1 shows an example of bunch by-bunch emittance evolution along a circulating bunch train acquired during the SB.The instability manifested itself few hours after the collisions were established leading to a sudden emittance blowup exclusively in the vertical plane for selected bunches of both beams.Based on these measurements, we could also investigate which bunches along the train were mostly affected.This is more clearly seen in Fig. 2 where we plot the bunch-by-bunch emittance pattern along four trains.In particular, the vertical dashed lines delimit the same train shown in Fig. 1.Two measurements were taken 2 h and 6 h after the beginning of the collisions, respectively.It is evident that the bunches becoming unstable are those at the tails of the bunch trains.More generally, Fig. 3 displays the occurrence of the vertical emittance blow-up as function of the bunch number within the trains, calculated over all physics fills in which the instability was observed.The plot shows a very clear instability pattern, which suggests that shorter trains (e.g., 48 bunches) could be more stable.
In order to gain a further insight into the origin of this phenomenon, we investigated the dependence of the instability onset on the bunch intensity.In fact, due to the collisions, an unavoidable beam intensity decay occurs when the beams are in collision.This mechanism includes the luminosity burn-off and possibly the effect of dynamic aperture limitations coming from head-on and long range beam-beam interactions.Figure 4 illustrates the evolution of the beam intensity for different physics fills.The red dots indicate the time at which instabilities occurred.The plot shows that most of the instabilities were observed when the beam intensity had decreased to values corresponding to bunch intensities between 1 × 10 11 and 0.7 × 10 11 protonper-bunch (ppb).
In order to mitigate and experimentally characterize this instability, the following strategies have been employed.Starting from the beginning of June 2016, the vertical chromaticity (Q 0 y ¼ dQ=ðdp=pÞ) for both beams was increased from Q 0 y ¼ 15 up to Q 0 y ¼ 22, right after the SB started.Following previous experience [7,8], this approach provided a sufficient mitigation against the beam instability and allowed storing stable 72-bunch trains into LHC.The beneficial impact of the chromaticity on the beam stability is shown in Fig. 5.The graphs display the bunch-by-bunch luminosity measured by the compact muon solenoid experiment (CMS) during two fills with different vertical chromaticities.The luminosity of each bunch is normalized to the value measured at the beginning of SB and a red point marks any sudden reduction in luminosity due to the emittance blowup.The plot on the top of the figure refers to a physics fill operated with Q 0 y ¼ 15, whereas the bottom one refers to a fill with Q 0 y ¼ 22.In the latter case, an overall reduction of the number of unstable bunches could be observed as well as a significant delay in the instability development.
At the end of June 2016, the production scheme in the injectors was changed to increase the beam brightness [9].With the new scheme the beam was made of trains of 48 consecutive bunches instead of the nominal 72 bunches.The first physics fills after the change were performed by following the same approach as described above.However, with this new injection setup, it was possible to gently reduce the vertical chromaticity, (from 22 → 18 → 16) until the nominal setting was restored (i.e., Q 0 y ¼ 15) without any instability observation.
In August 2016, dedicated tests were made during machine development sessions to better assess the impact of the bunch intensity and the chromaticity on the instability development.Three fills were performed in trains of 72 bunches with various bunch intensities (e.g., 1.1 × 10 11 , 0.9 × 10 11 , 0.7 × 10 11 ppb).The full operational cycle to bring the beams into collision was performed and, after 1.5 h in SB, the vertical chromaticity was reduced to Q 0 y ¼ 5 in small steps, without observing any detrimental effects on the beam stability.
Due to the distinguishing features described above, the EC build up was considered as a driving mechanism of this instability.

III. EC OBSERVATIONS AT THE LHC
The 2015 proton run (first year of luminosity production with 25 ns beams at top energy of 6.5 TeV) showed that the EC effects could significantly limit the LHC performance.At the beginning of the operation, since the LHC beam screens were exposed to air during the Long Shutdown 1 (LS1) in 2013-2014, their secondary electron yield (SEY) was found significantly degraded with respect to the pre-LS1 operation [10].For this reason, an extended scrubbing run of four weeks at injection energy took place with the aim of enabling physics production with beams having 25 ns bunch spacing.During the scrubbing run the beam conditions were deliberately chosen in order to lead to a significant EC, so that the resulting intense electron bombardment of the beam chambers could decrease the SEY of the surfaces (beam conditioning).Despite an evident conditioning which was observed during the scrubbing run, the full EC suppression was not achieved.Even by the end of the scrubbing run, the heat dissipated on the beam screen due to the interaction with the beam (beam induced heat load), as measured by the cryogenic system, revealed values larger than those expected from impedance and synchrotron radiation, showing that the dominant contribution was still due to the EC [11].
One of the first consequences of operating in a strong EC regime was the difficulty to ensure the beam stability and a good beam quality at injection.For this purpose, high chromaticity and high strength of the Landau-damping octupole magnets were needed together with a fully functional transverse feedback system [12].However, due to the large tune spread induced by the large chromaticities and the strong EC, the tunes of the machine had to be carefully placed far enough from the dangerous third order resonance line Q y ¼ 0.33.Indeed, incoherent losses which were observed when the vertical tune of the LHC was at its nominal value of 0.31 could be avoided by lowering its value to 0.295.The need of operating with an optimized working point to better accommodate the large tune spread has been fully confirmed via numerical simulations, as discussed in [13].Relying on the conditioning of the beam screen and the finer tuning of the machine and beam parameters, 2244 bunches per beam in short trains of 36 bunches were successfully brought into collision by the end of the 2015 proton run, despite the significant levels of EC still present in the machine [10].
In 2016, after only one day of scrubbing run, LHC went into physics production.Using the settings found to be beneficial during 2015, up to 2040 bunches per beam could be stored in the LHC in trains of 72 bunches.Throughout the year, the maximum number of bunches was limited by technical problems in the LHC injectors.Still, the heat load on the beam screens of the arcs was very close to the limit of the cooling capacity of the cryogenic system [14], which indicated that the EC was still present in the machine.
Pinning down the beam dynamics effects associated with the EC is therefore crucial to find the connection with the instability described in the previous section and explain all its features.For this purpose, an extensive simulation campaign has been carried out using the PYECLOUD-PYHEADTAIL suite, developed at CERN, explained in the next section.

IV. THE PYECLOUD-PYHEADTAIL SIMULATION SUITE
The understanding of the beam behavior in the presence of an EC can be simulated with the PYECLOUD-PYHEADTAIL simulation suite [15].PYECLOUD is a macroparticle code modeling the buildup of ECs in particle accelerators [16].PYHEADTAIL is a macroparticle tracking code used for simulating various collective effects in charged particle beams [17].These codes are the evolution of the ECLOUD and HEADTAIL codes developed at CERN and used for EC studies in the past decade [2].These tools are used extensively for studies of both current and future CERN accelerator complex and have been extensively benchmarked in the past [7,[18][19][20][21][22][23].Thanks to their modular structure and flexible nature, PYECLOUD and PYHEADTAIL can be coupled together to simulate in a combined way the dynamics of the EC and its effect on the beam.This new approach has several advantages with respect to the HEADTAIL code [24] such as the possibility to define the boundary conditions on an arbitrarily shaped chamber, to simulate the full EC dynamics using the model implemented in PYECLOUD including different magnetic field configurations previously unavailable (e.g., combined function and quadrupole magnets) [25] and the secondary emission process when electrons reach the chamber.
In this new setup, PYHEADTAIL is used to track the bunch along the machine.The action of the EC on the bunch is lumped at selected interaction points which in our simulation study have been equally spaced along the lattice.In each of these interaction points, the state of the bunch macroparticles is passed to PYECLOUD.At this point the bunch is longitudinally sliced and each slice successively interacts with the electrons (which can be initialized uniformly in the beam chamber or using the output from a dedicated PYECLOUD buildup simulation).Their dynamics is computed using the particle-in-cell (PIC) solver and the forces from the electrons are applied to the bunch particles.The updated beam phase-space coordinates are subsequently passed back to PYHEADTAIL to continue the tracking.
This kind of simulations is highly consuming in terms of time and computational resources.For this reason, new advanced features have been included in the codes and parallel computation has been exploited [15].This allowed simulating different effects of the EC on the beam dynamics in increasingly complex scenarios and for longer simulation spans, which were previously inaccessible.Nevertheless, the computational burden is still quite heavy.Indeed, a typical instability simulation study requires hundreds CPU cores organized in jobs using 8-16 cores each.Three to four weeks of computing time were needed to simulate 10 4 turns, which is the timescale of the instability rise-time experimentally observed.

V. SIMULATION RESULTS
The interaction with the EC has been simulated for a selected bunch at the tail of a bunch train.In order to best mimic the experimental conditions in which the instabilities were observed, we used the machine and beam parameters listed in Table I, unless stated otherwise.In the simulation model we included: (i) EC in the dipole magnets; (ii) EC in the quadrupole magnets; (iii) Chromaticity; (iv) Transverse amplitude detuning; (v) Bunch-by-bunch transverse feedback.
The numerical parameters used for the simulations were determined through convergence scans and are listed in Table II.
For the dipoles, which cover 65% of the total machine circumference, simulations were initialized with a uniform electron distribution within the chamber.This has been found to be a good approximation in the presence of a dipole magnetic field since the instability onset is strongly dependent on the central electron density seen by the beam [19].Conversely in the quadrupole magnets, which constitute about 7% of the LHC ring, simulations were initialized using the electron distribution as produced by the buildup code.In fact, due to the trapping effects from the magnetic field gradient [26], the EC pinch dynamics is very sensitive to the initial phase space distribution of electrons.More details can be found in [27].The EC in the drift regions of the insertion regions (IRs) has been neglected because the IRs are mainly NEG-coated and therefore have a low SEY.
To gain an insight into the physical mechanism that induced the observed instabilities, we have first scanned the electron density in the dipoles between 1 × 10 11 e − =m 3 and 22 × 10 11 e − =m 3 for a certain value of bunch current.This allowed identifying the value of electron density in the dipoles above which the beam becomes unstable (i.e., the instability threshold).The lower value of this range, 1 × 10 11 e − =m 3 , has been chosen to roughly match the number of electrons photoemitted from the wall due to the beam synchrotron radiation, while the higher value represents a typical saturated electron density under conditions of strong multipacting.In the quadrupoles, the electron distribution has been loaded from an earlier dedicated buildup simulation and kept unchanged.The distribution in the quadrupoles used as input for the instability simulations is shown in Fig. 6.Macroparticles coordinates and velocities were saved right before the passage of the last bunch of a 72-bunch train and for a maximum SEY of 1.3, as inferred from heat load measurements performed at the end of the 2016 scrubbing run.In fact the heat load data point to a maximum SEY of about 1.4 and 1.3, respectively for the dipoles and the quadrupoles.
Figure 7 shows the results obtained for a bunch intensity of 0.7 × 10 11 ppb.It is evident that the behavior in the two planes is strongly asymmetric.In fact, it can observed that the bunch becomes unstable only in the vertical plane.Such coherent motion, which leads to a strong emittance blowup, cannot be damped by the transverse bunch-by-bunch feedback system because it excites high order intra-bunch modes [13], as shown in Fig. 7 (right side).The instability threshold is found at around 5 × 10 11 e − =m 3 .For larger EC densities, vertical instabilities can occur in spite of the presence of a fully functional damper, high chromaticity and high strength of the Landau-damping octupole magnets, consistently with previous simulation studies [28].The simulations have been repeated for 1.0 × 10 11 ppb finding a very similar instability threshold.
Since the instability threshold is found not to depend on the bunch current, we have also investigated how the central electron density in the dipoles varies with the bunch intensity.Figure 8 shows the EC profiles estimated with PYECLOUD buildup simulations assuming different bunch intensities.These snapshots were taken for selected bunches along the bunch train.In dipole regions, the electron motion is confined along the lines of the magnetic field.Electrons trapped by different magnetic field lines will receive a different kick from the beam resulting in the characteristic vertical stripes, with increasing electron density along the bunch train.The position and the number of these side stripes mainly depends on the bunch current and on the value of the incident energy relative to the maximum in the SEY dependence on incident energy.At bunch intensities above 1 × 10 11 ppb, like in the range between the present operational value and the one required by the High Luminosity LHC (HL-LHC) upgrade (1 × 10 11 − 2.2 × 10 11 ppb) [29], the EC density within the chamber features a two-stripe structure and its density around the bunch is very low.In contrast, for lower bunch current a third stripe develops at the center of the chamber and the region around the horizontal beam position gets densely populated with electrons.The bunch intensity decay while the beams collide in SB spans this range.
To check whether the EC is responsible for the instability development, we compared the estimated central densities with the instability threshold.identified instability threshold (marked with a red line).This is significantly reduced at an SEY value of 1.35 and entirely absent at an SEY value of 1.30.This reveals that the conditioning of the beam screen due to the beam scrubbing could have a beneficial impact on the instability development.Indeed, the fact that the instability observed in 2016 disappeared later in the year, even with low chromaticity, could be ascribed to the conditioning of the central region of the beam screen accumulated with physics fills in the previous weeks.The beam stability can also be improved by increasing the chromaticity.In Fig. 9, the black dot shows the identified instability threshold for a bunch intensity of 1.0 × 10 11 assuming Q 0 y ¼ 22, instead of Q 0 y ¼ 15, and a SEY of 1.4 in the dipole magnets.As a result, the instability threshold becomes higher but it can still be crossed at lower bunch currents.This was also observed experimentally.In fact, as shown in Fig. 5, the chromaticity increase did not suppress completely the instability but it made it less severe and delayed it.
Finally, we compared the obtained simulation results with the experimental data for the fill shown in Fig. 1.The plot on the top of Fig. 10 shows the bunch intensity evolution measured during the SB, while the plot on the bottom displays the evolution of the electron density at the beam position in a dipole chamber, as inferred from buildup  simulations.The red line marks the identified instability threshold.It is clear that, when the beam intensity drops, the electron density at the beam position increases, reaching the threshold after several hours in SB.Thus, simulation results are shown to closely represent the experimental data.

VI. CONCLUSIONS
At the beginning of the 2016 run, a spontaneous beam instability was systematically observed at the CERN LHC, while the 25 ns beams were stably colliding at 6.5 TeV to provide data for the experiments.In particular, a few bunches at the end of the circulating trains became vertically unstable when the beam intensity had decreased due to the collisions.Such instability pattern, which was very reproducible from fill to fill, strongly suggested that the EC could be the main driving mechanism.
Simulations of beam dynamics show that the value of electron density that makes the beam unstable does not depend on the bunch current.However it was found that, for lower bunch current, the EC density in the immediate neighborhood of the beam increases, becoming sufficiently high as to exceed the instability threshold.The non-uniformity in the horizontal EC profile arises from dynamics in dipole magnetic fields.This range of bunch intensities are usually reached a few hours after the collisions are established.These numerical predictions are shown to be in good agreement with the LHC observations.Indeed, the comparison against the experimental data confirms that, even in the presence of high settings of chromaticity and strength of the Landaudamping octupole magnets, after few hours in collisions the EC at the beam location can exceed the instability threshold and drive the beam unstable.As observed also experimentally, the beam stability is improved by increasing further the vertical chromaticity and when the SEY of the beam screen surface is reduced by beam induced scrubbing.
These results are particularly relevant in the framework of the HL-LHC upgrade, showing that the source of this instability is weaker by two orders of magnitude in the range of intensities that will be covered by the HL-LHC upgrade.

FIG. 1 .
FIG. 1. Evolution of the horizontal and vertical bunch-bybunch emittance for one of the circulating bunch trains measured during the SB period.The bunch number is color-coded.The instability development is indicated by the arrows.

FIG. 4 .
FIG. 4. Beam intensity evolution over time for several physics fills.The red points indicate when emittance blowup was observed in each fill.Most of the instabilities occurred in a range of beam intensities which corresponds to bunch intensities between 1 × 10 11 and 0.7 × 10 11 ppb.

Figure 9
FIG. 7. Left: Evolution of the horizontal and vertical position of the modeled bunch centroid.Middle: Evolution of the horizontal and vertical normalized emittance (results are shown for different central cloud densities in the dipoles as labeled).Right: Simulated intrabunch oscillations in the horizontal and vertical plane for an EC density in the dipoles of 22 × 10 11 e − =m 3 .The traces have been selected for a time interval of 0.6 to 0.62 s (i.e., 250 consecutive turns marked with different colors).

FIG. 9 .
FIG. 9. Electron density estimated at AE2.5 mm around the horizontal beam position in a dipole magnet chamber as function of bunch intensity.The red line shows the estimated instability as obtained from PYECLOUD-PYHEADTAIL simulations for Q 0 y ¼ 15.For a bunch intensity of 1.0 × 10 11 the threshold has been estimated also for Q 0 y ¼ 22 (black dot).The arrows indicate the bunch intensity at the beginning of the collisions for the LHC (1.0 × 10 11 ppb) and for the HL-LHC (2.2 × 10 11 ppb).

FIG. 8 .
FIG.8.Horizontal electron density profile in a dipole magnet for different bunch intensities.These snapshots have been taken for selected bunches along the bunch train (as labeled).The vertical dashed lines delimit a distance of AE2.5 mm from the beam position (x ¼ 0).

TABLE I .
Parameters for an LHC bunch at high energy.

TABLE II .
Numerical parameters used in the simulations.
FIG.6.Electron distribution in an LHC arc quadrupole right before the passage of the last bunch of a 72-bunch train, for a maximum SEY of 1.3.