Difference between revisions of "Run Group B"

From clas12-run
Jump to: navigation, search
(Short Term Schedule)
(Short Term Schedule)
Line 98: Line 98:
| valign=top width=50% |
| valign=top width=50% |
== <font color=blue>''' Run Plan: Jan 14, 2020 '''</font>==  
== <font color=blue>''' Run Plan: Jan 14-15, 2020 '''</font>==  
* We continue with production running and have moved the beam current up to 45 nA. This is based on (1) the analysis of the tracking efficiency (thanks to Stepan) during the last week which does not show compelling evidence that we are doing much worse than past runs despite the higher DC occupancies and (2) from Yuri's analysis which shows we can run the CVT at 40-45 nA.
* We continue with production running and have moved the beam current up to 45 nA. This is based on (1) the analysis of the tracking efficiency (thanks to Stepan) during the last week which does not show compelling evidence that we are doing much worse than past runs despite the higher DC occupancies and (2) from Yuri's analysis which shows we can run the CVT at 40-45 nA.

Revision as of 23:18, 14 January 2020


Shift Schedule

Shift Checklist

Hot Checkout

Beam Time Accounting



JLab Logbooks

RC schedule

  • Nov 20 - Nov 27: S. Stepanyan
  • Nov 27 - Dec 4: M. Contalbrigo
  • Dec 4 - Dec 11: D. Sokhan
  • Dec 11 - Dec 18: S. Niccolai
  • Dec 18 - Dec 20 S. Stepanyan
  • Jan 7 - Jan 15: J. Gilfoyle
  • Jan 15 - Jan 22: F. Hauenstein /W. Phelps
  • Jan 22 - Jan 29: B. McKinnon

Person Phone Number
Run Coordinator (757) 575-7540 (cell)
Physics Division Liaison (757) 876-1789 (cell)
Shift Expert (757) 329-4846 (cell)
Remote Shifter (day/swing) (757) 310-7198 (cell)
MCC-OPS 7048
Crew Chief 7045
Crew Chief 9-876-3367 (cell)
Program Deputy 9-876-7997 (cell)
Engineering on-call 757 748-5048 (cell)
Engineering on-call secondary 757 897-9060 (cell)
Gate House Guard 5822
Location Phone Number
Hall-B Floor 5165
Hall-B Space Frame 5170, 5171
Hall-B Forward Carriage 5371
Hall-B Counting Room 5245 (Shift1), 5126 (Shift2)
Hall-B Counting Room 5244 (Expert)
Hall-B Gas Shed 7115
System Phone Number On-Call Person
DC (757) 748-5048 (cell) Engineering On-Call
SVT + MM (757) 541-7539 (cell) Yori Gotra Primary
SVT + MM (757) 753-7769 (cell) Rafo Paremuzyan Secondary
ECAL (757) 810-1489 (cell) Cole Smith
FTOF/CTOF (757) 344-7204 (cell) Jose Carvajal
CND (757) 344-7204 (cell) Silvia Niccolai
HTCC-LTCC (757) 344-7174 (cell) Youri Sharabian
FT (757) 344-1848 (cell) Raffaella De Vita
BAND (757) 310-7198 (cell) Florian Hauenstein
RICH (757) 344-3235 (cell) Valery Kubarovsky
DAQ (757) 232-6221 (cell) Sergey Boiarinov
Slow Controls (757) 748-6922 (cell) Nathan Baltzell
Beamline (757) 303-3996 (cell) Eugene Pasyuk
Target (757) 822-9586 (cell) Bob Miller Primary
Target secondary (757) 897-9060 (cell) Denny Insley

| valign=top |

Backup system to the On Call:
Use this ONLY if the On Call
phone is not answered
CLASCSS->Page Expert button


|} Click Here to edit Phone Numbers. Note, you then also have to edit this page to force a refresh.

CLAS12 Run Group B, December 2019 - January 2020
Beam energy ~10.4 GeV (5 pass)
Important: Document all your work in the logbook!

RC: Jerry Gilfoyle

PDL: Maurizio Ungaro

Run Plan: Jan 14-15, 2020

  • We continue with production running and have moved the beam current up to 45 nA. This is based on (1) the analysis of the tracking efficiency (thanks to Stepan) during the last week which does not show compelling evidence that we are doing much worse than past runs despite the higher DC occupancies and (2) from Yuri's analysis which shows we can run the CVT at 40-45 nA.
  • The reporting of the beam energy on the scalar gui has been fixed and now reads 10.4 GeV.
  • The Hall C 1-pass running is scheduled to be completed during owl shift tonight (Wed owl). When they are finished the Wien angle will be changed back to it's previous setting. Since we already have Moller runs from the last week for this Wien angle setting we will continue running without a Moller run today (do one next week). Let the RC know if there are any issues with this plan.
  • Please note that we have added two items to the list of tasks for each run.
    • (1) When you collect the gui snapshots for the logbook (scalar gui, FTC Asym plot, livetime, ROC rates, beamline stripcharts) include the DAQ gui AND make sure the snapshot is taken when you're running. We want to record the rate plots on the DAQ gui.
    • (2) Measure the trigger structure (trigger-time distribution) about every 30 minutes and at least once per run enter the results in the logbook. See below for instructions. This is needed to monitor the injector laser phase which created serious problems last week.
  • If the trigger-time distribution or region 1 DC occupancies get worse ( > 6.5%) call the RC. What we will likely do is reduce the beam and restart.
  • At some later time we need to do an empty target run. Current will be 150 nA for 50M triggers which should take 2-3 hours.

DAQ config: PROD66

Trigger file : rgb_inbending_v9_1.trg

Compare the monitoring histograms

Run database: when starting/stopping a run, enter (or check if already present) the requested current in the pop-up window and put RELEVANT COMMENTS (eg: "production run", "trigger test vXX, changed this and that" not just "run") -- please label junk runs if you get the chance. Please also keep a record of run numbers and their conditions in the shift summary. The information you enter in the dialogue box goes directly into the run database -- it's our primary source of run info and mistakes are hard to correct a posteriori.

Please run the script to look at trigger structure once per shift and check that you get a smooth distribution. Instructions are here: [1]

Keep an eye on CND channel S14L1R -- it's a bit temperamental. If it trips, switch HV off and on again for it and make an entry in the log book.

At the end of each run, follow the STANDARD DAQ RESTART SEQUENCE
This needs to be followed at the end of every run. Note: this might be different from the instructions coming up on RunControl.

  • "end run", "cancel", "reset"
  • if the run ended correctly:
    • "download", "prestart", "go"
  • if the run did not end correctly or if any ROCs had to be rebooted:
    • "configure", "download", "prestart", "go"
  • After each step, make sure it completed in the Run Control message window. If a ROC has crashed, find which one it is, issue a roc_reboot command ON JUST THAT ROC and try again. Contact the DAQ expert if there are any questions.

References and Standards:

(last update 1/10/2020- 11:00)

Nominal Beam Positions

  • 2H01, X: +1.4 mm, Y: -2.2 mm
  • 2C21, X: -0.0 mm, Y: 0.2 mm
  • 2C24, X: -0.3 mm, Y: -0.8 mm

FSD Thresholds

  • Upstream: 2000 Hz
  • Midstream: 2000 Hz
  • Downstream: 700000 Hz
  • BOM: 60000

Reference Harp Scans for Beam on Faraday Cup: 2C21 [2], 2C24 (tagger harp) [3], 2H01 [4]

Reference Harp Scans for Beam on Tagger dump: 2C21 [5], 2C24 (tagger harp) [6]

Reference Monitoring Histograms (40 nA, production trigger) [7]

Counter rates

  • Upstream counters integrated rates: 0-15 Hz (acceptable up to 100 Hz are acceptable) @50 nA.
  • Midstream counters: 10-20 Hz (acceptable up to 50 Hz) @50 nA.
  • Counting rates of ~ hundreds of Hz may indicate bad beam tune or bleed-through from other Halls.

Beamline vacuum: should be not higher than 5e-5.

General Instructions:

  • The main lights in the Hall (dome lights) and the Forward Carriage lights are being kept off because of light leaks affecting some of the detectors. If these lights are switched on during an access, they should be switched off when leaving the Hall. Note that the dome lights when switched off cannot be turned back on immediately because they require 10-15 min to cooldown.
  • Do not run more than 30 minutes above 30 nA with 5-pass beam without the beam blocker in front of Faraday cup . Put beam blocker in for long running at high currents for 5 pass operations.
  • Turn DC HV off only for beam tuning; if no beam is available or when beam is stable, keep them on even if you are not taking data.
  • In case of loss of communication with IOCBTARG , follow instructions at https://logbooks.jlab.org/entry/3502218
  • With any issue contact On-Call Experts or RC - do not spend more than 15-20 min trying to fix the problem. If the phone is not answered, try the "Page Experts" button on top of CLASCSS.
  • Check for and read any comments to log-book entries made during your shift!

Every Shift:

  • Follow run plan as outlined by RC
  • If any concern about beam stability, ask MCC if orbit locks are on (they should be).
  • Keep shift summary up to date in HBLOG. Record all that happens.
  • Document any beam condition change and send scaler GUIs to HBLOG
  • Fill out BTA hourly. Click "Load from EPICS" to automatically fill the left side. Run the script btaGet.py to print for you the correct BTA info for this hour -- edit the values on the BTA page if they are wrong!
  • Upload spectra from clas12mon at begin and end of shift and cross-check with reference spectra (make sure you select the SAME SECTOR as for the reference plots on tabs which have multiple sectors).
    • Reset clas12mon every ~2 hours and inspect spectra (some detectors have buttons at the bottom to toggle between the sectors -- please check all of them).
  • Fill and submit the shift checklist in the logbook
  • Run script to check smoothness of the trigger time structure: [8]
  • Monitor the SVT Slow Controls status, post plot of current stability in the HBSVT elog once per shift.

Every Run:

  • Check values at the start of every run and log screenshots of:
    • Main scaler GUI display
    • Trigger rate GUI
    • Beam strip charts
    • FTC scalers
  • At the start and end of every run, check that the data filled in the database run list is correct (especially fields which are filled in manually, like beam current and comments). This replaces the previously-used spreadsheet.
  • When checking the ECAL response, make sure to look at the histograms for each sector by using the sector buttons at the bottom of the clas12mon window - if adcecal* or/and adcpcal* ROC crashes, there will be no alarm and the histograms are the way to notice this issue.
  • Same holds for FTOF.

To access the Hall:

  • If you need to go down into Hall B alert Denny Insley (757-897-9060, cell/text: 757-810-9576) and the Run Coordinator (757-575-7540).
  • Check in with the shift personnel before entering the hall.

Notes / Troubleshooting :

If crate ADCBAND was rebooted: Contact BAND expert and make a note in the HBBAND logbook and in HBLOG.

FTOF HVs: The goal is to minimize the number of power cycles of the dividers.

    • OFF during: initial beam tuning to the Faraday Cup after CLAS12 has been off for a long shutdown, during a Moller run, harp scans, or if there is tuned/pulsed beam in the upstream beamline.
    • ON after an initial beam tune has been established and if there are only minor steering adjustments and “tweaks” being made.
    • If in doubt, call FTOF expert

FTT (mmft1) recovery:

    • power cycle FTT LV (FT->FTT->turn LV OFF, turn LV ON)
    • roc_reboot mmft1

Torus and/or Solenoid Fast Dump

    1. Notify MCC to request beam OFF and to drop Hall B status to Power Permit
    2. Call Engineering on-call
    3. Make separate log entry with copies to HBTORUS and HBSOLENOID logbooks. In the "Notify" field add Ruben Fair, Probir Goshal, Dave Kashy and esr-users@jlab.org
    4. Notify Run Coordinator
    5. Turn off all detectors

Beam to the Faraday Cup: Fast Shut Down elements

  • Upstream, Midstream, Downstream, BOM, and Solenoid FSD elements should always be in the state UNMASKED
  • No changes to the FSD threshold should be made without RC or beamline expert approval

Turning MVT off: do not press "All HV OFF". Instead go to 'Restore settings' from the MVT overview screen:

    1. SafeMode.snp for beam tuning and Moeller runs
    2. MV_HV_FullField.snp for full solenoid field
    3. MV_HV_MidField.snp wgeb solenoid < 4T

CND CAMAC crate switch on / reboot:

If CAMAC crate (camac1) holding CND CFD boards is switched off for any reason, always reset the associated CND CFD thresholds by typing, ssh into clondaq3 and type:

$CODA/src/rol/Linux_x86_64/bin/cnd-cfd-thresh -w 0

If this command is failing and the crate is not responding, first reboot it as follows: roc_reboot camac1

and then repeat the threshold reset command above.

Solutions for common SVT issues: see [9]

RICH recovery procedures: [10]

To be followed in case of:

  1. DAQ crash: rich4 is not responding or
  2. RICH alarms (LV,missing tile, temperature, etc).

If it does not work or you are uncertain about what to do, contact the RICH expert on call. Please note that missing tiles typically occur due to lost communication. Keep in mind that the recovery procedure will kill DAQ. If DAQ is running for purpose other than data taking (for which the RICH acceptance is important) do not initiate the recovery procedure. The most critical parameter for the RICH is the temperature of the photosensors. If the temperature rises above limits, an interlock will automatically turn the RICH HV and LV off. If this happens, notify the expert on call and keep taking data without RICH.

Observed asymmetry in FTCal ADC Scalers

When running at low current (<10 nA), FTC asymmetries can be affected by hot channels. If large asymmetries are detected in those conditions, cross check with other detectors (BOM rates, DC occupancies) before taking further action.

CED display not showing events

Sometimes, CED screens are up but not active (events not showing at all). To solve this:

  1. Close the CED sceen
  2. ssh to clondaq4
  3. Once in the right machine, type ced, and wait for the windows to appear.
  4. Go to "File" -> "Connect to ET ring". Make sure to select clondaq6, as the online data is being streamed.
  5. Go to "Events" (same level as "File") and check "Auto Next-Event Every 2.0 sec" (if it is not selected).

You should see the events now, if all is working well.