Difference between revisions of "Run Group E"
From clas12-run
Jump to navigation
Jump to search
| Line 279: | Line 279: | ||
* In case of '''RICH issues''', check the RICH manual for the current guidance: https://github.com/JeffersonLab/clas12-manuals/blob/master/rich/Novice/RICH_Controls.pdf | * In case of '''RICH issues''', check the RICH manual for the current guidance: https://github.com/JeffersonLab/clas12-manuals/blob/master/rich/Novice/RICH_Controls.pdf | ||
| − | * '''New MVT LV Reboot Procedure''' can be found in the {{#switchtablink:General_Information| | + | * '''New MVT LV Reboot Procedure''' can be found in the {{#switchtablink:General_Information|General Information}} tab. |
* At '''the end of the shift''', submit the '''strip charts''' to the logbook: | * At '''the end of the shift''', submit the '''strip charts''' to the logbook: | ||
Revision as of 00:22, 15 May 2024
[edit]
Daily RC Meetings at 13:00 in CH Room 200C, and on ZOOM: URL Link (Meeting ID: 1605705819), except on Wednesdays, which is @ 15:00 in CH Room 200C and via the same ZOOM link.
- General
- Phone Numbers
- Short Term Schedule
- General Information
- Shift Expert
- Shift Worker
- Run Coordinator
- Monitoring
- Important Links
GeneralLogbookShift ScheduleRG-E Run Coordinator ScheduleESAD, COO, RSADShift ChecklistBeam Time AccountingHot Checkout |
ManualsSystemsSoftwareDetectors |
Procedures
Logbooks
|
|
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
- Note, all non-JLab numbers must be dialed with an area code. When calling from a counting-house landline, dial "9" first.
- To call JLab phones from outside the lab, all 4-digit numbers must be preceded by 757-269
- Click Here to edit Phone Numbers. Note, you then also have to edit the current page to force a refresh.
Run Coordinator: Sebouh Paul (757) 575-7540
Current RP Coordinator: Lamiaa El Fassi
PDL: Nathan Baltzell (757) 876-1789, Office: x5902
Important Notes
- In case of medical or fire emergency, call 911 or 9-911 from a LANDLINE phone in the Counting House and inform the Crew Chief and RC.
- In case of evacuation of the Counting Room (e.g., due to a fire alarm), grab the shift expert phone and leave the room through the "EXIT" door leading to the parking lot. Do not leave through the hallway! Call the RC once outside so the latter can call MCC to turn the beam OFF for Hall B and inform other Hall B staff members.
- If you have symptoms or a positive COVID test, do NOT come to your shift. Instead, inform the PDL, the RC, and JLab medical services immediately to arrange replacements.
- In case of a problem with the HV Slow Controls, EPICS in general, please call the primary expert on-call, Nathan Baltzell, before reaching out to anyone else.
- Please put all essential information on the main Hall B logbook HBLOG. Some important alarms have been documented on HBRUN. Use the pulldown menu to choose where an error is reported.
Current Run Plan: May 18th to May 19th
Outbending Configuration Runs
Previous Dual-target Configuration Plans- DC firmware test plan (LD2+Pb): 1) Take one run each with 1M events with the trigger files and beam current at 70 nA
2) Take one run with 500k events and beam current at 35nA with trigger file rge_inb_v1.0_200MeV_dcrb17_2.trg 3) Take one run with 500k events and beam current at 35nA with trigger file rge_inb_v1.0_200MeV_dcrb17_4.trg
*** Notes ***
LD2+Pb runs
LD2+Sn runs
LD2+Al runs
Lumi Scan for LD2+Al on May 15th
Lumi Scan for LD2+Pb on May 15th - 16th
Streaming Readout Test
Empty-target runs on May 16th - 17th
Outbending Configuration Runs on May 17th - 19th =
|
References- BPM Positions
- Harp Scans- FSD Thresholds
- Charge Asymmetry- Upstream Halo counters
|
General Reminders
|
DAQ Operation
|
This tab contains information on
- Recent Changes
- Current Issues
- Beam Restoration
Recent Changes
New MVT LV Reboot Procedure
See https://logbooks.jlab.org/entry/4217011 for detailed information on rebooting the MVT LV when observing a drop in BMT strip occupancies. This gets done upon request of DAQ or MVT experts only.
- Turn LV OFF from MVT Overview->Low Voltage->MVT-All. LV status indicators should change from green to black;
- Wait 20 seconds, then turn LV back from the same menu.
- An alarm "NActiveCrates" can be created in the alarm handler with a 2-minute delay; the status is also visible on the MVT Overview GUI. This alarm will be cleared 2 minutes after LV is turned back ON.
- Reboot mvt1, mvt2, and mvt3 ROCS.
- Start a run with restart DAQ (Cancel → Reset → Configure → Download → Prestart → Go).
- If we have a beam and a reboot process takes more than 5 minutes, call the DAQ or MVT expert.
RICH issues
- The RICH fibers alarm leads to web guidance, aka logbook entry 3562273 - RICH recovery procedure.
- Check the RICH manual for the current guidance: https://github.com/JeffersonLab/clas12-manuals/blob/master/rich/Novice/RICH_Controls.pdf
- See below for current issues
Rapid/Controlled Access to the Hall
- Ask MCC to stop the beam. Wait for the beacon to stop flashing (15-20 minutes) before access. Rapid access is controlled access without RadCon.
- After a higher beam current or a Moller run, the beacon will not stop flashing anytime soon. Then, escorted access, viz the ARM, is needed. In that case, keep the beam until the ARM arrives at the Counting House.
24/7 ZOOM connection for Counting House
- Click on https://jlab-org.zoomgov.com/j/1618614939, which is included in "Remote Shifts" in the "Shift Worker" tab.
- choose the option to "join from browser" (Chrome is recommended) and ignore all other messages to update the ZOOM APP in the pop-up windows.
- Enter the passcode from the logbook entry in "Shift Worker" tab.
Current Issues
Lights in the Hall
- The main lights in the Hall (dome lights) and the Forward Carriage lights must be OFF because of light leaks from some of the detectors. If these lights are switched on during an access, they should be switched off when leaving the Hall.
- Refer to these photos: ON: [1], OFF: [2]
- We have a new dome light switch outside the hall in the labyrinth. See logbook entry: https://logbooks.jlab.org/entry/4204460
- When the dome lights are switched off, they cannot be turned back on immediately; they require 10-15 min to cool down.
Beamline
- Orbit locks are on BPMs 2C24A and 2H01, which should generally be on ("Running").
- - Beam positions at 2C24A and 2H01 must be kept within ±0.1 mm at the nominal beam positions listed under "References" at the Short Term Schedule tab.
- The beam position at 2C21A is not important for production runs but should be stable. Here, significantly drifting positions can indicate instabilities in beam energy.
- The beam position at 2C21A is important for initial beam tuning (on tagger dump).
- After the HWP change
- • The absolute values of the SLM and FCUP charge asymmetry (Q-Asym) should be less than 0.1%. If they are not, contact the Beamline Expert.
- • Start a new run
- Tagger Magnet Procedure: If the tagger magnet needs to be turned off, tell the operator to ONLY use the script for degaussing, NEVER do it "by hand"! The latter method might/will trip the circuit breaker for the power supply in the Hall.
BMT / SVT
- If new hot channels appear, call the SVT/MM expert Yuri Gotra (757-541-7539). These channels need to be addressed by the expert.
- BMT HV trips: no screenshots are necessary, power-cycle the channel HV as fast as possible (the underlying script will take care of ramping it up), note the information in the shift summary. If the same channel trips frequently, ask the expert on call for guidance.
- BMT HV S1 L2 STRIP in safe mode (low voltage) by expert - appears blue in MVT overview
- BMT HV S1 L5 STRIP and DRIFT are OFF (Vset= 0 V) - appears blue in MVT overview
- BMT HV S2 L6 STRIP and DRIFT are OFF (Vset= 0 V) - appears blue in MVT overview
- As of 05/10-12/24: BMT HV S2 [L4, L1] & S3 L5 strip frequent trips: The HV, Vset, was lowered by 10 V until it calmed down. If the problem persists, reduce the HV VSet value by an increment of 10 V after clicking the "Expert" button. Then, make a logbook entry and CC MVT/SVT/MM experts on it.
- If FMT gas errors appear, please follow the instructions in [3]
CND
- CAMAC crate alarm:
- • CAMAC crate (CND Constant Fraction Discriminators) needs rebooting, which can be done via the alarm handler.
- • After reboot, the CFD thresholds need to be set by running a script; see page 12 in the CND manual at the "General" tab: https://clasweb.jlab.org/wiki/images/9/97/Ops_manual_cnd.pdf
DAQ
- Some ROCs keep printing error messages while functioning properly; please see their list in this logbook entry # 4215299 to avoid unnecessary reboots since the printout in the ROC terminal won't change. The message will appear in the DAQ run control if they get disconnected!
- NEW STANDARD end and start of the run: End run, Cancel, Reset, Download, Prestart, Go. This will ensure proper compression of the FADCs and, therefore, a smaller data rate.
- ROC reboot instructions:
- • To reboot, i.e., adcecal1, or any noted ROC that became busy, type on a clasrun terminal: clasrun> roc_reboot adcecal1
- • To check that the crate is back alive, type:clasrun> ping adcecal1
- • The ping will start to answer as soon as the crate communicates again, then press Ctrl+C to exit ping.
DC
- Make sure to log every DC trip in the shift summary. Make sure to list WHERE and why, possibly, the trip occurred.
ECAL/PCAL
- "ECAL_SEC6_UO26 Noisy" https://logbooks.jlab.org/entry/4241957
- "PCAL_HV_SEC2_U25 turned off" https://logbooks.jlab.org/entry/4247187, https://logbooks.jlab.org/entry/4247400
FTOF
- FTOF S2 Paddle 19-left and S4 Paddle 16-left are known to be "weak" channels that tend to come and go depending on torus polarity and rates. This cannot be fixed during this run period; see https://logbooks.jlab.org/entry/4219555 and https://logbooks.jlab.org/entry/4219876.
- "FTOF HV Adjustment - S5 P2 2R" https://logbooks.jlab.org/entry/4249754
RICH
- Known problematic PMTs and tile:
- - You don't need to report about it.
- - PMT 2 (sector 1) and PMT 92 (sector 4) are not completely dead and may be visible depending on statistics.
- - Problematic tiles and PMTs:
- • Sector 1 Tile 21 dead (PMT 55,56,57)
- • Sector 1 Tile 1 PMT 2
- • Sector 4 Tile 34 PMT 92
Beam Restoration
- Study the manual: Establishing Physics-Quality Beam. If you do not feel confident doing this alone, contact the Beamline Expert.
- Find all Reference values on the CH whiteboard and the Short Term Schedule tab.
Restoring Beam after Short Downtime
- Ask for 5 nA beam current and check the 2C24A and 2H01 beam (x, y) positions against the Reference Values (tab "Short Term Schedule") after
- • beam steering or orbit change
- • change in beam current, e.g., after a target change
- • If BPM 2C24A and 2H01 positions are off (not within +/-0.1 mm), ask MCC to bring them to the reference values
Beam Tuning to Tagger Yoke Dump
- Study the manual: Establishing Physics-Quality Beam.
- Contact Eugene Pasyuk if issues arise.
- Ask MCC to energize the tagger magnet
- • blank collimator
- • beam type "Photon"
- All CLAS12 detectors HV off (BMT HV in safe mode)
- Ask MCC to mask the beam halo counters (FSDs: Upstream, Midstream, Downstream, BOM)
- Ask MCC for 5 nA beam current and make sure the Orbit Locks are OFF;
- If necessary, ask MCC to bring the 2C21A and 2C24A beam positions to the Reference Values
- Take harp scans:
- • at Harp 2C21A (Upstream Right Counter)
- • at Tagger Harp 2C24A Upstream Right Counter)
- If the beam trips, abort the scan and restart. The beam must be on during the scan, otherwise the fit to the beam profile will not work.
- Compare the results with the Harp Scan References
- The fit ranges might need to be adjusted manually to assess the fit results
- If good, log the new scan results.
Møller run
- Follow the standard procedure as described in the manual Performing Møller Runs.
- Contact Nathan Baltzell if issues arise
- A few points to note:
- • Request 20 nA , Orbit Locks should be off
- • The BPMs 2C21 and 2C24A may be out of range after the Møller configuration.
- • Keep running until the polarization uncertainty is < 1.5%
Beam Tuning to Faraday Cup
- Study the manual Establishing Physics-Quality Beam and consider the first steps mentioned therein.
- This is typically done after tuning to the tagger dump (and possibly taking a Moller run).
- All CLAS12 detectors HV OFF (MVT HV ON in SAFE Mode)
- Ask MCC to de-energize the tagger magnet
- • 20 mm collimator
- • beam type electron
- • beam blocker out
- If initial beam tune, the target should be "Empty"
- Ask MCC to unmask beam halos
- Set the halo counter FSD thresholds to 1 MHz and the integration time interval to 50 milliseconds
- Insert the Downstream Viewer, Chromax, in the beamline
- Ask MCC for 5 nA beam current
- If necessary, ask MCC to bring the beam positions at 2C24A and 2H01 to the Reference Values
- Take a harp scan at 2H01 (Midstream Left)
- If the beam trips, abort and restart. The beam must be on during the scan, otherwise the fit to the beam profile will not work.
- Compare the results with the Harp Scan References
- If good, log the new scan results.
- Once done and before resuming the production data-taking:
- • Retract the Downstream Viewer from the beamline
- • Put the FSDs thresholds and integration time to their reference values
- • Turn all detector HV ON, for MVT "HV ON - Full Voltage"
- • MCC might need to adjust the beam positions after going to the production beam current.
- Arrive at your shift in time for the shift change, 10-15 minutes before the start of your shift.
- • If it is your first shift or a shift after a long pause, do shadow an earlier shift for at least 4 hours to familiarize yourself with all shift duties and technicalities of the ongoing experiment.
- Follow the Short Term Schedule;
- • The RC must be notified before changing run conditions.
- Operate the Data Acquisition.
- Take action on and log alarms.
- Communicate with your co-shift person:
- • Remind them to read the tab 'Shift Worker'.
- • The shift worker should discuss an issue with the shift expert before making extra logbook entries.
- • During production, not in extended downtime, the shift expert should ensure the shift worker is filling their end-of-shift Checklist when the beam is ON, not in the period of beam trips.
- • Use the CS-Studio Logbook entry tool for all screenshots (but the strip charts)!
- • Note that the Logbook Entry tool defaults the entry in the HBRUN logbook. Other logbooks can be added or selected from the drop-down menu.
- Check for and read any comments made to log-book entries during your shift.
- Keep your shift summary up to date in HBLOG. Record all shift activities, issues, if any, and related communications with experts with details and/or reference to any associated logbook entries:
- - Refer to relevant logbook entries in the shift summary.
- - Include a list of runs taken during the shift with relevant run information;
- • See, for instance, this shift summary where all relevant run info, links, and shift activities are cited.
- Ensure that all MON12 monitoring plots are logged and checked against the reference plots for every single run; see the links under Monitoring tab;
- • Both, you and the shift worker should run MON12 and reset & check spectra frequently.
- If the worker shift is remote, maintain communication over Zoom.
- If you have any other questions or uncertainties about what to do, call the RC.
- In case of RICH issues, check the RICH manual for the current guidance: https://github.com/JeffersonLab/clas12-manuals/blob/master/rich/Novice/RICH_Controls.pdf
- New MVT LV Reboot Procedure can be found in the General Information tab.
- At the end of the shift, submit the strip charts to the logbook:
- - If they are not up already: (CS-Studio → Tools → Strip Charts → livePlot)
- - Right-click within the grid, choose "Load a Configuration", select "Beamline4Logbook_visibleHaloCounters"
- - Choose a time-span width of 10 hours (right-click below x-axis)
- - To make a logbook entry:
- • Right-click within the grid of one of the charts, choose "Make Logbook Entry"
- • The title of the logbook entry can be changed, eg to Strip charts
- • Un-click ELOG, select HBLOG and click "Make Entry" to submit.
- Fill out BTA hourly;
- Accelerator beam time (do not change these values):
- • ABU: acceptable beam used
- • BANU: beam acceptable but not used (should be reflected in our PCC and UED)
- • BNA: beam not acceptable (e.g., tuning, drifting, not focused)
- • ACC: Accelerator Configuration Change (e.g., pass change)
- • OFF: Accelerator is down
- Experiment beam time (do change these values):
- • ER: Experiment ready (should be 60 minutes - PCC - UED)
- • PCC: Planned configuration change (when the beam is masked while radiators are moved, target change, DAQ (stopping/starting runs, tests, etc.)
- • UED: Unplanned experiment down (hardware failure, DAQ crashes, our problems, ...)
- • Exception: BANU, due to time spent on beam tuning and BPM checks after beam down, should be counted as ER on our side.
- Enter fields at the "Shift Information" tab.
- To send information to MCC, you must save at 7:00, 15:00, and 23:00. To see the save button, click 'edit all' or 'Edit CC Hours.'
- Acknowledge the popup error message about the last hour not being completed by clicking OK.
- Sign, viz log completed 8-hour sheet.
- Note: we can change the time sheet up to 48 hours after completion.
- Come to your shift in time for the shift change, 10-15 minutes before the start of your shift;
- • If it is your first shift or a shift after a long pause, do shadow an earlier shift for at least 4 hours to familiarize yourself with all shift duties and technicalities of the ongoing experiment.
- Discuss any issue with the shift expert before making extra logbook entries; ask the shift expert to note the ongoing issue in the shift summary.
- Use the CS-Studio Logbook entry tool for all screen shots, except the strip charts!
Frequently
- Reset and check MON12 histograms frequently, preferably each 1 h, during a run;
- • Note that some plots have radio buttons at the bottom to choose the sector. Remember to cycle through and check all sectors.
Once per run
- Note that the CS-Studio Logbook Entry tool puts the entry in the HBRUN logbook by default. Other logbooks can be added or selected from the drop-down menu.
- Using the Logbook Entry tool, submit EPICS screenshots to the HBRUN logbook (one logbook entry) with beam ON:
- Beamline overview (remote shift: in VNC or Web Browser)
- DAQ status
- CLAS12 trigger rates
- Upload mon12 monitoring plots to the logbook with sufficient statistics and compare them to the reference and recent uploads (see Monitoring tab).
- At the end of the run:
- - Submit the strip charts to the logbook:
- • If they are not up already: (CS-Studio → Tools → Strip Charts → livePlot)
- • Right-click within the grid, choose "Load a Configuration", select "Beamline4Logbook_visibleHaloCounters"
- • Choose a time-span width of 4 hours, compatible with the run duration (right-click below x-axis)
- • To make a logbook entry:
- • Right-click within the grid of one of the charts, choose "Make Logbook Entry"
- • The title of the logbook entry can be changed, eg to Strip charts
- • Un-click ELOG, select HBLOG, and click on "Make Entry" to submit
- - Upload the clas12online plots to the logbook.
- • The program can continue through a standard end/start of a run. However, after a DAQ restart, you must back out and restart clas12online.
- • If the DAQ has to be restarted, you can still make the upload before backing out.
Once per shift
- Fill the shift checklist;
- • During production, get relevant values from the epics screens Beamline Overview, CLAS12 DAQ Status and CLAS12 Trigger;
- • During extended downtime, get relevant values from the latest EPICS screenshots that you made during production
For Remote Shifts
- Please inform the PDL that your shifts will be remote. . Remote shifts are listed here
- You must test your setup, including ZOOM and VNC connections, at least 24 hours before your first shift!
- ♦ ZOOM for 24-7 communication with Counting House:
- • In a Web-browser: https://jlab-org.zoomgov.com/j/1618614939; Or in the ZOOM APP: Enter the meeting #: 1618614939, then
- - The required passcode can be obtained from this log entry
- • In a Web-browser: https://jlab-org.zoomgov.com/j/1618614939; Or in the ZOOM APP: Enter the meeting #: 1618614939, then
- Remind the expert in the counting house to restart their ZOOM, if necessary
- Remote VNC connection instructions: https://logbooks.jlab.org/entry/3906039:
- • If you mistyped (username, pincode, passcode,...) a string of numbers will appear with text asking you to respond to this challenge. Back out with Ctrl_c, and you get another try! After N+1 mistypes, you will be locked out. For several hours after that, you will not even be able to get to the public JLab web pages.
- • All necessary applications should be available from the desktop menu (left/right click on the desktop)
- • EPICS GUIs should be accessed ONLY via the desktop menu or by running clascss-remote.
- • Some charts are also accessible via web browser: Upstream/Midstream Halo Counters, Downstream Halo Counters, Beam Currents, BPM Positions
- Check the upcoming shifts, and if necessary, coordinate with the PDL to contact shift takers about their training and site access status.
- Remote shifts are listed here.
- Backup shift personnel is listed here.
- Meeting ZOOM connections: see Important Links or Short Term Schedule tab
Daily
- Report at the PD/RC and attend the MCC/OPS meetings (weekdays 7:45 AM and 8 AM Daily, MCC building). The PD will communicate when and how meetings happen on the weekend.
- Update the run plan in the Short Term Schedule tab, as well as the whiteboard in CH, in coordination with the RG-E experiment Run Coordinators and communicate it to the shift crew.
- Log the RC Daily summary in the HBLOG.
- BTA time sheet: Make sure the "Beam Available Not Used" (BANU) time is accounted for as explained at the Shift Expert tab. The BTA information can be modified post-signature.
- • NOTE: "After 48 hours, you cannot modify a User signed timesheet unless you are the signer, Hall Manager, or Admin".
- Run the daily RC meetings in CH Room 200C and on ZOOM
- • Mo-Tu,Th-Fr at 13:00 ET in the Counting House Conference Room
- • Wednesdays at 3 PM, ZOOM only
- • Weekends upon agreement
- • Send a daily invitation for the RC meeting to mailing lists hallb, clas12_rge, clas_members.
- Connect to Zoom in CH Room 200C:
- • Turn on the projector with the white remote control, then log into your CUE account. If not listed yet, click "Not listed?" in the bottom left to add your account.
- • Once logged in, use the recommended Google Chrome to join the ZOOM meeting on the browser or use the ZOOM application.
- • Since the room doesn't have audio, don't connect with the computer audio, but Call the 1-833-xxx-xxxx number shown in the ZOOM session on the small Cisco answering device. Once prompted, enter the meeting ID followed by the participant ID displayed in the ZOOM session to join the meeting.
- • The room has a camera, which can be turned on via ZOOM.
Weekly
- MCC/OPS planning meeting, on Wednesdays at 13:30, MCC building.
- • Submit (save) the online form Run Coordinator Weekly Summary before the meeting and present the contents at the meeting.
- • The summary covers the data-taking status from 7 a.m. on the previous Wednesday to 7 a.m. on the current Wednesday.
- • Beam time accounting: Beam Time Manager; Physics Time Accounting
- • The total experiment data collected to date could be retrieved from the up-to-date Accumulated "Charge" or "Luminosity" depicted on the "RG-E 2024 Progress" plot. See the Monitoring tab
- Hall B meeting is on Mondays at 9 AM. Present your summary slides, which should be sent beforehand to the Hall B leader, Patrick Achenbach.
Webcams
|
EPICS |
Data Quality
|
Hall-B |
Accelerator |
ZOOM Meetings
|
|
|
