Difference between revisions of "Run Group K"
Jump to navigation
Jump to search
(Removed daily RICH recovery) |
|||
| Line 151: | Line 151: | ||
** There can be '''issues''' with missing tiles or the occupancy as seen in MON12. Put screenshots of scalers and temperatures or the occupancy spectra in the '''RICH logbook'''. Do not stop an ongoing run, but do the '''fast recovery between runs'''. | ** There can be '''issues''' with missing tiles or the occupancy as seen in MON12. Put screenshots of scalers and temperatures or the occupancy spectra in the '''RICH logbook'''. Do not stop an ongoing run, but do the '''fast recovery between runs'''. | ||
** If the issue is seen in the scaler but not in the temperature map of the tiles, then you should do a full recovery; see https://logbooks.jlab.org/entry/4214184. This might preempt the daily full recovery. | ** If the issue is seen in the scaler but not in the temperature map of the tiles, then you should do a full recovery; see https://logbooks.jlab.org/entry/4214184. This might preempt the daily full recovery. | ||
| − | ** '''Daily full RICH recovery''' | + | ** '''Daily full RICH recovery is no longer necessary.''' The new RICH firmware performs an automatic recovery at the beginning of each run. This might change so pay attention for updates. |
** '''Avoid unnecessary recoveries''', see https://logbooks.jlab.org/entry/4208477. The RICH overview GUI shows how many seconds have passed since the last full recovery. Also, there is a minor alarm when 24 hours have passed since the last full recovery. Acknowledge the alarm and wait for the end of the run before running the recovery. | ** '''Avoid unnecessary recoveries''', see https://logbooks.jlab.org/entry/4208477. The RICH overview GUI shows how many seconds have passed since the last full recovery. Also, there is a minor alarm when 24 hours have passed since the last full recovery. Acknowledge the alarm and wait for the end of the run before running the recovery. | ||
** Note that you have to restart the DAQ (cancel → reset → configure → download → Prestart → Go); see DAQ Operation under {{#switchtablink:Short_Term_Schedule|Short Term Schedule}}, after any RICH recovery. | ** Note that you have to restart the DAQ (cancel → reset → configure → download → Prestart → Go); see DAQ Operation under {{#switchtablink:Short_Term_Schedule|Short Term Schedule}}, after any RICH recovery. | ||
Revision as of 12:49, 13 January 2024
[edit]
- General
- Phone Numbers
- Short Term Schedule
- Run Coordinator
- Shift Expert
- Shift Worker
- Monitoring
- Important Links
GeneralLogbookShift ScheduleRun Coordinator ScheduleESAD, COO, RSADShift ChecklistBeam Time AccountingHot Checkout |
ManualsSystemsDetectorsSoftware |
Procedures
Logbooks
|
|
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
- Note, all non-JLab numbers must be dialed with an area code. When calling from a counting-house landline, dial "9" first.
- To call JLab phones from outside the lab, all 4-digit numbers must be preceded by 757-269
- Click Here to edit Phone Numbers. Note, you then also have to edit the current page to force a refresh.
Click to Edit solely this tab. P.S.: You must click on the top Edit, next to Read, followed by "Save changes" to force the current page to refresh.
Experiment Coordinator: Annalisa D'Angelo
Run Coordinator: Daniel Carman (757) 575-7540
PDL: Daniel Carman (757) 876-1789
Important Notes
Do NOT change FSD settings without approval from RC or beamline expert!
In case of medical or fire emergency, call 9-911 from a LANDLINE phone in the Counting House and inform the Crew Chief and RC.
In case of evacuation of the Counting Room (e.g., due to a fire alarm), grab the shift expert phone and leave the room through the "EXIT" door leading to the parking lot. Do not leave through the hallway! Call the RC once outside so the latter will call MCC to turn the beam OFF to Hall B and inform other Hall B staff members.
If you have symptoms or a positive COVID test, do NOT come to your shift - instead, inform the PDL and the RC and JLab medical services immediately to arrange replacements.
Remote shift: If needed, worker shifts can be taken from off-site. Inform the PDL and RC. Also, make sure that you follow the instructions under the "Shift Worker" tab, as you need some lead time to set up!
Note to Shift Takers:
- Read the ESAD, COO, RSAD, and ERG documents found on the "Current RG-K Run Info" as well as the yellow folder in the counting room. Before you start your first shift on the current experiment, you must sign on the list in the yellow folder. Thereby, you confirm that you have read and understand their content.
- Read at least the previous 24 hours logbook entries to be aware of recent run details and changes
- Arrive at least 10 minutes early for shift change
- If it is your first shift or a shift after a long pause, do shadow an earlier shift for at least 4 hours to familiarize yourself with all shift duties and technicalities of the ongoing experiment.
- Refresh the run wiki page/tabs; look over it/them every shift
- Follow the Shift Expert/Worker tab's instructions
- Do not spend more than 15-20 min trying to fix a problem. In case of any issue, contact the On-call Experts or the RC.
See tab GENERAL INFORMATION for
- Recent Changes
- Current Issues
- Beam Restoration (for beam restoration procedures and Møller runs)
Run Plan: Mar. 10 - Mar. 11, 2024
Production Running ... to the end!!
- DAQ configuration: PROD67, Trigger: rgk_noDC_v1.1_300MeV.trg - Beam current request: 75 nA - Run duration: 3 hrs
Note: RG-K run ends on Mar. 11 at 7 a.m.
- Turn all detectors off and put BMT HV in safe mode - Ramp torus and solenoid to zero field (contact Eugene to do this) - Call to put Hall B in Restricted Access
Related Information
Find detailed information at tab "GENERAL INFORMATION".
- The main lights in the Hall (dome lights) and the Forward Carriage lights must be OFF during run time.
- Orbit locks are on BPMs 2C24A and 2H01 and the locks should generally be on ("Running").
- Beam positions at 2C24A and 2H01 have to be kept, within ±0.1mm, at the nominal beam positions listed below ("Reference Values").
- The beam position at 2C21A is not important for production runs but should be stable. Here, significantly drifting positions can indicate instabilities in beam energy.
- The beam position at 2C21A is important for initial beam tuning (on tagger dump).
- If HWP is changed, start a new run. The absolute values of the SLM and FCUP charge asymmetry (Q-Asym (%)) should be less than 0.1%.
- Beam tuning and Møller runs
- Generally, Detector HV OFF but
- SVT: LV ON; HV OFF, and MVT: LV ON; HV ON - Safe Voltage
- Turn DC HV off only for beam tuning
Access to the Hall
- If you need to enter Hall B, alert Denny Insley (757-897-9060, cell/text: 757-810-9576) and the Run Coordinator (757-575-7540).
- Check in with the shift personnel before entering the hall.
DAQ Operation & Related Information
|
NEW STANDARD end and start of the run:
This will ensure proper compression of the FADCs and therefore a smaller data rate and perform a RICH recovery. Restart DAQ
|
|
Triggers files
| rgk_noDC_v1.0_300MeV.trg | DC roads are not in the trigger. Hold off time is 3 us. |
| rgk_noDC_v1.1_300MeV.trg | DC roads are not in the trigger. Hold off time is 2 us. |
| rgk_noDC_v1.2_300MeV.trg | DC roads are not in the trigger. Hold off time is 1 us. |
| rgk_out_v1.1_300MeV.trg | Outbending DC roads, Hold off time is 2 us. |
Reference Values
FSD Thresholds
- Upstream: 2 kHz
- Midstream: 2 kHz
- Downstream: 100 kHz
- BOM: 50 kHz
- 5 ms dwell time
Nominal Beam Positions
2C21A (not important for production)
- x = -0.4 mm
- y = +0.7 mm
2C24A
- x = -1.7 mm
- y = +0.5 mm
2H01
- x = -0.7 mm
- y = +0.6 mm
8.5 GeV Harp Scan References (Feb. 14, 2024)
2C21A: https://logbooks.jlab.org/entry/4252495
Tagger (2C24A): https://logbooks.jlab.org/entry/4252496
2H01: https://logbooks.jlab.org/entry/4252514
6.4 GeV Harp Scan References (Jan. 18, 2024)
2C21A: https://logbooks.jlab.org/entry/4240375
Tagger (2C24A): https://logbooks.jlab.org/entry/4240378
Responsibilities
- The RC duties are detailed in the COO.
- Chek the upcoming shifts, and if necessary, coordinate with the PDL to contact shifters about their training and site access status.
- Remote shifts are listed here.
- Backup shift personnel is listed here.
Daily
- Attend the in-person PD/RC and MCC/OPS meetings (7:45 AM and 8 AM Daily). The PD will communicate when and how meetings happen on the weekend.
- See the ZOOM connection info under Important Links tab.
- Update the run plan in the Short Term Schedule tab in coordination with the RG-K experiment coordinators (i.e., Annalisa, Daniel) and communicate it to the shift crew.
- Log the RC summary in the HBLOG.
- Update the list of runs taken on the Google Spreadsheet
- Send a daily invitation for the RC meeting to hallb and clas12_rgk mailing lists.
- Run the RC meetings at 13:00 in CH Room 200C and on ZOOM, except on Wednesdays.
- On Wednesdays, the RC meeting is held on Zoom only at 15:00, due to the conflict with the 13:30 MCC meeting and the unavailability of the CH room afterwards.
- To connect to Zoom in CH Room 200C, follow these steps:
- Turn on the projector with the white remote control, then log into your CUE account. If not listed yet, click "Not listed?" in the bottom left to add your account.
- Once logged in, use the recommended Google Chrome to join the ZOOM meeting on the browser or use the ZOOM application.
- Since the room doesn't have audio, don't connect with the computer audio, but choose "Call Me," then enter the number in the small Cisco answering device. Once it rings, answer the call, and you will be prompted to join the meeting.
- The room has a camera, which can be turned on via ZOOM.
Weekly
- Attend the in-person MCC/OPS planning meeting on Wednesdays at 13:30 PM and report the run status via the submitted Run Coordinator Weekly Summary.
- The report summarizes the data taking status (available beam used (ABU), any beam losses (BANU), CW on target (ABU+BANU), etc.) from 7 AM on the previous Wednesday to 7 AM on the current Wednesday, in addition to any standing and critical issues, plans for upcoming week, and general comments/requests.
- The beam time accounting can be retrieved from:
- OPS logbook entries with tag 'Beam Accounting': Timesheet Summaries
- Beam Time Manager; Physics Time Accounting
- To calculate the percentage of scheduled experiment hours to date contact Daniel Carman.
- The total experiment data collected to date could be retrieved from the up-to-date "Accumulated Charge" depicted at [1]
- Attend the Hall B meeting on Mondays at 9 AM and present your summary slides that should be sent beforehand to the Hall B leader, Patrick Achenbach.
Tasks
- Come to your shift in time for the shift change, 10-15 minutes before the start of your shift.
- If it is your first shift or a shift after a long pause, please shadow an earlier shift for at least 4 hours to familiarize yourself with all shift duties and technicalities of the ongoing experiment.
- Follow the Short Term Schedule
- The RC must be notified before changing run conditions
- Operate the Data Acquisition
- Take action on and log alarms
- Communicate with your co-shift person:
- Remind them to read the tab 'Shift Worker'.
- Discuss any issues with the shift worker before making extra logbook entries.
- Remind the shift worker of the end-of-shift Checklist.
- Check for and read any comments to log-book entries made during your shift.
- Keep your shift summary up to date in HBLOG. Record all significant happenings.
- Refer to relevant logbook entries in the shift summary.
- Include a run list for the shift with relevant run information.
- Ensure that all monitoring plots are logged and checked against the reference plots for every single run.
- Both, you and the shift worker should run MON12 and reset & check spectra frequently.
- If the worker shift is remote, maintain communication over Zoom.
- If you have any other questions or uncertainties about what to do, call the RC.
- New RICH recovery protocol (This protocol is being investigated - stay tuned for updates - contact RICH expert for questions)
- See https://logbooks.jlab.org/entry/4201728 for important details.
- There can be issues with missing tiles or the occupancy as seen in MON12. Put screenshots of scalers and temperatures or the occupancy spectra in the RICH logbook. Do not stop an ongoing run, but do the fast recovery between runs.
- If the issue is seen in the scaler but not in the temperature map of the tiles, then you should do a full recovery; see https://logbooks.jlab.org/entry/4214184. This might preempt the daily full recovery.
- Daily full RICH recovery is no longer necessary. The new RICH firmware performs an automatic recovery at the beginning of each run. This might change so pay attention for updates.
- Avoid unnecessary recoveries, see https://logbooks.jlab.org/entry/4208477. The RICH overview GUI shows how many seconds have passed since the last full recovery. Also, there is a minor alarm when 24 hours have passed since the last full recovery. Acknowledge the alarm and wait for the end of the run before running the recovery.
- Note that you have to restart the DAQ (cancel → reset → configure → download → Prestart → Go); see DAQ Operation under Short Term Schedule, after any RICH recovery.
- If the beam is ON, and any RICH recovery takes more than 5 min, please call the RICH Expert.
- New MVT LV Reboot Procedure
- See https://logbooks.jlab.org/entry/4217011 for detailed information on rebooting the MVT LV when observing a drop in BMT strip occupancies. In brief, it requires a reboot of LV as follows:
- Turn LV OFF from MVT Overview->Low Voltage->MVT-All. LV status indicators should change from green to black;
- Wait 20 seconds, then turn LV back from the same menu;
- In the alarm handler, with a 2-minute alarm delay, there can be alarm NActiveCrates, status also reported on the MVT Overview GUI. This alarm will be cleared 2 min after LV is turned back ON;
- At this point, mvt1, mvt2, and mvt3 ROCS can be rebooted;
- Start a run (Cancel → Reset → Configure → Download → Prestart → Go).
- If the beam is ON, and a reboot process takes more than 5 min, please call the DAQ and/or SVT/MVT/MM expert.
- Fill out BTA hourly;
- Accelerator beam time (do not change these values):
- ABU: acceptable beam used
- BANU: beam acceptable but not used (should be reflected in our PCC and UED)
- BNA: beam not acceptable (e.g., tuning, drifting, not focused)
- ACC: Accelerator Configuration Change (e.g., pass change)
- OFF: Accelerator is down
- Experiment beam time (do change these values):
- ER: Experiment ready (should be 60 minutes - PCC - UED)
- PCC: Planned configuration change (when the beam is masked while radiators are moved, target change, DAQ (stopping/starting runs, tests, etc.)
- UED: Unplanned experiment down (hardware failure, DAQ crashes, our problems, ...)
- Exception: BANU, due to time spent on beam tuning and BPM checks after beam down, should be counted as ER on our side.
- Enter fields at the "Shift Information" tab.
- You must save at 7:00, 15:00, and 23:00, to send info to MCC. To see the save button, click 'edit all' or 'edit cc hours.' You can check, https://ace.jlab.org/btm/reports/activity-audit, whether you saved successfully. Not sure this is true?
- Acknowledge the popup error message about the last hour not being completed by clicking OK.
- Sign, viz log completed 8-hour sheet.
- Note: we can change the time sheet up to 48 hours after completion. Still true?
- Accelerator beam time (do not change these values):
Tasks
- Come to your shift in time for the shift change, 10-15 minutes before the start of your shift;
- If it is your first shift or a shift after a long pause, please shadow a previous shift for at least 4 hours to familiarize yourself with all shift duties and technicalities of the ongoing experiment.
- Discuss any issue with the shift expert before making extra logbook entries; ask the shift expert to note the ongoing issue in the shift summary.
Frequently
- Reset and check MON12 histograms frequently, preferably each 1/2h
- Note that some plots have radio buttons at the bottom to choose the sector. Remember to cycle through and check all sectors.
Once per run
- Submit EPICS screenshots to the HBRUN logbook with beam ON
- Beamline overview, in VNC or Web Browser
- CLAS12 trigger rates and DAQ status
- Upload MON12 monitoring plots to the logbook with sufficient statistics and compare them to the reference and recent uploads (see Monitoring tab).
- Submit Beam current and position strip charts to the logbook at the end of the run
- If they are not up already: (CS-Studio → Beamline → Strip Charts)
- Choose the 'time-span width' to be compatible with the run duration (right-click below x-axis).
- To make the logbook entry:
- - In the blank area within the dotted lines in any chart, right-click, choose "Make Logbook Entry"
- - The title of the logbook entry can be changed
- - Un-click ELOG, select HBLOG, and click on "Make Entry" to submit
Once per shift
- Fill the shift checklist;
- During production, get relevant values from the epics screens Beamline Overview, CLAS12 DAQ Status and CLAS12 Trigger;
- During extended downtime, get relevant values from the latest EPICS screenshots that you made during production
- Upload by the end of the shift while data is being collected clas12online plots to the HBLOG logbook, with sufficient statistics
- Please refresh the clas12online program at the beginning of your shift in case the previous shift does not do it after uploading the snapshots. Also, do that while the DAQ is running!
- Note that the program must also be restarted after each DAQ crash!
For Remote Shifts
Please inform the PDL that your shifts will be remote. Remote shifts are listed here
You must test your setup, including Zoom and VNC connections, at least 24 hours before your first shift!
- Zoom for 24-7 communication with counting house:
- In the Zoom app: https://jlab-org.zoomgov.com/j/1618614939
- Or in a web browser: https://jlab-org.zoomgov.com/wc/1618614939/join?pwd=U003MnRuNVdiU1EzaFpXV0JuNWV1Zz09
- The passcode is required and contained in this log entry
- Remind the expert in the counting house to restart their ZOOM, if necessary
- Remote VNC connection instructions: https://logbooks.jlab.org/entry/3906039
- If you mistyped (username, pincode, passcode,...) a string of numbers will appear with text asking you to respond to this challenge. Back out with ctrl_c and you get another try! After N+1 mistypes you will be locked out. For several hours after that you will not even be able to get to the public JLab webpages.
- All necessary applications should be available from the desktop menu (left/right click on the desktop)
- EPICS GUIs should be accessed ONLY via the desktop menu or by running clascss-remote.
- Some charts are also accessible via web browser: Upstream/Midstream Halo Counters, Downstream Halo Counters, Beam Currents, BPM Positions
Webcams
|
EPICS |
Data Quality
|
Hall-B |
Accelerator |
ZOOM Meetings
|
|
|
