NIRC2

Introduction

The plan is to fully power down the NIRC2 before the ATS power shutdown. NIRC2 is scheduled to be on sky on Friday, November 7, so a speedy recovery is critical. We want to minimize the amount of time the motors are powered down because homing some of the stages (CAM, GRS, SLM, and SLS) can be challenging, but it has been successfully done in the past.

NIRC2 CCR system has two cold heads:

• A single stage head, aka CH1, to keep the temperature of the external radiation shield at ~150K.

• A two stage head, aka CH2, whose first stage is used to cool the optical bench to 50K and the second stage to cool the detector to 29K.

Both cold heads work at approximately half of their maximum operational speed (see https://www2.keck.hawaii.edu/inst/nirc2/technical/cnotes.html). Temperatures are monitored with a Lakeshore 218 and controlled with a Lakeshore 340.

Additionally, NIRC2 has an autonomous emergency vacuum system consisting of an automatic controller, and electrically operated solenoid valve on which a turbo pump is mounted, an oil-less roughing pump connected to the turbo pump, two full range Pfeiffer PKR 251 vacuum gages, and the Pfeiffer DCU 100 turbo-pump controller.

The emergency vacuum system will be triggered if the temperature of the first stage of CH2 reaches 55K (see https://www2.keck.hawaii.edu/inst/nirc2/technical/alarm55.html for details). The turbo pump will turn on and the solenoid valve will open after a few minutes, once the turbo pump is up to speed.

Based on previous experience, it takes about 2 hours since the CCR system is turned off for the first stage of CH2 to reach 55K. We are planning to lower the temperature of the optical bench and radiation shield a few days before the ATS power shutdown to gain a few hours before the CH2 first stage temperature reaches 55K.

Procedure Before power down

Mon, Nov 4: Increase the CH1 and CH2 speed settings and decrease CH2 first stage control temperature.

• Current cold head speed settings are:

  • waikoko-new{nirc1}81: gshow -s nirc2 ch1spd ch2spd
CH1SPD = 333.330 mvolts
CH2SPD = 250.000 mvolts

• Current cold head speed read backs are:

  • waikoko-new{nirc1}82: gshow -s nirc2 ch1spdrbv ch2spdrbv
CH1SPDRBV = 985.000 mvolts
CH2SPDRBV = 932.000 mvolts

• These are translated into RPM as follows, where CHSPD range is 100 to 3500 mV and CHSPDRBV range is 0 to 5000 mV:

  • Cold head speed setting in RPM: R [RPM] = 24 + 0.012 x CHSPD [mV]

    • CH1 = 28 RPM

    • CH2 = 27 RPM

  • Cold head back speed in RPM: R [RPM] = 0.024 x CHSPDRBV [mV]

    • CH1 = 23 RPM

    • CH2 = 22 RPM

• We will start by setting the both cold head speeds to 75% of their maximum operation speed and adjust depending on behavior:

  • modify -s nirc2 ch1spd=2400 ch2spd=2400

• Current temperature values are:

  • waikoko-new{nirc1}83: showtemps
tdetblck = 28.875 degK
tgetter = 4.678 degK
thead2lo = 11.149 degK
tcamera = 51.002 degK
tcoll = 50.515 degK
tbench = 50.624 degK
tshield = 155.980 degK
thead1 = 140.070 degK

    tc_setpa = 29.00
tempdet = 28.978 degK

tc_setpb = 50.00
thead2hi = 49.982 degK

htrdetblck = 24.300 percent
htrhead2hi = 16.900 percent

  • We will set the control temperatures as follows and monitor the behavior:

    • modify -s nirc2 thead1=100K

    • modify -s nirc2 thead2hi=40K

Tue, Nov 5: Record motor positions and power down instrument.

• Record motor positions, on a waikoko-new terminal type the following:

  • gshow -s nirc2 XXX% > ~/calvarez/ats/XXX.log

  • for each XXX = CAM GRS SHR FWI FWO PMS PMR SLS SLM PSI PSO ROT FOC MIN MAX

• Unplug NIRC2 motor power supply from PDU and plug it into an external battery.

• Home motors if needed.

Wed, Nov 6: Record motor positions and power down instrument.

• Power down sequence. When possible use the PDU interface available on any NIRC2 VNC pulldown menu (NIRC2 Engineering → NIRC2 power control GUI) and the PDU interface available on the TCS GUI → Tools → UNO Menu → UNO PDU GUI → Area: k2power)

1. K2AO enclosure:

   1. Archon controller (AO room)

   2. Signal Analyzer USB (AO room - no need to power this up)

   3. Calibration lamp power supply (AO vault)

   4. Private network switch (AO vault)

   5. DGH A1000 units 1, 2, and 3 (AO vault)

   6. Auto-pump system (AO room)

   7. Lakeshore 340 (AO vault - Temperature control)

   8. Lakeshore 218 (AO vault - Temperature monitoring)

   9. Motor power supply (AO vault) will be plugged into an external battery.

   10. Emulex TS (AO vault)

2. Keck-2 Machinery Room

   1. NIRC2 CCR speed controller

3. Keck-2 Computer Room:

   1. vm-nirc2

   2. waikoko-new

   3. waikoko

   4. waikoko Fortra 4-Slot disk U60

   5. waikoko Andataco GigaRAID 3000

   6. AT-MC15 10base converter

   7. Emulex TS for CCR

   8. DGH A1000 for CCR

   9. SCSI fiber optic extender (no need to power this up)

Procedure during power down

None because there will be nothing to monitor. Motor power supply will be plugged into an external battery.

Procedure to bring back the system

Once the power to the facility is restored, there are 3 possible scenarios:

1. If the power comes back up before the auto-pump system trigger temperature has been reached (55K on the CH2 first stage), then the NIRC2 temperature monitoring and control system will bring the temperatures down to their nominal values without any human intervention required.

2. If the power comes back up when the CH2 first stage temperature is slightly above the auto-pump system trigger temperature, then the auto-pump system will kick in and we can follow the procedure described in https://www2.keck.hawaii.edu/inst/nirc2/technical/alarm55.html to bring the system back to its normal state.

3. If the power comes back up when the CH2 when the pressure inside the dewar has reached 1E-5 mbar, then we will need to perform a full NIRC2 warm up procedure.

Power up sequence:

1. Keck-2 Computer Room:

   1. waikoko

   2. waikoko Fortra 4-Slot disk U60

   3. waikoko Andataco GigaRAID 3000

   4. AT-MC15 10base converter

   5. Emulex TS for CCR

   6. DGH A1000 for CCR

   7. waikoko-new

   8. vm-nirc2

2. Keck-2 Machinery Room:

   1. NIRC2 CCR Speed Controller

3. K2AO enclosure:

   1. Emulex TS (AO vault)

   2. Lakeshore 218 (AO vault - Temperature monitoring)

   3. Auto-pump system (AO room)

   4. Heater power supply (AO vault)

   5. Lakeshore 340 (AO vault - Temperature control)

   6. DGH A1000 Units 1, 2, and 3 (AO vault)

   7. Motor power supply (AO vault) will be unplugged from external battery and plugged into PDU.

   8. Private network switch (AO vault)

   9. Archon controller (AO room)

Once all the systems are running, evaluate which scenario we are on and determine how to proceed based on:

• The CH2 first stage temperature and other temperatures inside the dewar.

• Whether the auto-vacuum system has been started or not.

• Pressure inside the dewar.

One critical aspect of the decision making is to determine when the pressure inside the dewar is high enough that a full warm up will be required (Scenario #3). Based on the NIRC2 Maintenance Procedures (https://www2.keck.hawaii.edu/inst/nirc2/technical/NIRC2MaintainenceProcedures.pdf), sections “Evacuating D82” and “Cooling D82”, respectively: 

“The cold heads can be turned on when the vacuum has reached the desired state. Note that the pressure will never fall much below 1E-3 mbar at ambient temperature.” “After cooling, the getter should bring the pressure to about 1E-7 mbar or below.”

Therefore, I think ~1E-4 mbar would be a safe value for the “pressure of no return”, i.e. pressure beyond which we will need to fully warm up the instrument. Does this sound like a reasonable threshold?

Once the instrument is fully recovered, set the cold head speeds and control temperatures back to their normal operation values.

Appendix

This section includes insights provided by NIRC2 PI Keith Matthews (Caltech) with some edits and additional comments from me in orange.

I think the order of start up is fine, and cooling the bench using the first stage of CH2 is also fine. You could also cool the outer shields using the single stage cold head to about 100 K. What will happen when the CCR power is cut off, is that the second stage of CH2 will warm and the detector will warm and the getter box will also start to warm. It will release the small amount of He, H2, and Ne. Then after it gets up to around the bench temperature, it will release N2, O2 and CH3. It will have to get quite warm before CO2 and H2O get released. The big problem is H2O, which can affect the detector. That is why the pump is used when the instrument is warming up. If the pressure gets high then a full pump out and warmup is called for.

The only problem with cooling or having the bench at a different temperature from its regulated 50 K when observing is that because of differential contraction the slits, vortex masks and camera positions will be slightly off.  Also the filter characteristics will change slightly. These effects will probably be to too small to interfere with the observations, but you should ask.

There is no problem with moving the mechanisms at 40 K vs 50 K.

There is some chance that running the 2 stage cold head at high speed will exceed the available power from the 340 to keep the detector at 29 K.

If you think that any water vapor will out gas, it is best to warm up and pump in the standard manner.  You don’t want H20 to deposit on the first mirror of the collimator as it does with NIRSPEC producing an ice feature.   NIRC2’s design precludes that if the MFP of the residual gas is kept large.

My real concern is with the mechanisms. The motor power supply is on a UPS, and for all I know, the motors have not been homed for more than a decade [they have been homed at least twice in the last 5 years]. If the motors lose power then their status will be unknown. Also the computer will be down, so that it will not have the motor status. I would be good if you do not have to home the mechanisms, because they might get stuck.  Since the motors have power off brakes, they will hold position until they are commanded to move. I suggest that you interrogate all of the motors and record their positions on something nonvolatile, like paper. You should check the Smart Motor manual and also the low level motor command instructions on command computer and see if the positions and other parameters can be loaded on to the motors [we can do this via keywords and low-level commands are available on waikoko - see https://www2.keck.hawaii.edu/inst/nirc2/technical/motor_commands.html]. As I understand it, there have been operation system changes and software changes, so I hope that the fundamental commands are available [fundamental commands are available].

The most important thing is for the UPS to supply the motor PS for as long as possible, all of the other electronics can be shut down. If you have to home the mechanisms, good luck [we have powered down and homed motors in the recent past, so will not have an UPS for the motor Power Supply]. The most worrying are the roller screws, CAM, GRS, SLS, and SLM.

It is very important to use the positive homing script and not the original negative one.