KPF: Pre-Observing | Observing | Post-Observing

Power Outage Procedures

Power off components in K1 AO

Power off Ca HK detector

  • Disable Ca HK, so it will not crash scripts by setting kpfconfig.CA_HK_ENABLED to “No”.

    • This will only take effect when a new script is started, so if a current script is running, you must either end the script (using kpfconfig.SCRIPTSTOP) or let it finish.

  • Verify that no scripts are running (monitor kpfguide.SCRIPTNAME to see if one is running)

  • Stop cooling on the detector

    • modify -s kpf_hk COOLING=Off

  • Power off the HK detector systems

    • Port J1: “kpfexpose2 Galil ROI controller”

    • Port J2: “kpfexpose2 Galil output bank”

    • Port J5: “Andor Newton PS”

Power off CRED2

  • Power off the guide camera (CRED2)

    • Port K2: “kpffiu2 nPoint”

    • Port K3: “kpfguide1 CRED-2”

Basement Procedures

If power is out, the gate valve separating the vacuum pump from the vacuum chamber has likely closed. This was meant to be a fail safe to protect the vac chamber if the pump stopped, but since the compressed air (which holds the valve open) and the pump are on different power, it is not working as intended. As a result, the gate valve will close and the vacuum chamber will begin losing vacuum.

An example plot of the vacuum pump pressure and the vacuum chamber pressure (green) during a HELCO power outage.

It is important to restore air pressure to the gate valve as soon as possible after the outage. Fortunately, the air compressors come on automatically, so no action should be needed.

If the chamber vacuum gets close to 1e-2 torr, then we should turn the turbo pump off, so that it doesn't crash when power is restored and the gate valve suddenly opens.

Turning the Turbo Pump Off

Quoting Steve Gibson from the CPS Slack in the #kpf-testing channel:

Yes, the KTL dispatcher for the pump is telemetry only with no control.  We went that route because the pump has a built-in webserver for control, and also because the pump software has many built-in features and safety monitors that would have been time consuming to replicate in a way to keep the pump from destroying itself.  @Kodi Rider will no doubt include the pump shutdown in the procedures he's currently working on.  But looking at my notes from Berkeley, the steps are:
Close gate valve
Kodi will provide details on this.  I believe it is currently set to close when a PDU outlet is turned off.
Log into vacuum pump webserver
On a computer connected to the KPF private network, open a web browser and go to 192.168.23.125.
At top right of page, log in using the kpfdev account credentials.
Remain on the ‘Main’ page.
Turn off turbo pump
Turn off the vacuum pump using the red ‘STOP' button at bottom center of page.
Note pump will not instantaneously stop, but the pump frequency will slowly start to drop. The user has no control here, this process is internally regulated by the pump.
Note it might take 1.5 hours for it to gracefully slow from 1000 Hz to 0 Hz.
Once pump frequency slows down below 500 Hz the cart-side pressure will start to rise (when gate valve closed)
Ending point
The turbo has stopped rotating and is showing a frequency of 0 Hz, and a power of 0 W.
Roots pump will still be running, and vac cart gauges show a steady pressure of approximately 1.5 x 10-2 Torr.

After Power is Restored

Restore Power to Ca HK Detector

  • Power on HK systems:

    • Port J1: “kpfexpose2 Galil ROI controller”

    • Port J2: “kpfexpose2 Galil output bank”

    • Port J5: “Andor Newton PS”

  • Start the detector cooling

    • modify -s kpf_hk COOLING=On

    • modify -s kpf_hk COOLTARG=-60

    • Verify that the current temperature is going toward the set point: gshow -s kpf_hk CURRTEMP

  • It is worth testing the HK detector by taking a bias and a cal frame (e.g. etalon). Sometimes the system needs a reset after the first exposure.

Restore Power to CRED2

  • Power on the CRED2 systems

    • Port K2: “kpffiu2 nPoint”

    • Port K3: “kpfguide1 CRED-2”