2023-05-19 at 05:00 HST: KPF Green Detector deltaT=+11.3

The Green detector ceased communicating at about 5am HST. Similar to other events, the CCDPOWER keyword was non responsive, but restarting the service did not help. We power cycled the archon itself from the power strip.

Will was able to view camerad logs:

poweron 
ERROR Archon controller returned error processing command: POWERON
poweroff
ERROR Archon controller returned error processing command: POWEROFF

Furthermore, the camerad log (/kroot/var/log/ccd_green/camerad_2023)0519.log) is full of stuff like this:

2023-05-19T17:38:43.481123 (Camera::doit) thread 3 received command on fd 7: sensor 1 C AVG 
2023-05-19T17:38:43.481150 (Archon::Interface::sensor) ERROR: module 1 is not a heater board

That is, it doesn't even know there's a heater board at module 1. Also at startup, camerad logged that the "APPLYALL" Archon command failed.

Furthermore, the Archon system command returns sensible-looking values for the backplane itself, but about each individual 'module' it says:

SYSTEM:MOD1_TYPE=16
SYSTEM:MOD1_REV=0
SYSTEM:MOD1_VERSION=0.0.0
SYSTEM:MOD1_ID=0000000000000000

It repeats the above for MOD1 through MOD12

At about 9:15am, modify -s kpfgreen TEMPSET=-120 and modify -s kpfgreen STAT="Ramp Run" to prevent the CCD from overcooling.

The solution in this case was to make some changes on the archon boards which were performed the following day (2023-05-20). The detector temperature returned to nominal around 3pm on 2023-05-20.

It looks like the problem lay in one of the optional boards in the Archon itself (this was likely the cause of the slew of recent green side problems). We disabled the +/-100V supplies based on instructions from STA.