\section{Nightly debugging}
\label{sec:nightly_debug}

% -------------------------------------------------------------------------
\subsection{Overview}
\label{sec:nightly_debug:overview}
% -------------------------------------------------------------------------

A night-by-night list of targets can be found on the ARI apero profile page, under the “observation table” link in figure \ref{fig:ari_profile_page}

\begin{figure}[!htbp]
    \centering
    \includegraphics[width=\textwidth]{images/ari_profile_page.png}
    \caption{An example ARI profile page}
    \label{fig:ari_profile_page}
\end{figure}

This gives a table of all observations sorted by observation night directory:

\begin{figure}[!htbp]
    \centering
    \includegraphics[width=1\linewidth]{images/ari_observation_table.png}
    \caption{ARI Observation table}
    \label{fig:ari_observation_table}
\end{figure}

If there are observations for this night you must check all objects for that given night (briefly). 

To aid this there are links to the Target, Spectrum, LBL, CCF, Timeseries and Debug sections. 

Please familiarize yourself with the graphs in each section (Target, Spectrum, LBL, CCF) and what they should look like. 

Please report anything that looks weird to the following people: 

\begin{contacttable}
    \ccNeil{1} & \eeNeil \\
    \ccLison{} & \eeLison \\
    \ccEtienne{} & \eeEtienne \\
    \ccFred{} & \eeFred \\
    \ccThomas{} & \eeThomas \\
    \ccCharles{} & \eeCharles \\
    \ccUMail{} & \eeUMail \\
    \hline
\end{contacttable}

% -------------------------------------------------------------------------
\newpage
\subsection{ARI: Target section}
\label{sec:nightly_debug:ari_target_sec}
% -------------------------------------------------------------------------

No graphs in this section, however it is worth looking at the OBJECT name(s) in headers for any weird names that you think shouldn’t be there (compared to the actual name of the target and the aliases provided)

% -------------------------------------------------------------------------
\newpage
\subsection{ARI: Spectrum section}
\label{sec:nightly_debug:ari_spectrum_sec}
% -------------------------------------------------------------------------

There are graphs of the extracted SNR (as measured in APERO extract recipe), and BERV coverage plot and an example of the extracted and telluric corrected spectrum (with zoom-ins on three regions) see figure \ref{fig:ari_spectrum_section}.

\begin{figure}[!htbp]
    \centering
    \includegraphics[width=\textwidth]{images/ari_spectrum_section.png}
    \caption{Example of the plots in the ARI spectrum section}
    \label{fig:ari_spectrum_section}
\end{figure}

Just check that the SNR hasn’t dropped horribly, and that the telluric correction looks okay. 

You can also take a look at the number of files passing and failing QC (either in the BERV coverage plot - second panel or from the Spectrum information table) 


% -------------------------------------------------------------------------
\newpage
\subsection{ARI: LBL section}
\label{sec:nightly_debug:ari_lbl_sec}
% -------------------------------------------------------------------------

There are graphs of the LBL velocity, the SNR (again) and the RV in the blue vs red - see figure \ref{fig:ari_lbl_section}. 

For the LBL velocity if there are many possibly bad points (or many red arrows) report it. 

Note that there are various combinations of object and `friend' template so you may have several sets of graphs to look at (they are named \lstpath{\{OBJECT_NAME\}_\{TEMPLATE_NAME\}}) 

In the bottom panel just make sure that roughly that the blue and red points are centered around the black points and look equally distributed on either side (blue points are normally more uncertain than the red points). 

\begin{figure}[!htbp]
    \centering
    \includegraphics[width=\textwidth]{images/ari_lbl_section.png}
    \caption{Example of the plots in the ARI LBL section}
    \label{fig:ari_lbl_section}
\end{figure}


% -------------------------------------------------------------------------
\newpage
\subsection{ARI: CCF section}
\label{sec:nightly_debug:ari_ccf_sec}
% -------------------------------------------------------------------------

There are graphs for the velocity, the mean CCF, mean residuals and the residuals to the median. See figure \ref{fig:ari_ccf_section}. 

For the velocity graph (similar to LBL)y look out for lots of red outlier arrows. 

The mean CCF should look like a Gaussian curve with a continuum and a negative peak. 

The residual may have some wiggles to it but should general be small (compared to the scale of the peak in the CCF above). 

\begin{figure}[!htbp]
    \centering
    \includegraphics[width=\textwidth]{images/ari_ccf_section.png}
    \caption{Example of the plots in the ARI CCF section}
    \label{fig:ari_ccf_section}
\end{figure}

% -------------------------------------------------------------------------
\newpage
\subsection{ARI: Time-series section}
\label{sec:nightly_debug:ari_tso_sec}
% -------------------------------------------------------------------------

This is a table of observed points. 

You can roughly make sure the newest points (which should be the ones you reduced most recently) have similar values to other observations (though note that things such as weather and changes in observation plan can affect this – though ask if you are unsure!) 


% -------------------------------------------------------------------------
\newpage
\subsection{ARI: Debug section}
\label{sec:nightly_debug:ari_debug_sec}
% -------------------------------------------------------------------------

This section has a lot of graphs that it might be worth familiarizing yourself with. We try to show good examples below.

\subsubsection{Telluric map plot}

\begin{figure}[!htbp]
    \centering
    \includegraphics[width=\textwidth]{images/ari_debug_tellumap.png}
    \caption{Example of the ARI Debug telluric map plot}
    \label{fig:ari_debug_tellumap}
\end{figure}

In general, the bottom plot shouldn’t have large values and in an ideal case should just be noise. See figure \ref{fig:ari_debug_tellumap}. 

Vertical lines are what we want to remove - but currently we still have some in there. 

Note that the QC bar (purple and orange) shows which observations were rejected – so you can ignore horizontal rows with an orange flag. 

Some observations are just very log SNR – these will appear as noisier horizontal rows – you could look at the SNR in the other graphs but mostly these can be ignored. 

\subsubsection{Shape quality control plot}

\begin{figure}[!htbp]
    \centering
    \includegraphics[width=\textwidth]{images/ari_debug_shape_plot.png}
    \caption{Example of the ARI Debug shape quality plot}
    \label{fig:ari_debug_shape_plot}
\end{figure}

These show how the detector moved (dx and dy being horizontal and vertical shifts) and A,B,C,D being an affine transformation matrix describing rotation, sheer etc. See figure \ref{fig:ari_debug_shape_plot}. 

These should all be clustered around zero (A and D have been normalized around 0 their true value is close to 1), but most importantly these should be consistent across time. 

We do have jumps in the detector but if you see a large jump in the most recent observations you can report it. 

\subsubsection{WFPDRIFT and wave cavity plots}

\begin{figure}[!htbp]
    \centering
    \includegraphics[width=\textwidth]{images/ari_debug_wave_plots.png}
    \caption{Example of the ARI Debug wave drift plots}
    \label{fig:ari_debug_wave_plots}
\end{figure}

These show how the FP has changed over time. They should all be following the same overall trend. See figure \ref{fig:ari_debug_wave_plots}. 

We do have jumps in the detector but if you see a large jump in the most recent observations you can report it. 


\subsubsection{Saturation and Effective readout noise plots}

\begin{figure}[!htbp]
    \centering
    \includegraphics[width=\textwidth]{images/ari_debug_sat_noise.png}
    \caption{Example of the ARI Debug saturation and effective readout noise plots}
    \label{fig:ari_debug_sat_noise}
\end{figure}

Figure \ref{fig:ari_debug_sat_noise} show some metrics we are measuring when extracting a spectrum. 

They should all be following the same overall trend. However maximum saturation levels can vary with observing conditions but very large values could indicate something very wrong with the observation. 

We do have jumps in the detector but if you see a large jump in the most recent observations you can report it. 



\subsubsection{APERO Processing Debug Plot}

\begin{figure}[!htbp]
    \centering
    \includegraphics[width=\textwidth]{images/ari_debug_processing.png}
    \caption{Example of the APERO Debug processing plot}
    \label{fig:ari_debug_processing}
\end{figure}

The top panel of figure \ref{fig:ari_debug_processing} shows the version of the DRS used - it should not be different in most cases. Please report if it has changed. 

The bottom panel shows when the data was processed. This should be a flat line (for data originating from offline reductions) followed by a one-to-one slope with date (i.e. data has been reduced day by day after the last offline reduction) if this is not the case please report it (note you’ll also see when multiple day were reduced at the same time by horizonal lines – this is not a problem but long stretches may need to be reported. 


\subsubsection{APERO calibration times plot}

\begin{figure}[!htbp]
    \centering
    \includegraphics[width=\textwidth]{images/ari_debug_cal_time.png}
    \caption{Example of the APERO Debug calibration time plots}
    \label{fig:ari_debug_cal_time}
\end{figure}

This checks the calibration file's observed time against the observation time of the science target.  

Calibrations checked are as follows:
\begin{itemize}
    \item Dark (reference)
    \item Bad pixel file
    \item Order profile
    \item Localization
    \item Local shape file
    \item Shape X file (reference)
    \item Shape Y file (reference)
    \item Flat file
    \item Blaze file
    \item Wave file
    \item Wave solution
\end{itemize}

Reference objects may diverge, while all other calibrations should be within approximately one day,  
unless the calibrations were bad or not taken.  

A report should be generated for cases exceeding two days, unless there are comments about  
the calibration tests failing for multiple days and the babysitter was explicitly asked to  
ignore those failures.
