{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Selecting detector modes in METIS"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This notebook demonstrates the effect `detector_readout_parameters`, which selects between the different detector readout modes. These are `fast` and `slow` for the HAWAII2RG detectors, and `high_capacity` and `low_capacity` for the Geosnap detector."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "from astropy import units as u\n",
    "from matplotlib import pyplot as plt\n",
    "\n",
    "import scopesim as sim\n",
    "sim.bug_report()\n",
    "\n",
    "# Edit this path if you have a custom install directory, otherwise comment it out.\n",
    "sim.link_irdb(\"../../../../\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you haven't got the instrument packages yet, uncomment the following cell."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# sim.download_packages([\"METIS\", \"ELT\", \"Armazones\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cmd = sim.UserCommands(use_instrument=\"METIS\", set_modes=[\"img_lm\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The default readout mode for `img_lm` is the fast mode:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cmd[\"!OBS.detector_readout_mode\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We build the optical train using the default mode, and then check that the relevant parameters (`mindit`, `full_well`, `readout_noise` and `dark_current`) are taken over correctly, first in the `cmds` property of the optical train, then - most importantly - into the parameters of the affected `Effect` objects (demonstrated for the `readout_noise` effect)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "metis = sim.OpticalTrain(cmd)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "metis.cmds[\"!DET\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "metis['readout_noise'].meta['noise_std']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The \"bang string\" here means that the value is taken from the `cmds` structure in the previous cell, i.e."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "metis.cmds[metis['readout_noise'].meta['noise_std']]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "At this stage, we have access to the available detector modes and the parameter values that are set by them:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(metis['detector_readout_parameters'].list_modes())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can switch to the `slow` mode in the existing optical train by doing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "metis.cmds[\"!OBS.detector_readout_mode\"] = \"slow\"\n",
    "metis.update()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "metis.cmds[\"!DET\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "metis.cmds[metis['readout_noise'].meta['noise_std']]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Test: detector noise level (LSS-L)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To investigate the behaviour of the detector readout modes, we look at the L-band long-slit mode where the areas of the detector outside the spectral trace contain only readout noise and dark current. The default mode for long-slit spectroscopy is the `slow` mode, and we'll switch to the `fast` mode afterwards."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cmd = sim.UserCommands(use_instrument=\"METIS\", set_modes=[\"lss_l\"])\n",
    "metis = sim.OpticalTrain(cmd)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"Detector mode:\", metis.cmds[\"!OBS.detector_readout_mode\"])\n",
    "metis['psf'].include = False     # not needed for blank-sky observations\n",
    "metis.observe()\n",
    "hdul_slow = metis.readout(exptime=1000)[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We get the statistics in a strip at the left edge of the detector that is not covered by any source or background flux and compare to the expected values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ndit_slow = metis.cmds[\"!OBS.ndit\"]\n",
    "dit_slow = metis.cmds[\"!OBS.dit\"]\n",
    "\n",
    "bg_slow = hdul_slow[1].data[250:1750, 10:200].mean()\n",
    "bg_slow_expected = dit_slow * ndit_slow * metis.cmds[\"!DET.dark_current\"]\n",
    "\n",
    "noise_slow = hdul_slow[1].data[250:1750, 10:200].std()\n",
    "noise_slow_expected = np.sqrt(ndit_slow) * metis.cmds[\"!DET.readout_noise\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Do the same for the `fast` mode."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "hdul_fast = metis.readout(detector_readout_mode=\"fast\", exptime=1000)[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ndit_fast = metis.cmds[\"!OBS.ndit\"]\n",
    "dit_fast = metis.cmds[\"!OBS.dit\"]\n",
    "\n",
    "bg_fast = hdul_fast[1].data[250:1750, 10:200].mean()\n",
    "bg_fast_expected = dit_fast * ndit_fast * metis.cmds[\"!DET.dark_current\"]\n",
    "\n",
    "noise_fast = hdul_fast[1].data[250:1750, 10:200].std()\n",
    "noise_fast_expected = np.sqrt(ndit_fast) * metis.cmds[\"!DET.readout_noise\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(f\"\"\"\n",
    "Fast: ndit  = {ndit_fast}     dit = {dit_fast:.2f}\n",
    "      bg    = {bg_fast:5.1f}   expected: {bg_fast_expected:5.1f}\n",
    "      noise = {noise_fast:5.1f}  expected: {noise_fast_expected:5.1f}\"\"\")\n",
    "print(f\"\"\"\n",
    "Slow: ndit  = {ndit_slow}     dit = {dit_slow:.2f}\n",
    "      bg    = {bg_slow:5.1f}   expected: {bg_slow_expected:5.1f}   \n",
    "      noise = {noise_slow:5.1f}   expected: {noise_slow_expected:.1f}\"\"\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, we can let Scopesim automatically select the \"best\" mode."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "hdul_auto = metis.readout(detector_readout_mode=\"auto\", exptime=1000)[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The advantage of the \"slow\" mode of the H2RG detector is the low readout noise. The \"fast\" mode permit smaller DITs and would be selected for bright sources that would saturate the detector at the minimum DIT of the \"slow\" mode. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Test: Full well (IMG-N) \n",
    "This demonstrates the high- and low-capacity modes of the Geosnap detector. The setup uses a neutral-density filter to ensure that the background does not saturate the detector in the low-capacity mode. The source is a very bright star, which saturates in the low-capacity mode but does not in the high-capacity mode."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "star = sim.source.source_templates.star(flux=20 * u.Jy)\n",
    "\n",
    "cmd_n = sim.UserCommands(use_instrument=\"METIS\", set_modes=['img_n'],\n",
    "                        properties={\"!OBS.filter_name\": \"N2\", \"!OBS.nd_filter_name\": \"ND_OD1\"})\n",
    "metis_n = sim.OpticalTrain(cmd_n)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "metis_n.observe(star, update=True)\n",
    "print(\"--- high-capacity mode ---\")\n",
    "hdul_high = metis_n.readout(detector_readout_mode=\"high_capacity\", exptime=1)[0]\n",
    "fullwell_high = metis_n.cmds[\"!DET.full_well\"]\n",
    "gain_high = metis_n.cmds[\"!DET.gain\"]\n",
    "ndit_high = metis_n.cmds[\"!OBS.ndit\"]\n",
    "\n",
    "print(\"--- low-capacity mode ---\")\n",
    "hdul_low = metis_n.readout(detector_readout_mode=\"low_capacity\", exptime=1)[0]\n",
    "fullwell_low = metis_n.cmds[\"!DET.full_well\"]\n",
    "gain_low = metis_n.cmds[\"!DET.gain\"]\n",
    "ndit_low = metis_n.cmds[\"!OBS.ndit\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We take the average over NDIT exposures (units: electrons per DIT) so that we can immediately compare the counts to the full wells for the two modes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "detimg_high = hdul_high[1].data * gain_high\n",
    "detimg_low = hdul_low[1].data * gain_low"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.figure(figsize=(12, 4))\n",
    "plt.subplot(121)\n",
    "plt.plot(detimg_high[1024, 974:1074])\n",
    "plt.title(\"high-capacity mode\")\n",
    "plt.hlines(fullwell_high, 0, 100, colors='k', linestyles='dashed')\n",
    "plt.ylabel(\"Electrons per DIT\")\n",
    "\n",
    "plt.subplot(122)\n",
    "plt.plot(detimg_low[1024, 974:1074])\n",
    "plt.title(label=\"low-capacity mode\")\n",
    "plt.hlines(fullwell_low, 0, 100, colors='k', linestyles=\"dashed\")\n",
    "plt.ylabel(\"Electrons per DIT\");"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}