{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This notebook demonstrates the effect `detector_readout_parameters`, which selects between the different detector readout modes. These are `fast` and `slow` for the HAWAII2RG detectors, and `high_capacity` and `low_capacity` for the Geosnap detector."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import numpy as np\n",
    "from astropy import units as u\n",
    "from matplotlib import pyplot as plt\n",
    "from matplotlib.colors import LogNorm\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import scopesim as sim\n",
    "sim.bug_report()\n",
    "\n",
    "# Edit this path if you have a custom install directory, otherwise comment it out.\n",
    "sim.rc.__config__[\"!SIM.file.local_packages_path\"] = \"../../../../\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you haven't got the instrument packages yet, uncomment the following cell."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# sim.download_packages(['METIS', 'ELT', 'Armazones'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cmd = sim.UserCommands(use_instrument=\"METIS\", set_modes=[\"img_lm\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The default readout mode for `img_lm` is the fast mode:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cmd[\"!OBS.detector_readout_mode\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We build the optical train using the default mode, and then check that the relevant parameters (`mindit`, `full_well`, `readout_noise` and `dark_current`) are taken over correctly, first in the `cmds` property of the optical train, then - most importantly - into the parameters of the affected `Effect` objects (demonstrated for the `readout_noise` effect)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "metis = sim.OpticalTrain(cmd)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "metis.cmds[\"!DET\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "metis.cmds[metis['readout_noise'].meta['noise_std']]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "At this stage, we have access to the available detector modes and the parameter values that are set by them:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(metis['detector_readout_parameters'].list_modes())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can switch to the `slow` mode in the existing optical train by doing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "metis.cmds[\"!OBS.detector_readout_mode\"] = \"slow\"\n",
    "metis.update()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "metis.cmds[\"!DET\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "metis.cmds[metis['readout_noise'].meta['noise_std']]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Test: detector noise level (LSS-L)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To investigate the behaviour of the detector readout modes, we look at the L-band long-slit mode where the areas of the detector outside the spectral trace contain only readout noise and dark current. The default mode for long-slit spectroscopy is the `slow` mode, and we'll switch to the `fast` mode afterwards."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sky = sim.source.source_templates.empty_sky()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cmd = sim.UserCommands(use_instrument=\"METIS\", set_modes=[\"lss_l\"],\n",
    "                      properties={\"!OBS.exptime\": 1000})\n",
    "metis = sim.OpticalTrain(cmd)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"Detector mode:\", metis.cmds[\"!OBS.detector_readout_mode\"])\n",
    "metis.observe(sky, update=True)\n",
    "hdul_slow = metis.readout()[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We get the statistics in a strip at the left edge of the detector that is not covered by any source or background flux and compare to the expected values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ndit_slow = metis.cmds[\"!OBS.ndit\"]\n",
    "dit_slow = metis.cmds[\"!OBS.dit\"]\n",
    "\n",
    "bg_slow = hdul_slow[1].data[250:1750, 10:200].mean()\n",
    "bg_slow_expected = dit_slow * ndit_slow * metis.cmds[\"!DET.dark_current\"]\n",
    "\n",
    "noise_slow = hdul_slow[1].data[250:1750, 10:200].std()\n",
    "noise_slow_expected = np.sqrt(ndit_slow) * metis.cmds[\"!DET.readout_noise\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Do the same for the `fast` mode."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "hdul_fast = metis.readout(detector_readout_mode=\"fast\")[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ndit_fast = metis.cmds[\"!OBS.ndit\"]\n",
    "dit_fast = metis.cmds[\"!OBS.dit\"]\n",
    "\n",
    "bg_fast = hdul_fast[1].data[250:1750, 10:200].mean()\n",
    "bg_fast_expected = dit_fast * ndit_fast * metis.cmds[\"!DET.dark_current\"]\n",
    "\n",
    "noise_fast = hdul_fast[1].data[250:1750, 10:200].std()\n",
    "noise_fast_expected = np.sqrt(ndit_fast) * metis.cmds[\"!DET.readout_noise\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(f\"\"\"\n",
    "Fast: ndit  = {ndit_fast}  \n",
    "      bg    = {bg_fast:5.1f}  expected: {bg_fast_expected:5.1f}\n",
    "      noise = {noise_fast:5.1f}  expected: {noise_fast_expected:5.1f}\"\"\")\n",
    "print(f\"\"\"\n",
    "Slow: ndit  = {ndit_slow}  \n",
    "      bg    = {bg_slow:5.1f}  expected: {bg_slow_expected:5.1f}   \n",
    "      noise = {noise_slow:5.1f}  expected: {noise_slow_expected:.1f}\"\"\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, we can let Scopesim automatically select the \"best\" mode."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "hdul_auto = metis.readout(detector_readout_mode=\"auto\")[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Test: Full well (IMG-N) \n",
    "This demonstrates the high- and low-capacity modes of the Geosnap detector. The setup uses a neutral-density filter to ensure that the background does not saturate the detector in the low-capacity mode. The source is a very bright star, which saturates in the low-capacity mode but does not in the high-capacity mode."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "star = sim.source.source_templates.star(flux=20 * u.Jy)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cmd_n = sim.UserCommands(use_instrument=\"METIS\", set_modes=['img_n'],\n",
    "                        properties={\"!OBS.filter_name\": \"N2\", \"!OBS.nd_filter_name\": \"ND_OD1\"})\n",
    "metis_n = sim.OpticalTrain(cmd_n)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "metis_n.observe(star, update=True)\n",
    "print(\"--- high-capacity mode ---\")\n",
    "hdul_high = metis_n.readout(detector_readout_mode=\"high_capacity\")[0]\n",
    "fullwell_high = metis.cmds[\"!DET.full_well\"]\n",
    "ndit_high = metis.cmds[\"!OBS.ndit\"]\n",
    "print(\"--- low-capacity mode ---\")\n",
    "hdul_low = metis_n.readout(detector_readout_mode=\"low_capacity\")[0]\n",
    "ndit_low = metis.cmds[\"!OBS.ndit\"]\n",
    "fullwell_low = metis.cmds[\"!DET.full_well\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "detimg_high = hdul_high[1].data / ndit_high\n",
    "detimg_low = hdul_low[1].data / ndit_low"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.figure(figsize=(12, 4))\n",
    "plt.subplot(121)\n",
    "plt.plot(detimg_high[1024, 974:1074])\n",
    "plt.title(\"high-capacity mode\")\n",
    "plt.hlines(fullwell_high, 0, 100, colors='k', linestyles='dashed')\n",
    "plt.ylabel(\"Electrons per DIT\")\n",
    "\n",
    "plt.subplot(122)\n",
    "plt.plot(detimg_low[1024, 974:1074])\n",
    "plt.title(label=\"low-capacity mode\")\n",
    "plt.hlines(fullwell_low, 0, 100, colors='k', linestyles=\"dashed\")\n",
    "plt.ylabel(\"Electrons per DIT\");"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}