Merge pull request #847 from int-brain-lab/docs

Docs

Author: mayofaulkner

examples/loading_data/loading_photometry_data.ipynb

@@ -0,0 +1,279 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "c1fdd473",
+   "metadata": {},
+   "source": [
+    "# Loading Fiber Photometry Data\n",
+    "\n",
+    "Calcium activity recorded using a fiber photometry."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "6e1ba7b2",
+   "metadata": {
+    "nbsphinx": "hidden"
+   },
+   "outputs": [],
+   "source": [
+    "# Turn off logging and disable tqdm this is a hidden cell on docs page\n",
+    "import logging\n",
+    "import os\n",
+    "\n",
+    "logger = logging.getLogger('ibllib')\n",
+    "logger.setLevel(logging.CRITICAL)\n",
+    "\n",
+    "os.environ[\"TQDM_DISABLE\"] = \"1\""
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "a406ed50",
+   "metadata": {},
+   "source": [
+    "## Relevant ALF objects\n",
+    "* photometry\n",
+    "* photometryROI\n",
+    "\n",
+    "\n",
+    "## More details\n",
+    "* [Description of photometry datasets](https://docs.google.com/document/d/1OqIqqakPakHXRAwceYLwFY9gOrm8_P62XIfCTnHwstg/edit#heading=h.3o4nwo63tny)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "02482b24",
+   "metadata": {},
+   "source": [
+    "## Finding sessions with photometry data\n",
+    "Sessions that contain photometry data can be found by searching for sessions with a corresponding photometry dataset"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "06f43a20",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from one.api import ONE\n",
+    "one = ONE()\n",
+    "sessions = one.search(dataset='photometry.signal.pqt')\n",
+    "print(f'{len(sessions)} sessions with photometry data found')"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "dea30e9f",
+   "metadata": {},
+   "source": [
+    "## Loading photometry data\n",
+    "The photometry data for a single session can be loaded in the following way"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "41fae7ad",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Get the first returned sessions with photometry data\n",
+    "eid = sessions[0]\n",
+    "# Load the photometry signal dataset\n",
+    "photometry = one.load_dataset(eid, 'photometry.signal.pqt')\n",
+    "print(photometry.columns)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "5a19d768",
+   "metadata": {},
+   "source": [
+    "The data returned is a table that contains photometry data for all ROIS (Region0G, Region1G, ...) recorded simultaneously in a single session. The number of rows in the table give the number of imaging frames in the dataset. The timestamps for each frame is stored in the `times` column are in seconds from session start and are aligned to other times from the session, e.g behavioral or video events.\n",
+    "\n",
+    "The wavelength of light used to collect each imaging frame can be found using either the `wavelength` or the `name` column. For example if we want to limit our table to only frames collected at 470 nm we can do the following"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "62f0bc6c",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Limit signal to frames collected at 470 nm\n",
+    "photometry = photometry[photometry['wavelength'] == 470]"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "d544f651",
+   "metadata": {},
+   "source": [
+    "The photometry data also contains a column called `include` which contains a manually selected interval of the signal that is free from artefacts"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "acc946cf",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Restrict signal to artefact free intervals\n",
+    "photometry = photometry[photometry['include']]"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "a9f91adb",
+   "metadata": {},
+   "source": [
+    "## Associating ROIs to Brain Regions"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "462c2242",
+   "metadata": {},
+   "source": [
+    "We can associate each Region with a brain region by loading in the photometryROI dataset. This contains a lookup table from `ROI` to a `fiber` stored on the openalyx database and a `brain_region`"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "5ff081d7",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "rois = one.load_dataset(eid, 'photometryROI.locations.pqt')\n",
+    "rois"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "bbdfb2fc",
+   "metadata": {},
+   "source": [
+    "We can rename our columns in our photometry data with the brain regions"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "4420a657",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "photometry = photometry.rename(columns=rois.to_dict()['brain_region'])\n",
+    "print(photometry.columns)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "9c3978e5",
+   "metadata": {},
+   "source": [
+    "Please see the associated [publication](https://doi.org/10.1101/2024.02.26.582199) for these datasets for more information about the definition of the given brain regions."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "d2116ab7",
+   "metadata": {},
+   "source": [
+    "## QC of the ROIs"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "a4cc1d09",
+   "metadata": {},
+   "source": [
+    "Each ROI has an associated fiber insertion registered on the openalyx database. The fiber contains information about the brain region targeted and also a `QC` value indicating if the signal is good or not. The associated [publication](https://doi.org/10.1101/2024.02.26.582199) contains more information about the defintion of a passing QC value.\n",
+    "\n",
+    "For a session we can find the QC for each ROI in the following way"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "4937fd8c",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from iblutil.util import Bunch\n",
+    "\n",
+    "QC = Bunch()\n",
+    "for roi, info in rois.iterrows():\n",
+    "    fiber = one.alyx.rest('insertions', 'list', session=eid, name=info.fiber)[0]\n",
+    "    QC[info.brain_region] = fiber['json']['qc']\n",
+    "\n",
+    "print(QC)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "eb061d87",
+   "metadata": {},
+   "source": [
+    "## Computing dF / F"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "b36aef53",
+   "metadata": {},
+   "source": [
+    "Here we show an example of how to compute the dF/F signal from the photometry data using the defintion in associated [publication](https://doi.org/10.1101/2024.02.26.582199)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "5fa5ca08",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Compute df/F signal for brain region DMS\n",
+    "# Baseline signal is the +- 30s rolling average of the raw signal\n",
+    "\n",
+    "# Get the frame rate of the data\n",
+    "fr = (1 / photometry.times.diff().mean()).round()\n",
+    "# Define rolling average window of 30 s\n",
+    "window = 30\n",
+    "\n",
+    "F = photometry['DMS']\n",
+    "F0 = F.rolling(int(fr * window), center=True).mean()\n",
+    "dF = (F - F0) / F0\n"
+   ]
+  }
+ ],
+ "metadata": {
+  "celltoolbar": "Edit Metadata",
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.9.16"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}