3.4.1

Changelog

All notable changes to this project will be documented in this file.

The format is based on Keep a Changelog,
and this project adheres to Semantic Versioning.

[3.4.1] 2025-07-02

Fixed

• SpikeSortingLoader takes into account revisions
• There is a specific test for spikeinterface CI in our integration tests

Release Note 3.4.0

features

• Add revision as a kwarg to the spike sorting loader
• Register bin files to flatiron
• Ensure pre-existing compressed files are verified and registered
• Only update relevant collections in update_collections
• Update extended QC with task number prefix
• Update trials extractors to handle clean first trial
• Pin numpy to >=1.18,<=2.2 (numpy 2.3 is not compatible with numba)
• oneibl.patcher.patch_dataset: revert from clobber to force
• Ensure session paths are ALFPath instances
• Update dataset QC
• Handle None session path
• Add options for SDSC and EC2 patchers
• Use ALFPath objects in Task and DataHandler classes

Release Note 3.3.1

Bugfixes

• SpikeSortingLoader: returns channels from 'electrodeSites' if available (it always was the intention...)

Release Note 3.3.0

features

• ONE 3
• Handle extraction of sessions where first Bpod trial missed on FPGA
• Support for multi depth sessions by passing slices kwarg to suite2p fork
• Selectively remove old FOV datasets in MesoscopePreprocess
• Added starplot function
• Improvements to LightningPose and SpikeSorting tasks

Bugfixes

• Add back PROJECT_EXTRACTION_VERSION and TASK_VERSION to session JSON
• Fix error in MesoscopeCompress when compressing multiply imaging bouts
• Fix error in MesoscopeSync.setUp with remote location

Release Note 3.2.0

features

• Add session delay info during registration of Bpod session
• Add detailed criteria info to behaviour plots
• Add column filtering, sorting and color-coding of values to metrics table of
  task_qc_viewer

Bugfixes

• Read in updated json keys from task settings to establish ready4recording
• Handle extraction of sessions with dud spacers

Release Note 3.1.0

features

• Add narrative during registration of Bpod session

Release Note 3.0.0

features

• Support for py3.8 and 3.9 dropped
• JobCreator Alyx task
• Photometry sync task
• Remove deprecated code, incl. iblatlas and brainbox/quality
• Bugfix: badframes array dtype change int8 -> uint32
• Brain wide map release information
• Limited Numpy 2.0 and ONE 3.0 support

Release Note 2.40.0

features

• iblsorter >= 1.9 sorting tasks with waveform extraction and channel sorting
• s3 patcher prototype

2.40.1

• Bugfix: ibllib.io.sess_params.merge_params supports tasks extractors key

2.40.2

• Bugfix: badframes array dtype change int8 -> uint32

Release Note 2.39.0

features

• register inputs without rename in registerSnapshots
• iterate only plane folders in Suite2P
• removed old pipeline

bugfixes

• NeuromodulatorCW is a subclass of Advanced CW regarding debiasing
• Behaviour QC: StimFreeze QC due to indexing errors, ITI duration and negative feedback delay for nogo trials
• Camera QC fixes

2.39.1

• Bugfix: brainbox.metrics.single_unit.quick_unit_metrics fix for indexing of n_spike_below2

2.39.2

• Bugfix: routing of protocol to extractor through the project repository checks that the
  target is indeed an extractor class.

Release Note 2.38.0

features

• Support running large pipeline jobs within multiple python environments
• Extraction and QC of of task pause periods
• Register animated GIF snapshots and tiff images
• Load waveforms low level changes in spike sorting loader
• Loading of additional channel object attributes (qc labels) in spike sorting loader
• Stimulus times now extracted in a more robust manner

bugfixes

• Fix in training plots when no trials in block

Release Note 2.37.0

features

• Add code in preparation for retirement of old training and ephys local server pipelines

bugfixes

• Change handling of trials extraction of iblrigv8 sessions such that pregenerated session is not used

Release Note 2.36.0

features

• Adding spike sorting iblsort task

Release Note 2.35.0

features

• Adding LightningPose task

bugfixes

• SessionLoader can now handle trials that are not in the alf collection
• Extraction of trials from pre-generated sequences supports iblrigv8 keys

2.35.1

• Ensure no REST cache used when searching sessions in IBLRegistationClient

2.35.2

• Flag to allow session registration without water administrations
• Support extraction of repNum for advancedChoiceWorld
• Support matplotlib v3.9; min slidingRP version now 1.1.1

2.35.3

• Use correct task for timeline acquisitions in make_pipeline

Release Note 2.34.0

features

• Task assert_expected_input now take into account revisions
• Camera qc and video motion now take into account dynamic pipeline folder

bugfixes

• Typo in raw_ephys_data documentation
• oneibl.register_datasets accounts for non existing sessions when checking protected dsets

2.34.1

• Ensure mesoscope frame QC files are sorted before concatenating
• Look for SESSION_TEMPLATE_ID key of task settings for extraction of pre-generated choice world sequences
• Download required ap.meta files when building pipeline for task_qc command

Release Note 2.33.0

features

• Datasets no longer registered by default if protected, need to force registration
• Tasks now allows datasets to be registered before qc computation
• Histology channel upload now reads in channel map from data when available

bugfixes

• PostDLC tasks looks in alf folder for lick datasets

Release Notes 2.32

features

• SDSC patcher automatically support revisions

other

• Add extra key to alignment qc with manual resolution for channel upload

2.32.3

• FpgaTrials supports alignment of Bpod datasets not part of trials object
• Support chained protocols in BehaviourPlots task

2.32.4

• Add support for variations of the biaseCW task in the json task description

2.32.5

• Minor fixes to IBL registration client, including use of SESSION_END_TIME key

Release Notes 2.31

features

• Training status uses new extractor map
• Refactor neurodsp to ibldsp
• ITI qc check for iblrig v8
• Support habituationChoiceWorld extraction in iblrig v8.15.0

bugfixes

• NP2 waveforms extracted with correct dtype
• Sorted cluster ids in single unit metrics

Release Notes 2.30

features

• Task QC viewer
• Raw ephys data loading documentation

other

• Pandas 3.0 support

Release Notes 2.29

features

• Added raw data loaders and synchronisation tools in brainbox.io.one.SpikeSortingLoader, method ssl.raw_electrophysiology()

Release Notes 2.28

features

• Added ibllib.pipes.dynamic_pipeline.get_trials_tasks function

bugfixes

• Fix ibllib.io.extractors.ephys_fpga.extract_all for python 3.8

other

• Change behavior qc to pass if number of trials > 400 (from start) can be found for which easy trial performance > 0.9

2.28.1

• Typo in ibllib.pipes.video_tasks.EphysPostDLC class
• ibllib.io.raw_data_loaders.patch_settings works with iblrigv8 settings files

2.28.2

• Fix loading of personal projects extractor map

Release Notes 2.27

features

• Add full video wheel motion alignment code to ibllib.io.extractors.video_motion module
• Change FPGA camera extractor to attempt wheel alignment if audio alignment fails
• Flexible FpgaTrials class allows subclassing for changes in hardware and task
• Task QC thresholds depend on sound card
• Extractor classes now return dicts instead of tuple
• Support extraction of habituationChoiceWorld with FPGA
• New IBLGlobusPatcher class allows safe and complete deletion of datasets

bugfixes

• Fix numpy version dependent error in io.extractors.camera.attribute_times
• Fix for habituationChoiceWorld stim off times occuring outside of trial intervals
• Improvements to Timeline trials extractor, especially for valve open times
• trainingPhaseChoiceWorld added to Bpod protocol extractor map fixture
• Last trial of FPGA sessions now correctly extracted
• Correct dynamic pipeline extraction of passive choice world trials

2.27.1

• Correct handling of missing TTLs in FpgaTrialsHabituation

other

• Removed deprecated pyschofit module
• Deprecated oneibl.globus module in favour of one.remote.globus
• Deprecated qc.task_extractors in favour of behaviour pipeline tasks

Release Notes 2.26

features

• Deprecate ibllib.atlas. Code now contained in package iblatlas

bugfixes

2.26.1

• fix typo in criterion_ephys where lapse high for 0.8 blocks only passed when non-zero.

Release Notes 2.25

features

• Training status pipeline now compatible with dynamic pipeline
• Dynamic DLC task using description file
• Full photometry lookup table

bugfixes

• fix for untrainable, unbiasable don't repopulate if already exists

2.25.1

• relax assertion on Neuropixel channel mappings to allow for personal projects

2.25.2

• listing of all collections does not skip repeat task protocols anymore for copy/extraction

Release Notes 2.23

Release Notes 2.23.1 2023-06-15

features

• split swanson areas

bugfixes

• trainig plots
• fix datahandler on SDSC for ONEv2

Release Notes 2.23.0 2023-05-19

• quiescence period extraction
• ONEv2 requirement

Release Notes 2.22

Release Notes 2.22.3 2023-05-03

same as 2.22.2 (I messed up the release sorry)

Release Notes 2.22.2 2023-05-03

bugfixes

• training plots

features

• can change download path for atlas

Release Notes 2.22.1 2023-05-02

bugfixes

• use session path in training status task

Release Notes 2.22.0

features

• trials extractors support omissions trials from neuromodulator tasks
• SpikeSortingLoader and EphysSessionLoader utils functions to load related objects such as drift
• Training status tasks upload training date and session to subject json
• Query training criterion function added to brainbox.training
• Swanson vector - option to annotate top/bottom 5 regions based on values, or pass in a list of acronyms
• EphysQC can run without connection to ONE

Release Notes 2.21

Release Notes 2.21.3 2023-03-22

features

• show atlas names in swanson plot
• allow user to define mask regions in swanson plot

bugfixes

• register_session: handle lack of taskData in passive sessions

Release Notes 2.21.2 2023-02-24

bugfixes

• get_lab function now gets lab name from session path subject name
• create_jobs now returns pipelines

Release Notes 2.21.1 2023-02-21

bugfixes

• remove unused jupyter and jupyterhub requirements
• fix mapping for atlas svg plots

Release Notes 2.21.0 2023-02-14

features

• support for multiple task protocols within a session
• extract protocol period from spacer signals
• function for changing subject, collection, number or date in a settings file
• function to retrieve task protocol number from experiment description
• dataset type validation supports null filename patterns
• oneibl.registration uses one.registration client as superclass
• experiment description files are created and registered in legacy pipelines
• QC sign-off keys are added to session JSON field
• class-based note template API for attaching session notes for QC sign-offs
• protocol and procedures now taken from experiment description file
• handle truncated first spacer in passive extraction
• fix the Perirhinal area layer color in Allen Atlas
• fix region volume computation
• vectorised swanson flatmap
• brainbox trial ITI loader
• vectorised atlas plots

Release 2.20

Release Notes 2.20.0 2023-01-23

features

• ephys session loader
• bwm release documentation

bugfixes

• mock gpu lock in tests
• use cluster_ids in metrics

Release Notes 2.19

Release Notes 2.19.0 2022-11-30

features

• camera qc allows for string values
• deprecate brainbox.io.spikeglx.stream instead use Streamer
• copy logics take into account presence of experiment description files
• option for error bars in training plots

bugfixes

• cam qc for dynamic pipeline uses sync type to determine number of cameras

Release 2.18

Release Notes 2.18.0 2022-11-23

features

• brainbox.modeling removed
• brainbox.io.one.load_trials_df removed, use one.load_object(...).to_df() instead
• AWS DataHandler refactored (ibllib.oneibl.aws removed)
• raise error when saving an empty dataset during extraction
• removed deprecated dsp packages (ibllib.dsp, ibllib.ephys.neuropixel, ibllib.io.spikeglx)
• register datasets as a revision if protected on Alyx

bugfixes

• frameData and timestamps both optional datasets for camera QC
• for 3B probes don't glob the sync files as fails for NP2 case
• support iblutil 1.4 (iblutil.util.get_logger -> iblutil.util.setup_logger)

Release Notes 2.17

Release Notes 2.17.2 2022-10-14

• Compute wheel velocity using a low-pass filter instead of a Gaussian window smoothing
• Save widefield output plots in png format
• Widefield Sync task uses sync_collection argument
• Improve boundary display for atlas slice plots

Release Notes 2.17.1 2022-10-04

• adjust ONE-api requirement to redownload on AWS backend when md5sum mismatch

Release Notes 2.17.0 2022-10-04

• units quality metrics use latest algorithms for refractory period violations and noise cut-off

Release Notes 2.16

Release Notes 2.16.1 2022-09-28

bugfixes

• photometry extraction: recover from corrupt DAQ signal and reversed polarity of voltage pulses

Release Notes 2.16.0 2022-09-27

features

• swanson flatmap: the algorithm to propagate down the hierarchy has been refined

bugfixes

• set exists flag to false for all data repos when registering datasets with tasks

Release Notes 2.15

Release Notes 2.15.3 - 2022-09-26

• SessionLoader error handling and bug fix

Release Notes 2.15.2 - 2022-09-22

• extraction pipeline: fix unpacking of empty arguments field from alyx dict that prevents running task

Release Notes 2.15.1 - 2022-09-21

• atlas: gene-expression backend and MRI Toronto atlas stretch and squeeze factors (Dan/Olivier)
• FDR correction (Benjamin-Hochmann) to correct for multiple testing optional (Guido)
• SpikeSortingLoader can be used with ONE local mode (Julia)

Release Notes 2.15.0 2022-09-20

features

• dynamic pipelines creation: if the acquisition description file exists, task sequences get created
• session registration procedures and projects, only add a couple of fields in the json by default
• new modalities:
  • photometry extraction (Mainen lab)
  • widefield extraction (Churchland lab)

bugfixes

• Spike sorting task: parse new pykilosort log format
• Session loader

Release Notes 2.14

Release Notes 2.14.0 2022-08-17

• Adding brainbox.io.one.SessionLoader for standardized loading of session data
• Changes to TaskQC, VideoQC and DLCQC thresholds and aggregation
• Unfreezing numpy version
• Adding probabilistic brain regions lookup in ibllib.atlas.atlas.BrainAtlas.get_labels
• Removing two voxelless areas from beryl atlas
• Minor updates to audio events detection and audio welchogram

Release Notes 2.13

Release Notes 2.13.6 2022-08-02

• Hotfix: don't overwrite full settings if iblrig on untagged version

Release Notes 2.13.5 2022-07-27

• Hotfix: pseudo session biased generation contrast distribution

Release Notes 2.13.4 2022-07-22

• Hotfix: Density displays had non-existing colormaps in snapshots QC tasks
• Generate extra training plots based on psychometric curves

Release Notes 2.13.3 2022-07-01

• Hotfix: fix previous hotfix with incorrect package version number

Release Notes 2.13.2 2022-07-01

• Hotfix: datahandler sets local paths in init

Release Notes 2.13.1 2022-07-01

• Hotfix: globus imports were mixed one.globus and one.remote

Release Notes 2.13.0 2022-06-30

• Deprecated ibllib.version
• Fix Globus patcher
• Add SpikeSorting Loader samples2times function
• Fix atlas.BrainCoordinate.xyz2i functions to not quietly wrap indices out of volume bounds.
• Set jobs to Held if parent jobs are Started or Abandoned as well
• Reverse matplotlib colorbars in density displays

Release Notes 2.12

Release Notes 2.12.2 2022-05-27

• Fixes to plotting in training_status

Release Notes 2.12.1 2022-05-26

• ibllib.pipes.training_status: pipeline to compute training status of mice on local servers, new TrainingStatus task (Mayo)
• Fix swanson regions (Olivier)
• Swanson to Beryl mapping

Release Notes 2.12.0 2022-05-10

• ibllib.atlas: add the Swanson flatmap backend (Olivier)
• ibllib.io.extractors: output of task extractions are trial tables, not individual datasets (Miles)
• Documentation: data release examples (Mayo)
• ibl-neuropixel new repository contains ibllib.dsp, illlib.ephys.neuropixel and ibllib.io.spikeglx modules (Olivier)
• brainbox.task.closed loop get impostor targets to evaluate null distribution (Brandon)
• minimum supported version of Python is 3.8 (Michele)

Release Notes 2.11

Release Notes 2.11.1 2022-04-12

• Set video compression jobs to priority 90
• Set jobs to Held if parents are Waiting

Release Notes 2.11.0 2022-04-08

• brainbox.io.one.SpikeSortingLoader: option to load using collection argument
• Restructuring of how jobs are run on local servers, run large jobs as service

Release Notes 2.10

Release Notes 2.10.6 2022-03-15

• Allow parent tasks to be 'Incomplete' to run task on local server
• Change one base_rul for dlc_qc_plot on cortexlab

Release Notes 2.10.5 2022-03-11

• Fix moot release accident

Release Notes 2.10.4 2022-03-11

• Data handler connects to correct alyx database on cortexlab

Release Note 2.10.3 2022-03-09

• Fixes to EphysPostDLC
• Small change to storing in dsp.voltage.decompress_destripe_cbin function

Release Notes 2.10.2 2022-02-28

• Small fixes to local server task queues

Release Notes 2.10.1 2022-02-22

• Authenticate alyx user in Task class
• Some fixes to make dlc_qc_plot in EphysPostDLC more reliable
• SpikeGlx:
  • supports reading of flat binary files without meta data
• atlas.Regions:
  • add support for region ordering in BrainRegions object
  • bugfix in transposing of RGB image data for coronal slice plots
• voltage: decompress cbin
  • add support for custom spikeglx.Reader

Release Notes 2.10.0 2022-02-11

• Fix in EphysDLC task to not return multiple copies of outputs
• Loading examples for different IBL data types
• Fix for probe syncing when Nidq and probe pulses don't match
• Account for new ONE tables in ond datahandler
• Add bad channels plots into RawEphysQc task

Release Notes 2.9

Release Notes 2.9.1 2022-01-24

• deprecation warnings and documentation for spike sorting loading method
• bugfix: remove lru_cache on AllenAtlas class for iblviewer

Release Notes 2.9.0 2022-01-24

• Adding EphysDLC task in ephys_preprocessing pipeline
• NOTE: requires DLC environment to be set up on local servers!
• Fixes to EphysPostDLC dlc_qc_plot

Release Notes 2.8

Release Notes 2.8.0 2022-01-19

• Add lfp, aprms, spike raster and behaviour report plots to task infastructure
• Computation of apRMS in decompress_destripe_cbin before conversion to normalised units
• Add SpikeSortingLoader class in brainbox.io.one

Release Notes 2.7

Release Notes 2.7.1 2022-01-05

• Fixes and better logging for EphysPostDLC task

Release Notes 2.7.0 2021-12-20

• Remove atlas instantiation from import of histology module
• Write electodeSites dataset at end of histology and spikesorting pipelines if insertion resolved
• Removal of native test train support and improved documentation / tests for GLMs
• LFP destriping functions
• Version number is now obtained from ibllib/init.py

Release Notes 2.6

Release Notes 2.6.0 2021-12-08

• New ReportSnapshot class
• DLC QC plots, as part of EphysPostDLC task
• BadChannelsAP plots for ephys QC
• Fix typo in camera extractor

Release Notes 2.5

Release Notes 2.5.1 2021-11-25

• SpikeSorting task overwrites old tar file on rerun

Release Notes 2.5.0 2021-11-24

• Snapshot class to register images as notes in Alyx
• Ephys pipeline: RawEphysQC: outputs channels labels as part of pipeline
• Ephys pipeline: spike sorting
  • dsp.voltage: destripe detects bad channels and interpolate them
  • synchronisation 3B: gives a 10% leeway to throw exception
  • spikes sorting task outputs the RMS plot

Release Notes 2.4

Release Notes 2.4.0

• Setting tasks to Waiting if they encountered lock (status -2)
• Setting tasks to Incomplete if they return status -3
• Completed tasks set held dependent tasks to waiting
• Adding PostDLC task to compute pupil diameters, licks and DLC QC

Release Notes 2.3

Release Notes 2.3.2 2021-11-08

• Trial wheel extraction: use alternative sync method when first on fails
• bugfix: timer for tasks was returning None

Release Notes 2.3.1 2021-11-5

• Add setup method in tasks.py into try except to catch globus download errors

Release Notes 2.3.0 2021-11-4

• Add input and output signatures to all ephys tasks
• Add datahandler to task to download and upload data based on location where task is run
• Spike sorting and EphysVideoSyncQc download data on local servers if not available
• brainbox.io.one load_spike_sorting_fast: bugfix returns acronyms
• creates sequence files for spikesorting
• GPU tasks have a lock - local data handler doesn't instantiate one

Release Notes 2.2

Release Notes 2.2.1 2021-11-02

• allows more than 2 probes in ephys computer probe creation

Release Notes 2.2.0 2021-10-29

• brainbox.io.one load_spike_sorting fast: merge channel info in clusters
• brainbox.io.one generic function to interpolate channels after alignment
• dsp: spike detection by voltage thresholding and cadzow filtering docs
• ibllib.io.spikeglx: get geometry from metadata
• RawEphysQC outputs basic detection spike rates
• ibllib.atlas.regions: add new mapping cosmos and revise Beryl

Release Notes 2.1

Release Notes 2.1.0 2021-10-05

• destriping as pykilosort internal pre-processing
• NP2 probe framework for splitting shanks and LFP band
• Extension of task module to rerun from different locations

Release Notes 2.1.1 2021-10-06

• RawEphysQC tasks computes median RMS from samples for .ap data (stored in _iblqc_ephysChannels.RMS)
• New EphysQC class

Release Notes 2.1.2 2021-10-14

• Fix issue with RawEphysQC that was not looking in local Subjects folder for data
• Fix ensure_required_data in DlcQc

Release Notes 2.1.3 2021-10-19

• Split jobs.py run function in two, one running large tasks (video compression, dlc, spike sorting), one the rest
• Ensure RawEphysQC runs both probes if one fails

Release Notes 2.0

Release Notes 2.0.1 2021-08-07

• pykilosort error handling

Release Notes 2.0.2 2021-08-31

• passive extraction robust to frame2ttl flickers

Release Notes 2.0.3 2021-09-03

• pykilosort bugfix after low yield results

Release Notes 2.0.4 2021-09-10

• ephys trials extraction when audio FPGA starts on up state

Release Notes 2.0.5 2021-09-13

• pykilosort pipeline: output correct version number / fix log file name

Release Notes 2.0.6 2021-09-14

• extraction fixes: passive extraction spacers and camera times format
• small bugfix for the sequential selection

Release Notes 2.0.7 2021-09-15

• ephys trials extraction: audio from FPGA: up-state and TTLs cleanup from camera wiring
• passive extraction: improved spacers detection
• ephyscellsqc: bugfix on probe creation for pykilosort subfolder
• task version 6.2.5 specific audio extractor
• camera times: index by video length when GPIO unstable

Release Notes 2.0.6 - 2021-09-14

• extraction fixes: passive extraction spacers and camera times format
• small bugfix for the sequential selection

Release Notes 2.0.5 - 2021-09-13

• pykilosort pipeline: output correct version number / fix log file name

Release Notes 2.0.4 - 2021-09-10

• ephys trials extraction when audio FPGA starts on up state

Release Notes 2.0.3 - 2021-09-03

• pykilosort bugfix after low yield results

Release Notes 2.0.2 - 2021-08-31

• passive extraction robust to frame2ttl flickers

Release Notes 2.0.1 - 2021-08-07

• pykilosort error handling

Release Notes 2.0.0 - 2021-08-04

• ONE2 released on the master branch
• Pykilosort is the new default spike sorter in the pipeline

Release Notes 1.12 (not released yet)

Release Notes 1.12.0

• oneibl and alf now deprecated; moved to separate repository
• other ibllib miscellanea moved to iblutil repository
• test database is now configurable with an env var

Release Notes 1.11

Release Notes 1.11.0

• brainbox.modeling: New API for using linear and poisson models, doing
  sequential feature selection.
• bugfix: spike sorting continues even if probe information not entered (personal projects)
• TaskQC iteration: aggregation excludes iti delays, frame2ttl clean signal used to compute QC instead of raw

Release Notes 1.10

Release Notes 1.10.4 - 2021-05-15

• optogenetics for UCL task variant
• check Nvidia status before launching spike sorting

Release Notes 1.10.2 / 1.10.3

• fix public one params

Release Notes 1.10.1

• fix ks2 logging output
• set default one params to public credentials

Release Notes 1.10.0 - 2021-05-04

• TaskQC: exclude stim_freeze from overall session qc
• Get modality from task type to include personal projects
• Bugfix Atlas ccf coordinates order
• Tool to label Critical sessions / insertions reasons

Release Notes 1.9

Release Notes 1.9.1 - 2021-04-19

• Successful ONE setup on instantiation when params file doesn't exist

Release Notes 1.9.0 - 2021-04-15

• Video QC optimization
• New task types for widefield imaging
• Add revision and default dataset to register dataset
• Fix for camera extraction failure for novel protocols
• CameraQC wheel alignment check more robust to short videos and large alignment failures
• Parameters raises FileNotFound error if no defaults are provided

Release Notes 1.8

Release Notes 1.8.0 - 2021-03-30

• opto-ephys tasks
• DLC QC
• Do not block session creation if error

Release Notes 1.7

Release Notes 1.7.0 - 2021-03-16

• Removed deprecated numpy dtypes
• Fix trajectory query bug
• GPIO pin states loaded as bool array
• Machine info and log append in Task class
• Only delete session flag file after processing
• Extractor support for widefield imaging protocol

Release Notes 1.6

Release Notes 1.6.2 - 2021-03-18

• hotfix: fix test_modelling error after removing pytorch

Release Notes 1.6.1 - 2021-03-17

• hotfix: maintain compatibility with Python 3.7

Release Notes 1.6.0 - 2021-03-16

• ibllib.atlas: backend region mapping is a lateralized version of the Allen atlas "Allen-lr"
  The default behaviour in the Atlas is to remap according to the original mapping "Allen"
• camera timestamps: extraction of the exact camera time stamps by cross-examination of bonsai timestamps, audio pulses
  and camera pulses

Release Notes 1.5

Release Notes 1.5.39

• ephys extraction: remove short TTL pulses of frame2ttl in task extraction

Release Notes 1.5.38

• ibllib.atlas.regions remapping option
• optogenetics pybpod: extraction of laser probabilities

Release Notes 1.5.37

• MANIFEST.in fix
  • Added fixtures file for passive protocol extraction

Release Notes 1.5.36

• Amplitudes fix:
  • sync_probes doesn't require raw binary file and looks for meta-data files instead
  • tar file of intermediate spike sorting results gets uploaded on flatiron
• Ephys Task extraction: ephys extraction doesn't fail when bpod started before ephys

Release Notes 1.5.35

• histology: brain atlas can handle insertion on the sides
• optogenetics dataset type _ibl_trials.laser_stimulation for training sessions

Release Notes 1.5.34

• spikeglx analog sync thresholding removes DC offset option
• ephys FPGA behaviour extraction: bpod sync with FPGA assumes possible missing FPGA fronts

Release Notes 1.5.32

• Ephys pipeline:
  • passive extractor
  • units QC

Release Notes 1.5.31/1.5.33 Hotfix

• add the available space of system and raid volumes in local servers reports.

Release Notes 1.5.30

• following flatiron server change, add auto-redirect to https

Release Notes 1.5.29

• add widefieldChoiceworld tasks for pipeline

Release Notes 1.5.28

• add opto laser tasks for pipeline

Release Notes 1.5.27

• register ch when mtscomp runs properly
• probes_description runs even if .cbin doesn't exist

Release Notes 1.5.26 Hotfix

• allows dataset registration on errored task

Release Notes 1.5.25 Hotfix

• ks2task import conflict fix

Release Notes 1.5.24 Hotfix

• Ks2 task does not depend on ephys pulses

Release Notes 1.5.23

• Ephys mtscomp
  • ks2 task registers first probe even if one failing
  • mtscomp task register .ch and .meta even if .cbin doesn't exist
  • move ibllib tests to tests_ibllib
  • brainbox atlas plot functions

Release Notes 1.5.22

• Ephys extraction:
  • synchronisation between probes computed in the ephysPulses job
  • spike sorting resync done directly after KS2 output
  • unit-based metrics have their own task

Release Notes 1.5.21: hotfix

• create local server tasks only on raw_session.flag

Release Notes 1.5.20

• ephys alignment QC
• hotfix: ibl errors inherit Exception, not BaseException
• hotfix: partial qc task extractor keeps FPGA stim times

Release Notes 1.5.19

• create tasks looks for create_me.flags

Release Notes 1.5.18

• add Karolina's optogenetics tasks for extractions

Release Notes 1.5.17

• histology probe QC pipeline and final locations dataset export

Release Notes 1.5.16 Hotfix

• numpy needs upgrading >= 1.18

Release Notes 1.5.15 Hotfix

• session creation skips alyx procedure for unknown task protocol (custom projects)

Release Notes 1.5.14

• task extraction:
  • Habituation QC
  • ephys extraction StimOffTimes fix

Release Notes 1.5.13

• ibllib.atlas
  • allen csv Atlas part of package is not installed in dev mode
  • improved slicing performance
• ephys extraction: mtscomp registers ch file on run and re-runs bis

Release Notes 1.5.12 Hotfix

• mtscomp registers ch file on run and re-runs

Release Notes 1.5.11 Hotfix

• ffmpeg nostdin option as jobs were stopped in background on a server

Release Notes 1.5.10

• QC base class
• Support for task QC on FPGA data
• TaskQC run during task extraction

Release Notes 1.5.9

• local server: catches error when subject is not registered in Alyx
• ibllib.atlas.AllenAtlas
  • re-ordered the volumes in c-order contiguous ml-ap-dv efficient coronal shapes
  • top/bottom surface extraction

Release Notes 1.5.8 Hotfix

• Ephys extraction SyncSpikeSorting: specify different dir for ks2 ouput and raw ephys data

Release Notes 1.5.7 Hotfix

• Ephys extraction ks2: mkdir for scratch more robust

Release Notes 1.5.6

Ephys extraction bugfixes:

• RawEphysQC: No object "ephysTimeRmsAP" found
• EphysMtsComp,RawEphysQC, EphysPulses : ValueError: mmap length is greater than file size
• Ks2: ks2 job cleans-up temp dir

Release Notes 1.5.5

• ONE offline mode and cache dataset table to speed up reloading of large datasets (Olivier)
• ALF io naming conventions on loading objects (Miles)
• KS2 Matlab ephys pipeline tasks (Olivier)
• Support for running QC on biased and training sessions (Nico)
• metrics_df and passed_df properties in BpodQC obj for qcplots (Nico)
• Added missing unittest to stim_move_before_goCue metric (Nico)

Release Notes 1.5.4 - 29/07/2020 hotfix

• ibllib.pipes.training_preprocessing.TrainingAudio

Release Notes 1.5.3 - 28/07/2020

• ibllib.pipes.training_preprocessing.TrainingAudio: returns files for registration and proper status. (Olivier)
• ibllib.atlas: compute nearest region from probe trajectory (Mayo)

Release Notes 1.5.2 - 23/07/2020

• Local server jobs:
  • fix wheel moves size mismatch extractor error
  • only look for raw_session.flag for ephys extraction to avoid race conditions

Release Notes 1.5.1 - 25/05/2020

• Ephys extraction:
  • spike amplitudes in Volts
  • added waveforms samples dataset to use Phy from Flatiron datasets
• ONE performance:
  • Metaclass implementation of UniqueSingletons for AlyxClient
  • Multi-threaded downloads
  • Added JSON fields methods to AlyxClient
• QCs: Bpod and ONE QC features, basic plotting, examples

Release Notes 1.4

Release Notes 1.4.14 - 2020-03-09

• Added permutation test, comparing a metric on two sets of datasets, by shuffling labels and seeing how plausible the observed actual difference is
  Sped up calculation of firing_rate
• ephys extraction: updated extracted metrics, including a new contamination estimate and drift metrics.
• ibllib.io.spikeglx

Release Notes 1.4.13 - 2020-03-07

• Hotfix: rig transfer - create probes. One variable used before assignation.

Release Notes 1.4.12 - 2020-03-06

• ONE.load overwrites local file if filesizes different or hash mismatch
• ephys extraction:
  • registration sets the session.procedure field to acute recording
  • bugfix synchronization on re-extraction: always recompute spike.times from spike.samples
  • ephys registration sets the session.procedure field to acute recording
• training extraction:
  • added biasedVisOffChoiceWorld as training extractor
• wheel data
  • dropping support for wheel velocity, not extracted anymore

Release Notes 1.4.11 - 2020-02-14

• bugfix: Include sessions data files for ephys mock
• bugfix: Single probe 3B gets synchronized

Release Notes 1.4.10 - 2020-02-10

• bugfix: Include sessions data files in pip package

Release Notes 1.4.9 - 2020-02-09

• Big brainbox merge and release
• bugfix: clusters.metrics spiking rates accurate
• probability left for ephys choice world contain generative probabilities, not outcomes

Release Notes 1.4.8 - 2020-01-27

• ONE Light Windows fixes
• Installation documentation separates conda and virtualenv options
• Conda yaml environement file for ibllib

Release Notes 1.4.7

• ONE Light for behaviour paper data release
• ONE() standard syntax matching the one light examples

Release Notes 1.4.6

• Alyx registration: add md5, version and filesize to the pipeline registration
• Data Patcher: allows to register data from anywhere through FTP/SSH/GLobus
• ONE Light for behaviour paper data release

Release Notes 1.4.5 Hotfix

• Ephys extraction: left probability bug when sequence was 0 - fixed

Release Notes 1.4.4

• Ephys extraction:
  • left probability extracted properly
  • add robustness to audio fronts extraction in FPGA
• Passive stimulus: raw data registered in pipeline
• Training extraction: microphone extraction for habituation sessions
• ALF: specific to_dataframe method for Bunch

Release Notes 1.4.3 Hotfix

• Ephys extraction: handle fringe case where recording is interrupted in the middle
• Wheel extraction: if rotary encoder version is outdated and stores data in the wrong unit, auto-detect and output in seconds even for new versions

Release Notes 1.4.2

• FPGA/bpod events synchronization performed even when their counts do not match
• Updated requirement versions for mtscomp and phylib

Release Notes 1.4.1

• wheel extraction outputs a timestamps attribute, not times
• make the wheel extraction more robust

Release Notes 1.4.0

• Ephys extraction:
  • un-synchronized spike sortings not uploaded on flat-iron
  • reaction times extracted
  • valve-open times bugfix
• Wheel extraction:
  • training wheel position and timing are now correct
  • ephys & training: units: radians mathematical convention
• ONE:
  • Alyx client handles pagination seamlessly

Release Notes 1.3

Release Notes 1.3.10 Hotfix

• cross-platform get of session folder for rig computer copy to server

Release Notes 1.3.9

• spikeglx.verify_hash() method to check file integrity after transfers/manipulations
• create wirings settings files on ephys computer transfer to server

Release Notes 1.3.8

• Ephys Extraction:
  • duplicate probe.trajectories bugfix
  • extraction works with unoperational fram2ttl at beginning of ephys session
  • clusters.metrics.csv has consistent size with npy cluster objects
  • ephys transfer: create ephys extraction flags after the transfer is complete

Release Notes 1.3.7 hotfixes- 21-NOV-2019

• Rename spike.times on failed sync to reflect the clock as per ALF convention
• sync 3A fails if first cam event whithin 200ms of start
• compress ephys goes through a tempfile to not interfere with transfers/globbing

Release Notes 1.3.6 - 20-NOV-2019

• Ephys Extraction (phylib)
  • convert ks2 amplitudes to volts for spikes.amps, clusters.amps. templates.waveforms, clusters.waveforms to get uV
  • generates Cluster UUIDs file
  • individual spike depths computed from PC features
• Ephys Synchronization
  • use frame2TTL split for 3A by default, if not found look for right_camera
  • output individual probe sync in ALF timestamps format

Release Notes 1.3.5 - 18-NOV-2019

• registration ignores ks2alf probes subfolders
• fix typo in raw qc dataset types

Release Notes 1.3.4 - 18-NOV-2019

• Alyx registration adds the relative path to session root as dataset.subcollection
• Ephys extraction:
  • split probe folders output alf/probe00 and alf/probe01 instead of merge
  • outputs templates.waveforms and clusters.waveforms in sparse arrays
  • outputs probes.description and probes.trajectory
  • renamed the raw ephys QC output
  • outputs clusters.metrics
• Bugfixes:
  • 3B raw ephys QC output ap.file not found on nidq object
  • 3A sync probe threshold set to 2.1 samples

Release Notes 1.3.1 - 12-NOV-2019

• transfer scripts from ephys/video/rig computers to local servers
• bugfix spigeglx.glob_ephys_files when metadata file without ap.bin file

Release Notes 1.3.0 - 11-NOV-2019

• Transfer rig data takes into account session type (ephys/training) to create flags
• Ephys video compression in pipeline
• Ephys audio compression in pipeline

Release Notes 1.2

Release Notes 1.2.9

• Ephys extraction: provide full 3B default wirings if files do not exist.

Release Notes 1.2.8

• Ephys extraction: merge sync ephys in the pipeline overwrites ks2_alf directory if it already exists

Release Notes 1.2.7

• Added biasedScanningChoiceWorld task to biased extractor for Zador lab

Release Notes 1.2.6

feature/ephys_compression

• spikeglx.Reader supports mtscomp ephys binaries
• server pipeline for compression of ephys files

feature/peths

• brainbox.singlecell.peths with tests
• brainbox.processing.bincount2D supports aggregation on fixed scale
• simple examples script and notebook

Release Notes 1.2.5

• examples/brainbox/plot_peths.py: by Matt. W.
• examples/brainbox/rasters by Michaël S.
• Allen Atlas framework and probe registration base functions
• server pipeline for audio extraction of training sessions