Skip to content

Fitting the CRFTagger on NER task with a lot of training data fails on too small memory #9

@FelixKirsch

Description

@FelixKirsch

Fitting the CRFTagger on NER tasks with large training data needs a lot of memory. The active learning fails if the memory assigned to the docker container is too small. To fix this behavior training data has to be used in batches or maybe something like numpy memmap can be used. Or check how other ml models tackle this issue.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingenhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions