Skip to content

Use all available cores #21

@simonschmidt

Description

@simonschmidt

json2csv uses a lot of CPU, but only one core. json.Unmarshal seems to be taking up most of the time so I made a forked version that launches a whole bunch of them in parallel

A 207Mb json file with 241804 lines takes 15s to convert to csv with the single-threaded version, using all cores (2 real + 2 hyperthreading) it takes 7s. However using just one core on the multi-threaded version takes 17s It might be worth to keep the single-threaded version in the code and use that when running on one core.

parallel version

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions