Skip to content

implement 'schema translate' without copying data #33

@thomasdfischer

Description

@thomasdfischer

Problem

There is an edge case in schema translate where, if any part of the input data is parsed as a map[interface{}]interface{}, then data must be copied. This is because all of the JSON/YAML/TOML/etc. packages do not support encoding map[interface{}]interface{} objects. To allow translate to still work, the map[interface{}]interface{} object is converted to a map[string]interface{} object by

  1. Making a new map[string]interface{}
  2. Looping through the original map, inserting all the values into the new map (which has string keys instead of interface{} keys)

The conversion is so costly that translating an 8MB file takes 8 seconds on my ThinkPad T450s. This is not acceptable.

Possible Solutions

  1. Write custom decoders and encoders which automatically attempt to convert interface{} keys to strings.

  2. Find someone who knows more about Go's type system to refactor the current code so that data is cast to the correct type without copying it.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions