Skip to content

Fix slow loading of policy revisions #326

@romangithub1024

Description

@romangithub1024

Steps to reproduce:

  1. Run ./tools/stress.sh, wait until it generates 500 policy revisions
  2. See that it start running a bit slow (takes a couple of seconds to process a change)
  3. Grab bolt.db, copy it over to /var/lib/aptomi/db.bolt.large, stop stress test
  4. Run stress again using /var/lib/aptomi/db.bolt.large as a starting point and generate tracing information
    DB=/var/lib/aptomi/db.bolt.large TRACE_PROFILE=aptomi.trace ./tools/stress.sh
  5. Run go tool trace aptomi.trace to see the issue

It spends 99% of the time here, performing N^2 operations:

github.com/Aptomi/aptomi/vendor/gopkg.in/yaml%2ev2.Unmarshal:84
github.com/Aptomi/aptomi/pkg/runtime/codec/yaml.(*yamlCodec).decodeOneOrMany:73
github.com/Aptomi/aptomi/pkg/runtime/codec/yaml.(*yamlCodec).DecodeOne:43
github.com/Aptomi/aptomi/pkg/runtime/store/generic/bolt.(*boltStore).List.func1:133
github.com/Aptomi/aptomi/vendor/github.com/boltdb/bolt.(*DB).View:629
github.com/Aptomi/aptomi/pkg/runtime/store/generic/bolt.(*boltStore).List:124
github.com/Aptomi/aptomi/pkg/runtime/store/generic/bolt.(*boltStore).ListGenerations:151
github.com/Aptomi/aptomi/pkg/runtime/store/core.(*defaultStore).GetFirstRevisionForPolicy:29
github.com/Aptomi/aptomi/pkg/api.(*coreAPI).handleRevisionGetByPolicy:37

We need to fix GetFirstRevisionForPolicy as well as GetAllRevisionsForPolicy

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions