Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 23 additions & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ on: [ push, pull_request ]

jobs:

build:
tests:
runs-on: ubuntu-latest
strategy:
matrix:
Expand Down Expand Up @@ -35,3 +35,25 @@ jobs:
with:
token: ${{ secrets.CODECOV_TOKEN }}

pre-commit:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

- name: Set up Python 3.11
uses: actions/setup-python@v5
with:
python-version: '3.11'

- name: Install poetry
run: pipx install poetry

- name: Install python dependencies
run: |
poetry env use 3.11
poetry install --with pre-commit,dev

- name: Run pre-commit
run: |
poetry run pre-commit install
poetry run pre-commit run --all-files || ( git status --short ; git diff ; exit 1 )
20 changes: 20 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
# Install pre-commit hooks via:
# pre-commit install
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v5.0.0
hooks:
- id: end-of-file-fixer
- id: mixed-line-ending
- id: trailing-whitespace
- id: check-json

- repo: https://github.com/astral-sh/ruff-pre-commit
# Ruff version.
rev: v0.9.6
hooks:
# Run the linter.
- id: ruff
args: [ --fix ]
# Run the formatter.
- id: ruff-format
2 changes: 0 additions & 2 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -13,5 +13,3 @@ RUN python setup.py develop
#CMD ["/bin/bash"]
CMD [ "python", "abcd/server/__init__.py" ]
#CMD ["gunicorn", "-w 4", "-b 0.0.0.0:8000", "server:app"]


24 changes: 12 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,17 +3,17 @@
[![Doc](https://img.shields.io/badge/docs-master-green.svg)](https://libatoms.github.io/abcd/)
[![Build Status](https://travis-ci.org/libAtoms/abcd.svg?branch=master)](https://travis-ci.org/libAtoms/abcd)

Database storage and discovery of atomistic data.
Database storage and discovery of atomistic data.

Take a look at the `examples.md` file for.. examples!

Main features:

- Configurations that consist of atom positions, elements, forces, and various metadata are stored as a dictionary by a MongoDB backend.
- There is no predefined schema, any combination of keys are allowed for all configurations.
- Two modes: "discovery" and "download". Both use filter-type queries, but in "discovery" mode, summary statistics of the configurations that pass the filter are reported. In "download" mode, the matching configurations are downloaded and exported to a file.
- The "discovery" mode can be used to learn what keys exist in the set of configurations that have passed the current quiery filter. The user can use this to refine the query.
- Complex queries on dictionary key-value pairs are allowed, and their logical combinations.
- Configurations that consist of atom positions, elements, forces, and various metadata are stored as a dictionary by a MongoDB backend.
- There is no predefined schema, any combination of keys are allowed for all configurations.
- Two modes: "discovery" and "download". Both use filter-type queries, but in "discovery" mode, summary statistics of the configurations that pass the filter are reported. In "download" mode, the matching configurations are downloaded and exported to a file.
- The "discovery" mode can be used to learn what keys exist in the set of configurations that have passed the current quiery filter. The user can use this to refine the query.
- Complex queries on dictionary key-value pairs are allowed, and their logical combinations.

## Installation

Expand All @@ -24,13 +24,13 @@ $ pip install git+https://github.com/libAtoms/abcd.git

## Setup

If you have an already running mongo server, or install your own, they you are ready to go. Alternatively,
If you have an already running mongo server, or install your own, they you are ready to go. Alternatively,

```
docker run -d --rm --name abcd-mongodb -v <path-on-your-machine-to-store-database>:/data/db -p 27017:27017 mongo
```

will download and install a docker and run a database in it.
will download and install a docker and run a database in it.

To connect to a mongodb that is already running, use
```
Expand All @@ -56,7 +56,7 @@ You can set up an `abcd` user on your machine where the database is running, and
command="/path/to/abcd --remote ${SSH_ORIGINAL_COMMAND}",no-port-forwarding,no-X11-forwarding,no-agent-forwarding,no-pty ssh-rsa <public-key> your@email
```

Then you'll be able to access the database remotely using, e.g.
Then you'll be able to access the database remotely using, e.g.
```
ssh abcd@your.machine summary
```
Expand All @@ -74,18 +74,18 @@ To access it:
docker pull stenczelt/projection-abcd:latest
```

2. create a docker network, which enables the containers to communicate with each other and the outside world as well
2. create a docker network, which enables the containers to communicate with each other and the outside world as well
```
docker network create --driver bridge abcd-network
```

3. run the mongo (ABCD) and the visualiser as well
```
docker run -d --rm --name abcd-mongodb-net -v <path-on-your-machine-to-store-database>:/data/db -p 27017:27017 --network abcd-network mongo

docker run -it --rm --name visualiser-dev -p 9999:9999 --network abcd-network stenczelt/projection-abcd
```
NB: You need a a directory where the database files are kept locally and you need to connect this to the mongo
NB: You need a a directory where the database files are kept locally and you need to connect this to the mongo
container. More info about this can be found in the original ABCD repo

This will start the visualiser with ABCD integration! Have fun!
Expand Down
26 changes: 8 additions & 18 deletions abcd/__init__.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
from enum import Enum
import logging
from urllib import parse
from enum import Enum

logger = logging.getLogger(__name__)

Expand All @@ -10,7 +10,7 @@ class ConnectionType(Enum):
http = 2


class ABCD(object):
class ABCD:
@classmethod
def from_config(cls, config):
# Factory method
Expand All @@ -24,7 +24,6 @@ def from_url(cls, url, **kwargs):
logger.info(r)

if r.scheme == "mongodb":

conn_settings = {
"host": r.hostname,
"port": r.port,
Expand All @@ -39,20 +38,19 @@ def from_url(cls, url, **kwargs):
from abcd.backends.atoms_pymongo import MongoDatabase

return MongoDatabase(db_name=db, **conn_settings, **kwargs)
elif r.scheme == "mongodb+srv":
if r.scheme == "mongodb+srv":
db = r.path.split("/")[1] if r.path else None
db = db if db else "abcd"
from abcd.backends.atoms_pymongo import MongoDatabase

return MongoDatabase(db_name=db, host=r.geturl(), uri_mode=True, **kwargs)
elif r.scheme == "http" or r.scheme == "https":
if r.scheme == "http" or r.scheme == "https":
raise NotImplementedError("http not yet supported! soon...")
elif r.scheme == "ssh":
if r.scheme == "ssh":
raise NotImplementedError("ssh not yet supported! soon...")
else:
raise NotImplementedError(
"Unable to recognise the type of connection. (url: {})".format(url)
)
raise NotImplementedError(
f"Unable to recognise the type of connection. (url: {url})"
)


if __name__ == "__main__":
Expand All @@ -62,11 +60,3 @@ def from_url(cls, url, **kwargs):
url = "mongodb://mongoadmin:secret@localhost:27017/abcd_new"
abcd = ABCD.from_url(url)
abcd.print_info()

# from ase.io import iread
# for atoms in iread('../tutorials/data/bcc_bulk_54_expanded_2_high.xyz', index=slice(1)):
# # Hack to fix the representation of forces
# atoms.calc.results['forces'] = atoms.arrays['force']
#
# abcd.push(atoms)
# print(atoms)
20 changes: 8 additions & 12 deletions abcd/backends/atoms_http.py
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
import json
import logging
import requests
from os import linesep
from typing import List

import ase
import requests

from abcd.backends.abstract import Database

logger = logging.getLogger(__name__)
Expand Down Expand Up @@ -44,17 +44,15 @@ def pull(self, query=None, properties=None):
def query(self, query_string):
pass

def search(self, query_string: str) -> List[str]:
results = requests.get(self.url + "/calculation").json()
return results
def search(self, query_string: str) -> list[str]:
return requests.get(self.url + "/calculation").json()

def get_atoms(self, id: str) -> Atoms:
data = requests.get(self.url + "/calculation/{}".format(id)).json()
atoms = Atoms.from_dict(data)
return atoms
data = requests.get(self.url + f"/calculation/{id}").json()
return Atoms.from_dict(data)

def __repr__(self):
return "ABCD(type={}, url={}, ...)".format(self.__class__.__name__, self.url)
return f"ABCD(type={self.__class__.__name__}, url={self.url}, ...)"

def _repr_html_(self):
"""jupyter notebook representation"""
Expand All @@ -67,9 +65,7 @@ def print_info(self):
[
"{:=^50}".format(" ABCD Database "),
"{:>10}: {}".format("type", "remote (http/https)"),
linesep.join(
"{:>10}: {}".format(k, v) for k, v in self.db.info().items()
),
linesep.join(f"{k:>10}: {v}" for k, v in self.db.info().items()),
]
)

Expand Down
Loading