Skip to content

Move to blazegraph #83

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 33 commits into
base: dev
Choose a base branch
from
Open

Move to blazegraph #83

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
0d044a8
move to blazegraph
anitacaron Jun 27, 2023
8a18679
create blazegraph dockerfile
anitacaron Jul 18, 2023
7627563
update blazegraph docker image
anitacaron Jul 21, 2023
bd03c53
create RWStore.properties
anitacaron Aug 9, 2023
4753b12
update upload-triplestore docker image
anitacaron Aug 9, 2023
2b1049e
change data to ttl to load into blazegraph
anitacaron Aug 9, 2023
9991b3b
update blazegraph config and dataloader
anitacaron Aug 11, 2023
10b7fe6
update obo context
anitacaron Sep 11, 2023
2dc246d
update data volume directory
anitacaron Sep 11, 2023
8a21d5a
update registry parser
anitacaron Sep 11, 2023
b164840
search entities endpoint by confidence (min and max)
anitacaron Sep 11, 2023
4576c1d
enable search mapping_justification by curie
anitacaron Sep 12, 2023
a50660f
fix lint errors
anitacaron Sep 12, 2023
67b14c3
fix lint errors
anitacaron Sep 12, 2023
1c52908
add PyLD to poetry
anitacaron Sep 12, 2023
8793106
fix lint error in registry_parser
anitacaron Sep 12, 2023
49602f9
fix lint errors in utils
anitacaron Sep 12, 2023
36c0f4e
add mapping_justification_curie to response in mappings endpoints
anitacaron Sep 21, 2023
a7811a1
update packages
anitacaron Nov 16, 2023
a35b010
update query method
anitacaron Nov 16, 2023
c5d57d5
Merge pull request #84 from mapping-commons/anitacaron/issue35
Nov 16, 2023
f9add0f
remove rdf4j config files
anitacaron Nov 17, 2023
9995994
move data to .gitignore
anitacaron Nov 17, 2023
8330801
move config files to config folder
anitacaron Nov 17, 2023
e2c4d7b
improve query
anitacaron Nov 17, 2023
71e2e41
parse local_name
anitacaron Nov 17, 2023
6c9ea4e
update obo context
anitacaron Dec 1, 2023
8c381b7
add rule to update context
anitacaron Dec 1, 2023
be0cf4a
clean docker-compose file
anitacaron Dec 1, 2023
94e65d4
use the env variable `JAVA_OPTS` when running blazegraph and remove n…
anitacaron Jun 27, 2024
3c57ef9
remove unneeded code in the `api.dockerfile` that was moved to `docke…
anitacaron Jun 27, 2024
890c3ff
update sssom dependency to fix numpy `np.NAN` issue
anitacaron Jun 27, 2024
d9163a9
improve query results efficiency for the `entities`, `mappings` and `…
anitacaron Jun 27, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 12 additions & 6 deletions docker-compose.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,14 +15,20 @@ services:
- triplestore
- upload-triplestore
environment:
- SPARQL_ENDPOINT=http://triplestore:8080/rdf4j-server/repositories/sssom
- SPARQL_ENDPOINT=http://triplestore:8889/bigdata/sparql
triplestore:
image: eclipse/rdf4j-workbench:latest
image: lyrasis/blazegraph:2.1.5
environment:
- JAVA_OPTS=-Xms10g -Xmx20g
- RDF4J_DATA=/data
- JVM_MEM=10g
ports:
- 8080:8080
- 8889:8080
volumes:
- ./data:/data:ro
healthcheck:
test: ["CMD", "curl", "-f", "http://triplestore:8889/"]
interval: 3s
timeout: 5s
retries: 3
upload-triplestore:
build:
context: .
Expand All @@ -32,6 +38,6 @@ services:
links:
- triplestore
environment:
- SERVER=http://triplestore:8080
- SERVER=http://triplestore:8889
volumes:
- ./data:/data
16 changes: 8 additions & 8 deletions process.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,22 +7,22 @@ echo "Start: upload-triplestore"
echo "TIME:"
date

SETUP=${WORKSPACE}/rdf4j_sssom.txt
RDF4J=/opt/eclipse-rdf4j-${RDF4J_VERSION}
RDF4JSERVER=${SERVER}/rdf4j-server
# SETUP=${WORKSPACE}/rdf4j_sssom.txt
#BLAZEGRAPH_UID=/opt/eclipse-rdf4j-${RDF4J_VERSION}
BLAZEGRAPHSERVER=${SERVER}/
DATA=/data/ols

if [ `ls $DATA/*.jsonld.gz | wc -l` -lt 1 ]; then echo "ERROR: No data in data directory! Aborting.. " && exit 1; fi

echo 'Waiting for RDF4J server..'
until $(curl --output /dev/null --silent --head --fail ${RDF4JSERVER}); do
echo 'Waiting for BLAZEGRAPH WORKBENCH..'
until $(curl --output /dev/null --silent --head --fail ${BLAZEGRAPHSERVER}); do
printf '.'
sleep 5
done

echo "connect "${RDF4JSERVER}|cat - ${SETUP} > /tmp/out && mv /tmp/out ${SETUP}
cat ${SETUP}
cat ${SETUP} | sh ${RDF4J}/bin/console.sh
# echo "connect "${RDF4JSERVER}|cat - ${SETUP} > /tmp/out && mv /tmp/out ${SETUP}
# cat ${SETUP}
# cat ${SETUP} | sh ${RDF4J}/bin/console.sh

ls -lh $DATA

Expand Down
6 changes: 3 additions & 3 deletions upload.Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
FROM yyz1989/rdf4j:latest
FROM lyrasis/blazegraph:2.1.5

VOLUME /data

ENV WORKSPACE=/opt/SSSOM
WORKDIR /opt/SSSOM

ENV BUILD_OUTPUT=${WORKSPACE}/build.out
# ENV BUILD_OUTPUT=${WORKSPACE}/build.out

COPY process.sh /opt/SSSOM/process.sh
COPY rdf4j_sssom.txt /opt/SSSOM/rdf4j_sssom.txt
# COPY rdf4j_sssom.txt /opt/SSSOM/rdf4j_sssom.txt

RUN chmod +x /opt/SSSOM/*.sh

Expand Down