|
1 | | -# Template for interTwin repositories |
| 1 | + |
2 | 2 |
|
3 | | -This repository is to be used as a repository template for creating a new interTwin |
4 | | -repository, and is aiming at being a clean basis promoting currently accepted |
5 | | -good practices. |
| 3 | +## :information_source: Overview |
6 | 4 |
|
7 | | -It includes: |
| 5 | +### Introduction |
| 6 | +InterLink aims to provide an abstraction for the execution of a Kubernetes pod on any remote resource capable of managing a Container execution lifecycle. |
| 7 | +We target to facilitate the development of provider specific plugins, so the resource providers can leverage the power of virtual kubelet without a black belt in kubernetes internals. |
8 | 8 |
|
9 | | -- License information |
10 | | -- Copyright and author information |
11 | | -- Code of conduct and contribution guidelines |
12 | | -- Templates for PR and issues |
13 | | -- Code owners file for automatic assignment of PR reviewers |
14 | | -- [GitHub actions](https://github.com/features/actions) workflows for linting |
15 | | - and checking links |
| 9 | +The project consists of two main components: |
16 | 10 |
|
17 | | -Content is based on: |
| 11 | +- __A Kubernetes Virtual Node:__ based on the [VirtualKubelet](https://virtual-kubelet.io/) technology. Translating request for a kubernetes pod execution into a remote call to the interLink API server. |
| 12 | +- __The interLink API server:__ a modular and pluggable REST server where you can create your own Container manager plugin (called sidecars), or use the existing ones: remote docker execution on a remote host, singularity Container on a remote SLURM batch system. This repo aims to maintain the SLURM sidecar as a standalone plugin. |
18 | 13 |
|
19 | | -- [Contributor Covenant](http://contributor-covenant.org) |
20 | | -- [Semantic Versioning](https://semver.org/) |
21 | | -- [Chef Cookbook Contributing Guide](https://github.com/chef-cookbooks/community_cookbook_documentation/blob/master/CONTRIBUTING.MD) |
| 14 | +The project got inspired by the [KNoC](https://github.com/CARV-ICS-FORTH/knoc) and [Liqo](https://github.com/liqotech/liqo/tree/master) projects, enhancing that with the implemention a generic API layer b/w the virtual kubelet component and the provider logic for the container lifecycle management. |
22 | 15 |
|
23 | | -## GitHub repository management rules |
| 16 | +## :electron: Usage |
24 | 17 |
|
25 | | -All changes should go through Pull Requests. |
| 18 | +### :bangbang: Requirements |
| 19 | +- __[Our Kubernetes Virtual Node and the interLink API server](https://github.com/interTwin-eu/interLink)__ |
| 20 | +- __[The Go programming language](https://go.dev/doc/install)__ (to build binaries) |
| 21 | +- __[Docker Engine](https://docs.docker.com/engine/)__ (optional) |
26 | 22 |
|
27 | | -### Merge management |
| 23 | +Note: if you want a quick start setup (using a Docker container), Go is not necessary |
28 | 24 |
|
29 | | -- Only squash should be enforced in the repository settings. |
30 | | -- Update commit message for the squashed commits as needed. |
| 25 | +### :fast_forward: Quick Start |
| 26 | +Just run: |
| 27 | +```bash |
| 28 | +cd docker && docker compose up -d |
| 29 | +``` |
31 | 30 |
|
32 | | -### Protection on main branch |
| 31 | +### :hammer: Building binaries |
| 32 | +It is of course possible to use binaries as a standalone application. Just run |
| 33 | +```bash |
| 34 | +make all |
| 35 | +``` |
| 36 | +and you will be able to find the built slurm-sd binary inside the bin directory. Before executing it, remember to check if the configuration file is correctly set according to your needs. You can find an example one under examples/config/InterLinkConfig.yaml. Do not forget to set the INTERLINKCONFIGPATH environment variable to point to your config. |
33 | 37 |
|
34 | | -To be configured on the repository settings. |
| 38 | +### :pencil2: Annotations |
| 39 | +It is possible to specify Annotations when submitting Pods to the K8S cluster. A list of all Annotations follows: |
| 40 | +| Annotation | Description| |
| 41 | +|--------------|------------| |
| 42 | +| slurm-job.vk.io/singularity-commands | Used to add specific Commands to be executed before the actual SLURM Job starts. It adds Commands on the Singularity exection line, in the SLURM bastch file | |
| 43 | +| slurm-job.vk.io/pre-exec | Used to add commands to be executed before the Job starts. It adds a command in the SLURM batch file after the #SBATCH directives | |
| 44 | +| slurm-job.vk.io/singularity-mounts | Used to add mountpoints to the Singularity Containers | |
| 45 | +| slurm-job.vk.io/singularity-options | Used to specify Singularity arguments | |
| 46 | +| slurm-job.vk.io/image-root | Used to specify the root path of the Singularity Image | |
| 47 | +| slurm-job.vk.io/flags | Used to specify SLURM flags. These flags will be added to the SLURM script in the form of #SBATCH flag1, #SBATCH flag2, etc | |
| 48 | +| slurm-job.vk.io/mpi-flags | Used to prepend "mpiexec -np $SLURM_NTASKS \*flags\*" to the Singularity Execution | |
35 | 49 |
|
36 | | -- Require pull request reviews before merging |
37 | | - - Dismiss stale pull request approvals when new commits are pushed |
38 | | - - Require review from Code Owners |
39 | | -- Require status checks to pass before merging |
40 | | - - GitHub actions if available |
41 | | - - Other checks as available and relevant |
42 | | - - Require branches to be up to date before merging |
43 | | -- Include administrators |
| 50 | +### :wrench: InterLink Config file |
| 51 | +Detailed explanation of the InterLink config file key values. Edit the config file before running the binary or before building the docker image (`docker compose up -d --build --force-recreate` will recreate and re-run the updated image) |
| 52 | +| Key | Value | |
| 53 | +|--------------|-----------| |
| 54 | +| InterlinkURL | the URL to allow the Virtual Kubelet to contact the InterLink module. | |
| 55 | +| SidecarURL | the URL to allow InterLink to communicate with the Sidecar module (docker, slurm, etc). Do not specify port here | |
| 56 | +| InterlinkPort | the Interlink listening port. InterLink and VK will communicate over this port. | |
| 57 | +| SidecarPort | the sidecar listening port. Sidecar and Interlink will communicate on this port. Set $SIDECARPORT environment variable to specify a custom one | |
| 58 | +| SbatchPath | path to your Slurm's sbatch binary | |
| 59 | +| ScancelPath | path to your Slurm's scancel binary | |
| 60 | +| CommandPrefix | here you can specify a prefix for the programmatically generated script (for the slurm plugin). Basically, if you want to run anything before the script itself, put it here. | |
| 61 | +| ExportPodData | Set it to true if you want to export Pod's ConfigMaps and Secrets as mountpoints in your Singularity Container | |
| 62 | +| DataRootFolder | Specify where to store the exported ConfigMaps/Secrets locally | |
| 63 | +| Namespace | Namespace where Pods in your K8S will be registered | |
| 64 | +| Tsocks | true or false values only. Enables or Disables the use of tsocks library to allow proxy networking. Only implemented for the Slurm sidecar at the moment. | |
| 65 | +| TsocksPath | path to your tsocks library. | |
| 66 | +| TsocksLoginNode | specify an existing node to ssh to. It will be your "window to the external world" | |
| 67 | +| BashPath | Path to your Bash shell | |
| 68 | +| VerboseLogging | Enable or disable Debug messages on logs. True or False values only | |
| 69 | +| ErrorsOnlyLogging | Specify if you want to get errors only on logs. True or false values only | |
| 70 | + |
| 71 | +### :wrench: Environment Variables list |
| 72 | +Here's the complete list of every customizable environment variable. When specified, it overwrites the listed key within the InterLink config file. |
| 73 | + |
| 74 | +| Env | Value | |
| 75 | +|--------------|-----------| |
| 76 | +| VK_CONFIG_PATH | VK config file path | |
| 77 | +| INTERLINKURL | the URL to allow the Virtual Kubelet to contact the InterLink module. Do not specify a port here. Overwrites InterlinkURL. | |
| 78 | +| INTERLINKPORT | the InterLink listening port. InterLink and VK will communicate over this port. Overwrites InterlinkPort. | |
| 79 | +| INTERLINKCONFIGPATH | your InterLink config file path. Default is `./kustomizations/InterLinkConfig.yaml` | |
| 80 | +| SIDECARURL | the URL to allow InterLink to communicate with the Sidecar module (docker, slurm, etc). Do not specify port here. Overwrites SidecarURL. | |
| 81 | +| SIDECARPORT | the Sidecar listening port. Docker default is 4000, Slurm default is 4001. | |
| 82 | +| SBATCHPATH | path to your Slurm's sbatch binary. Overwrites SbatchPath. | |
| 83 | +| SCANCELPATH | path to your Slurm's scancel binary. Overwrites ScancelPath. | |
| 84 | +| VKTOKENFILE | path to a file containing your token fot OAuth2 proxy authentication. Overwrites VKTokenFile. | |
| 85 | +| SHARED_FS | set this env to "true" to save configmaps values inside files directly mounted to Singularity containers instead of using ENVS to create them later | |
| 86 | +| CUSTOMKUBECONF | path to a service account kubeconfig | |
| 87 | +| TSOCKS | true or false, to use tsocks library allowing proxy networking. Working on Slurm sidecar at the moment. Overwrites Tsocks. | |
| 88 | +| TSOCKSPATH | path to your tsocks library. Overwrites TsocksPath. | |
0 commit comments