diff --git a/docs/README.md b/docs/README.md new file mode 100644 index 00000000..5d183cc9 --- /dev/null +++ b/docs/README.md @@ -0,0 +1,141 @@ + +# Comify docs + +Hi, welcome to the Comify documentation. 👋 + +Here you'll find all the technical details on how we are building the application. We'll start with the fundamentals below. From there you can zoom in at any part for the details. + +## Architecture + +At the heart of any application lies architecture. To explain ours, we found inspiration from the [C4 model](https://c4model.com/) and added our own sauce. + +### Context + +Comify runs as a stand-alone application and hasn't many external dependencies (yet?). The most important one is the `Creator` that represents the end-user. Without it, the application doesn't have any value. Secondly we're outsourcing the identity and access management (IAM) to save on development that doesn't add business value. + +Application context + +**Note:** We use OpenID Connect for the IAM integration. + +### Shadows + +The application has two shadows. Each shadow contains content related to the application. + +Application shadows + +If you're reading this, then you've already found the docs. This is the darkest shadow, because it's the furthest away from the code. The tests are more code related, and can be found in the `test` folder in the project root. The application itself is placed in the `src` folder also in the project root. + +The structure of the shadows mimic the application structure to make navigating them easy. This structure is explained in the containers section below. + +### Containers + +The application is divided into multiple parts, called containers. We have these four. + +Application containers + +Let's look at their usages and responsibilities. + +* [**Web UI**](./webui/) - Contains the web based interface for user interaction (SPA). +* [**Assets**](./assets/) - Contains shared things like images, downloads, etc... +* [**Domain**](./domain/) - Contains business related components like business logic, data entities, etc.. +* [**Integrations**](./integrations/) - Contains implementations for IAM, database, etc.. + +Each part has its own folder in the application (and its shadows). For more detailed information you can click on their names. + +## Technology stack + +The project is 100% [TypeScript](https://www.typescriptlang.org/) based. + +**Runtime requirements** + +* [Node.js](https://nodejs.org/) as runtime (version 20+). +* [Docker](https://www.docker.com/) for additional services. + +**Main dependencies** + +* [Jitar](http://jitar.dev) for E2E communication and scaling concerns. +* [React](https://react.dev) for the frontend. + +All backend components are written in pure TypeScript without a framework. + +**Additional services** + +* [MongoDB](https://www.mongodb.com/) as database. +* [MinIO](https://min.io/) as file store (images). +* [Keycloak](https://www.keycloak.org/) as identity and access manager. + +## Configuration + +The application has several configuration options. You can find them here. + +### Environment + +For both the application and Docker services we've set up environment variables that can be managed from a single configuration file. An example of this file is located in the root of the project with the name `example.env`. + +**Note:** The project uses the `development.env` file for development. + +### Segments + +The domain container can be split into multiple segments for scalability reasons like deployment, load balancing and fail over. This is a feature provided by Jitar, more information on this can be found in the [documentation](https://docs.jitar.dev/deploy/segmentation.html). The configuration files can be found in the `segments` folder in the project root. + +We use a backend for frontend (BFF) segmentation strategy. The `bff.json` file exposes all functions that can be remotely accessed by the frontend. The other segment files are used to demonstrate two deployment strategies: + +1. vertical slicing (`notification.json`) +1. horizontal slicing (`reads.json` and `writes.json`) + +Based on this segmentation model, the application can run in a monolithic and microservice fashion. + +### Resources + +The application uses resources like a database, file store, event broker, etc. that need to be defined in Jitar for sharing them across segments that are running on the same node. More information on this can be found in the [documentation](https://docs.jitar.dev/deploy/resources.html). The configuration files can be found in the `resources` folder in the project root. + +### Services + +Jitar provides [multiple services](https://docs.jitar.dev/fundamentals/runtime-services.html) to deploy applications with different strategies. Currently, we have configurations for running the application as a monolith and as microservices. The configuration files can de found in the `services` folder in the project root. + +The `standalone.json` configuration file configures the monolithic deployment. All other files configure the microservice setup. + +## Getting started + +Before you start, make sure you meet all the runtime requirements. Follow these instructions next. + +1. Run `npm install` from a terminal to install the dependencies. +1. Copy the `example.env` file to a `development.env` file. +1. Run `npm run build` to build the application. +1. Run `npm run standalone` to run Jitar at port 3000. + +To start the application you need to open a new terminal and run `npm run dev`. Now a [Vite](https://vitest.dev/) dev server is running on port 5173. You can then navigate to to open the application. + +To stop the application you can press CONTROL+C in both terminals. + +### Build & run + +To build and run the application you can keep using the `npm run build` and `npm run standalone` commands. For convenience both commands are combined in the following command: `npm run rebuild`. + +### Test & lint + +To run the tests you can use the `npm run test` command. You can also run the linter with the `npm run lint` command. For convenience the **build**, **test** and **lint** commands are combined in the following command: `npm run review`. + +### Run without Vite + +If you want to run the application without a Vite dev server, you can change the port number in the `AUTHENTICATION_CLIENT_URI` environment variable to `3000`. After that you can start the application again as standalone and navigate to to open the application. + +### Run as microservices + +For running the application as microservices, you need to start some additional helper services: + +* Proxy - central access point for the web browser. +* Repository - provider for static assets. +* Gateway - locator and load balancer for all procedures. + +After that, the segment services can be started. You can follow these instructions: + +1. Start the proxy by running `npm run proxy`. +1. Start the repository by running `npm run repository`. +1. Start the gateway by running `npm run gateway`. +1. Start the BFF service by running `npm run bff`. +1. Start the Notification service by running `npm run notification`. +1. Start the Reads service by running `npm run reads`. +1. Start the Writes service by running `npm run writes`. + +You can then navigate to to open the application via the proxy service. diff --git a/docs/assets/README.md b/docs/assets/README.md new file mode 100644 index 00000000..62f52f61 --- /dev/null +++ b/docs/assets/README.md @@ -0,0 +1,6 @@ + +# Assets | Comify docs + +The assets container contains all kind of non-code related things that can be use by the [Web UI](../webui/) and / or the [Domain](../domain/) containers. Currently it only holds the application icon, but sooner or later it will be accompanied by other files like the *privacy statement* and the *terms of use* documents. + +This shadow container serves the same goal. It contains all the images used for the documentation. diff --git a/docs/assets/images/architecture/containers.png b/docs/assets/images/architecture/containers.png new file mode 100644 index 00000000..ab9a08eb Binary files /dev/null and b/docs/assets/images/architecture/containers.png differ diff --git a/docs/assets/images/architecture/context.png b/docs/assets/images/architecture/context.png new file mode 100644 index 00000000..e76451cb Binary files /dev/null and b/docs/assets/images/architecture/context.png differ diff --git a/docs/assets/images/architecture/shadows.png b/docs/assets/images/architecture/shadows.png new file mode 100644 index 00000000..b81b047d Binary files /dev/null and b/docs/assets/images/architecture/shadows.png differ diff --git a/docs/assets/images/domain/modules.png b/docs/assets/images/domain/modules.png new file mode 100644 index 00000000..31cb794e Binary files /dev/null and b/docs/assets/images/domain/modules.png differ diff --git a/docs/assets/images/webui/example.png b/docs/assets/images/webui/example.png new file mode 100644 index 00000000..d7121335 Binary files /dev/null and b/docs/assets/images/webui/example.png differ diff --git a/docs/assets/images/webui/modules.png b/docs/assets/images/webui/modules.png new file mode 100644 index 00000000..fa63f997 Binary files /dev/null and b/docs/assets/images/webui/modules.png differ diff --git a/docs/domain/README.md b/docs/domain/README.md new file mode 100644 index 00000000..17986ca1 --- /dev/null +++ b/docs/domain/README.md @@ -0,0 +1,69 @@ + +# Domain | Comify docs + +The domain contains the implementation of all business related components. It leverages a procedural approach using [Jitar](https://jitar.dev). + +## Module structure + +We're using a feature-oriented approach. Our root module structure looks like this. + +Web UI module structure + +Basically, it boils down to a root module per domain concept. Each module contains its features. + +A module is implemented as a folder containing an index file that exposes its essentials for consumers. + +## Features + +A feature is a sub-module containing all components required for its implementation. Features vary in size, ranging from small (single components) to large (multiple components), depending on the requirements. + +Every feature exposes its main function as the default export through the index file. Additionally, its types, errors, etc., can be exposed if their consumers require them. + +We aim to keep features as autonomous as possible by encapsulating their internal components. If another feature requires the same component (such as retrieving data by ID), we tend to make a copy on the first occurrence and move the component to its own feature upon multiple occurrences. This decision heavily depends on the complexity of the component. + +## Components + +Components can have different types depending on their responsibilities. Currently, we distinct the following types: + +* **Process** - a series of steps, where each step is a separate component. +* **Task** - handles calculations, data manipulations, etc. +* **Persistence** - responsible for querying, inserting, and updating the database. +* **Aggregation** - gathers all related data. +* **Validation** - ensures incoming data meets the required criteria. +* **Event** - handles publications and subscriptions. + +Each component is implemented in its own file and exported as the default. + +A feature typically contains a combination of these component types. Process components that perform multiple database writes follow the [SAGA pattern](https://microservices.io/patterns/data/saga.html) as described in the [Jitar documentation](https://docs.jitar.dev/develop/data-consistency.html). + +## Data + +Data is defined per domain concept. We distinguish between two types: + +* **Persistent** - data stored in the database. +* **Aggregated** - a data view containing all referenced data. + +Both types are defined as TypeScript types with read-only fields to ensure immutability. + +Persistent data is defined in the `types.ts` file in the root of the domain concept folder, as it is used by multiple features. Aggregated data is defined in the `types.ts` file of the aggregation feature, as it is more specific. + +Persistent data is read and written by persistence components using the database integration. + +## Events + +We use a publish/subscribe model for side effects such as updating counters, creating notifications, etc. + +Features that trigger side effects contain both a **publish** and **subscribe** component. Both components use the event broker integration to manage events. + +The publish component is used as the last step in the process. The subscribe component is exported in the feature's index file for use by other features. + +Features leverage a **subscriptions** component that imports and uses the subscribe components of other features. The subscriptions component is also exported in the feature's index file. + +## Definitions + +Every module (domain, concept, and feature) can have two types of definitions: + +* **Types** - TypeScript types such as data and event definitions, defined in the `types.ts` file. +* **Definitions** - constants such as enums, validation schemas, record types, etc., defined in the `definitions.ts` file. + +The location of a definition depends on its scope of use. For example, the base data model is located in the domain root module, a record type is defined per domain concept, and a validation schema is defined per validation feature. diff --git a/docs/integrations/AUTHENTICATION.md b/docs/integrations/AUTHENTICATION.md new file mode 100644 index 00000000..64aff9c1 --- /dev/null +++ b/docs/integrations/AUTHENTICATION.md @@ -0,0 +1,109 @@ + +# Authentication | Comify docs + +The authentication integration provides a universal interaction layer with an actual identity provider solution. + +This integration is based on the following authentication flow: + +1. Browser redirects to the IDP login page. +2. User authenticate at the IDP. +3. IDP provides identity information to this integration. +4. This integration starts a session based on the provided identity. +5. Sessions can be refreshed via this integration. +6. Until the user logs out via this integration. + +## Implementations + +Currently, there is only one implementation: + +* **OpenID** - persistent document storage (used in production). + +## Configuration + +The used implementation needs to be configured in the `.env` file, together with the client URL. + +```env +AUTHENTICATION_IMPLEMENTATION="openid" +AUTHENTICATION_CLIENT_URI="http://localhost:5173/identify" +``` + +In case of OpenID, additional configuration is required. + +```env +OPENID_ISSUER="http://localhost:8080/realms/comify" +OPENID_CLIENT_ID="openid" +OPENID_CLIENT_SECRET="" +OPENID_REDIRECT_URI="http://localhost:3000/rpc/domain/authentication/login" +OPENID_ALLOW_INSECURE_REQUESTS=true +``` + +## How to use + +An instance of the configured identity provider implementation can be imported for performing authentication operations. + +```ts +import identityProvider from '^/integrations/authentication'; + +// Perform operations with the identityProvider instance +``` + +### Operations + +```ts +import identityProvider, { Session } from '^/integrations/authentication'; + +// Open connection +await identityProvider.connect(); + +// Close connection +await identityProvider.disconnect(); + +// Request the URL of the login page +const loginUrl: string = await identityProvider.getLoginUrl(); + +// Handle a login and get a session +// Throws LoginFailed if not successful +const firstSession: Session = await identityProvider.login(providedIdentity); + +// Refresh a session +// Throws LoginFailed if not successful +const secondSession: Session = await identityProvider.refresh(firstSession); + +// Logout +await identityProvider.logout(secondSession); +``` + +### Session structure + +The session has the following structure. + +```ts +type Session = { + key?: string; + requester?: unknown; + identity: Identity; + accessToken: Token; + refreshToken: Token; + expires: Date; +}; +``` + +Every session has a unique key that will be provided to external clients. This key is an unrelated hash value that contains no session information. It's ony used for referencing and storage. + +The requester represents the actual logged in user retrieved from the identity information (email), and can be stored in the session for quick reference. The full Identity structure looks like this. + +```ts +type Identity = { + name: string; + nickname: string | undefined; + picture: string | undefined; + email: string; + email_verified: boolean; +}; +``` + +The access and refresh tokens can be of any type, but is always represented as string. This depends on the configured implementation. In most cases this will be a JWT. + +```ts +type Token = string; +``` diff --git a/docs/integrations/DATABASE.md b/docs/integrations/DATABASE.md new file mode 100644 index 00000000..3f24a7b5 --- /dev/null +++ b/docs/integrations/DATABASE.md @@ -0,0 +1,120 @@ + +# Database | Comify docs + +The database integration provides a universal interaction layer with an actual data storage solution. + +This integration is based on simple CRUD operations and purposely does NOT support relational querying. + +## Implementations + +Currently, there are two implementations: + +* **Memory** - non-persistent in memory storage (used for testing). +* **MongoDB** - persistent document storage (used in production). + +## Configuration + +The used implementation needs to be configured in the `.env` file. + +```env +DATABASE_IMPLEMENTATION="mongodb" # (memory | mongodb) +``` + +In case of MongoDB, additional configuration is required. + +```env +MONGODB_CONNECTION_STRING="mongodb://development:development@localhost:27017" +MONGODB_DATABASE_NAME="comify" +``` + +## How to use + +An instance of the configured implementation can be imported for performing database operations. + +```ts +import database from '^/integrations/database'; + +// Perform operations with the database instance +``` + +### Operations + +```ts +import database, { RecordData, RecordQuery, RecordSort, SortDirections } from '^/integrations/database'; + +// Open connection +await database.connect(); + +// Close connection +await database.disconnect(); + +// INSERT INTO items (name, quantity) VALUES (?, ?) +const id: string = await database.createRecord('items', { name: 'Popcorn', quantity: 3 }); + +// SELECT * FROM items WHERE id = ? +// Throws RecordNotFound if not found +const record: RecordData = await database.readRecord('items', id); + +// SELECT name FROM items WHERE id = ? +const record: RecordData = await database.readRecord('items', id, ['name']); + +// SELECT * FROM items WHERE id = ? LIMIT 1 OFFSET 0 +const records: RecordData | undefined = await database.findRecord('items', { id }); + +// SELECT * FROM items +const records: RecordData[] = await database.searchRecords('items', {}); + +// SELECT name FROM items +const records: RecordData[] = await database.searchRecords('items', {}, ['name']); + +// UPDATE items SET name = ? WHERE id = ? +// Throws RecordNotFound if not found +await database.updateRecord('items', item.id, { 'name': item.name }); + +// DELETE FROM items WHERE id = ? +// Throws RecordNotFound if not found +await database.deleteRecord('items', item.id); + +// SELECT * FROM items WHERE name LIKE "%?%" ORDER BY name ASC LIMIT ? OFFSET ? +const query: RecordQuery = { name: { CONTAINS: name }}; +const sort: RecordSort = { name: SortDirections.ASCENDING }; +const records: RecordData[] = await database.searchRecords('items', query, undefined, sort, limit, offset); + +// SELECT name FROM items WHERE name LIKE "?%" OR name LIKE "%?" ORDER BY name ASC, quantity DESC LIMIT ? OFFSET ?; +const query: RecordQuery = { OR: [ { name: { STARTS_WITH: name } }, { name: { ENDS_WITH: name } } ] }; +const sort: RecordSort = { name: SortDirections.ASCENDING, quantity: SortDirections.DESCENDING }; +const records: RecordData[] = await database.searchRecords('items', query, ['name'], sort, limit, offset); +``` + +### Query options + +A basic query has the following structure. + +```ts +const query: RecordQuery = { fieldName1: { OPERATOR: value }, fieldName2: { OPERATOR: value }, ... } +``` + +The following operators are supported: `EQUALS`, `NOT_EQUALS`, `LESS_THAN`, `LESS_THAN_OR_EQUALS`, `GREATER_THAN`, `GREATER_THAN_OR_EQUALS`, `IN`, `NOT_IN`, `CONTAINS`, `STARTS_WITH`, `ENDS_WITH` + +Multiple queries can be grouped using the logical operators: `AND`, `OR`. + +```ts +const andQuery: RecordQuery = { AND: [ query1, query2, ...] } +const orQuery: RecordQuery = { OR: [ query1, query2, ...] } +``` + +### Sort options + +A basic query has the following structure. + +```ts +const sort: RecordSort = { fieldName1: DIRECTION, fieldName2: DIRECTION, ... }; +``` + +The following directions are supported: `ASCENDING`, `DESCENDING`. Both are defined in the `SortDirections` enum. + +```ts +const sort: RecordSort = { fieldName1: SortDirections.ASCENDING, fieldName2: SortDirections.DESCENDING, ... }; +``` + +The sort will be performed in the configured order. diff --git a/docs/integrations/EVENT_BROKER.md b/docs/integrations/EVENT_BROKER.md new file mode 100644 index 00000000..efe62e6c --- /dev/null +++ b/docs/integrations/EVENT_BROKER.md @@ -0,0 +1,55 @@ + +# Event Broker | Comify docs + +The event broker integration provides a universal interaction layer with an actual event broker solution. + +This integration is based on a publish / subscribe model. + +## Implementations + +Currently, there is only one implementation: + +* **Memory** - non-persistent event broker based on the Node.js `EventEmitter` (used in test and production). + +We have plans to add a Kafka implementation later on. + +## Configuration + +The used implementation needs to be configured in the `.env` file. + +```env +EVENT_BROKER_IMPLEMENTATION="memory" +``` + +## How to use + +An instance of the configured event broker implementation can be imported for performing event operations. + +```ts +import eventBroker from '^/integrations/eventbroker'; + +// Perform operations with the eventBroker instance +``` + +### Operations + +```ts +import eventBroker, { Publication, Subscription } from '^/integrations/eventbroker'; + +// Open connection +await eventBroker.connect(); + +// Close connection +await eventBroker.disconnect(); + +// Subscribe to an event +const subscription: Subscription = { channel: 'post', name: 'updated', handler: (postId: string) => { ... } }; +await eventBroker.subscribe(subscription); + +// Publish an event +const publication: Publication = { channel: 'post', name: 'updated', data: { postId: '123' } }; +await eventBroker.publish(publication); + +// Unsubscribe from an event +await eventBroker.unsubscribe(subscription); +``` diff --git a/docs/integrations/FILE_STORE.md b/docs/integrations/FILE_STORE.md new file mode 100644 index 00000000..ba4b081d --- /dev/null +++ b/docs/integrations/FILE_STORE.md @@ -0,0 +1,66 @@ + +# File Store | Comify docs + +The file store integration provides a universal interaction layer with an actual file storage solution. + +## Implementations + +Currently, there are two implementations: + +* **Memory** - non-persistent in memory storage (used for testing). +* **Minio** - persistent S3 compatible object storage (used in production). + +## Configuration + +The used implementation needs to be configured in the `.env` file. + +```env +FILE_STORE_IMPLEMENTATION="minio" # (memory | minio) +``` + +In case of Minio, additional configuration is required. + +```env +MINIO_END_POINT="localhost" +MINIO_PORT_NUMBER=9000 +MINIO_USE_SSL=false +MINIO_ACCESS_KEY="development" +MINIO_SECRET_KEY="development" +``` + +## How to use + +An instance of the configured file storage implementation can be imported for performing file operations. + +```ts +import fileStorage from '^/integrations/filestorage'; + +// Perform operations with the fileStorage instance +``` + +### Operations + +```ts +import fileStorage from '^/integrations/filestorage'; + +// Open connection +await fileStorage.connect(); + +// Close connection +await fileStorage.disconnect(); + +// Check if a file exists +const exists: boolean = await fileStorage.hasFile('path/to/file.txt'); + +// Write a file to the storage +const data: Buffer = Buffer.from('Something interesting'); +await fileStorage.writeFile('path/to/file.txt', data); + +// Read a file from storage +// Throws FileNotFound if not found +const data: Buffer = await fileStorage.readFile('path/to/file.txt'); + +// Delete a file from storage +// Throws FileNotFound if not found +await fileStorage.deleteFile('path/to/file.txt'); +``` diff --git a/docs/integrations/HTTP.md b/docs/integrations/HTTP.md new file mode 100644 index 00000000..3e4fd6f1 --- /dev/null +++ b/docs/integrations/HTTP.md @@ -0,0 +1,82 @@ + +# HTTP | Comify docs + +The HTTP integration provides a universal interaction layer with an actual HTTP client solution. + +## Implementations + +Currently, there is only one implementation: + +* **Fetch** - Node.js fetch implementation. + +## Configuration + +The used implementation needs to be configured in the `.env` file. + +```env +HTTP_IMPLEMENTATION="fetch" +``` + +## How to use + +An instance of the configured HTTP client implementation can be imported for performing HTTP operations. + +```ts +import httpClient from '^/integrations/http'; + +// Perform operations with the httpClient instance +``` + +### Operations + +```ts +import httpClient, { HTTP_METHODS } from '^/integrations/http'; + +// Set a cached response +const response: Response = new Response(); +httpClient.setCache(HTTP_METHODS.GET, url, response); + +// Get a cached response +const response: Response | undefined = httpClient.getCache(HTTP_METHODS.GET, url); + +// Remove a cached response +httpClient.removeCache(method: string, url: string) + +// Clear all cache +httpClient.clearCache() + +// Perform a GET request +const response: Response = await httpClient.get(url); + +// Perform a GET request with optional headers +const headers: Record = { 'Accept': 'application/json' }; +const response: Response = await httpClient.get(url, headers); + +// Perform a POST request with optional headers +const headers: Record = { 'Content-Type': 'application/json' }; +const response: Response = await httpClient.post(url, data, headers); + +// Perform a PUT request with optional headers +const headers: Record = { 'Content-Type': 'application/json' }; +const response: Response = await httpClient.put(url, data, headers); + +// Perform a PATCH request with optional headers +const headers: Record = { 'Content-Type': 'application/json' }; +const response: Response = await httpClient.patch(url, data, headers); + +// Perform a DELETE request with optional headers +const headers: Record = { }; +const response: Response = await httpClient.delete(url, headers); + +// Perform a HEAD request with optional headers +const headers: Record = { }; +const response: Response = await httpClient.head(url, headers); +``` + +### Response model + +The result of every request is a standard [ECMAScript Response](https://developer.mozilla.org/en-US/docs/Web/API/Response) object. + +### Caching mechanism + +All requests are cached by URL. To prevent this behavior, the cache for the URL must be deleted before performing the request. diff --git a/docs/integrations/LOGGING.md b/docs/integrations/LOGGING.md new file mode 100644 index 00000000..7188dc7f --- /dev/null +++ b/docs/integrations/LOGGING.md @@ -0,0 +1,66 @@ + +# Logging | Comify docs + +The logging integration provides a universal interaction layer with an actual logging solution. + +## Implementations + +Currently, there are two implementations: + +* **Void** - dummy implementation that doesn't log anything (used for testing). +* **Console** - implementation based on the Node.js console (used in production). + +## Configuration + +The used implementation needs to be configured in the `.env` file with the debug enabled setting. + +```env +LOGGING_IMPLEMENTATION="console" # (void | console) +LOGGING_DEBUG_ENABLED=true +``` + +## How to use + +An instance of the configured logger implementation can be imported for performing logging operations. + +```ts +import logger from '^/integrations/logging'; + +// Perform operations with the logger instance +``` + +### Operations + +```ts +import logger from '^/integrations/logging'; + +// Log info +await logger.logInfo(message); + +// Log warning +await logger.logWarn(message); + +// Log error +await logger.logError(message); + +// Log debug information +await logger.logDebug(message); + +// Log multiple messages (works for all levels) +await logger.logInfo(message1, message2, ...); + +// Logging multiple types of values (works for all levels) +await logger.logInfo('string', new Error('Oops...'), 42, [ 'a', 3.14 ], { name: 'John Doe', age: null }); +``` + +### Value interpretation + +Currently, the logger has support for the following types of values: + +* All primitive types +* Null / undefined +* Errors (its stack if available or else its message) +* Arrays (all values will be interpreted and concatenated with a space between them) +* Objects (will be stringyfied) + +In case multiple messages are given, they will be concatenated with a space between them. diff --git a/docs/integrations/NOTIFICATION.md b/docs/integrations/NOTIFICATION.md new file mode 100644 index 00000000..9b945444 --- /dev/null +++ b/docs/integrations/NOTIFICATION.md @@ -0,0 +1,62 @@ + +# Notification | Comify docs + +The notification integration provides a universal interaction layer with an actual notification solution. + +This integration is based on a push notification model. + +## Implementations + +Currently, there are two implementations: + +* **Memory** - non-persistent in memory notifications (used for testing). +* **WebPush** - web browser based push notifications (used in production). + +## Configuration + +The used implementation needs to be configured in the `.env` file with the debug enabled setting. + +```env +NOTIFICATION_IMPLEMENTATION="webpush" # (memory | webpush) +``` + +In case of WebPush, additional configuration is required. + +```env +WEBPUSH_VAPID_SUBJECT="..." +WEBPUSH_VAPID_PUBLIC_KEY="..." +WEBPUSH_VAPID_PRIVATE_KEY="..." +``` + +## How to use + +An instance of the configured notification service implementation can be imported for performing notification operations. + +```ts +import notificationService from '^/integrations/notification'; + +// Perform operations with the notificationService instance +``` + +### Operations + +```ts +import notificationService from '^/integrations/notification'; + +// Open connection +await notificationService.connect(); + +// Close connection +await notificationService.disconnect(); + +// Subscribe to receive notifications +await notificationService.subscribe(recipientId); + +// Unsubscribe from receiving notifications +// Throws SubscriptionNotFound if subscription not found. +await notificationService.unsubscribe(recipientId); + +// Send a notification to a recipient +// Throws SubscriptionNotFound if subscription not found. +await notificationService.sendNotification(recipientId, title, body); +``` diff --git a/docs/integrations/README.md b/docs/integrations/README.md new file mode 100644 index 00000000..4d4516be --- /dev/null +++ b/docs/integrations/README.md @@ -0,0 +1,25 @@ + +# Integrations | Comify docs + +The integrations contain interfaces and implementations for decoupling technical concerns and external dependencies from the domain and Web UI containers. + +## Concerns + +Currently, we have dependency integrations for the following concerns: + +* [**Authentication**](./AUTHENTICATION.md) - handles access and identity management. +* [**Database**](./DATABASE.md) - handles storing and retrieving data. +* [**Event broker**](./EVENT_BROKER.md) - handles event management. +* [**File store**](./FILE_STORE.md) - handles storing and retrieving files (comic images). +* [**HTTP**](./HTTP.md) - handles outgoing HTTP requests. +* [**Logging**](./LOGGING.md) - handles application logging. +* [**Notification**](./NOTIFICATION.md) - handles sending notifications to devices. +* [**Runtime**](./RUNTIME.md) - handles the runtime configuration for setting up the infrastructure. +* [**Validation**](./VALIDATION.md) - handles incoming data validation. + +Besides these, we have some smaller integrations for the following utilities: + +* [**Crypto**](../../src/integrations/utilities/crypto.ts) - handles generating hashes and IDs. +* [**Dates**](../../src/integrations/utilities/dates.ts) - handles complex date querying like calculating time elapsed. +* [**Sanitize**](../../src/integrations/utilities/sanitize.ts) - handles sanitizing incoming data. +* [**Web browser**](../../src/integrations/utilities/webbrowser.ts) - handles common web browser-related functions. diff --git a/docs/integrations/RUNTIME.md b/docs/integrations/RUNTIME.md new file mode 100644 index 00000000..a9f32db1 --- /dev/null +++ b/docs/integrations/RUNTIME.md @@ -0,0 +1,48 @@ + +# Runtime | Comify docs + +The runtime integration provides a universal interaction layer with Jitar and the configuration for setting up the other integrations. + +## Errors + +Both the domain and web UI containers do not depend on Jitar to operate. However, there are certain aspects of Jitar that we want to utilize for better debuggability—specifically, its error handling. + +Jitar automatically creates API endpoints based on the segment configuration. By default, it translates every application error into an Internal Server Error, which is transported with a 500 status code over HTTP. To help Jitar translate application errors into more meaningful status codes, we can leverage its error-handling mechanisms. + +To decouple these errors from the domain, a separate set of errors has been added to this integration, which are used by the domain. Each error is an extension of a Jitar error: + +* **BadRequest** - status code 400 +* **Unauthorized** - status code 401 +* **NotFound** - status code 404 +* **ValidationError** - status code 400 (specialization of BadRequest) +* **ServerError** - status code 500 + +If we decide to migrate from Jitar to another solution, these errors will need to be remapped. + +## Health checks + +Jitar provides a [health check](https://docs.jitar.dev/deploy/health-checks.html) system for monitoring the health of worker nodes. We leverage this system to check the availability of the services behind the integrations: + +* Database +* File store +* Notification + +For each health check, we implemented Jitar's interface. These implementations are located in the `healthchecks` folder. An instance is created and provided to Jitar from the similarly named health check file in the runtime root folder. + +## Middleware + +Jitar provides a [middleware](https://docs.jitar.dev/develop/middleware.html) system for intercepting RPC requests. We leverage this system for authentication. For this we've created two types of middleware: + +* Authentication - server side middleware hooked to the authentication integration +* Requester - client side middleware for browser control + +For each middleware, we implemented Jitar's interface. These implementations are located in the `middleswares` folder. An instance is created and provided to Jitar from the similarly named middleware file in the runtime root folder. + +## Set up and tear down + +Jitar provides a [set-up and tear-down](https://docs.jitar.dev/develop/setup-and-teardown.html) system for executing tasks before starting and stopping a service. We leverage this system for connection / disconnecting the services behind the integrations. We have script for the following two Jitar services: + +* Gateway (authentication) +* Node (database, event broker, file store, notification) + +For each service, we implemented a separate set-up and tear-down script in the runtime root folder. diff --git a/docs/integrations/VALIDATION.md b/docs/integrations/VALIDATION.md new file mode 100644 index 00000000..e3df929e --- /dev/null +++ b/docs/integrations/VALIDATION.md @@ -0,0 +1,96 @@ + +# Validation | Comify docs + +The validation integration provides a universal interaction layer with an actual data validation solution. + +## Implementations + +Currently, there is only one implementation: + +* **Zod** - implementation for the currently popular Zod library (used in test and production). + +## Configuration + +The used implementation needs to be configured in the `.env` file. + +```env +VALIDATION_IMPLEMENTATION="zod" +``` + +## How to use + +An instance of the configured validator implementation can be imported for performing validation operations. + +```ts +import validator from '^/integrations/validation'; + +// Perform operations with the validator instance +``` + +### Operations + +```ts +import validator, { ValidationSchema, ValidationResult } from '^/integrations/validation'; + +const data = { + name: 'John Doe', + age: '42' +}; + +const schema: ValidationSchema = { + name: { message: 'Invalid name', STRING: { required: true, minLength: 4, maxLength: 40 } }, + nickname: { message: 'Invalid nickname', STRING: { required: false, , pattern: '^[a-z]+$' } }, + age: { message: 'Invalid age', NUMBER: { required: true, minValue: 18, maxValue: 99 } } +}; + +// Validate data +const result: ValidationResult = validator.validate(data, schema); +``` + +### Validation scheme options + +A basic validation scheme has the following structure. + +```ts +const schema: ValidationSchema = { + fieldName1: { TYPE: { /* type options */ } }, + fieldName2: { TYPE: { /* type options */ } }, + ... +} +``` + +**Note** that a custom validation error `message` can optionally be set per field. + +The following types are supported: + +* **STRING** + * `required: boolean` + * `minLength?: number` + * `maxLength?: number` + * `pattern?: string` +* **NUMBER** + * `required: boolean` + * `minValue?: number` + * `maxValue?: number` +* **ARRAY** + * `required: boolean` + * `minLength?: number` + * `maxLength?: number` + * `validations?: Partial` +* **BOOLEAN** + * `required: boolean` +* **DATE** + * `required: boolean` +* **UUID** + * `required: boolean` +* **EMAIL** + * `required: boolean` +* **URL** + * `required: boolean` + +### Validation result structure + +The validation result has two fields: + +* **invalid** - boolean indicating if at least one of the fields is invalid. +* **messages** - map containing the validation error messages per field. diff --git a/docs/webui/DESIGNSYSTEM.md b/docs/webui/DESIGNSYSTEM.md new file mode 100644 index 00000000..10dac4a8 --- /dev/null +++ b/docs/webui/DESIGNSYSTEM.md @@ -0,0 +1,4 @@ + +# Design System | Web UI | Comify docs + +Yet to be written... diff --git a/docs/webui/README.md b/docs/webui/README.md new file mode 100644 index 00000000..463907b6 --- /dev/null +++ b/docs/webui/README.md @@ -0,0 +1,105 @@ + +# Web UI | Comify docs + +The Web UI container contains the web based interface for user interaction. It's set up as a single page application (SPA) using [React](https://react.dev). + +## Module structure + +We're using a modular composable approach. Our primary module structure looks like this. + +Web UI module structure + +Let's look at their usages and responsibilities. + +* **Layouts** - templates for building pages. +* **Pages** - composition of features. +* **Features** - interactive composition of components. +* **Components** - small reusable building blocks. + +Each module has its own folder in the `webui` folder and exports its components in an `index.ts` file. + +Besides these, there are some additional modules. + +* **Context** - custom React context providers +* **Design system** - our own simple design system (details provided later) +* **Editor** - implementation of the comic editor +* **Hooks** - custom React hooks +* **Utils** - custom React utils + +## Composition model + +The following example shows the graphical design of the UI showing the users timeline containing comics from followed users. + +Web UI page example + +We'll use this design to explain the composition from layout till components. + +### Layout + +A layout only provide containers for content and do not contain any content themselves. We currently have two layouts: + +1. **Centered** - centers all content both vertically and horizontally. +2. **Sidebar** - two-column layout with a 100% height and centered horizontally. + +The example clearly shows the **Sidebar** layout. + +### Page + +Each page picks a layout and fills its containers. We have a single page per layout because we're using feature based routing: + +1. **Guest** - for non-logged in users (centered layout). +1. **Application** - for logged in users (sidebar layout). + +The example shows the **Application** page. This page has a fixed sidebar coming from the components module. The content is a feature coming from the router. + +### Feature + +Features compose components and make them work as a functional whole. For the largest part the features are implementations of a use-case, like the **Timeline** showed in the example. The other main features are: + +* **Explore** - posts and creators the user isn't following yet. +* **Notifications** - of likes, reactions, followers, etc.. +* **Create** - a new post. +* **Profile** - the users posts, followers and following. + +These features can link to, or contain, sub-features like **PostDetails**, **PostReactions**, etc.. + +At the composition of a feature, actions need to be defined for the components. For example, what needs to happen when the ordering of the overview has changed, or when the follow button is pressed. All available actions are provided as hooks (from the `hooks` module), so they only need to be imported and linked. For example, the `useEstablishRelation` hook is linked as action for the follow button. + +A feature is also responsible for loading data. For this, hooks are also available like the `useNotifications` hook used for the notifications feature. + +### Components + +Components are small reusable building blocks grouped per domain concept. To keep them small, we break them down into smaller pieces. Conceptually we distinguish two types of components: + +1. **Primary** - public (shared) components for building features. +1. **Elementary** - private components for building primary components. + +Our basic design rule: components are dumb! A component can define actions, but does not implement them. Features will provide the implementation, because they know the context of use. + +Also, primary components can be bound to a domain model. Elementary components cannot, because they are even dumber. + +### Design system + +At the core of the design lies a design system that we use for building the pages, features and components. When done correctly, you won't find a trace of standard HTML in them. + +The design system consists for the most part of a set of elements like a button, panel, etc.. But it also has interactive components like a dropdown and tabs. A full description of the design system can be found in a [separate document](./DESIGNSYSTEM.md). + +**Note:** In the future we might move the design system to its own project so we can reuse it for other projects. + +## Other topics + +### Routing + +The routes can be found in the `Routes.tsx` file in the root of the webui folder. They are used in the `App` components that is also placed in the root folder. + +### Contexts + +Currently we only a single context for the application. The `AppContext` only provides the identity of the user. We try to limit the number of contexts, so we hope we can leave it like this. + +### Server communication + +All server communication comes from hooks. A hook is allowed to import and execute a domain function, like establishing a relation. + +Many of the domain functions require a requester object for authorization purposes. We've created a `requester` object to act as placeholder when calling these functions. The authentication middleware will replace this object with the actual requester. + +**Note:** Because we're using Jitar, we can import and call the functions directly and don't have to build API requests. diff --git a/src/integrations/database/index.ts b/src/integrations/database/index.ts index 26955018..49ceef20 100644 --- a/src/integrations/database/index.ts +++ b/src/integrations/database/index.ts @@ -1,48 +1,3 @@ -/************************************************************************************* - -# INTRODUCTION - -This module provides a consistent interface for the domain layer to interact with the database layer. -It's designed as a lightweight wrapper around the database implementation, which is currently MongoDB. - -# EXAMPLES OF USAGE - -```ts -import { - RecordQuery, RecordSort, SortDirections, - createRecord, readRecord, updateRecord, deleteRecord, findRecord, searchRecords -} from 'path/to/integrations/database/module'; - -// INSERT INTO items (name, quantity) VALUES (?, ?) -const id = await createRecord(ITEM_COLLECTION, { 'name': name, 'quantity': quantity }); - -// SELECT * FROM items WHERE id = ? -const record = await readRecord(ITEM_COLLECTION, id); - -// SELECT * FROM items -const records = await searchRecords(ITEM_COLLECTION, {}); - -// SELECT id FROM items -const records = await searchRecords(ITEM_COLLECTION, {}, ['id']); - -// UPDATE items SET name = ? WHERE id = ? -await updateRecord(ITEM_COLLECTION, item.id, { 'name': name }); - -// DELETE FROM items WHERE id = ? -await deleteRecord(ITEM_COLLECTION, item.id); - -// SELECT * FROM items WHERE name LIKE "%?%" ORDER BY name ASC LIMIT ? OFFSET ? -const query: RecordQuery = { 'name': { CONTAINS: name }}; -const sort: RecordSort = { 'name': SortDirections.ASCENDING }; -const records = await searchRecords(ITEM_COLLECTION, query, undefined, sort, limit, offset); - -// SELECT * FROM items WHERE name LIKE "?%" OR name LIKE "%?" ORDER BY name ASC, quantity DESC LIMIT ? OFFSET ?; -const query: RecordQuery = { OR: [ { 'name': { STARTS_WITH: name } }, { 'name': { ENDS_WITH: name } } ] }; -const sort: RecordSort = { 'name': SortDirections.ASCENDING, 'quantity': SortDirections.DESCENDING }; -const records = await searchRecords(ITEM_COLLECTION, query, undefined, sort, limit, offset); -``` - -*************************************************************************************/ import Database from './Database'; import implementation from './implementation';