Implement HTTP and ES DataLoaders (#5)

bboure · web-flow · commit 8f62faa46c4e · 2019-12-13T22:53:20.000+01:00
diff --git a/README.md b/README.md
@@ -52,10 +52,13 @@ Put options under `custom.appsync-simulator` in your `serverless.yml` file
 | port                     | 20002                 | AppSync operations port                                                                                                        |
 | wsPort                   | 20003                 | AppSync subscriptions port                                                                                                     |
 | location                 | . (base directory)    | Location of the lambda functions handlers.                                                                                     |
+| refMap | {}        | A mapping of [resource resolutions](#resource-cloudformation-functions-resolution) for the `Ref` function |
+| getAttMap | {}        | A mapping of [resource resolutions](#resource-cloudformation-functions-resolution) for the `GetAtt` function |
 | dynamoDb.endpoint        | http://localhost:8000 | Dynamodb endpoint. Specify it if you're not using serverless-dynamodb-local. Otherwise, port is taken from dynamodb-local conf |
 | dynamoDb.region          | localhost             | Dynamodb region. Specify it if you're connecting to a remote Dynamodb intance.                                                 |
 | dynamoDb.accessKeyId     | DEFAULT_ACCESS_KEY    | AWS Access Key ID to access DynamoDB                                                                                           |
-| dynamoDb.secretAccessKey | DEFAULT_SECRET        | AWS Secret Key to access DynamoDB                                                                                              |
+| dynamoDb.secretAccessKey | DEFAULT_SECRET        | AWS Secret Key to access DynamoDB |
+
 Example:
 
 ````yml
@@ -67,13 +70,115 @@ custom:
 
 ````
 
-# Caveats
+# Resource CloudFormation functions resolution
+
+This plugin supports *some* resources resolution from the `Ref` and `Fn::GetAtt` functions
+in your yaml file. It also supports *some* other Cfn functions such as `Fn::Join`, `Fb::Sub`, etc.
+
+**Note:** Under the hood, this features relies on the [cfn-resolver-lib](https://github.com/robessog/cfn-resolver-lib) package. For more info on supported cfn functions, refer to [the documentation](https://github.com/robessog/cfn-resolver-lib/blob/master/README.md)
+
+## Basic usage
+
+You can reference resources in your functions' environment variables (that will be accessible from your lambda functions) or datasource definitions.
+The plugin will automatically resolve them for you.
+
+````yaml
+provider:
+  environment:
+    BUCKET_NAME:
+      Ref: MyBucket # resolves to `my-bucket-name`
+
+resources:
+  Resources:
+    MyDbTable:
+      Type: AWS::DynamoDB::Table
+      Properties:
+        TableName: myTable
+      ...
+    MyBucket:
+      Type: AWS::S3::Bucket
+      Properties:
+        BucketName: my-bucket-name
+    ...
+
+# in your appsync config
+dataSources:
+  - type: AMAZON_DYNAMODB
+    name: dynamosource
+    config:
+      tableName:
+        Ref: MyDbTable # resolves to `myTable`
+````
+
+## Override (or mock) values
+
+Sometimes, some references **cannot** be resolved, as they come from an *Output* from Cloudformation; or you might want to use mocked values in your local environment.
+
+In those cases, you can define (or override) those values using the `refMap` and `getAttMap` options.
+
+- `refMap` takes a mapping of *resource name* to *value* pairs
+- `getAttMap` takes a mapping of *resource name* to *attribute/values* pairs
+
+Example:
+
+````yaml
+custom:
+  serverless-appsync-simulator:
+    refMap:
+      # Override `MyDbTable` resolution from the previous example.
+      MyDbTable: 'mock-myTable'
+    getAttMap:
+      # define ElasticSearchInstance DomainName
+      ElasticSearchInstance:
+        DomainEndpoint: "localhost:9200"
+
+# in your appsync config
+dataSources:
+  - type: AMAZON_ELASTICSEARCH
+    name: elasticsource
+    config:
+      # endpoint resolves as 'http://localhost:9200'
+      endpoint:
+        Fn::Join:
+          - ""
+          - - https://
+            - Fn::GetAtt:
+                - ElasticSearchInstance
+                - DomainEndpoint
+````
+
+## Limitations
+
+This plugin only tries to resolve the following parts of the yml tree:
+- `provider.environment`
+- `functions[*].environment`
+- `custom.appSync`
+
+If you have the need of resolving others, feel free to open an issue and explain your use case.
+
+For now, the supported resources to be automatically resovled by `Ref:` are:
+- DynamoDb tables
+- S3 Buckets
 
-This plugin currently only supports resolvers implemented by `amplify-appsync-simulator`.
-At the time of writing, this is:
+Feel free to open a PR or an issue to extend them as well.
 
+# Supported Resolver types
+
+This plugin supports resolvers implemented by `amplify-appsync-simulator`, as well as custom resolvers.
+
+**From Aws Amplify:**
 - NONE
 - AWS_LAMBDA (*)
 - AMAZON_DYNAMODB
+- PIPELINE
+
+**Implemented by this plugin**
+- AWS_LAMBDA (*)
+- AMAZON_ELASTIC_SEARCH
+- HTTP
+
+(*) The `AWS_LAMBDA` dataloader has been partially copied from Aws Amplify but has been extended
+to support the *BatchInvoke* operations
 
-(*) Note: This plugin also supports `AWS_LAMBDA`'s `BatchInvoke` (which Amplify Simulator doesn't)
+**Not Supported / TODO**
+- RELATIONAL_DATABASE
diff --git a/package.json b/package.json
@@ -5,6 +5,7 @@
   "author": "bboure",
   "license": "MIT",
   "private": false,
+  "description": "Offline support for serverless-appsync-plugin",
   "repository": {
     "type": "git",
     "url": "git+https://github.com/bboure/serverless-appsync-simulator.git"
@@ -21,6 +22,8 @@
     "amplify-appsync-simulator": "^1.5.0",
     "amplify-util-mock": "^3.5.0",
     "aws-sdk": "^2.585.0",
+    "axios": "^0.19.0",
+    "cfn-resolver-lib": "^1.1.2",
     "dataloader": "^2.0.0",
     "lodash": "^4.17.15"
   },
diff --git a/src/data-loaders/ElasticDataLoader.js b/src/data-loaders/ElasticDataLoader.js
@@ -0,0 +1,26 @@
+import axios from 'axios';
+
+export default class ElasticDataLoader {
+  constructor(config) {
+    this.config = config;
+  }
+
+  async load(req) {
+    try {
+      const { data } = await axios.request({
+        baseURL: this.config.endpoint,
+        url: req.path,
+        headers: req.params.headers,
+        params: req.params.queryString,
+        method: req.operation.toLowerCase(),
+        data: req.params.body,
+      });
+
+      return data;
+    } catch (err) {
+      console.log(err);
+    }
+
+    return null;
+  }
+}
diff --git a/src/data-loaders/HttpDataLoader.js b/src/data-loaders/HttpDataLoader.js
@@ -0,0 +1,26 @@
+import axios from 'axios';
+
+export default class HttpDataLoader {
+  constructor(config) {
+    this.config = config;
+  }
+
+  async load(req) {
+    try {
+      const { data } = await axios.request({
+        baseURL: this.config.endpoint,
+        url: req.resourcePath,
+        headers: req.params.headers,
+        params: req.params.query,
+        method: req.method.toLowerCase(),
+        data: req.params.body,
+      });
+
+      return data;
+    } catch (err) {
+      console.log(err);
+    }
+
+    return null;
+  }
+}
diff --git a/src/data-loaders/NotImplementedDataLoader.js b/src/data-loaders/NotImplementedDataLoader.js
@@ -1,6 +1,12 @@
 /* eslint-disable class-methods-use-this */
 export default class NotImplementedDataLoader {
+  constructor(config) {
+    this.config = config;
+  }
+
   async load() {
-    throw new Error('Data Loader not implemented');
+    console.log(`Data Loader not implemented for ${this.config.type} (${this.config.name})`);
+
+    return null;
   }
 }
diff --git a/src/getAppSyncConfig.js b/src/getAppSyncConfig.js
@@ -3,7 +3,7 @@ import {
 } from 'amplify-appsync-simulator';
 import { invoke } from 'amplify-util-mock/lib/utils/lambda/invoke';
 import fs from 'fs';
-import { find, get } from 'lodash';
+import { find } from 'lodash';
 import path from 'path';
 
 export default function getAppSyncConfig(context, appSyncConfig) {
@@ -30,22 +30,6 @@ export default function getAppSyncConfig(context, appSyncConfig) {
       type: source.type,
     };
 
-    /**
-     * Returns the tableName resolving reference. Throws exception if reference is not found
-     */
-    const getTableName = (table) => {
-      if (table && table.Ref) {
-        const tableName = get(context.serverless.service, `resources.Resources.${table.Ref}.Properties.TableName`);
-
-        if (!tableName) {
-          throw new Error(`Unable to find table Reference for ${table} inside Serverless resources`);
-        }
-
-        return tableName;
-      }
-      return table;
-    };
-
     switch (source.type) {
       case 'AMAZON_DYNAMODB': {
         const {
@@ -62,28 +46,49 @@ export default function getAppSyncConfig(context, appSyncConfig) {
             region,
             accessKeyId,
             secretAccessKey,
-            tableName: getTableName(source.config.tableName),
+            tableName: source.config.tableName,
           },
         };
       }
       case 'AWS_LAMBDA': {
         const { functionName } = source.config;
-        if (context.serverless.service.functions[functionName] === undefined) {
+        if (functionName === undefined) {
+          context.plugin.log(
+            `${source.name} does not have a functionName`,
+            { color: 'orange' },
+          );
           return null;
         }
-
-        const [fileName, handler] = context.serverless.service.functions[functionName].handler.split('.');
+        const func = context.serverless.service.functions[functionName];
+        if (func === undefined) {
+          context.plugin.log(
+            `The ${functionName} function is not defined`,
+            { color: 'orange' },
+          );
+          return null;
+        }
+        const [fileName, handler] = func.handler.split('.');
         return {
           ...dataSource,
           invoke: (payload) => invoke({
             packageFolder: context.serverless.config.servicePath,
             handler,
             fileName: path.join(context.options.location, fileName),
             event: payload,
-            environment: context.serverless.service.provider.environment || {},
+            environment: {
+              ...context.serverless.service.provider.environment,
+              ...func.environment,
+            },
           }),
         };
       }
+      case 'AMAZON_ELASTICSEARCH':
+      case 'HTTP': {
+        return {
+          ...dataSource,
+          endpoint: source.config.endpoint,
+        };
+      }
       default:
         return dataSource;
     }
diff --git a/src/index.js b/src/index.js
diff --git a/yarn.lock b/yarn.lock

Original file line number	Diff line number	Diff line change
`@@ -1,6 +1,12 @@`
`1`	`1`	`/* eslint-disable class-methods-use-this */`
`2`	`2`	`export default class NotImplementedDataLoader {`
	`3`	`+ constructor(config) {`
	`4`	`+ this.config = config;`
	`5`	`+ }`
	`6`	`+`
`3`	`7`	`async load() {`
`4`		`- throw new Error('Data Loader not implemented');`
	`8`	+ console.log(`Data Loader not implemented for ${this.config.type} (${this.config.name})`);
	`9`	`+`
	`10`	`+ return null;`
`5`	`11`	`}`
`6`	`12`	`}`