set up pylint with sane defaults, black as pre-commit hook

Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>

Author: varun-edachali-dbx

.github/workflows/code-quality-checks.yml

@@ -156,6 +156,11 @@ jobs:
       #----------------------------------------------
       - name: Black
         run: poetry run black --check src
+      #----------------------------------------------
+      #              pylint the code
+      #----------------------------------------------
+      - name: Pylint
+        run: poetry run pylint --rcfile=pylintrc src
 
   check-types:
     runs-on: ubuntu-latest

.pre-commit-config.yaml

@@ -0,0 +1,13 @@
+repos:
+-   repo: https://github.com/pre-commit/pre-commit-hooks
+    rev: v4.5.0
+    hooks:
+    -   id: trailing-whitespace
+    -   id: end-of-file-fixer
+    -   id: check-added-large-files
+
+-   repo: https://github.com/psf/black
+    rev: 22.3.0
+    hooks:
+    -   id: black
+        exclude: '/(\.eggs|\.git|\.hg|\.mypy_cache|\.nox|\.tox|\.venv|\.svn|_build|buck-out|build|dist|thrift_api)/'

CONTRIBUTING.md

@@ -145,9 +145,11 @@ The `PySQLStagingIngestionTestSuite` namespace requires a cluster running DBR ve
 The suites marked `[not documented]` require additional configuration which will be documented at a later time.
 
 
-### Code formatting
+### Code formatting and linting
 
-This project uses [Black](https://pypi.org/project/black/).
+This project uses [Black](https://pypi.org/project/black/) for code formatting and [Pylint](https://pylint.org/) for linting.
+
+#### Black
 
 ```
 poetry run python3 -m black src --check
@@ -157,6 +159,25 @@ Remove the `--check` flag to write reformatted files to disk.
 
 To simplify reviews you can format your changes in a separate commit.
 
+#### Pylint
+
+```
+poetry run pylint --rcfile=pylintrc src
+```
+
+#### Pre-commit hooks
+
+We use [pre-commit](https://pre-commit.com/) to automatically run Black and other checks before each commit.
+
+To set up pre-commit hooks:
+
+```bash
+# Set up the git hooks
+poetry run pre-commit install
+```
+
+This will set up the hooks defined in `.pre-commit-config.yaml` to run automatically on each commit.
+
 ### Change a pinned dependency version
 
 Modify the dependency specification (syntax can be found [here](https://python-poetry.org/docs/dependency-specification/)) in `pyproject.toml` and run one of the following in your terminal: