From 30fd5ac79cd14a3499fafb7118cfe983d7546691 Mon Sep 17 00:00:00 2001 From: msukkari Date: Wed, 18 Sep 2024 16:11:19 -0700 Subject: [PATCH 1/6] add instructions on adding GITHUB_TOKEN as env variable on machine --- README.md | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 4b644a67b..088b1a252 100644 --- a/README.md +++ b/README.md @@ -53,7 +53,15 @@ Sourcebot also supports indexing GitLab & BitBucket. Checkout the [index.json](.
GitHub - Generate a GitHub Personal Access Token (PAT) [here](https://github.com/settings/tokens/new). If you are indexing public repositories only, you can select the `public_repo` scope, otherwise you will need the `repo` scope. + Generate a GitHub Personal Access Token (PAT) [here](https://github.com/settings/tokens/new). If you're only indexing public repositories select the `public_repo` scope; otherwise, select the `repo` scope. + + You'll need to pass this PAT each time you run Sourcebot, so we recommend adding it as an environment variable. In this guide, we'll add the Github PAT as an environment variable called `GITHUB_TOKEN`: + ```sh + export GITHUB_TOKEN= + ``` + + If you'd like to persist this environment variable across shell sessions, please add this line to your shell config file (ex. `~/.bashrc`, `~/.bash_profile`, etc) +
@@ -78,8 +86,9 @@ Sourcebot also supports indexing GitLab & BitBucket. Checkout the [index.json](.
GitHub + Run the `sourcebot` docker image, passing in the Github PAT you generated in the previous step as an environment variable called `GITHUB_TOKEN`: ```sh - docker run -p 3000:3000 --rm --name sourcebot -v $(pwd):/data -e GITHUB_TOKEN= ghcr.io/taqlaai/sourcebot:main + docker run -p 3000:3000 --rm --name sourcebot -v $(pwd):/data -e GITHUB_TOKEN=$GITHUB_TOKEN ghcr.io/taqlaai/sourcebot:main ```
From 5eb059435dd6e894fad739ac50259188f6cb0c74 Mon Sep 17 00:00:00 2001 From: msukkari Date: Wed, 18 Sep 2024 17:11:34 -0700 Subject: [PATCH 2/6] add .github-token to gitignore and add additional info in build instructions --- .gitignore | 4 +++- README.md | 6 +++--- 2 files changed, 6 insertions(+), 4 deletions(-) diff --git a/.gitignore b/.gitignore index e33810831..e417cd3b2 100644 --- a/.gitignore +++ b/.gitignore @@ -41,4 +41,6 @@ next-env.d.ts .sourcebot /bin -/config.json \ No newline at end of file +/config.json + +.github-token \ No newline at end of file diff --git a/README.md b/README.md index 088b1a252..cdec27c33 100644 --- a/README.md +++ b/README.md @@ -128,14 +128,14 @@ Sourcebot also supports indexing GitLab & BitBucket. Checkout the [index.json](. ## Building Sourcebot -0. Install go and NodeJS +1. Install go and NodeJS. Note that a NodeJS version of at least `21.1.0` is required. -1. Clone the repository with submodules: +2. Clone the repository with submodules: ```sh git clone --recurse-submodules https://github.com/TaqlaAI/sourcebot.git ``` -2. Run make to build zoekt and install dependencies: +3. Run make to build zoekt and install dependencies: ```sh cd sourcebot make From e72fa8f31d4a8a18d028552318ee47cfe4a3cdfd Mon Sep 17 00:00:00 2001 From: msukkari Date: Wed, 18 Sep 2024 17:22:28 -0700 Subject: [PATCH 3/6] remove github-token from gitignore because its meant to be in home dir. Bold this part in the README so its more clear --- .gitignore | 2 -- README.md | 2 +- 2 files changed, 1 insertion(+), 3 deletions(-) diff --git a/.gitignore b/.gitignore index e417cd3b2..1cbc37353 100644 --- a/.gitignore +++ b/.gitignore @@ -42,5 +42,3 @@ next-env.d.ts .sourcebot /bin /config.json - -.github-token \ No newline at end of file diff --git a/README.md b/README.md index cdec27c33..7bc1c609d 100644 --- a/README.md +++ b/README.md @@ -166,7 +166,7 @@ The zoekt binaries and web dependencies are placed into `bin` and `node_modules` Generate a GitHub Personal Access Token (PAT) [here](https://github.com/settings/tokens/new). If you are indexing public repositories only, you can select the `public_repo` scope, otherwise you will need the `repo` scope. - Create a text file named `.github-token` in your home directory and paste the token in it. The file should look like: + Create a text file named `.github-token` **in your home directory** and paste the token in it. The file should look like: ```sh ghp_... ``` From f74ae98a1f2ebe3ecc3d9313ff609efb5960dd9a Mon Sep 17 00:00:00 2001 From: msukkari Date: Wed, 18 Sep 2024 17:48:22 -0700 Subject: [PATCH 4/6] add ctags installation instructions to readme --- README.md | 8 ++++++-- package.json | 2 +- 2 files changed, 7 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 7bc1c609d..6bba0a061 100644 --- a/README.md +++ b/README.md @@ -130,12 +130,16 @@ Sourcebot also supports indexing GitLab & BitBucket. Checkout the [index.json](. 1. Install go and NodeJS. Note that a NodeJS version of at least `21.1.0` is required. -2. Clone the repository with submodules: +2. Install ctags (required by zoekt-indexserver): + Mac: `brew install universal-ctags` + Ubuntu: `apt-get install universal-ctags` + +3. Clone the repository with submodules: ```sh git clone --recurse-submodules https://github.com/TaqlaAI/sourcebot.git ``` -3. Run make to build zoekt and install dependencies: +4. Run make to build zoekt and install dependencies: ```sh cd sourcebot make diff --git a/package.json b/package.json index d90d64d6b..204b79cdc 100644 --- a/package.json +++ b/package.json @@ -5,7 +5,7 @@ "scripts": { "dev": "npm-run-all --print-label --parallel next:dev zoekt:webserver zoekt:indexserver", "zoekt:webserver": "export PATH=\"$(PWD)/bin:$PATH\" && zoekt-webserver -index .sourcebot/index -rpc", - "zoekt:indexserver": "export PATH=\"$(PWD)/bin:$PATH\" && zoekt-indexserver -data_dir .sourcebot -mirror_config config.json", + "zoekt:indexserver": "export PATH=\"$(PWD)/bin:$PATH\" && export CTAGS_COMMAND=ctags && zoekt-indexserver -data_dir .sourcebot -mirror_config config.json", "next:dev": "next dev", "build": "next build", "start": "next start", From 348df5c14b3b904e13685b5f78323b01e612efae Mon Sep 17 00:00:00 2001 From: msukkari Date: Thu, 19 Sep 2024 13:26:19 -0700 Subject: [PATCH 5/6] link to ctags repo in README --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 6bba0a061..301606ff8 100644 --- a/README.md +++ b/README.md @@ -130,7 +130,7 @@ Sourcebot also supports indexing GitLab & BitBucket. Checkout the [index.json](. 1. Install go and NodeJS. Note that a NodeJS version of at least `21.1.0` is required. -2. Install ctags (required by zoekt-indexserver): +2. Install [ctags](https://github.com/universal-ctags/ctags) (required by zoekt-indexserver): Mac: `brew install universal-ctags` Ubuntu: `apt-get install universal-ctags` From 43a0499606d3228d5c063e32eda546339f0c75e1 Mon Sep 17 00:00:00 2001 From: msukkari Date: Thu, 19 Sep 2024 13:27:16 -0700 Subject: [PATCH 6/6] remove unnecessary line change in gitignore --- .gitignore | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.gitignore b/.gitignore index 1cbc37353..e33810831 100644 --- a/.gitignore +++ b/.gitignore @@ -41,4 +41,4 @@ next-env.d.ts .sourcebot /bin -/config.json +/config.json \ No newline at end of file