updating dev branch with main changes #118
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ation (#113) * Develop to Main (#77) * updating develop with latest main changes (#58) * Readme and demo update (closes issue #52) (#53) * Readme and demo update (closes issue #52) * D3js fix for graph visualization (closes #54) (#55) * Readme update * fixing HF broken link ref in README.md (#56) --------- Co-authored-by: Vagner Santana <santanavagner@users.noreply.github.com> * Readme and demo update (closes issue #52) (#53) * Readme and demo update (closes issue #52) * D3js fix for graph visualization (closes #54) (#55) * Readme update * update in requirements.txt (#59) downgrading tensorflow * update in requirements.txt (#60) fixing version * update requirements.txt (#61) * update requirements.txt (#62) leaving version problems to pip * update requirements.txt (#63) leaving torch without version * reverting requirements.txt (#64) removing libs added with autofix and package versions * Adding error message about ajax header setup * adding recipes folder with first cookbook (#70) * adding all cookbooks (#71) * Cookbook notebooks (closes #69) * Update README.md * Adding flags for running the notebook on colab (#69) * Adding the open in colab button (#69) * Fix for loading files from github (issue #69) * Adjustments for infovis notebook to work on colab (issue #69) * Adding populate embeddings to open in colab (issue #69) * Adding a recommend threshold notebook (#69) * Readme update * Readme update * Simplifying recommend threshold notebook (#69) * Comment update (#69) * Map keys TYPO * Removing deprecated models and adding SentenceTransformer as fallback (#75) * Adding sentences and updating umap models (closes #76) * Preventing a numpy error when saving json * Added a recipe for testing prompt dataset (closes #78) * Readme update --------- Co-authored-by: Cássia Sampaio <4005687+cassiasamp@users.noreply.github.com> Co-authored-by: Cássia Sanctos <csamp@ibm.com> * brazilian portuguese translation Signed-off-by: Thiago Nebuloni <thiago.nebuloni@gmail.com> * corrected 'your are' to 'you are' Signed-off-by: Thiago Nebuloni <thiago.nebuloni@gmail.com> * updated torch version in requirements.txt (#79) (#89) Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> Co-authored-by: Ashwath V A <73862377+Mystic-Slice@users.noreply.github.com> * Recipe for Responsible Prompting using IBM Granite Embedding model (#88) * updated torch version in requirements.txt Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> * added recipe for responsible prompting using granite embedding model Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> * renamed and included run outputs Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> * added more examples and set better thresholds Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> --------- Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> * Added inference to the demo website (#86) * updated torch version in requirements.txt Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> * added inference to demo along with an endpoint in the Flask API to service inference requests Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> * updated README.md Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> * updated README and added route to swagger Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> --------- Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> * Added the prompt-sentences file for `ibm-granite/granite-embedding-278m-multilingual` model populated with embeddings (#87) * updated torch version in requirements.txt Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> * populated embeddings file for granite-embedding-278m-multilingual Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> * added visualization plot for ibm-granite/granite-embedding-278m-multilingual Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> --------- Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> * added the granite embedding notebook to readme (#93) Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> * Switch to non parametric umap (#94) * updated torch version in requirements.txt Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> * switched to non-parametric umap Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> --------- Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> * Added Multi-turn conversation in the Demo (#91) * added multi-turn conversation in the demo Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> * made some ui fixes to mimic the granite playground more Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> * added model selection + few other ui changes Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> * added svgs and css assets Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> * model switch feature added + graph for each recommendation chosen Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> * refactored code Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> * loading tags + graph repositioned + model select repositioned + chat scroll issue fixed Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> * markdown rendering for user and assistant messages Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> * fix small ui issues Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> * shortened value tags under user message Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> * final touches Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> --------- Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> * updated cookbooks and prompt json files to use non-parametric umap (#98) Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> * Refactored code + Vectorized similarity & search operations (#95) * refactored recommend_prompt function to allow custom embeddings as input Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> * refactored to make recommend_prompt function independent of embedding function Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> * cleanup + model_id fix Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> * vectorized similarity search Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> * moved loading of umap model out of recommend_prompt method + other clean up Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> * reverse debug settings --------- Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> * Delete control/recommendation_handler.py * Delete cookbook/README.md * Added Replicate support for `demo_inference` (#111) * added replicate support for demo_inference Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> * updated readme with replicate support info Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> * updated readme Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> * fixed readme Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> --------- Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> --------- Signed-off-by: Thiago Nebuloni <thiago.nebuloni@gmail.com> Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> Co-authored-by: Vagner Santana <santanavagner@users.noreply.github.com> Co-authored-by: Thiago Nebuloni <thiago.nebuloni@gmail.com> Co-authored-by: Ashwath V A <73862377+Mystic-Slice@users.noreply.github.com>
* Develop to Main (#77) * updating develop with latest main changes (#58) * Readme and demo update (closes issue #52) (#53) * Readme and demo update (closes issue #52) * D3js fix for graph visualization (closes #54) (#55) * Readme update * fixing HF broken link ref in README.md (#56) --------- Co-authored-by: Vagner Santana <santanavagner@users.noreply.github.com> * Readme and demo update (closes issue #52) (#53) * Readme and demo update (closes issue #52) * D3js fix for graph visualization (closes #54) (#55) * Readme update * update in requirements.txt (#59) downgrading tensorflow * update in requirements.txt (#60) fixing version * update requirements.txt (#61) * update requirements.txt (#62) leaving version problems to pip * update requirements.txt (#63) leaving torch without version * reverting requirements.txt (#64) removing libs added with autofix and package versions * Adding error message about ajax header setup * adding recipes folder with first cookbook (#70) * adding all cookbooks (#71) * Cookbook notebooks (closes #69) * Update README.md * Adding flags for running the notebook on colab (#69) * Adding the open in colab button (#69) * Fix for loading files from github (issue #69) * Adjustments for infovis notebook to work on colab (issue #69) * Adding populate embeddings to open in colab (issue #69) * Adding a recommend threshold notebook (#69) * Readme update * Readme update * Simplifying recommend threshold notebook (#69) * Comment update (#69) * Map keys TYPO * Removing deprecated models and adding SentenceTransformer as fallback (#75) * Adding sentences and updating umap models (closes #76) * Preventing a numpy error when saving json * Added a recipe for testing prompt dataset (closes #78) * Readme update --------- Co-authored-by: Cássia Sampaio <4005687+cassiasamp@users.noreply.github.com> Co-authored-by: Cássia Sanctos <csamp@ibm.com> * brazilian portuguese translation Signed-off-by: Thiago Nebuloni <thiago.nebuloni@gmail.com> * corrected 'your are' to 'you are' Signed-off-by: Thiago Nebuloni <thiago.nebuloni@gmail.com> * updated torch version in requirements.txt (#79) (#89) Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> Co-authored-by: Ashwath V A <73862377+Mystic-Slice@users.noreply.github.com> * Recipe for Responsible Prompting using IBM Granite Embedding model (#88) * updated torch version in requirements.txt Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> * added recipe for responsible prompting using granite embedding model Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> * renamed and included run outputs Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> * added more examples and set better thresholds Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> --------- Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> * Added inference to the demo website (#86) * updated torch version in requirements.txt Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> * added inference to demo along with an endpoint in the Flask API to service inference requests Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> * updated README.md Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> * updated README and added route to swagger Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> --------- Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> * Added the prompt-sentences file for `ibm-granite/granite-embedding-278m-multilingual` model populated with embeddings (#87) * updated torch version in requirements.txt Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> * populated embeddings file for granite-embedding-278m-multilingual Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> * added visualization plot for ibm-granite/granite-embedding-278m-multilingual Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> --------- Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> * added the granite embedding notebook to readme (#93) Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> * Switch to non parametric umap (#94) * updated torch version in requirements.txt Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> * switched to non-parametric umap Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> --------- Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> * Added Multi-turn conversation in the Demo (#91) * added multi-turn conversation in the demo Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> * made some ui fixes to mimic the granite playground more Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> * added model selection + few other ui changes Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> * added svgs and css assets Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> * model switch feature added + graph for each recommendation chosen Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> * refactored code Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> * loading tags + graph repositioned + model select repositioned + chat scroll issue fixed Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> * markdown rendering for user and assistant messages Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> * fix small ui issues Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> * shortened value tags under user message Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> * final touches Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> --------- Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> * updated cookbooks and prompt json files to use non-parametric umap (#98) Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> * Refactored code + Vectorized similarity & search operations (#95) * refactored recommend_prompt function to allow custom embeddings as input Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> * refactored to make recommend_prompt function independent of embedding function Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> * cleanup + model_id fix Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> * vectorized similarity search Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> * moved loading of umap model out of recommend_prompt method + other clean up Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> * reverse debug settings --------- Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> * Delete control/recommendation_handler.py * Delete cookbook/README.md * Added Replicate support for `demo_inference` (#111) * added replicate support for demo_inference Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> * updated readme with replicate support info Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> * updated readme Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> * fixed readme Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> --------- Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> * updated links in CONTRIBUTING.md (#116) Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> --------- Signed-off-by: Thiago Nebuloni <thiago.nebuloni@gmail.com> Signed-off-by: Ashwath Vaithinathan Aravindan <ashwath.vaithina@ibm.com> Co-authored-by: Vagner Santana <santanavagner@users.noreply.github.com> Co-authored-by: Thiago Nebuloni <thiago.nebuloni@gmail.com> Co-authored-by: Ashwath V A <73862377+Mystic-Slice@users.noreply.github.com>
Added back umap files
* Add functionality to load existing embeddings from output JSON file thereby optimizing the process of generating embeddings for new data. Signed-off-by: Ayesha Imran <ayesha.i1505@gmail.com> Signed-off-by: GitHub <noreply@github.com> * Add functionality to load existing embeddings from output JSON file thereby optimizing the process of generating embeddings for new data. Signed-off-by: Ayesha Imran <ayesha.i1505@gmail.com> Signed-off-by: GitHub <noreply@github.com> --------- Signed-off-by: Ayesha Imran <ayesha.i1505@gmail.com> Signed-off-by: GitHub <noreply@github.com>
* Recipe and Cookbook to evaluate embedding models Signed-off-by: ArionDas <ariondasad@gmail.com> * Updated all metrics and added a short synopsis. Signed-off-by: ArionDas <ariondasad@gmail.com> --------- Signed-off-by: ArionDas <ariondasad@gmail.com>
…d generate embeddings (prompt_sentences_ur-multilingual-e5-large.json) for them using multilingual-e5-large embedding model from Hugging Face. (#114) Signed-off-by: Ayesha Imran <ayesha.i1505@gmail.com>
#130) * Added a new endpoint to compute closest positive and negative values for individual sentences Signed-off-by: ArionDas <ariondasad@gmail.com> * Added validations and implemented cache compatibility to optimize subsequent calls Signed-off-by: ArionDas <ariondasad@gmail.com> --------- Signed-off-by: ArionDas <ariondasad@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.