Skip to content

Commit 9d91665

Browse files
ygwpzflesher0813harrisonyhqhek14propanone1006
authored
0.0.2 release merge main (#157)
* fix issue#26 and issue#36 (#55) * [Doc] Add vllm institution (#61) * [CI] Add issue and pull request template; [Fix][Doc] Fix nfs doc error. (#62) (#64) * [CI] Add issue template * [CI] Add pr template * [Fix][Doc] Fix nfs doc error, close #57 Co-authored-by: harrisonyhq <harrisonyhq@gmail.com> * [Doc] update install doc using patch to build from source code (#68) * [Feat] Merge 0.0.1 back into develop (#72) * [CI] Add issue and pull request template; [Fix][Doc] Fix nfs doc error. (#62) * [CI] Add issue template * [CI] Add pr template * [Fix][Doc] Fix nfs doc error, close #57 * [CI][Style] Add Github workflow for pre commit and format the codestyle (#70) * [CI] Add github flow for pre-commit and unittest * [Style] Fix typo and sytle problem in repo --------- Co-authored-by: harrisonyhq <harrisonyhq@gmail.com> * [Style] Fix codestyle problems and typo in develop (#75) * [Style] Fix codestyle problems and typo * [Fix] Fix CI bug * [CI] Add workflow trigger on push * [CI] Add support pyproject.toml to enable using python -m build to compile whl package * ucm_sparse framework v1.0 (#79) * [Fix] Fix cant find cmake error when using pip install -e . * Revert "ucm_sparse framework v1.0 (#79)" (#82) This reverts commit b965dc8. * [Feature] add Mooncake Store * [Fix bug] fix docker build err and installation.md (#87) * adapt deepseek (#89) * [Feature][P/D] add example for disaggregated prefill (#90) * [Perf] Pipelined ucmnfsstore (#97) * pipelined ucmnfsstore * update default stream number * Revert "[Feature] add Mooncake Store" (#98) * [Fix bug] fix uc_connector ut and change hash generation method * [Fix] Fix .so build error (#104) [Fix] Fix so file import error in build and edit mode [Fix] format the code [Feat] Add device recognize function * [Fix] Fix ascend compile error (#106) * ESA 1.0 fix typo ESA: add vllm and vllm-ascend patch add vllm and vllm_ascend patch * fix typo * [fix] compatible with prefix cache * add sparse_attn example * add sparse_attn docs * Modify start_load_kv (#103) * [Fix] Fix duplicate create/commit errors upon preemption (#109) * [refact] format * adapt for vllm 0.9.1 (#113) Co-authored-by: y00945504 <yuhui87@huawei.com> * add patch * fix: uc_connector,rm .gitkeep ucm_oceanstor.py * rename vllm-adapt-2 to vllm-adapt-sparse * [Fix] Fix spelling issues with PR templates (#119) * remove load_tasks * [bugfix] bugfix in ucmnfsstore (#123) * trans task timeout support * [Fix] posix file open interface bugfix * add config parameter * Fix rank handling in multi-node PP setup (#129) * [Feat]Support UCM Sparse on cuda (#126) * [Feat]Support UCM Sparse on cuda * [DOCS]Add doc for format code. * [Feature] Add mooncake store (#117) * 暂存 * [Feature] Monncake connector support both config and file * [Doc] Add docs for Ucm Mooncake Connector * [Feature] Add mooncake to ucm factory * [Doc][Fix] Modify the description of configuration to match usage. * [Feature] [Fix] Load Mooncake config from dict, when lack params, load from env config file. * [Doc] update the performance and modify description. * [Test] Example config file for Mooncake test `test_mooncake_env.py`. * [Test] [Del] Removed unnecessary tests that do not match the current functionality * [Feat!] [Del] Adjust the mooncake configuration method, remove the configuration file method, and only retain the parameter transmission method * [Doc] [Fix] modifiy the performance figure of Mooncake Store. * [Feat] add __del__() to shutdown all the mooncake components --------- Co-authored-by: z00452769 <zhangyichen@huawei.com> Co-authored-by: propanone1006 <1035097916@qq.com> Co-authored-by: propanone1006 <1035067916@qq.com> * [bugfix]modify mla dump (#128) * modify mla dump * fix ci problem * [BugFix] aggregate work ouputs to decide dumped blocks * [BugFix] Modify npu worker for aggregating modelrunner_outputs * [CI] Add vllm patch for sparse in dockerfile (#134) * [CI] Add vllm patch for sparse in dockerfile * [Fix] Add patch in dockerfile and pip mirror. * [Fix] Update version 0.0.2 * ESA: skip processing for short requests (#147) * ucm_sparse: skip processing for short requests * add comments --------- Co-authored-by: flesher0813 <33923823+flesher0813@users.noreply.github.com> Co-authored-by: harrisonyhq <harrisonyhq@gmail.com> Co-authored-by: hek14 <1023129548@qq.com> Co-authored-by: Chen Deng <120033622+propanone1006@users.noreply.github.com> Co-authored-by: propanone1006 <1035067916@qq.com> Co-authored-by: qyh111 <qiuyuhao1@huawei.com> Co-authored-by: Mag1c.H <hemajun815@163.com> Co-authored-by: t00939662 <tianxuehan@huawei.com> Co-authored-by: Fate469434 <58885253+Fate469434@users.noreply.github.com> Co-authored-by: y00945504 <yuhui87@huawei.com> Co-authored-by: Zbm1996 <370478722@qq.com> Co-authored-by: NaganooMei <290992347@qq.com> Co-authored-by: NaganooMei <104300720+NaganooMei@users.noreply.github.com> Co-authored-by: f00943869 <fenghao0720@outlook.com> Co-authored-by: hufumans <113507465+hufumans@users.noreply.github.com> Co-authored-by: z00452769 <zhangyichen@huawei.com> Co-authored-by: propanone1006 <1035097916@qq.com> Co-authored-by: zhou-haitao <74044944+zhou-haitao@users.noreply.github.com> Co-authored-by: flesher0813 <1208954694@qq.com> Co-authored-by: AooooooA-C <chenaozhu@outlook.com>
1 parent 7bc6d75 commit 9d91665

File tree

118 files changed

+6491
-1825
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

118 files changed

+6491
-1825
lines changed

.github/PULL_REQUEST_TEMPLATE.md

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,9 @@ BEFORE SUBMITTING, PLEASE READ OUR OFFICIAL WEBSITE.
44
55
-->
66

7-
# Prupose
7+
8+
# Purpose
9+
810

911
What this PR does / why we need it?
1012
<!--
Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
{
2+
"problemMatcher": [
3+
{
4+
"owner": "actionlint",
5+
"pattern": [
6+
{
7+
"regexp": "^(?:\\x1b\\[\\d+m)?(.+?)(?:\\x1b\\[\\d+m)*:(?:\\x1b\\[\\d+m)*(\\d+)(?:\\x1b\\[\\d+m)*:(?:\\x1b\\[\\d+m)*(\\d+)(?:\\x1b\\[\\d+m)*: (?:\\x1b\\[\\d+m)*(.+?)(?:\\x1b\\[\\d+m)* \\[(.+?)\\]$",
8+
"file": 1,
9+
"line": 2,
10+
"column": 3,
11+
"message": 4,
12+
"code": 5
13+
}
14+
]
15+
}
16+
]
17+
}
Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
{
2+
"problemMatcher": [
3+
{
4+
"owner": "mypy",
5+
"pattern": [
6+
{
7+
"regexp": "^(.+):(\\d+):\\s(error|warning):\\s(.+)$",
8+
"file": 1,
9+
"line": 2,
10+
"severity": 3,
11+
"message": 4
12+
}
13+
]
14+
}
15+
]
16+
}
Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
{
2+
"problemMatcher": [
3+
{
4+
"owner": "ruff",
5+
"pattern": [
6+
{
7+
"regexp": "^(.+?):(\\d+):(\\d+): (\\w+): (.+)$",
8+
"file": 1,
9+
"line": 2,
10+
"column": 3,
11+
"code": 4,
12+
"message": 5
13+
}
14+
]
15+
}
16+
]
17+
}

.github/workflows/pre-commit.yml

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
name: pre-commit
2+
3+
on:
4+
workflow_call:
5+
6+
permissions:
7+
contents: read
8+
9+
jobs:
10+
pre-commit:
11+
runs-on: ubuntu-latest
12+
steps:
13+
- name: Checkout repository
14+
uses: actions/checkout@v4
15+
16+
- name: Set up Python
17+
uses: actions/setup-python@v5
18+
with:
19+
python-version: "3.12"
20+
21+
- name: Add matchers for better error display
22+
run: |
23+
echo "::add-matcher::.github/workflows/matchers/actionlint.json"
24+
echo "::add-matcher::.github/workflows/matchers/mypy.json"
25+
26+
- name: Run pre-commit checks on all files
27+
uses: pre-commit/action@v3.0.1
28+
env:
29+
SHELLCHECK_OPTS: "--exclude=SC2046,SC2006,SC2086" # Exclude SC2046, SC2006, SC2086 for actionlint
30+
with:
31+
extra_args: --all-files --hook-stage manual
Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,55 @@
1+
# This starter workflow is for a CMake project running on a single platform. There is a different starter workflow if you need cross-platform coverage.
2+
# See: https://github.com/actions/starter-workflows/blob/main/ci/cmake-multi-platform.yml
3+
name: ucmnfsstore-ut
4+
5+
on:
6+
push:
7+
branches: [ "dev*", "main", "*release" ]
8+
pull_request:
9+
branches: [ "dev*", "main", "*release" ]
10+
11+
env:
12+
# Customize the CMake build type here (Release, Debug, RelWithDebInfo, etc.)
13+
BUILD_TYPE: Debug
14+
15+
jobs:
16+
ci:
17+
# The CMake configure and build commands are platform agnostic and should work equally well on Windows or Mac.
18+
# You can convert this to a matrix build if you need cross-platform coverage.
19+
# See: https://docs.github.com/en/free-pro-team@latest/actions/learn-github-actions/managing-complex-workflows#using-a-build-matrix
20+
runs-on: ubuntu-latest
21+
22+
steps:
23+
- uses: actions/checkout@v4
24+
25+
- name: Install googletest
26+
run: |
27+
git clone https://github.com/google/googletest.git --depth=1 --branch=release-1.11.0
28+
cd googletest
29+
mkdir build && cd build
30+
cmake -DCMAKE_CXX_FLAGS="-fPIC" -DCMAKE_C_FLAGS="-fPIC" -DCMAKE_CXX_STANDARD=17 -DCMAKE_CXX_STANDARD_REQUIRED=True ..
31+
sudo make install -j
32+
33+
- name: Install mockcpp
34+
run: |
35+
git clone https://github.com/sinojelly/mockcpp.git --depth=1
36+
cd mockcpp
37+
mkdir build && cd build
38+
cmake -DCMAKE_CXX_FLAGS="-fPIC" -DCMAKE_C_FLAGS="-fPIC" -DCMAKE_CXX_STANDARD=17 -DCMAKE_CXX_STANDARD_REQUIRED=True -DMOCKCPP_XUNIT="gtest" -DMOCKCPP_XUNIT_HOME=/usr/local/ ..
39+
sudo make install -j
40+
41+
- name: Configure CMake
42+
# Configure CMake in a 'build' subdirectory. `CMAKE_BUILD_TYPE` is only required if you are using a single-configuration generator such as make.
43+
# See https://cmake.org/cmake/help/latest/variable/CMAKE_BUILD_TYPE.html?highlight=cmake_build_type
44+
working-directory: ${{github.workspace}}/unifiedcache/csrc/ucmnfsstore
45+
run: cmake -B ${{github.workspace}}/build -DCMAKE_BUILD_TYPE=${{env.BUILD_TYPE}} -DBUILD_TESTS=ON -DCOVERAGE_ENABLED=ON -DDOWNLOAD_DEPENDENCE=ON -DRUNTIME_ENVIRONMENT=simu
46+
47+
- name: Build
48+
# Build your program with the given configuration
49+
run: cmake --build ${{github.workspace}}/build --config ${{env.BUILD_TYPE}}
50+
51+
- name: Test
52+
working-directory: ${{github.workspace}}/build
53+
# Execute tests defined by the CMake configuration.
54+
# See https://cmake.org/cmake/help/latest/manual/ctest.1.html for more detail
55+
run: ctest -C ${{env.BUILD_TYPE}}
Lines changed: 71 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,71 @@
1+
name: 'test'
2+
3+
on:
4+
push:
5+
branches:
6+
- 'main'
7+
- 'dev*'
8+
- '*release'
9+
pull_request:
10+
branches:
11+
- 'main'
12+
- 'dev*'
13+
- '*release'
14+
15+
jobs:
16+
call-lint:
17+
uses: ./.github/workflows/pre-commit.yml
18+
19+
unit-test:
20+
needs: call-lint
21+
name: Run Unittests
22+
runs-on: ubuntu-latest
23+
container:
24+
image: vllm/vllm-openai:v0.9.2
25+
env:
26+
VLLM_USE_PRECOMPILED: 1
27+
PLATFORM: cuda
28+
steps:
29+
- name: Install system deps
30+
run: |
31+
apt-get update -y
32+
apt-get install -y python3-pip git build-essential
33+
34+
- name: Clone vLLM
35+
run: |
36+
git clone --depth 1 --branch v0.9.2 \
37+
https://github.com/vllm-project/vllm.git \
38+
/vllm-workspace/vllm
39+
40+
- name: Install vLLM (CPU)
41+
working-directory: /vllm-workspace/vllm
42+
run: |
43+
pip uninstall -y vllm
44+
VLLM_TARGET_DEVICE=empty \
45+
pip install -v -e . \
46+
--extra-index-url https://download.pytorch.org/whl/cpu/
47+
48+
- name: Checkout unified-cache-management repo
49+
uses: actions/checkout@v4
50+
51+
- name: Print debug info
52+
run: |
53+
echo "GITHUB_WORKSPACE is $GITHUB_WORKSPACE"
54+
ls -l $GITHUB_WORKSPACE/unifiedcache/patch
55+
56+
- name: Debug repo layout
57+
run: |
58+
echo "GITHUB_WORKSPACE=$GITHUB_WORKSPACE"
59+
find "$GITHUB_WORKSPACE" -type f -name "*.patch"
60+
ls -l "$GITHUB_WORKSPACE"
61+
- name: Apply patch
62+
run: |
63+
git -C /vllm-workspace/vllm apply $GITHUB_WORKSPACE/unifiedcache/patch/0.9.2/vllm-adapt.patch
64+
65+
66+
- name: Install unified-cache-management
67+
run: pip install -v -e .
68+
69+
- name: Run ut
70+
run: python3 -m unittest discover -s test
71+

.gitignore

Lines changed: 46 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,52 @@
1-
# Development Enviroment
1+
# Prerequisites
2+
*.d
3+
4+
# Compiled Object files
5+
*.slo
6+
*.lo
7+
*.o
8+
*.obj
9+
10+
# Precompiled Headers
11+
*.gch
12+
*.pch
13+
14+
# Linker files
15+
*.ilk
16+
17+
# Debugger Files
18+
*.pdb
19+
20+
# Compiled Dynamic libraries
21+
*.so
22+
*.dylib
23+
*.dll
24+
25+
# Fortran module files
26+
*.mod
27+
*.smod
28+
29+
# Compiled Static libraries
30+
*.lai
31+
*.la
32+
*.a
33+
*.lib
34+
35+
# Executables
36+
*.exe
37+
*.out
38+
*.app
39+
40+
# Debug information files
41+
*.dwo
42+
43+
# Development environment files
44+
*.code-workspace
245
.vscode/**
346
.idea/**
447
.git/**
548
**/build/**
649
**/output/**
750
.venv/**
51+
**/__pycache__/**
52+
*.egg-info/**

.pre-commit-config.yaml

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
repos:
2+
- repo: https://github.com/codespell-project/codespell
3+
rev: v2.4.1
4+
hooks:
5+
- id: codespell
6+
args: [
7+
'--skip', 'unifiedcache/csrc/**,./unifiedcache.egg-info/**,.github/**',
8+
'-L', 'CANN,cann,NNAL,nnal,ASCEND,ascend,EnQue,CopyIn'
9+
]
10+
- repo: https://github.com/psf/black
11+
rev: 24.4.2
12+
hooks:
13+
- id: black
14+
language_version: python3
15+
- repo: https://github.com/PyCQA/isort
16+
rev: 6.0.1
17+
hooks:
18+
- id: isort
19+
args:
20+
- "--profile=black"
21+
- repo: https://github.com/rhysd/actionlint
22+
rev: v1.7.7
23+
hooks:
24+
- id: actionlint
25+
default_stages:
26+
- pre-commit
27+
- manual

MANIFEST.in

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
recursive-include unifiedcache/csrc *

0 commit comments

Comments
 (0)