[WIP] Implementation of DRAM connector #285
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ChenyuZhu1
requested review from
harrisonyhq,
mag1c-h and
ygwpz
as code owners
harrisonyhq
reviewed
Oct 15, 2025
| void Commit(const std::string& block, const bool success) override {} | ||
| int32_t Setup(const Config& config) { | ||
| // config里之前没有blockSize，稳妥起见还是先打桩 | ||
| int32_t blockSize = 128; |
There was a problem hiding this comment.
can get blocksize here from vllm.config?
There was a problem hiding this comment.
Yes you are right, but currently blockSize is not added into config. If possible, it will be added into the Config class soon.
qyh111
reviewed
Oct 16, 2025
| void Commit(const std::string& block, const bool success) override {} | ||
| int32_t Setup(const Config& config) { | ||
| // config里之前没有blockSize，稳妥起见还是先打桩 | ||
| int32_t blockSize = 128; |
There was a problem hiding this comment.
Do we need blocksize? config.def_readwrite("ioSize", &UC::DRAMStorePy::Config::ioSize) we have iosize here
| this->_device = DeviceFactory::Make(deviceId, 0, 0); // 这里不需要buffer，暂时都先传0吧 | ||
| if (!this->_device) { return Status::OutOfMemory(); } | ||
| } | ||
| if (!this->_streamOper.Setup([this](DramTsfTask& task) { this->StreamOper(task); })) { |
There was a problem hiding this comment.
I think this->_device->Setup() shoulld be called here
ucm/store/infra/memory/memory_pool.h
Outdated
| uint32_t GetNextAvailableOffset() const { return head_; } | ||
|
|
||
| Status NewBlock(std::string blockId) { | ||
| if (head_ >= capacity_) { |
There was a problem hiding this comment.
Do we need to consider concurrency issues here?
There was a problem hiding this comment.
memory pool is rewritten today
| this->memPool_ = std::make_unique<MemoryPool>(config.capacity, blockSize).release(); | ||
| // 初始化memPool的办法是否正确？如果失败的话怎么办？ | ||
| int32_t streamNumber = 60; // 这个参数是否需要，以及怎么传，还要讨论 | ||
| int32_t timeoutMs = 10000; // 这个参数是否需要，以及怎么传，还要讨论 |
There was a problem hiding this comment.
We need to add these two variables to the config.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Purpose
What this PR does / why we need it?
Implementation of DRAM connector
Modifications
A new infrastructure, the MemoryPool class is implemented and referenced in the dramstore implementation class, to store the KVCache data. The memory pool is initialized at the beginning with capacity and blockSize, and NewBlock(), LookupBlock(), GetAddress(), CommitBlock() interfaces are implemented here. It acts like an LRU-Cache. The storage pool is divided into slotNum=capacity/blockSize slots with dummy block names in the format
__slot_{index}. During the initialization stage, all slots are pushed back into the LRU queue. Each time an allocation request (blockId) is requested, the last element of the LRU queue is popped, with the corresponding address assigned to the new blockId, which is soonly added intoaddressMap_.Each time a block is committed, whether true or false, the block will be pushed into the LRU queue. If success==true, then it will be pushed to the front of the LRU queue (so that it is unlikely to be popped out soon); and if success==false, then it will be pushed to the tail of the LRU queue (so that it will be soonly popped out next time an allocation request comes). If success==true, the blockId will be added into the
availableBlocks_set.Also, in the dramstore/domain directory, DramTsfTaskManager is implemented with a transfer task queue similar to the counterpart in the nfsstore.
Test
under folder infra/mem_pool_test.cc and passed in CI