|
1 | | ---- |
2 | | -title: map-in-map 管理增强 |
3 | | -authors: |
4 | | -- "@nlgwcy" |
5 | | -reviewers: |
6 | | -- "@hzxuzhonghu" |
7 | | -- "@supercharge-xsy" |
8 | | -- "@bitcoffeeiux" |
9 | | -approvers: |
10 | | -- "@robot" |
11 | | -- TBD |
12 | | - |
13 | | -creation-date: 2024-07-20 |
14 | | - |
15 | | ---- |
16 | | - |
17 | | -## map-in-map 管理增强 |
18 | | - |
19 | | -### 摘要 |
20 | | - |
21 | | -在 ads 模式下,支持基于 map-in-map 记录的弹性伸缩,以满足大规模集群的流量管理需求。 |
22 | | - |
23 | | -### 动机 |
24 | | - |
25 | | -正如 [optimizing_bpf_map_update_in_xDS_mode](https://github.com/kmesh-net/kmesh/blob/main/docs/proposal/optimizing_bpf_map_update_in_xDS_mode-en.md) 中提到的,为了解决 map-in-map 记录更新缓慢的问题,Kmesh 通过以空间换时间的方式在启动时一次性创建所有记录。在小规模集群场景下不会出现这个问题,但是,当支持大规模集群(例如,5000 个服务和 100,000 个 Pod)时,map-in-map 表中定义的大小非常大,并且 `BPF_MAP_TYPE_ARRAY_OF_MAPS` 类型的 map 不支持 `BPF_F_NO_PREALLOC`,这会导致大量的内存浪费。必须支持 map-in-map 记录的弹性伸缩,以满足大规模集群的流量管理需求。 |
26 | | - |
27 | | -#### 目标 |
28 | | - |
29 | | -- 支持大规模集群中的流量管理。 |
30 | | -- 考虑配置恢复场景。 |
31 | | - |
32 | | -### 提案 |
33 | | - |
34 | | -Kmesh 在用户模式下管理 map-in-map 的使用。为了支持弹性伸缩,管理结构扩展如下: |
35 | | - |
36 | | -```c |
37 | | -struct inner_map_mng { |
38 | | - int inner_fd; |
39 | | - int outter_fd; |
40 | | - struct bpf_map_info inner_info; |
41 | | - struct bpf_map_info outter_info; |
42 | | - struct inner_map_stat inner_maps[MAX_OUTTER_MAP_ENTRIES]; |
43 | | - int elastic_slots[OUTTER_MAP_ELASTIC_SIZE]; |
44 | | - int used_cnt; // real used count |
45 | | - int alloced_cnt; // real alloced count |
46 | | - int max_alloced_idx; // max alloced index, there may be holes. |
47 | | - int init; |
48 | | - sem_t fin_tasks; |
49 | | - int elastic_task_exit; // elastic scaling thread exit flag |
50 | | -}; |
51 | | - |
52 | | -struct inner_map_stat { |
53 | | - int map_fd; |
54 | | - unsigned int used : 1; |
55 | | - unsigned int alloced : 1; |
56 | | - unsigned int resv : 30; |
57 | | -}; |
58 | | -``` |
59 | | - |
60 | | -Map-in-map 伸缩过程: |
61 | | - |
62 | | - |
63 | | - |
64 | | -以下是 map-in-map 缩容和扩容的示例: |
65 | | - |
66 | | - |
| 1 | +--- |
| 2 | +title: map-in-map ������ǿ |
| 3 | +authors: |
| 4 | +- "@nlgwcy" |
| 5 | +reviewers: |
| 6 | +- "@hzxuzhonghu" |
| 7 | +- "@supercharge-xsy" |
| 8 | +- "@bitcoffeeiux" |
| 9 | +approvers: |
| 10 | +- "@robot" |
| 11 | +- TBD |
| 12 | + |
| 13 | +creation-date: 2024-07-20 |
| 14 | + |
| 15 | +--- |
| 16 | + |
| 17 | +## map-in-map ������ǿ |
| 18 | + |
| 19 | +### ժҪ |
| 20 | + |
| 21 | +�� ads ģʽ�£�֧�ֻ��� map-in-map ��¼�ĵ�����������������ģ��Ⱥ�������������� |
| 22 | + |
| 23 | +### ���� |
| 24 | + |
| 25 | +���� [optimizing_bpf_map_update_in_xDS_mode](https://github.com/kmesh-net/kmesh/blob/main/docs/proposal/optimizing_bpf_map_update_in_xDS_mode-en.md) ���ᵽ�ģ�Ϊ�˽�� map-in-map ��¼���»��������⣬Kmesh ͨ���Կռ任ʱ��ķ�ʽ�����ʱһ���Դ������м�¼����С��ģ��Ⱥ�����²������������⣬���ǣ���֧�ִ��ģ��Ⱥ�����磬5000 ������� 100,000 �� Pod��ʱ��map-in-map ���ж���Ĵ�С�dz����� `BPF_MAP_TYPE_ARRAY_OF_MAPS` ���͵� map ��֧�� `BPF_F_NO_PREALLOC`����ᵼ�´������ڴ��˷ѡ�����֧�� map-in-map ��¼�ĵ�����������������ģ��Ⱥ�������������� |
| 26 | + |
| 27 | +#### Ŀ�� |
| 28 | + |
| 29 | +- ֧�ִ��ģ��Ⱥ�е���������� |
| 30 | +- �������ûָ������� |
| 31 | + |
| 32 | +### � |
| 33 | + |
| 34 | +Kmesh ���û�ģʽ�¹��� map-in-map ��ʹ�á�Ϊ��֧�ֵ�������������ṹ��չ���£� |
| 35 | + |
| 36 | +```c |
| 37 | +struct inner_map_mng { |
| 38 | + int inner_fd; |
| 39 | + int outter_fd; |
| 40 | + struct bpf_map_info inner_info; |
| 41 | + struct bpf_map_info outter_info; |
| 42 | + struct inner_map_stat inner_maps[MAX_OUTTER_MAP_ENTRIES]; |
| 43 | + int elastic_slots[OUTTER_MAP_ELASTIC_SIZE]; |
| 44 | + int used_cnt; // real used count |
| 45 | + int alloced_cnt; // real alloced count |
| 46 | + int max_alloced_idx; // max alloced index, there may be holes. |
| 47 | + int init; |
| 48 | + sem_t fin_tasks; |
| 49 | + int elastic_task_exit; // elastic scaling thread exit flag |
| 50 | +}; |
| 51 | + |
| 52 | +struct inner_map_stat { |
| 53 | + int map_fd; |
| 54 | + unsigned int used : 1; |
| 55 | + unsigned int alloced : 1; |
| 56 | + unsigned int resv : 30; |
| 57 | +}; |
| 58 | +``` |
| 59 | + |
| 60 | +Map-in-map �������̣� |
| 61 | + |
| 62 | + |
| 63 | + |
| 64 | +������ map-in-map ���ݺ����ݵ�ʾ���� |
| 65 | + |
| 66 | + |
| 67 | + |
0 commit comments