nebula3.0 内存占用

storage 部分日志

6589 I20220929 11:26:41.838034  5449 SlowOpTracker.h:31] [Port: 9780, Space: 4, Part: 25] total time:933ms, Write WAL, total 2
6590 I20220929 11:26:42.826150  5449 SlowOpTracker.h:31] [Port: 9780, Space: 6, Part: 35] total time:822ms, Write WAL, total 2
6591 I20220929 11:27:35.846838  5471 SlowOpTracker.h:31] [Port: 9780, Space: 5, Part: 43] total time:143ms, Write WAL, total 2
6592 I20220929 11:27:37.837945  5471 SlowOpTracker.h:31] [Port: 9780, Space: 6, Part: 87] total time:612ms, Write WAL, total 2
6593 I20220929 11:27:38.836521  5471 SlowOpTracker.h:31] [Port: 9780, Space: 6, Part: 3] total time:957ms, Write WAL, total 2
6594 I20220929 11:27:39.841322  5471 SlowOpTracker.h:31] [Port: 9780, Space: 4, Part: 52] total time:938ms, Write WAL, total 2
6595 I20220929 11:27:40.840849  5471 SlowOpTracker.h:31] [Port: 9780, Space: 4, Part: 38] total time:339ms, Write WAL, total 2
6596 I20220929 11:27:46.871877  5449 SlowOpTracker.h:31] [Port: 9780, Space: 3, Part: 60] total time:402ms, Write WAL, total 2
6597 I20220929 11:27:47.856581  5446 SlowOpTracker.h:31] [Port: 9780, Space: 6, Part: 74] total time:921ms, Write WAL, total 2
6598 I20220929 11:30:04.936359  5449 SlowOpTracker.h:31] [Port: 9780, Space: 3, Part: 74] total time:708ms, Write WAL, total 2
6599 I20220929 11:31:02.955296  5448 SlowOpTracker.h:31] [Port: 9780, Space: 4, Part: 63] total time:742ms, Write WAL, total 2
6600 I20220929 11:34:05.115072  5456 SlowOpTracker.h:31] [Port: 9780, Space: 4, Part: 79] total time:330ms, Write WAL, total 2
6601 I20220929 11:35:06.154806  5449 SlowOpTracker.h:31] [Port: 9780, Space: 4, Part: 26] total time:436ms, Write WAL, total 2
6602 I20220929 11:36:06.172777  5456 SlowOpTracker.h:31] [Port: 9780, Space: 4, Part: 91] total time:544ms, Write WAL, total 2
6603 I20220929 11:40:09.324720  5449 SlowOpTracker.h:31] [Port: 9780, Space: 3, Part: 71] total time:208ms, Write WAL, total 2
6604 I20220929 11:40:10.321213  5456 SlowOpTracker.h:31] [Port: 9780, Space: 3, Part: 83] total time:353ms, Write WAL, total 2
6605 I20220929 11:40:12.323447  5448 SlowOpTracker.h:31] [Port: 9780, Space: 3, Part: 17] total time:763ms, Write WAL, total 2
6606 I20220929 11:44:11.474040  5446 SlowOpTracker.h:31] [Port: 9780, Space: 5, Part: 42] total time:832ms, Write WAL, total 2
6607 I20220929 12:55:28.459122  5505 EventListener.h:18] Rocksdb start compaction column family: default because of LevelL0FilesNum, status: OK, compacted 5 files into 0, ba     se level is 0, output level is 1
6608 I20220929 12:55:28.460000  5505 CompactionFilter.h:54] Do full/manual compaction!
6609 I20220929 12:55:29.591056  5505 EventListener.h:28] Rocksdb compaction completed column family: default because of LevelL0FilesNum, status: OK, compacted 5 files into 1     , base level is 0, output level is 1
6610 I20220929 18:15:37.203197  5471 AdminTask.cpp:19] createAdminTask (7, 0)
6611 I20220929 18:15:37.238875  5471 AdminTaskManager.cpp:153] enqueue task(7, 0), con req=2147483647
6612 I20220929 18:15:37.240670  7081 AdminTaskManager.cpp:253] dequeue task(7, 0)
6613 I20220929 18:15:37.240902  7081 AdminTaskManager.cpp:298] run task(7, 0), 100 subtasks in 10 thread
6614 I20220929 18:15:37.241179 17419 StatsTask.cpp:102] Start stats task
6615 I20220929 18:15:37.241336 17420 StatsTask.cpp:102] Start stats task
6616 I20220929 18:15:37.241464 17421 StatsTask.cpp:102] Start stats task
6617 I20220929 18:15:37.241601 17422 StatsTask.cpp:102] Start stats task
6618 I20220929 18:15:37.241712 17423 StatsTask.cpp:102] Start stats task
6619 I20220929 18:15:37.241858 17424 StatsTask.cpp:102] Start stats task
6620 I20220929 18:15:37.241998 17425 StatsTask.cpp:102] Start stats task
6621 I20220929 18:15:37.242154 17426 StatsTask.cpp:102] Start stats task
6622 I20220929 18:15:37.242300 17427 StatsTask.cpp:102] Start stats task
6623 I20220929 18:15:37.242435 17428 StatsTask.cpp:102] Start stats task
6624 I20220929 18:15:37.242455  7081 AdminTaskManager.cpp:241] waiting for incoming task
6625 I20220929 18:15:37.409646 17426 StatsTask.cpp:279] Stats task finished
6626 I20220929 18:15:37.409679 17426 AdminTaskManager.cpp:334] subtask of task(7, 0) finished, unfinished task 99
6627 I20220929 18:15:37.409699 17426 StatsTask.cpp:102] Start stats task
6628 I20220929 18:15:37.410228 17427 StatsTask.cpp:279] Stats task finished
6629 I20220929 18:15:37.410252 17427 AdminTaskManager.cpp:334] subtask of task(7, 0) finished, unfinished task 98
6630 I20220929 18:15:37.410269 17427 StatsTask.cpp:102] Start stats task
6631 I20220929 18:15:37.411021 17428 StatsTask.cpp:279] Stats task finished
6632 I20220929 18:15:37.411046 17428 AdminTaskManager.cpp:334] subtask of task(7, 0) finished, unfinished task 97
6633 I20220929 18:15:37.411062 17428 StatsTask.cpp:102] Start stats task
6634 I20220929 18:15:37.415784 17426 StatsTask.cpp:279] Stats task finished
6635 I20220929 18:15:37.415804 17426 AdminTaskManager.cpp:334] subtask of task(7, 0) finished, unfinished task 96
6636 I20220929 18:15:37.415822 17426 StatsTask.cpp:102] Start stats task

默认的配置下如果5个space都有写入,光这部分内存最大可能会达到5x100x16M = 8G。你单个节点配置也不太高的情况下,把分片数量都降到20看下。
目前分片数量不能改,只好重新创建图空间来导入了。

2 个赞

16M 指是的什么呢,每个分片 wal log的缓存的大小?

还有内存持续增长, 不释放吗?

是缓存的最大值,每个分片默认不会超过16M

1 个赞

此话题已在最后回复的 30 天后被自动关闭。不再允许新回复。