关于Exchange SST 导入成功但是 在图中查不到数据

2022-06-29 16:12:08 INFO NebulaSSTWriter:29 - Loading RocksDB successfully
2022-06-29 16:12:08 INFO deprecation:1173 - fs.default.name is deprecated. Instead, use fs.defaultFS
2022-06-29 16:12:08 INFO NebulaSSTWriter:29 - Loading RocksDB successfully
2022-06-29 16:12:08 INFO deprecation:1173 - fs.default.name is deprecated. Instead, use fs.defaultFS
2022-06-29 16:12:09 INFO NebulaSSTWriter:29 - Loading RocksDB successfully
2022-06-29 16:12:09 INFO deprecation:1173 - fs.default.name is deprecated. Instead, use fs.defaultFS
2022-06-29 16:12:10 INFO NebulaSSTWriter:29 - Loading RocksDB successfully
2022-06-29 16:12:10 INFO deprecation:1173 - fs.default.name is deprecated. Instead, use fs.defaultFS
2022-06-29 16:12:10 INFO NebulaSSTWriter:29 - Loading RocksDB successfully
2022-06-29 16:12:10 INFO deprecation:1173 - fs.default.name is deprecated. Instead, use fs.defaultFS
2022-06-29 16:12:11 INFO NebulaSSTWriter:29 - Loading RocksDB successfully
2022-06-29 16:12:11 INFO deprecation:1173 - fs.default.name is deprecated. Instead, use fs.defaultFS
2022-06-29 16:12:11 INFO NebulaSSTWriter:29 - Loading RocksDB successfully
2022-06-29 16:12:11 INFO deprecation:1173 - fs.default.name is deprecated. Instead, use fs.defaultFS
2022-06-29 16:12:11 INFO Executor:54 - Finished task 1.0 in stage 7.0 (TID 71). 3209 bytes result sent to driver
2022-06-29 16:12:11 INFO TaskSetManager:54 - Finished task 1.0 in stage 7.0 (TID 71) in 3467 ms on localhost (executor driver) (2/2)
2022-06-29 16:12:11 INFO TaskSchedulerImpl:54 - Removed TaskSet 7.0, whose tasks have all completed, from pool
2022-06-29 16:12:11 INFO DAGScheduler:54 - ResultStage 7 (foreachPartition at EdgeProcessor.scala:143) finished in 5.021 s
2022-06-29 16:12:11 INFO DAGScheduler:54 - Job 3 finished: foreachPartition at EdgeProcessor.scala:143, took 5.760292 s
2022-06-29 16:12:11 INFO Exchange$:172 - import for edge account_account cost time: 6.08 s
2022-06-29 16:12:11 INFO Exchange$:178 - SST-Import: failure.account_account: 0
2022-06-29 16:12:11 INFO AbstractConnector:318 - Stopped Spark@311203f0{HTTP/1.1,[http/1.1]}{0.0.0.0:4040}
2022-06-29 16:12:11 INFO SparkUI:54 - Stopped Spark web UI at http://192.168.0.107:4040
2022-06-29 16:12:11 INFO MapOutputTrackerMasterEndpoint:54 - MapOutputTrackerMasterEndpoint stopped!
2022-06-29 16:12:11 INFO MemoryStore:54 - MemoryStore cleared
2022-06-29 16:12:11 INFO BlockManager:54 - BlockManager stopped
2022-06-29 16:12:11 INFO BlockManagerMaster:54 - BlockManagerMaster stopped
2022-06-29 16:12:11 INFO OutputCommitCoordinator$OutputCommitCoordinatorEndpoint:54 - OutputCommitCoordinator stopped!
2022-06-29 16:12:11 INFO SparkContext:54 - Successfully stopped SparkContext
2022-06-29 16:12:11 INFO ShutdownHookManager:54 - Shutdown hook called
2022-06-29 16:12:11 INFO ShutdownHookManager:54 - Deleting directory /private/var/folders/fq/fpzfw1md7mn_6f_qvb92tt800000gn/T/spark-d59ce346-bb3a-41de-9588-99c91c730c05
2022-06-29 16:12:11 INFO ShutdownHookManager:54 - Deleting directory /private/var/folders/fq/fpzfw1md7mn_6f_qvb92tt800000gn/T/spark-5b17865f-d869-4736-988f-91c27dfa4f71

这是生成sst文件的日志,sst文件生成之后,你还需要在console中执行download和ingest操作,ingest是否有成功?

1 个赞


还是不行

执行了 , 还是查不到数据

JOB 成功了么?新版本里这个过程好像是异步的了

Show jobs

1 个赞

我不太熟悉 exchange,不知道会不会和 https://github.com/vesoft-inc/nebula-exchange/issues/71 有关。

有没有试过 repartitionWithNebula: True 生成sst 看看?

试过了 , 还是没数据

能在帮忙看下吗 , 还是不行 , 同样的 csv 文件 ,用csv 的方式导入就可以 , 用 sst 就不行 , 也没报错什么的

请问 vertexid 的长度 不是刚好是8吧?

什么版本的exchange?vid type是fixed string吗?是不是长度正好是8?

3.0 的 exchange ,vid type 是 INT64

vid type 是 INT64


这两个类型的 vid type 我都试过

@nicole 这个太邪门了,用 nebula sink 是可以的,sst 就是空的,vid字段是数字类型的,所以int64/fixed_string 都试过了也。

@Tianwen 能给 head 一下你的 csv 数据的几行么,我想看看 vid 那一列长什么样子?

image

还是用的这个数据试了一下导入 hive 数据 , 都是可以的 , 就 sst 还是不行

1 个赞

确认一下哈,在 download 之后,ingest 之前,在 storage 节点上,在 hdfs 上的文件已经下载到 data 目录之下了对吧?

正在上传:481656904560_.pic.jpg…


没有 , /data 目录下没有