私有部署版数据集成中无法停止和删除同步任务

Lee Lv.3修习发布于 2023-12-09 10:26 最后回复 2024-01-26 20:28:40 问答

15 回复1071 浏览

私有部署版数据集成中无法删除同步任务，因为 MySQL 数据库中断一次，数据同步就没同步上去，但是显示仍在同步中，数据库中也没有数据，数据同步任务无法停止，导致无法删除同步任务，没办法重新启动。 [图片]

αппəх Lv.6传道 2024-01-26 20:28:40

井正天 2024-01-26 19:15:09

没影响

没影响

好的多谢

井正天 Lv.2入道明道云员工 2024-01-26 19:15:09

αппəх 2024-01-26 19:12:59

始终连不上 flink 服务，我将 flink 单独部署了，微服务按照教程修改了参数配置，加入 fink 服务的 swarn 作为 worker，每次重启微服务都会提示 Image loaded: registry.cn-hangzhou.aliyuncs.com/mdpublic/mingdaoyun-community:5.1.0 Image loaded: registry.cn-hangzhou.aliyuncs.com/mdpublic/mingdaoyun-doc:1.2.0 WARNING: The Docker Engine you're using is running in swarm mode. Compose does not use swarm mode to deploy services to multiple nodes in a swarm. All containers will be scheduled on the current node. To deploy your application across the swarm, use `docker stack deploy`. 这个会有什么影响吗
始终连不上 flink 服务，我将 flink 单独部署了，微服务按照教程修改了参数配置，加入 fink 服务的 swarn 作为 worker，每次重启微服务都会提示
Image loaded: registry.cn-hangzhou.aliyuncs.com/mdpublic/mingdaoyun-community:5.1.0
Image loaded: registry.cn-hangzhou.aliyuncs.com/mdpublic/mingdaoyun-doc:1.2.0
WARNING: The Docker Engine you're using is running in swarm mode.

Compose does not use swarm mode to deploy services to multiple nodes in a swarm. All containers will be scheduled on the current node.

To deploy your application across the swarm, use `docker stack deploy`.
这个会有什么影响吗

没影响

αппəх Lv.6传道 2024-01-26 19:12:59

井正天 2024-01-26 18:20:35

docker exec -it $(docker ps|grep mingdaoyun-flink|awk '{print $1}') bash -c "mv /data/flink/zookeeper /data/flink/zookeeper.bakcup-1" 重启 docker restart $(docker ps|grep mingdaoyun-flink|awk '{print $1}') 等 5 分钟

docker exec -it $(docker ps|grep mingdaoyun-flink|awk '{print $1}') bash -c "mv /data/flink/zookeeper /data/flink/zookeeper.bakcup-1"

重启
docker restart $(docker ps|grep mingdaoyun-flink|awk '{print $1}')
等 5 分钟

始终连不上 flink 服务，我将 flink 单独部署了，微服务按照教程修改了参数配置，加入 fink 服务的 swarn 作为 worker，每次重启微服务都会提示

Image loaded: registry.cn-hangzhou.aliyuncs.com/mdpublic/mingdaoyun-community:5.1.0
Image loaded: registry.cn-hangzhou.aliyuncs.com/mdpublic/mingdaoyun-doc:1.2.0
WARNING: The Docker Engine you're using is running in swarm mode.

Compose does not use swarm mode to deploy services to multiple nodes in a swarm. All containers will be scheduled on the current node.

To deploy your application across the swarm, use `docker stack deploy`.

这个会有什么影响吗

井正天 Lv.2入道明道云员工 2024-01-26 18:20:35

docker exec -it $(docker ps|grep mingdaoyun-flink|awk '{print $1}') bash -c "mv /data/flink/zookeeper /data/flink/zookeeper.bakcup-1"
重启
docker restart $(docker ps|grep mingdaoyun-flink|awk '{print $1}')
等 5 分钟

αппəх Lv.6传道 2024-01-26 17:59:52

有什么方式可以强制删除任务吗

αппəх Lv.6传道 2024-01-26 17:53:22

井正天 2024-01-26 17:45:15

根据输出是数据集成进程没有了，可以重启下服务，建议服务器配置参考和明道云服务器部署在一起的话参考这里（需要额外资源，推荐是 16C40G 以上且单运行明道云服务时内存冗余 8G 以上）： https://docs.pd.mingdao.com/faq/integrate/flink/combine 新增服务器部署参考这里： https://docs.pd.mingdao.com/faq/integrate/flink/split 新增服务器资源推荐： https://docs.pd.mingdao.com/deployment/source#%E6%95%B0%E6%8D%AE%E9%9B%86%E6%88%90%E6%9C%8D%E5%8A%A1

根据输出是数据集成进程没有了，可以重启下服务，建议服务器配置参考
和明道云服务器部署在一起的话参考这里（需要额外资源，推荐是 16C40G 以上且单运行明道云服务时内存冗余 8G 以上）：https://docs.pd.mingdao.com/faq/integrate/flink/combine
新增服务器部署参考这里：https://docs.pd.mingdao.com/faq/integrate/flink/split
新增服务器资源推荐：https://docs.pd.mingdao.com/deployment/source#%E6%95%B0%E6%8D%AE%E9%9B%86%E6%88%90%E6%9C%8D%E5%8A%A1

[Y]                                           User: root
[Y]                        Selinux: getenforce not found
[Y]                               Firewalld: not running
[Y]                                              CPU: 12
[Y]                                           Memory: 62
[Y]                                   docker: executable
[Y]                                  dockerd: executable
[Y]                                  MaxMapCount: 262144
[Y]                                   SysFileNr: 2048000
[Y]                                       IPv4Forward: 1
[Y]                          DockerCgroupDrive: cgroupfs
[Y]                                DockerdFileNr: 524288
[Y]                               DockerdVersion: 25.0.1
[Y]                                          Umask: 0022
--------------------------------------------------------
Start ServerId service in script mode
--------------------------------------------------------
Image loaded: registry.cn-hangzhou.aliyuncs.com/mdpublic/mingdaoyun-command:node1018-python36
Image loaded: registry.cn-hangzhou.aliyuncs.com/mdpublic/mingdaoyun-community:5.1.0
Image loaded: registry.cn-hangzhou.aliyuncs.com/mdpublic/mingdaoyun-doc:1.2.0
Image loaded: registry.cn-hangzhou.aliyuncs.com/mdpublic/mingdaoyun-flink:1.17.1.500
Image loaded: registry.cn-hangzhou.aliyuncs.com/mdpublic/mingdaoyun-sc:1.0.0
Creating network "script_default" with the default driver
Creating script_sc_1      ... done
Creating script_doc_1     ... done
Creating script_app_1     ... done
Creating script_flink_1   ... done
Creating script_command_1 ... done
The complete service startup is expected to take 5 minutes, please do not access the service during startup!
[###################################################][100%][|]
Service started normally
______  _______               _________
___   |/  /__(_)_____________ ______  /_____ ______
__  /|_/ /__  /__  __ \_  __ `/  __  /_  __ `/  __ \
_  /  / / _  / _  / / /  /_/ // /_/ / / /_/ // /_/ /
/_/  /_/  /_/  /_/ /_/_\__, / \__,_/  \__,_/ \____/
                      /____/

root@R440:/usr/local/MDPrivateDeployment# docker exec -it $(docker ps|grep mingdaoyun-flink|awk '{print $1}') bash -c "ps -ef"
UID          PID    PPID  C STIME TTY          TIME CMD
root           1       0  0 09:22 ?        00:00:00 /bin/bash /run.sh
root         256       1  0 09:27 ?        00:00:04 /opt/java/openjdk/bin/java -Dzookeeper.jmx.log4j.disable=true -Duser.timezone=Asia/Shanghai -Dlog.file=/opt/flink/log/flink--zookeeper-0-572b15c8422
root         999       1  0 09:27 ?        00:00:00 sleep 1000
root        1346       0  0 09:40 pts/0    00:00:00 ps -ef

重启之后还是这样，还有其他解决办法吗

井正天 Lv.2入道明道云员工 2024-01-26 17:45:15

αппəх 2024-01-26 17:42:38

请问执行之后如何处理呢，我也遇到同样的问题，更新了社区版，服务器重启，数据集成不工作，也删不了 root@R440:/usr/local/MDPrivateDeployment# docker exec -it $(docker ps|grep mingdaoyun-flink|awk '{print $1}') bash -c "ps -ef" UID PID PPID C STIME TTY TIME CMD root 1 0 0 09:22 ? 00:00:00 /bin/bash /run.sh root 256 1 0 09:27 ? 00:00:04 /opt/java/openjdk/bin/java -Dzookeeper.jmx.log4j.disable=true -Duser.timezone=Asia/Shanghai -Dlog.file=/opt/flink/log/flink--zookeeper-0-572b15c8422 root 999 1 0 09:27 ? 00:00:00 sleep 1000 root 1346 0 0 09:40 pts/0 00:00:00 ps -ef root@R440:/usr/local/MDPrivateDeployment#

请问执行之后如何处理呢，我也遇到同样的问题，更新了社区版，服务器重启，数据集成不工作，也删不了
root@R440:/usr/local/MDPrivateDeployment# docker exec -it $(docker ps|grep mingdaoyun-flink|awk '{print $1}') bash -c "ps -ef"
UID PID PPID C STIME TTY TIME CMD
root 1 0 0 09:22 ? 00:00:00 /bin/bash /run.sh
root 256 1 0 09:27 ? 00:00:04 /opt/java/openjdk/bin/java -Dzookeeper.jmx.log4j.disable=true -Duser.timezone=Asia/Shanghai -Dlog.file=/opt/flink/log/flink--zookeeper-0-572b15c8422
root 999 1 0 09:27 ? 00:00:00 sleep 1000
root 1346 0 0 09:40 pts/0 00:00:00 ps -ef
root@R440:/usr/local/MDPrivateDeployment#

根据输出是数据集成进程没有了，可以重启下服务，建议服务器配置参考
和明道云服务器部署在一起的话参考这里（需要额外资源，推荐是 16C40G 以上且单运行明道云服务时内存冗余 8G 以上）：https://docs.pd.mingdao.com/faq/integrate/flink/combine
新增服务器部署参考这里：https://docs.pd.mingdao.com/faq/integrate/flink/split
新增服务器资源推荐：https://docs.pd.mingdao.com/deployment/source#%E6%95%B0%E6%8D%AE%E9%9B%86%E6%88%90%E6%9C%8D%E5%8A%A1

αппəх Lv.6传道 2024-01-26 17:42:38

井正天 2023-12-11 13:41:57

在数据集成服务器上操作下这个看看呢 docker exec -it $(docker ps|grep mingdaoyun-flink|awk '{print $1}') bash -c "ps -ef"

在数据集成服务器上操作下这个看看呢
docker exec -it $(docker ps|grep mingdaoyun-flink|awk '{print $1}') bash -c "ps -ef"

请问执行之后如何处理呢，我也遇到同样的问题，更新了社区版，服务器重启，数据集成不工作，也删不了
root@R440:/usr/local/MDPrivateDeployment# docker exec -it $(docker ps|grep mingdaoyun-flink|awk '{print $1}') bash -c "ps -ef"
UID PID PPID C STIME TTY TIME CMD
root 1 0 0 09:22 ? 00:00:00 /bin/bash /run.sh
root 256 1 0 09:27 ? 00:00:04 /opt/java/openjdk/bin/java -Dzookeeper.jmx.log4j.disable=true -Duser.timezone=Asia/Shanghai -Dlog.file=/opt/flink/log/flink--zookeeper-0-572b15c8422
root 999 1 0 09:27 ? 00:00:00 sleep 1000
root 1346 0 0 09:40 pts/0 00:00:00 ps -ef
root@R440:/usr/local/MDPrivateDeployment#