私有部署版数据集成中无法删除同步任务,因为 MySQL 数据库中断一次,数据同步就没同步上去,但是显示仍在同步中,数据库中也没有数据,数据同步任务无法停止,导致无法删除同步任务,没办法重新启动。
井正天(Team) 2024-01-26 19:15:09没影响
好的 多谢
Yanc 2024-01-26 19:12:59始终连不上 flink 服务,我将 flink 单独部署了,微服务按照教程修改了参数配置,加入 fink 服务的 swarn 作为 worker,每次重启微服务都会提示
Image loaded: registry.cn-hangzhou.aliyuncs.com/mdpublic/mingdaoyun-community:5.1.0 Image loaded: registry.cn-hangzhou.aliyuncs.com/mdpublic/mingdaoyun-doc:1.2.0 WARNING: The Docker Engine you're using is running in swarm mode. Compose does not use swarm mode to deploy services to multiple nodes in a swarm. All containers will be scheduled on the current node. To deploy your application across the swarm, use `docker stack deploy`.
这个会有什么影响吗
没影响
井正天(Team) 2024-01-26 18:20:35
docker exec -it $(docker ps|grep mingdaoyun-flink|awk '{print $1}') bash -c "mv /data/flink/zookeeper /data/flink/zookeeper.bakcup-1"
- 重启
docker restart $(docker ps|grep mingdaoyun-flink|awk '{print $1}')
等 5 分钟
始终连不上 flink 服务,我将 flink 单独部署了,微服务按照教程修改了参数配置,加入 fink 服务的 swarn 作为 worker,每次重启微服务都会提示
Image loaded: registry.cn-hangzhou.aliyuncs.com/mdpublic/mingdaoyun-community:5.1.0
Image loaded: registry.cn-hangzhou.aliyuncs.com/mdpublic/mingdaoyun-doc:1.2.0
WARNING: The Docker Engine you're using is running in swarm mode.
Compose does not use swarm mode to deploy services to multiple nodes in a swarm. All containers will be scheduled on the current node.
To deploy your application across the swarm, use `docker stack deploy`.
这个会有什么影响吗
Yanc 2024-01-26 18:12:06
从日志来看确实因为 ip 变了导致连不上数据库,这种情况下如何强制删除任务呢
docker exec -it $(docker ps|grep mingdaoyun-flink|awk '{print $1}') bash -c "mv /data/flink/zookeeper /data/flink/zookeeper.bakcup-1"
- 重启
docker restart $(docker ps|grep mingdaoyun-flink|awk '{print $1}')
等 5 分钟
有什么方式可以强制删除任务吗
井正天(Team) 2024-01-26 17:45:15根据输出是数据集成进程没有了,可以重启下服务,建议服务器配置参考
和明道云服务器部署在一起的话参考这里(需要额外资源,推荐是 16C40G 以上且单运行明道云服务时内存冗余 8G 以上):https://docs.pd.mingdao.com/faq/integrate/flink/combine
新增服务器部署参考这里:https://docs.pd.mingdao.com/faq/integrate/flink/split
新增服务器资源推荐:https://docs.pd.mingdao.com/deployment/source#%E6%95%B0%E6%8D%AE%E9%9B%86%E6%88%90%E6%9C%8D%E5%8A%A1
[Y] User: root
[Y] Selinux: getenforce not found
[Y] Firewalld: not running
[Y] CPU: 12
[Y] Memory: 62
[Y] docker: executable
[Y] dockerd: executable
[Y] MaxMapCount: 262144
[Y] SysFileNr: 2048000
[Y] IPv4Forward: 1
[Y] DockerCgroupDrive: cgroupfs
[Y] DockerdFileNr: 524288
[Y] DockerdVersion: 25.0.1
[Y] Umask: 0022
--------------------------------------------------------
Start ServerId service in script mode
--------------------------------------------------------
Image loaded: registry.cn-hangzhou.aliyuncs.com/mdpublic/mingdaoyun-command:node1018-python36
Image loaded: registry.cn-hangzhou.aliyuncs.com/mdpublic/mingdaoyun-community:5.1.0
Image loaded: registry.cn-hangzhou.aliyuncs.com/mdpublic/mingdaoyun-doc:1.2.0
Image loaded: registry.cn-hangzhou.aliyuncs.com/mdpublic/mingdaoyun-flink:1.17.1.500
Image loaded: registry.cn-hangzhou.aliyuncs.com/mdpublic/mingdaoyun-sc:1.0.0
Creating network "script_default" with the default driver
Creating script_sc_1 ... done
Creating script_doc_1 ... done
Creating script_app_1 ... done
Creating script_flink_1 ... done
Creating script_command_1 ... done
The complete service startup is expected to take 5 minutes, please do not access the service during startup!
[###################################################][100%][|]
Service started normally
______ _______ _________
___ |/ /__(_)_____________ ______ /_____ ______
__ /|_/ /__ /__ __ \_ __ `/ __ /_ __ `/ __ \
_ / / / _ / _ / / / /_/ // /_/ / / /_/ // /_/ /
/_/ /_/ /_/ /_/ /_/_\__, / \__,_/ \__,_/ \____/
/____/
root@R440:/usr/local/MDPrivateDeployment# docker exec -it $(docker ps|grep mingdaoyun-flink|awk '{print $1}') bash -c "ps -ef"
UID PID PPID C STIME TTY TIME CMD
root 1 0 0 09:22 ? 00:00:00 /bin/bash /run.sh
root 256 1 0 09:27 ? 00:00:04 /opt/java/openjdk/bin/java -Dzookeeper.jmx.log4j.disable=true -Duser.timezone=Asia/Shanghai -Dlog.file=/opt/flink/log/flink--zookeeper-0-572b15c8422
root 999 1 0 09:27 ? 00:00:00 sleep 1000
root 1346 0 0 09:40 pts/0 00:00:00 ps -ef
重启之后还是这样,还有其他解决办法吗
Yanc 2024-01-26 17:42:38请问执行之后如何处理呢,我也遇到同样的问题,更新了社区版,服务器重启,数据集成不工作,也删不了
root@R440:/usr/local/MDPrivateDeployment# docker exec -it $(docker ps|grep mingdaoyun-flink|awk '{print $1}') bash -c "ps -ef"
UID PID PPID C STIME TTY TIME CMD
root 1 0 0 09:22 ? 00:00:00 /bin/bash /run.sh
root 256 1 0 09:27 ? 00:00:04 /opt/java/openjdk/bin/java -Dzookeeper.jmx.log4j.disable=true -Duser.timezone=Asia/Shanghai -Dlog.file=/opt/flink/log/flink--zookeeper-0-572b15c8422
root 999 1 0 09:27 ? 00:00:00 sleep 1000
root 1346 0 0 09:40 pts/0 00:00:00 ps -ef
root@R440:/usr/local/MDPrivateDeployment#
根据输出是数据集成进程没有了,可以重启下服务,建议服务器配置参考
和明道云服务器部署在一起的话参考这里(需要额外资源,推荐是 16C40G 以上且单运行明道云服务时内存冗余 8G 以上):https://docs.pd.mingdao.com/faq/integrate/flink/combine
新增服务器部署参考这里:https://docs.pd.mingdao.com/faq/integrate/flink/split
新增服务器资源推荐:https://docs.pd.mingdao.com/deployment/source#%E6%95%B0%E6%8D%AE%E9%9B%86%E6%88%90%E6%9C%8D%E5%8A%A1
井正天(Team) 2023-12-11 13:41:57在数据集成服务器上操作下这个看看呢
docker exec -it $(docker ps|grep mingdaoyun-flink|awk '{print $1}') bash -c "ps -ef"
请问执行之后如何处理呢,我也遇到同样的问题,更新了社区版,服务器重启,数据集成不工作,也删不了
root@R440:/usr/local/MDPrivateDeployment# docker exec -it $(docker ps|grep mingdaoyun-flink|awk '{print $1}') bash -c "ps -ef"
UID PID PPID C STIME TTY TIME CMD
root 1 0 0 09:22 ? 00:00:00 /bin/bash /run.sh
root 256 1 0 09:27 ? 00:00:04 /opt/java/openjdk/bin/java -Dzookeeper.jmx.log4j.disable=true -Duser.timezone=Asia/Shanghai -Dlog.file=/opt/flink/log/flink--zookeeper-0-572b15c8422
root 999 1 0 09:27 ? 00:00:00 sleep 1000
root 1346 0 0 09:40 pts/0 00:00:00 ps -ef
root@R440:/usr/local/MDPrivateDeployment#
在数据集成服务器上操作下这个看看呢
docker exec -it $(docker ps|grep mingdaoyun-flink|awk '{print $1}') bash -c "ps -ef"
听人劝吃饱饭 2023-12-09 12:34:37动不动就这个命令,有业务在跑你咋办 😕
(万能重启大法,管他呢 😄 )
他们有后台可以进去的
有业务再跑照样重启,哈哈哈,找个借口
Lee 2023-12-09 12:43:45我昨晚更新了最新的私有社区版 好像是 3.8.1 吧 目前最新的,因为白天业务在跑,没办法 resrarall,等晚上再试试吧
遇到问题,可以先看日志,基于日志进行问题定位。
听人劝吃饱饭 2023-12-09 12:32:28老问题了,前后端接不上
你们现在是什么版本?
我昨晚更新了最新的私有社区版 好像是 3.8.1 吧 目前最新的,因为白天业务在跑,没办法 resrarall,等晚上再试试吧
Alvin.Zhang 2023-12-09 12:05:30restartall 命令试一下呢
动不动就这个命令,有业务在跑你咋办 😕
(万能重启大法,管他呢 😄 )
他们有后台可以进去的
老问题了,前后端接不上
你们现在是什么版本?
restartall 命令试一下呢