Kubernetes中Nginx配置热加载的全过程
Nginx已经是互联网IT业界一个无敌的存在,作为反向代理、负载均衡、Web服务器等多种角色的扮演者,下面这篇文章主要给大家介绍了关于Kubernetes中Nginx配置热加载的相关资料,需要的朋友可以参考下
目录
前言
使用方法
总结
前言
Nginx本身是支持热更新的,通过nginx -s reload指令,实际通过向进程发送HUB信号实现不停服重新加载配置,然而在Docker或者Kubernetes中,每次都需要进容器执行nginx -s reload指令,单docker容器还好说,可以在外面通过exec指定容器执行该指令进行热加载,Kubernetes的话,就比较难受了
今天介绍一下Kubernetes中Nginx热加载配置的处理方法——reloader
reloader地址:https://github.com/stakater/Reloader
reloader主要就是用来监测ConfigMap或Secret的变化,然后对相关DeploymentConfig的Deployment、DaemonSet执行滚动升级
reloader需要kubernetes1.9以上的版本才支持
使用方法
首先是安装部署reloader
1 2 | # 直接通过官方yaml文件部署 kubectl apply -f https: //raw .githubusercontent.com /stakater/Reloader/master/deployments/kubernetes/reloader .yaml |
默认情况下reloader是部署在default命名空间,但是它是监控所有命名空间的configmaps和secrets
当然,如果不想监控某个configmap或secret,可以通过--resources-to-ignore=configMaps/secrets来忽略某个资源
部署成功后,就可以直接使用了,我提前部署了nginx和configmap
这是目前的配置,看一下Nginx目前的配置
接着,我修改Nginx的Deployment,添加reloader,监听nginx-config这个ConfigMap,执行reload
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 | { "kind" : "Deployment" , "apiVersion" : "extensions/v1beta1" , "metadata" : { "name" : "nginx" , "namespace" : "default" , "selfLink" : "/apis/extensions/v1beta1/namespaces/default/deployments/nginx" , "uid" : "7eee5fa8-7514-11ec-a916-0210d5e9ca3b" , "resourceVersion" : "286141" , "generation" : 10, "creationTimestamp" : "2022-01-14T08:32:23Z" , "labels" : { "k8s-app" : "nginx" }, "annotations" : { "deployment.kubernetes.io/revision" : "9" , "description" : "nginx应用" # 主要是这行 "reloader.stakater.com/reload" : "nginx-config" } }, "spec" : { "replicas" : 1, "selector" : { "matchLabels" : { "k8s-app" : "nginx" } } …… |
然后apply该Deployment,之后我们去更新ConfigMap,更新nginx配置文件
更新完成,去掉proxy_redirect,然后去看nginx容器是否执行滚动更新
可以看到,nginx执行了滚动更新,接着看下nginx配置文件是否更新
这样很简单的通过reloader就可以实现Nginx的配置热加载
除了这种方法,常见的方法还有使用sidecar,通过sidecar去做的话,需要自己写监听脚本,比较麻烦,但是有时候也相对灵活,这里也附一个sidecar的python脚本
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 | #!/usr/bin/env python # -*- encoding: utf8 -*- """ 需求:nginx配置文件变化,自动更新配置文件,类似nginx -s reload 实现: 1、用pyinotify实时监控nginx配置文件变化 2、如果配置文件变化,给系统发送HUP来reload nginx """ import os import re import pyinotify import logging from threading import Timer # Param LOG_PATH = "/root/python/log" CONF_PATHS = [ "/etc/nginx" , ] DELAY = 5 SUDO = False RELOAD_COMMAND = "nginx -s reload" if SUDO: RELOAD_COMMAND = "sudo " + RELOAD_COMMAND # Log logger = logging.getLogger(__name__) logger.setLevel(level = logging.INFO) log_handler = logging.FileHandler(LOG_PATH) log_handler.setLevel(logging.INFO) log_formatter = logging.Formatter( '%(asctime)s - %(levelname)s - %(message)s' ) log_handler.setFormatter(log_formatter) logger.addHandler(log_handler) # Reloader def reload_nginx(): os.system(RELOAD_COMMAND) logger.info( "nginx is reloaded" ) t = Timer(DELAY, reload_nginx) def trigger_reload_nginx(pathname, action): logger.info( "nginx monitor is triggered because %s is %s" % (pathname, action)) global t if t.is_alive(): t.cancel() t = Timer(DELAY, reload_nginx) t.start() else : t = Timer(DELAY, reload_nginx) t.start() events = pyinotify.IN_MODIFY | pyinotify.IN_CREATE | pyinotify.IN_DELETE watcher = pyinotify.WatchManager() watcher.add_watch(CONF_PATHS, events, rec = True , auto_add = True ) class EventHandler(pyinotify.ProcessEvent): def process_default( self , event): if event.name.endswith( ".conf" ): if event.mask = = pyinotify.IN_CREATE: action = "created" if event.mask = = pyinotify.IN_MODIFY: action = "modified" if event.mask = = pyinotify.IN_DELETE: action = "deleted" trigger_reload_nginx(event.pathname, action) handler = EventHandler() notifier = pyinotify.Notifier(watcher, handler) # Start logger.info( "Start Monitoring" ) notifier.loop() |
如果喜欢用go的,这里也提供go脚本
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 | package main import ( "log" "os" "path/filepath" "syscall" "github.com/fsnotify/fsnotify" proc "github.com/shirou/gopsutil/process" ) const ( nginxProcessName = "nginx" defaultNginxConfPath = "/etc/nginx" watchPathEnvVarName = "WATCH_NGINX_CONF_PATH" ) var stderrLogger = log.New(os.Stderr, "error: ", log.Lshortfile) var stdoutLogger = log.New(os.Stdout, "", log.Lshortfile) func getMasterNginxPid() (int, error) { processes, processesErr := proc.Processes() if processesErr != nil { return 0, processesErr } nginxProcesses := map[int32]int32{} for _, process := range processes { processName, processNameErr := process.Name() if processNameErr != nil { return 0, processNameErr } if processName == nginxProcessName { ppid, ppidErr := process.Ppid() if ppidErr != nil { return 0, ppidErr } nginxProcesses[process.Pid] = ppid } } var masterNginxPid int32 for pid, ppid := range nginxProcesses { if ppid == 0 { masterNginxPid = pid break } } stdoutLogger.Println("found master nginx pid:", masterNginxPid) return int(masterNginxPid), nil } func signalNginxReload(pid int) error { stdoutLogger.Printf("signaling master nginx process (pid: %d) -> SIGHUP\n", pid) nginxProcess, nginxProcessErr := os.FindProcess(pid) if nginxProcessErr != nil { return nginxProcessErr } return nginxProcess.Signal(syscall.SIGHUP) } func main() { watcher, watcherErr := fsnotify.NewWatcher() if watcherErr != nil { stderrLogger.Fatal(watcherErr) } defer watcher.Close() done := make(chan bool) go func() { for { select { case event, ok := <-watcher.Events: if !ok { return } if event.Op&fsnotify.Create == fsnotify.Create { if filepath.Base(event.Name) == "..data" { stdoutLogger.Println("config map updated") nginxPid, nginxPidErr := getMasterNginxPid() if nginxPidErr != nil { stderrLogger.Printf("getting master nginx pid failed: %s", nginxPidErr.Error()) continue } if err := signalNginxReload(nginxPid); err != nil { stderrLogger.Printf("signaling master nginx process failed: %s", err) } } } case err, ok := <-watcher.Errors: if !ok { return } stderrLogger.Printf("received watcher.Error: %s", err) } } }() pathToWatch, ok := os.LookupEnv(watchPathEnvVarName) if !ok { pathToWatch = defaultNginxConfPath } stdoutLogger.Printf("adding path: `%s` to watch\n", pathToWatch) if err := watcher.Add(pathToWatch); err != nil { stderrLogger.Fatal(err) } <-done } |
ok,今天的内容就到这里
总结
到此这篇关于Kubernetes中Nginx配置热加载的文章就介绍到这了
原文链接:https://mp.weixin.qq.com/s/dcqprANA0m6gGEvnF2l4uQ