Kubernetes .Net6 Webhook

• Kubernetes .Net6 Webhook
  • Admission Webhook的介绍
  • MutatingAdmissionWebhook 与 ValidatingAdmissionWebhook
  • 先决条件
  • 需求分析
  • 创建K8sWebhook项目
  • 编写MutatingAdmissionWebhook控制器
  • 编写ValidatingAdmissionWebhook控制器
  • 编写Program.cs
  • 打包
  • 编写部署文件
  • 进行部署与测试
  • 相关参数说明

本文主要是学习陈计节大佬讲的 使用 .NET Core 开发 Kubernetes 基础组件记录的笔记。

Admission Webhook的介绍

Admission Webhook 是 api-server 对外提供的一个扩展能力，把K8s中的ApiServer当作我们.net的网站的话，它就像是其中的Filter过滤器一样可以拦截并修改这些请求信息。

MutatingAdmissionWebhook 与 ValidatingAdmissionWebhook

我们可以通过MutatingAdmissionWebhook 与 ValidatingAdmissionWebhook控制器进行强制实施自定义的策略。
流程如下图所示：

MutatingAdmissionWebhook：进行变更处理
ValidatingAdmissionWebhook：负责验证处理

先决条件

确保启用了MutatingAdmissionWebhook和ValidatingAdmissionWebhook资源类型，以及admissionregistration.k8s.io/v1API。
可通过如下命令进行检查。

kubectl api-resources | egrep "ValidatingWebhookConfiguration|MutatingWebhookConfiguration"

需求分析

我们想实现如下的一个需求：
1.我们不允许来源于abcd.com网站的镜像源进行创建容器。(ValidatingAdmissionWebhook)
2.我们对创建Podannotations:k8s.jijiechen.com/inject-dotnet-helper: "true"的进行创建一个边车容器，像dapr、istio注入一样。

创建K8sWebhook项目

创建一个名为K8sWebhook API项目（添加Docker支持）。
并添加KubernetesClient.Models依赖包。

<PackageReference Include="KubernetesClient.Models" Version="8.0.12" />

然后我们就需要去找到K8s的OpenAPI了，我提供了如下连接请根据k8s的版本需要对应swagger.json。

https://github.com/kubernetes/kubernetes/blob/master/api/openapi-spec/swagger.json
https://raw.githubusercontent.com/kubernetes/kubernetes/release-1.20/api/openapi-spec/swagger.json

下载下来后，将json文件放入项目中的AdminReviewModels目录下。

然后我们进行添加服务。

点击完成后，我们需要对生成的类做一些修改，我们找到项目目录的obj下找到admission.swaggerClient.cs文件。
在该目录下打开一个bash窗口，并执行如下命令。
该命令可以将项目中的序列化类从Newtonsoft.Json更改为System.Text.Json。

sed -i 's/public RawExtension/public System.Text.Json.JsonElement/g' ./admission.swaggerClient.cs
sed -i 's/\[Newtonsoft.Json.JsonProperty/[System.Text.Json.Serialization.JsonPropertyName/g' ./admission.swaggerClient.cs
sed -i 's/,\ Required.*/)]/g' ./admission.swaggerClient.cs

更改完成后，我们项目创建算是完成了。

编写MutatingAdmissionWebhook控制器

创建WebhookMutate.cs文件，编写如下代码：

public static class WebhookMutate
{
    const string InjectAnnotationKeySetting = "k8s.jijiechen.com/inject-dotnet-helper";
    public static AdmissionReview InjectDotnetHelper(AdmissionReview reviewRequest)
    {
        // 创建通行结果
        var allowedResponse = CreateInjectResponse(reviewRequest, true);
        // 判断请求的资源类型是不是Pod，是不是创建的请求，如果不是直接放行
        if (reviewRequest.Request.Kind.Kind != "Pod" || reviewRequest.Request.Operation != "CREATE")
        {
            return allowedResponse;
        }
        // 序列化成一个Pod资源对象
        var podJson = reviewRequest.Request.Object.GetRawText();
        Console.WriteLine($"原来的 Pod Json:{podJson}");
        var pod = JsonSerializer.Deserialize<V1Pod>(podJson,
            new JsonSerializerOptions() { PropertyNamingPolicy = JsonNamingPolicy.CamelCase });
        Console.WriteLine($"序列化后 Pod Json:{JsonSerializer.Serialize(pod)}");
        // 判断是否需要注入
        var _DisabledByAnnotation = DisabledByAnnotation(pod!.Metadata.Annotations);
        Console.WriteLine($"判断是否需要注入 :{!_DisabledByAnnotation}");
        if (_DisabledByAnnotation)
        {
            return allowedResponse;
        }
        // 如果该容器是我们注入的边车容器则跳过直接放行
        var _AlreadyInjected = AlreadyInjected(pod);
        Console.WriteLine($"判断是否已经注入 :{_AlreadyInjected}");
        if (_AlreadyInjected)
        {
            return allowedResponse;
        }
        // 创建我们的边车容器
        var container = new V1Container("dotnet-helper")
        {
            Args = new[] { "infinity" },
            Command = new[] { "sleep" },
            Image = "mcr.microsoft.com/dotnet/runtime:6.0",
            ImagePullPolicy = "IfNotPresent"
        };
        // 获取pod名称
        var podName = string.IsNullOrEmpty(reviewRequest.Request.Name)
            ? pod.Metadata.GenerateName + "?"
            : reviewRequest.Request.Name;
        // 获取完整的pod名称
        var fullPodName = $"{reviewRequest.Request.Namespace}/{podName}";
        Console.WriteLine($"正在向此 Pod 中注入 dotnet helper：{fullPodName}");
        // 发起添加的请求
        var patch = new
        {
            op = "add",
            path = "/spec/containers/-",
            value = container
        };
        var patches = new[] { patch };
        var patchResponse = new AdmissionResponse()
        {
            Allowed = true,
            PatchType = "JSONPatch",
            Patch = Encoding.Default.GetBytes(JsonSerializer.Serialize(patches))
        };
        var reviewResponse = CreateInjectResponse(reviewRequest, true);
        // 编写请求的结果
        reviewResponse.Response = patchResponse;
        Console.WriteLine($"获取请求结果 :{JsonSerializer.Serialize(patchResponse)}");
        return reviewResponse;
    }
    private static bool AlreadyInjected(V1Pod pod)
    {
        return pod.Spec.Containers.Any(c => c.Name == "dotnet-helper");
    }
    /// <summary>
    /// 判断是否需要注入
    ///
    /// annotation: k8s.jijiechen.com/inject-dotnet-helper:true 可以通过
    /// annotation: k8s.jijiechen.com/inject-dotnet-helper:false 不可以通过
    /// </summary>
    /// <param name="annotations"></param>
    /// <returns></returns>
    private static bool DisabledByAnnotation(IDictionary<string, string>? annotations)
    {
        // 当创建时没有annotations我们便不进行注入
        if (annotations == null)
        {
            return true;
        }
        var falseValues = new[] { "no", "false", "0" };
        // 判断为false我们也不进行注入
        return annotations.TryGetValue(InjectAnnotationKeySetting, out var annotation) &&
               falseValues.Contains(annotation.ToLower());
    }
    static AdmissionReview CreateInjectResponse(AdmissionReview originalRequest, bool allowed)
    {
        var res = new AdmissionResponse
        {
            Allowed = allowed,
            Status = new Status
            {
                Code = (int)HttpStatusCode.OK,
                Message = string.Empty
            }
        };
        res.Uid = originalRequest.Request.Uid;
        return new AdmissionReview
        {
            ApiVersion = originalRequest.ApiVersion,
            Kind = originalRequest.Kind,
            Response = res
        };
    }
}

编写ValidatingAdmissionWebhook控制器

创建WebhookValidate.cs文件，编写如下代码：

public static class WebhookValidate
{
    private static string[]? _blockedRepos = null;
    public static AdmissionReview CheckImage(AdmissionReview reviewRequest)
    {
        if (_blockedRepos == null)
        {
            // 获取BLOCKED_REPOS环境变量值，拒接这些网址的容器
            _blockedRepos = (Environment.GetEnvironmentVariable("BLOCKED_REPOS") ?? "").Split(",");
        }
        var allowed = true;
        // 判断请求的资源类型是不是Pod，是不是创建的请求，如果不是直接放行
        if (reviewRequest.Request.Kind.Kind != "Pod" || reviewRequest.Request.Operation != "CREATE")
        {
            return CreateReviewResponse(reviewRequest, allowed);
        }
        // 序列化成一个Pod资源对象
        var pod = JsonSerializer.Deserialize<V1Pod>(reviewRequest.Request.Object.GetRawText(),
            new JsonSerializerOptions() { PropertyNamingPolicy = JsonNamingPolicy.CamelCase });
        // 判断序列化是否成功，如果序列化失败直接放行
        if (pod == null)
        {
            return CreateReviewResponse(reviewRequest, allowed);
        }
        // 获取请求的Pod名称
        var podName = string.IsNullOrEmpty(reviewRequest.Request.Name) ? pod.Metadata.GenerateName + "?" : reviewRequest.Request.Name;
        // 完整的pod名称
        var fullPodName = $"{reviewRequest.Request.Namespace}/{podName}";
        // 获取初始化容器与运行容器的镜像
        var usedImages = new List<string>()
            .Concat(pod.Spec.Containers.NotEmpty().Select(c => c.Image))
            .Concat(pod.Spec.InitContainers.NotEmpty().Select(c => c.Image))
            .Distinct()
            .ToList();
        var blockedImages = new List<string>();
        // 拦截不允许的镜像地址
        var blockedCount = usedImages.Select(img =>
        {
            var shouldBlock = ImageBlocked(img);
            if (shouldBlock)
            {
                blockedImages.Add(img);
                Console.WriteLine($"Pod {fullPodName} 被拦截，因为使用被禁止的镜像 {img}");
            }
            return shouldBlock;
        }).Count(b => b);
        // 判断是否有不可以通行的镜像
        allowed = blockedCount == 0;
        // 创建放行结果
        var res = CreateReviewResponse(reviewRequest, allowed);
        // 判断剩余的通行镜像
        if (!allowed)
        {
            // 如果这些镜像不允许，将会进行如下操作
            var imageList = string.Join(",", blockedImages);
            res.Response.Status.Message = $"dotnet webhook: Pod should not use these images: {imageList}";
        }
        return res;
    }
    /// <summary>
    /// 判断不允许使用的镜像地址
    /// </summary>
    /// <param name="imageLocation"></param>
    /// <returns></returns>
    static bool ImageBlocked(string imageLocation)
    {
        if (_blockedRepos!.Length == 1 && string.IsNullOrWhiteSpace(_blockedRepos[0]))
        {
            return false;
        }
        if (!imageLocation.Contains('/'))
        {
            imageLocation = "docker.io/" + imageLocation;
        }
        return _blockedRepos
            .Select(r => r.EndsWith('/') ? r : string.Concat(r, '/'))
            .Any(r => imageLocation.StartsWith(r));
    }
    /// <summary>
    /// 放行
    /// </summary>
    /// <param name="originalRequest"></param>
    /// <param name="allowed"></param>
    /// <returns></returns>
    static AdmissionReview CreateReviewResponse(AdmissionReview originalRequest, bool allowed)
    {
        var res = new AdmissionResponse
        {
            Allowed = allowed,
            Status = new Status
            {
                Code = (int)HttpStatusCode.OK,
                Message = string.Empty
            }
        };
        res.Uid = originalRequest.Request.Uid;
        return new AdmissionReview
        {
            ApiVersion = originalRequest.ApiVersion,
            Kind = originalRequest.Kind,
            Response = res
        };
    }
}

关于放行输出的格式可以参考如下文档：https://kubernetes.io/zh-cn/docs/reference/access-authn-authz/webhook/

编写Program.cs

我们设置路由/validate将进行验证处理，/mutate则进行变更处理。

using K8sWebhook;
var builder = WebApplication.CreateBuilder(args);
builder.Services.AddControllers();
builder.Services.AddEndpointsApiExplorer();
builder.Services.AddSwaggerGen();
var app = builder.Build();
app.UseSwagger();
app.UseSwaggerUI();
app.MapGet("/", () => "Hello .NET Kubernetes Webhook!");
app.MapPost("/validate", (AdmissionReview review) => Task.FromResult(WebhookValidate.CheckImage(review)));
app.MapPost("/mutate", (AdmissionReview review) => Task.FromResult(WebhookMutate.InjectDotnetHelper(review)));
app.Run();

打包

Dockerfile如下：

FROM mcr.microsoft.com/dotnet/aspnet:6.0 AS base
WORKDIR /app
EXPOSE 80
FROM mcr.microsoft.com/dotnet/sdk:6.0 AS build
WORKDIR /src
COPY ["K8sWebhook/K8sWebhook.csproj", "K8sWebhook/"]
RUN dotnet restore "K8sWebhook/K8sWebhook.csproj"
COPY . .
WORKDIR "/src/K8sWebhook"
RUN dotnet build "K8sWebhook.csproj" -c Release -o /app/build
FROM build AS publish
RUN dotnet publish "K8sWebhook.csproj" -c Release -o /app/publish
FROM base AS final
WORKDIR /app
COPY --from=publish /app/publish .
ENTRYPOINT ["dotnet", "K8sWebhook.dll"]

当然也可以使用陈计节大佬编写的高山发布。

FROM mcr.microsoft.com/dotnet/sdk:7.0.100-preview.4-alpine3.15-amd64 as Builder
WORKDIR /src/
COPY *.csproj /src/
RUN dotnet restore
COPY . /src/
RUN mkdir /app && dotnet publish -c Release -p:PublishSingleFile=true -p:PublishTrimmed=true -r linux-musl-x64  --self-contained -o /app
FROM alpine:3.15
RUN apk add bash icu-libs krb5-libs libgcc libintl libssl1.1 libstdc++ zlib
WORKDIR /app
COPY --from=Builder /app/* ./
ENTRYPOINT ["/app/K8sWebhook"]

docker build -t aidasi/webhook:v1 -f Dockerfile ..
docker push aidasi/webhook:v1

编写部署文件

编写deploy.yaml部署文件。

---
apiVersion: v1
kind: Namespace
metadata:
  name: kncs-system
---
apiVersion: v1
kind: Secret
metadata:
  name: kncs-server-certs
  namespace: kncs-system
type: kubernetes.io/tls
data:
  tls.crt: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUV6VENDQXJXZ0F3SUJBZ0lKQU4wTjdiTGtxM1hSTUEwR0NTcUdTSWIzRFFFQkN3VUFNRzh4Q3pBSkJnTlYKQkFZVEFrTk9NUkF3RGdZRFZRUUlEQWRDWldscWFXNW5NUkV3RHdZRFZRUUhEQWhEYUdGdmVXRnVaekVQTUEwRwpBMVVFQ2d3R1JHVjJUM0J6TVF3d0NnWURWUVFMREFOUVMwa3hIREFhQmdOVkJBTU1FMFJsZGs5d2N5QkhaVzVsCmNtbGpJRkp2YjNRd0hoY05Nakl3TlRFMU1EZzBNRE15V2hjTk1qTXdOVEUxTURnME1ETXlXakI0TVFzd0NRWUQKVlFRR0V3SkRUakVSTUE4R0ExVUVDQXdJVTJobGJucG9aVzR4RURBT0JnTlZCQWNNQjA1aGJuTm9ZVzR4RHpBTgpCZ05WQkFvTUJrUmxkazl3Y3pFTU1Bb0dBMVVFQ3d3RFVFdEpNU1V3SXdZRFZRUUREQnhyYm1OekxYZGxZbWh2CmIyc3VhMjVqY3kxemVYTjBaVzB1YzNaak1JSUJJakFOQmdrcWhraUc5dzBCQVFFRkFBT0NBUThBTUlJQkNnS0MKQVFFQXV5aTJwdGM3NWVxVG5BWDRHQnZmUXpqOGt3SWVQcVRIU3hVOWtpWFpKL1JMUWdheHBUUmROL3F0SkNFVwpaSUhOUHFJVU5wNnRpeXF4d3poQzR5UHNkS3JIREl4eGQreHFPMEp3VmNMNUk4dEhsT0RIY0pJUDM2L1ZKS0VBClNlT2lqMm91T3pkYXBZRFh2dm5wVkdMQzRkKzNiTVM5cXNVMjFUS2VqU1VNZGZhdVdObXdzNGk1eVRlVVA1WUkKeWdBZWRYRGs5ZWM2WmtUNTBFV2JhU1VQdkNEWEZHY3NnZVQxSlNkclgxemw5dXl3bjFZa1h2d3gyd0VYdTlHUApqdVRMYmxGSEkzYnRKTTJVNldkbXJvTWlESmY4V1dPeXNCdDMvczVjZWduRTUrSDgxVzV0ZWR4WWc3MUMvUzVmCjgrWFpsbG9GL3FteVhwZnhEM0x6QUtFbGV3SURBUUFCbzJNd1lUQUpCZ05WSFJNRUFqQUFNQXNHQTFVZER3UUUKQXdJRjREQkhCZ05WSFJFRVFEQStnaHhyYm1OekxYZGxZbWh2YjJzdWEyNWpjeTF6ZVhOMFpXMHVjM1pqZ2g0cQpMbXR1WTNNdGQyVmlhRzl2YXk1cmJtTnpMWE41YzNSbGJTNXpkbU13RFFZSktvWklodmNOQVFFTEJRQURnZ0lCCkFKTmlEbGp0WEM4RldPT0dsMmdnODg3RExLTmROQ3NuYzhRRUR3NXNJWUhUN2FkSFAzQlRPSG0xdGN6dW5uL2sKWEVTVEd1MzQwZmhpanljeXViQjJsaCsvYXBKZDNPQkNwdkNjLzg0c3l5Z3lnenFUWGxKUmowUzBiaWhWQ2xLNwpnZHVSYWpxQnpBWE55RkcrM25LWkpEQVk3NklMNlVFejdzWXlFSTdsMlFtQmN4VTM5VmJuU0dnMCt1czZhWWtnCjF6WXA3c3NUcEpMS013cHZYOG96TnY1TjFKOGFHVjJJYldUdHh2cUR1N01xZ2FHbzBlZUl2K3NwNVRSRiswWUoKYXBXS1Bzd2wxMW1hMEpPS1Z5RG8zOEUrT3Q1K2FFdjF6V3BEaGU0NHpaOVB4U01uaGVTMTBySHRFdkZ4bmxpWApGeWlwVkJBK2lFWUQ2b0ZVeWIxVzJUN2Zna1M2UWdrSzVQeGRITFg3U2VqaHh3OTRzbzllWjBqSGZLd012WWNwCktHenhnOUJYMnpKSmcwM0sramUwemhIenkzMy9KYVlzVzlXVTF1UjlqenlvMWtMWFZDZWhXNlFlbDV6bzFMKzAKYkZPVHc3bnhtTW1sdTJ6NTFUVld0OHlvUWdWVEJzRFFxZDg1MlVHMGwzYVp6ZFlLUmN6MlpwZ3VqVjRCZjlmaAp0RjFXNXZsYnlPR05wM0ZqV2NkdTNxZW5PU0RtaUd3V0dYZjNKK0o4bmUwa0dKajBabHljekxTbUdHYlo1YlBGClZIaWJISlc4YnAwMGpYNVVTVFY4blQxLzVFT3k3blRpRUJXRDZpNE1SUGl5ZmwvNHVGdGFUeTlNRm5KRDYrSnQKWGVmR3VvWXVFUDNqN1RNWmJYZnJzSFk1dDcvZGV1bldyWVh6K3JkQ25PVUEKLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=
  tls.key: LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQpNSUlFb3dJQkFBS0NBUUVBdXlpMnB0Yzc1ZXFUbkFYNEdCdmZRemo4a3dJZVBxVEhTeFU5a2lYWkovUkxRZ2F4CnBUUmROL3F0SkNFV1pJSE5QcUlVTnA2dGl5cXh3emhDNHlQc2RLckhESXh4ZCt4cU8wSndWY0w1STh0SGxPREgKY0pJUDM2L1ZKS0VBU2VPaWoyb3VPemRhcFlEWHZ2bnBWR0xDNGQrM2JNUzlxc1UyMVRLZWpTVU1kZmF1V05tdwpzNGk1eVRlVVA1WUl5Z0FlZFhEazllYzZaa1Q1MEVXYmFTVVB2Q0RYRkdjc2dlVDFKU2RyWDF6bDl1eXduMVlrClh2d3gyd0VYdTlHUGp1VExibEZISTNidEpNMlU2V2Rtcm9NaURKZjhXV095c0J0My9zNWNlZ25FNStIODFXNXQKZWR4WWc3MUMvUzVmOCtYWmxsb0YvcW15WHBmeEQzTHpBS0VsZXdJREFRQUJBb0lCQUQwUE1rL0tKbk9ERFRjNAp4MUR1UHUrS2R2UnJHM3pxZTA1bWxwaklta2tyclNYVVV6NkhqK1lFZFZvMUpUNFREdWZoTHVFRzhhMVdkM291Ckw3dzA2eDdBM0lHZWpDSkkwZnVWV0ZyU2FqK2dRVEUwQ0QwVW1mTXJSVWxXOFdZcHlzNHBJUDRXdUE4SXN0cE8KWkM0d3JrM01rK1g3WmJtQjc3cXNjZ2V3VDVsb0d6OGdKNG14dVF5MnR5SWF6d3VUQnBkbm5VUWh0QkJLZ2J1YwplYnpTS0luZ3RXa2o3SHlDWmVYY0xTdmpORzNGcEpWUXZZK1pncC9rbno1S042RFZXUkZNWThFV3BpTVVaVE1HCmVVVCtuSER4NkZlSVllNVBhRG41TWROTWhPOGZRYWdBVG9GcDRkRU5xb0l6RWVIWkZwOEJTODgwTXpTQSt0UTgKanNMaEt1RUNnWUVBNXYvTFptNXVaQ1RwNnY0NXJjQlRIM2VadXVmOVJ1Z01HYXJMS2tTS1RHZGZLMEg3dlVMZQpBSDlxalVuMExtSUxscVdmcHJiMC90eVBTUEFaSVhYY0FYQVA1TDJBdlJNNnQ1c0JXWk5HcnV5aVcyWkxTQzhQCnROQ3RHTkxmK0ZGQzJnZjlBTzZtZjhFRzlkYW9uNFdvKzZWU1FYMlRiOUZMYkxHWnByeDZjL0VDZ1lFQXoycEIKcnU1TE1ENS9abHpwY2xJNGg4RytpZ3dDcjM4K3V1NmdXRmZFZVRkYjk0cm1RclptM2xPQkxqVTBteExUTlUrVQp3NEVVQ3piR2xyYlNWRVZZL0s0Zi9ZZXBXazB4cjY3aUN1amd5ZzVSbDM5WFlMYXhxMVkvUzZ0TlNQdVQ2RjYzCkJMY1RYM3YrQ2RDR0Q5L0V1TWFEcTBJM0h5TjlOVkVPaCsrR2JDc0NnWUVBMk9MZ3Vnc0RrUGxydTl6NGtOL2IKNjlhaXUyK29TZFFEMEhHaEVjMkt3RlBxY2pZZ3c4R3RxVy80dmpIcWwwWXROVVBLazRDQ3BXeTNCN2VQRVBDVgpJYkJ5NjhUVnhERHkxNE10RUVxTWVoN3FEY0VNKy9oYjJkeDRPYTk4NUt4L2hURXM1cHdzTGhVeGtNNzhRZE1BCkowNUEzZ2FtMEwwRkFVZjdTU2I4SGpFQ2dZQTUrN2xxL3NEVU50U0V1RHFtcytlTHhCVFJJTFJyZlVYN0doU0gKUGRuMkRRelBzZXZYQUlqWFpENjd2VEg4bkJHaFdLTDgySXZTNnJndmorSlNucVJXMXhLb1hKRnlaaHdhd2VmOQpKc2NZbFZJbjZQaHpWLzlwSjQ1QVNCNHQ1ZTZlU2tRZHRGUmRJQnVQZ05USmdVUE1aK3FOS05DaUN0akkyK1VWCkNWZnB5d0tCZ0hvbERiZVRvbTZKbHppcnN0TXFlNUZwZ0NBQ0VIcDR1dkw1Tjk4TmlzWmNPRm05d1ZKcnErYVIKUVJDT1pXRnY0OTM1a2tFLzJnRDVzRnZYZzUzZmVjbmV2UFFWRytsbU1sUXRoSlR0NzRpTkZpRlBqRDR5Y0lDNQpCZG5yWkNDTWJMWHVUdEhiZmtIbXphRlExaUFKUTV0SkN3dG04TFVEd0YvMzM2bk5FN1RYCi0tLS0tRU5EIFJTQSBQUklWQVRFIEtFWS0tLS0tCg==
---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: kncs-webhook
    kncs: webhook
  name: kncs-webhook
  namespace: kncs-system
spec:
  progressDeadlineSeconds: 600
  replicas: 1
  selector:
    matchLabels:
      kncs: webhook
  template:
    metadata:
      annotations:
        sidecar.istio.io/inject: "false"
        k8s.jijiechen.com/inject-dotnet-helper: "false"
      labels:
        kncs: webhook
    spec:
      containers:
        - env:
            - name: BLOCKED_REPOS
              value: abcd.com
            - name: ASPNETCORE_URLS
              value: 'https://+:443'
            - name: ASPNETCORE_Kestrel__Certificates__Default__Path
              value: /etc/kncs/certs/tls.crt
            - name: ASPNETCORE_Kestrel__Certificates__Default__KeyPath
              value: /etc/kncs/certs/tls.key
#            - name: Logging__LogLevel__Default
#              value: Debug
#            - name: Logging__LogLevel__Microsoft.AspNetCore
#              value: Debug
#            - name: DEBUG
#              value: 'true'
#            - name: Microsoft__AspNetCore__HttpLogging__HttpLoggingMiddleware
#              value: Information
          image: aidasi/webhook:v1
          imagePullPolicy: IfNotPresent
          name: webhook
          ports:
            - containerPort: 443
              protocol: TCP
          resources:
            requests:
              cpu: 50m
              memory: 128Mi
          volumeMounts:
            - mountPath: /etc/kncs/certs
              name: certs
      restartPolicy: Always
      schedulerName: default-scheduler
      volumes:
        - name: certs
          secret:
            defaultMode: 420
            secretName: kncs-server-certs
---
apiVersion: v1
kind: Service
metadata:
  labels:
    app: kncs-webhook
  name: kncs-webhook
  namespace: kncs-system
spec:
  ports:
    - name: https
      port: 443
      protocol: TCP
      targetPort: 443
  selector:
    kncs: webhook
  sessionAffinity: None
  type: ClusterIP
---
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingWebhookConfiguration
metadata:
  name: validating-webhook-kncs
webhooks:
  - admissionReviewVersions:
      - v1beta1
      - v1
    clientConfig:
      caBundle: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUZzVENDQTVtZ0F3SUJBZ0lKQU1LNnNwL3N3Uzl5TUEwR0NTcUdTSWIzRFFFQkN3VUFNRzh4Q3pBSkJnTlYKQkFZVEFrTk9NUkF3RGdZRFZRUUlEQWRDWldscWFXNW5NUkV3RHdZRFZRUUhEQWhEYUdGdmVXRnVaekVQTUEwRwpBMVVFQ2d3R1JHVjJUM0J6TVF3d0NnWURWUVFMREFOUVMwa3hIREFhQmdOVkJBTU1FMFJsZGs5d2N5QkhaVzVsCmNtbGpJRkp2YjNRd0hoY05Nakl3TlRFMU1EZzBNRE14V2hjTk16SXdOVEV5TURnME1ETXhXakJ2TVFzd0NRWUQKVlFRR0V3SkRUakVRTUE0R0ExVUVDQXdIUW1WcGFtbHVaekVSTUE4R0ExVUVCd3dJUTJoaGIzbGhibWN4RHpBTgpCZ05WQkFvTUJrUmxkazl3Y3pFTU1Bb0dBMVVFQ3d3RFVFdEpNUnd3R2dZRFZRUUREQk5FWlhaUGNITWdSMlZ1ClpYSnBZeUJTYjI5ME1JSUNJakFOQmdrcWhraUc5dzBCQVFFRkFBT0NBZzhBTUlJQ0NnS0NBZ0VBeFZHdUwrcnoKOGJXWWtoME5SbDUyOHJFcHlhNmVoU2tPdUU1QWN6MERGMGJCSjRyZzVCaE8xS1NyWlhzd1krVFp6c1ZqL0pKbwpWYnd5YXdxcTZRZ2tXWEhUTHcya3d3UE5MQ1RYakk4TUZ0TUlKdkdYaktJZnp6aXE1c2hxYVVWWU8vNm13WFBmCjdTMVhVOFFrelErNnJ0aEgvUFdYYkFjMWlRbXhvWGgvVXVRZkRQMHcrM3djelZwc0VYSm5xVmFnYUlBbDdSbXkKMEhYM0JoV281WTQwMFFVSEF3bnJpUUVXWmRhYUEwaExGaFZqMHl4WW5HUnBQOWJjUEZmc0poOUFyam04M2pSRQp3RU02MVFJOU81TExmMlJTTlFmMnB5ZmNKZm1KZWVvK2pVQTEwZjJWOC8wY3YvRGJVZitjZDFRWGhUQllEek9iCnlNWWZsMUFIMGIxZit0YU9WYlFmT0QwalRyWXNLS1hrYmtYNUpPZzBXMndvd3VYbURXa1A1aGl3MUZNRGN5MngKU0dsRG8wQjNQMXVCSkJzc0ErUXlvT05YWDBoYWRsekR0NDhpbk5hdTF5L1Exb1daMHJvZDFBa0t0TnJBRk1WYQpxekhVRVRtTDNaRUtXbDFwbjBadE40ZUNVSEZhOGZDZ0MwVDUvK0VFU1ZoV1dPQ2FjU1VJbkIyUFlxZnBUWnlMCmZ6UnRRR2hYcE1xcWVCbDNvcjJtUUI5REZKdHZNZWl4Y3VHTVdSTGIyOWkxKy9qVkdzVTU5ck9tTDJIaXJsTjIKU2Izcmd5RkpXb3FyQmN5L215dEJ5VG9sc1RPc1BTRGRORE81ZGRiVkVLMk9IZC9YZ29MUTh1TVpCUng4Y3Y1dQo1a0grNS9xanRGbi9rcDg0Sy9yY0k2T2V4eWFielRsQm51Y0NBd0VBQWFOUU1FNHdIUVlEVlIwT0JCWUVGQnlmCmllYnU3N3pHQ1lwbU9EU3pmWmMzUXRuQ01COEdBMVVkSXdRWU1CYUFGQnlmaWVidTc3ekdDWXBtT0RTemZaYzMKUXRuQ01Bd0dBMVVkRXdRRk1BTUJBZjh3RFFZSktvWklodmNOQVFFTEJRQURnZ0lCQUNCZkpzMXNtYnZDUHYxYQorcWZQTy8vbmpPTHBOVm1ybzZZcVp6SkJqbHo0VmpVZ1JOMGxrQ1ltTk9oM0orRnVMRGNwMi9XRjJ5RnlkZlhLClBORGZQb2xkQ2RrdHh5OVk2SlpHM3dOQ3pqTEdJWVpwZStET2RjSmswellqYy9HRWFNTkVmQVVJN09zSFlDcTUKLy8ycWNQTWpOOG1TTnJiM0FwZkZaNmJsMUFGSW9NNzBZUUFDbHRKTWdkNFRSSWhISlBibmdaVjE0b1kyNWp6UApscFBMUVVRV1B2V00wbm43QW1OMldmdG5mNGl2ZzludHczN1pvdm1nc0JhaHhockRCWTBUbmdYNWFlVElHOEFICmFvZjhneUxvUVIzQjhZeURBU2N5Y0l0ZUhNdDgrNVp3YkdFTVdJanVFNUdKcUtBeWZDOUlIWjdFMHBIbUg2ck0KY1hoaVFNTjdLZm1LaGwvbGZtbmlIeEpTNnE2eUlsbmU4NDRpK0l3Y3crL0sxbUJDdFZDZ0R4OUc3TWo2QUljeApWekJyYlVrSUdQKzRxc2NmbDRGTDRIbGdjOE9vMnpYbmc1T2VpQ2JTSGRXK21zMFdXeVM1cGRZd29BbXFzaisyCnYzdmtMS3llR0lWNHBscWl4ZjMybXdrTW9NTXRPQ1NNR3FzZmhwTFFLUUhSaXNxMjNhc0tOYVZXV0RtcHFqa2YKbmZPWFJKdkpDbktFMXRJNkpPbjViN3pCc1JjaC9maVRRcmZ3Zk1sU3N2N2ZCRGZEbE8zU09STkhHTU0zRndUdgpZYjhsMllaTEpTNmpoeVZsUVZESjRUVW5TeXZJVURxOWJhZVJKTU1vZ3g2QmhTY2tGbFhwRmJiT2FVMmMzaktsCmtlUFVKZEUzU1ZnbWt6RW10YmJVTHdnOENPV0UKLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=
      service:
        name: kncs-webhook
        namespace: kncs-system
        path: /validate
        port: 443
    failurePolicy: Fail
    name: image-validator.kncs.io
    namespaceSelector: {}
    objectSelector: {}
    rules:
      - apiGroups:
          - ""
        apiVersions:
          - v1
        operations:
          - CREATE
          - UPDATE
          - DELETE
        resources:
          - pods
        scope: '*'
    sideEffects: None
    timeoutSeconds: 30
---
apiVersion: admissionregistration.k8s.io/v1
kind: MutatingWebhookConfiguration
metadata:
  name: mutating-webhook-kncs
webhooks:
  - admissionReviewVersions:
      - v1beta1
      - v1
    clientConfig:
      caBundle: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUZzVENDQTVtZ0F3SUJBZ0lKQU1LNnNwL3N3Uzl5TUEwR0NTcUdTSWIzRFFFQkN3VUFNRzh4Q3pBSkJnTlYKQkFZVEFrTk9NUkF3RGdZRFZRUUlEQWRDWldscWFXNW5NUkV3RHdZRFZRUUhEQWhEYUdGdmVXRnVaekVQTUEwRwpBMVVFQ2d3R1JHVjJUM0J6TVF3d0NnWURWUVFMREFOUVMwa3hIREFhQmdOVkJBTU1FMFJsZGs5d2N5QkhaVzVsCmNtbGpJRkp2YjNRd0hoY05Nakl3TlRFMU1EZzBNRE14V2hjTk16SXdOVEV5TURnME1ETXhXakJ2TVFzd0NRWUQKVlFRR0V3SkRUakVRTUE0R0ExVUVDQXdIUW1WcGFtbHVaekVSTUE4R0ExVUVCd3dJUTJoaGIzbGhibWN4RHpBTgpCZ05WQkFvTUJrUmxkazl3Y3pFTU1Bb0dBMVVFQ3d3RFVFdEpNUnd3R2dZRFZRUUREQk5FWlhaUGNITWdSMlZ1ClpYSnBZeUJTYjI5ME1JSUNJakFOQmdrcWhraUc5dzBCQVFFRkFBT0NBZzhBTUlJQ0NnS0NBZ0VBeFZHdUwrcnoKOGJXWWtoME5SbDUyOHJFcHlhNmVoU2tPdUU1QWN6MERGMGJCSjRyZzVCaE8xS1NyWlhzd1krVFp6c1ZqL0pKbwpWYnd5YXdxcTZRZ2tXWEhUTHcya3d3UE5MQ1RYakk4TUZ0TUlKdkdYaktJZnp6aXE1c2hxYVVWWU8vNm13WFBmCjdTMVhVOFFrelErNnJ0aEgvUFdYYkFjMWlRbXhvWGgvVXVRZkRQMHcrM3djelZwc0VYSm5xVmFnYUlBbDdSbXkKMEhYM0JoV281WTQwMFFVSEF3bnJpUUVXWmRhYUEwaExGaFZqMHl4WW5HUnBQOWJjUEZmc0poOUFyam04M2pSRQp3RU02MVFJOU81TExmMlJTTlFmMnB5ZmNKZm1KZWVvK2pVQTEwZjJWOC8wY3YvRGJVZitjZDFRWGhUQllEek9iCnlNWWZsMUFIMGIxZit0YU9WYlFmT0QwalRyWXNLS1hrYmtYNUpPZzBXMndvd3VYbURXa1A1aGl3MUZNRGN5MngKU0dsRG8wQjNQMXVCSkJzc0ErUXlvT05YWDBoYWRsekR0NDhpbk5hdTF5L1Exb1daMHJvZDFBa0t0TnJBRk1WYQpxekhVRVRtTDNaRUtXbDFwbjBadE40ZUNVSEZhOGZDZ0MwVDUvK0VFU1ZoV1dPQ2FjU1VJbkIyUFlxZnBUWnlMCmZ6UnRRR2hYcE1xcWVCbDNvcjJtUUI5REZKdHZNZWl4Y3VHTVdSTGIyOWkxKy9qVkdzVTU5ck9tTDJIaXJsTjIKU2Izcmd5RkpXb3FyQmN5L215dEJ5VG9sc1RPc1BTRGRORE81ZGRiVkVLMk9IZC9YZ29MUTh1TVpCUng4Y3Y1dQo1a0grNS9xanRGbi9rcDg0Sy9yY0k2T2V4eWFielRsQm51Y0NBd0VBQWFOUU1FNHdIUVlEVlIwT0JCWUVGQnlmCmllYnU3N3pHQ1lwbU9EU3pmWmMzUXRuQ01COEdBMVVkSXdRWU1CYUFGQnlmaWVidTc3ekdDWXBtT0RTemZaYzMKUXRuQ01Bd0dBMVVkRXdRRk1BTUJBZjh3RFFZSktvWklodmNOQVFFTEJRQURnZ0lCQUNCZkpzMXNtYnZDUHYxYQorcWZQTy8vbmpPTHBOVm1ybzZZcVp6SkJqbHo0VmpVZ1JOMGxrQ1ltTk9oM0orRnVMRGNwMi9XRjJ5RnlkZlhLClBORGZQb2xkQ2RrdHh5OVk2SlpHM3dOQ3pqTEdJWVpwZStET2RjSmswellqYy9HRWFNTkVmQVVJN09zSFlDcTUKLy8ycWNQTWpOOG1TTnJiM0FwZkZaNmJsMUFGSW9NNzBZUUFDbHRKTWdkNFRSSWhISlBibmdaVjE0b1kyNWp6UApscFBMUVVRV1B2V00wbm43QW1OMldmdG5mNGl2ZzludHczN1pvdm1nc0JhaHhockRCWTBUbmdYNWFlVElHOEFICmFvZjhneUxvUVIzQjhZeURBU2N5Y0l0ZUhNdDgrNVp3YkdFTVdJanVFNUdKcUtBeWZDOUlIWjdFMHBIbUg2ck0KY1hoaVFNTjdLZm1LaGwvbGZtbmlIeEpTNnE2eUlsbmU4NDRpK0l3Y3crL0sxbUJDdFZDZ0R4OUc3TWo2QUljeApWekJyYlVrSUdQKzRxc2NmbDRGTDRIbGdjOE9vMnpYbmc1T2VpQ2JTSGRXK21zMFdXeVM1cGRZd29BbXFzaisyCnYzdmtMS3llR0lWNHBscWl4ZjMybXdrTW9NTXRPQ1NNR3FzZmhwTFFLUUhSaXNxMjNhc0tOYVZXV0RtcHFqa2YKbmZPWFJKdkpDbktFMXRJNkpPbjViN3pCc1JjaC9maVRRcmZ3Zk1sU3N2N2ZCRGZEbE8zU09STkhHTU0zRndUdgpZYjhsMllaTEpTNmpoeVZsUVZESjRUVW5TeXZJVURxOWJhZVJKTU1vZ3g2QmhTY2tGbFhwRmJiT2FVMmMzaktsCmtlUFVKZEUzU1ZnbWt6RW10YmJVTHdnOENPV0UKLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=
      service:
        name: kncs-webhook
        namespace: kncs-system
        path: /mutate
        port: 443
    failurePolicy: Fail
    name: dotnet-helper-injector.kncs.io
    namespaceSelector: {}
    objectSelector: {}
    rules:
      - apiGroups:
          - ""
        apiVersions:
          - v1
        operations:
          - CREATE
          - UPDATE
          - DELETE
        resources:
          - pods
        scope: '*'
    sideEffects: None
    timeoutSeconds: 30

关于细节我在后面会讲到。

进行部署与测试

kubectl apply -f deploy.yaml
kubectl get all -n kncs-system

先测试不注入的情况。

kubectl run nginx --image=nginx
kubectl get pod

测试需要注入的情况，我们写道一个pod.yaml的文件下面。

apiVersion: v1
kind: Pod
metadata:
  annotations:
    k8s.jijiechen.com/inject-dotnet-helper: 'true'
  labels:
    app: test
  name: webhook-tester-mutating
  namespace: default
spec:
  containers:
    - args:
        - infinity
      command:
        - sleep
      image: centos:7
      imagePullPolicy: IfNotPresent
      name: tester

kubectl apply -f pod.yaml
kubectl get pod -w

测试注入成功。
同样我们可以测试注入的日志信息。

kubectl get pod -n kncs-system
kubectl logs <pod名称> -n kncs-system -f

接下来我们创建一个镜像源来自abcd.com看看能不能创建

kubectl run aidasi --image=abcd.com/aidasi:v1

我们发现直接报错了。
接着我们来看看日志，发现有一条被禁止的镜像。

到这里我们的Webhook就算完成了，下面我们来讲讲部署时ValidatingWebhookConfiguration和MutatingWebhookConfiguration所使用到的参数。

相关参数说明