update bugfix/1.24-GA

2024-03-13 23:33:49 +08:00 · 2024-03-13 15:02:21 +00:00 · 2024-03-13 11:54:15 +00:00 · 2024-03-13 11:52:02 +00:00 · 2024-03-13 11:51:19 +00:00 · 2024-03-13 10:05:53 +08:00
1061 changed files with 76275 additions and 178 deletions
--- a/ReadME.MD
+++ b/ReadME.MD
@ -1,64 +1,33 @@
-## 安装 Charm 2.x
+# Kubernetes 1.24
+
+## Breaking Change
+
+### Transition to Charmhub
+
+https://charmhub.io/
+
+Starting with this release, charms and bundles will be published to Charmhub instead of the Charm Store.
+
+## Important Change
+
+### Calico is now the default CNI
+
+The charmed-kubernetes and kubernetes-core reference bundles have been updated to use Calico for pod networking instead of Flannel. We recommend Calico as the default CNI choice for all new deployments due to the rich set of advanced networking features that it provides.
+
+While we do recommend Calico as the default choice, we will continue to support new and existing deployments that use Flannel as well.
+
+### Docker support deprecated
+
+The default container runtime in Charmed Kubernetes has been containerd for some time. The Docker container runtime is no longer supported.

 ```Bash
-# 如果是使用 1.24 版本以及以后, 此步骤可以跳过
-snap install charm --channel=2.x --classic
-```
-
-## 获取特定版本的 Bundle
-
-```Bash
-wget https://raw.githubusercontent.com/charmed-kubernetes/bundle/main/releases/1.21/bundle.yaml
-```
-
-或是
-
-```Bash
-# 对于 1.23 版本以及以前
-charm pull cs:~containers/charmed-kubernetes-657
-
-# 对于 1.24 版本以及以后
-juju download ch:charmed-kubernetes --channel 1.24/stable --series focal
-```
-
-## 拉取 Bundle 中的相关组件
-
-```Bash
-# 对于 1.23 版本以及以前
-charm pull cs:~containers/etcd-607
-
-# 对于 1.24 版本以及以后
 juju download ch:etcd --channel 1.24/stable --series focal
-```
-
-## 拉取相关组件的资源包
-
-```Bash
-# 对于 1.23 版本以及以前
-charm list-resources cs:~containers/calico-812
-wget https://api.jujucharms.com/charmstore/v5/~containers/calico-812/resource/calico/1027 -O calico.tgz
-wget https://api.jujucharms.com/charmstore/v5/~containers/calico-812/resource/calico-arm64/1026 -O calico-arm64.tgz
-wget https://api.jujucharms.com/charmstore/v5/~containers/calico-812/resource/calico-node-image/709 -O calico-node-image.tgz
-wget https://api.jujucharms.com/charmstore/v5/~containers/calico-812/resource/calico-upgrade/854 -O calico-upgrade.tgz
-wget https://api.jujucharms.com/charmstore/v5/~containers/calico-812/resource/calico-upgrade-arm64/854 -O calico-upgrade-arm64.tgz
-
-# 对于 1.24 版本以及以后
-unzip xxx.charm
-build-xxx-resource.sh
-# 或
-前往 https://charmhub.io/calico/resources/calico?channel=1.24/stable 下载对应的 Revision  
-```
-
-## 向 Controller 上传资源
-
-```Bash
-# ex: juju attach-resource <charm-name> resource-name=<filepath>
-juju attach-resource calico calico=/home/sa/charm/calico/calico-amd64.tar.gz
-juju attach-resource calico calico-upgrade=/home/sa/charm/calico/calico-upgrade-amd64.tar.gz
-```
-
-## 回收节点
-
-```None
-https://git.motofans.club/Motofans/Charmed-Kubernetes/src/branch/Guide/cleanup-k8s-node.py
+juju download ch:easyrsa --channel 1.24/stable --series focal
+juju download ch:kubernetes-control-plane --channel 1.24/stable --series focal
+juju download ch:kubernetes-worker --channel 1.24/stable --series focal
+juju download ch:calico --channel 1.24/stable --series focal
+juju download ch:containerd --channel 1.24/stable --series focal
+# Extend
+juju download ch:kubeapi-load-balancer --channel 1.24/stable --series focal
+juju download ch:keepalived --channel 1.24/stable --series focal
 ```
--- a/calico.charm
+++ b/calico.charm
--- a/cleanup-k8s-node.py
+++ b/cleanup-k8s-node.py
@ -1,50 +0,0 @@
-#!/usr/bin/env python
-
-import subprocess
-
-# Purge snap packages
-snap_packages = [
-    "cdk-addons",
-    "kube-apiserver",
-    "kube-controller-manager",
-    "kube-proxy",
-    "kube-scheduler",
-    "kubelet",
-    "kubectl",
-    "etcd"
-]
-
-for package in snap_packages:
-    subprocess.run(["snap", "remove", package, "--purge"])
-
-# Purge Debian packages
-packages_to_purge = [
-    "nginx-full",
-    "containerd",
-    "keepalived"
-]
-
-subprocess.run(["apt", "purge", "-y"] + packages_to_purge + ["--allow-change-held-packages"])
-subprocess.run(["apt", "autoremove", "-y", "--purge"])
-subprocess.run(["/sbin/remove-juju-services"])
-
-# Remove directories and files
-directories = [
-    "/var/log/juju",
-    "/var/lib/juju",
-    "/home/ubuntu/config*",
-    "/root/cdk",
-    "/etc/juju-proxy*",
-    "/var/run/calico",
-    "/var/lib/calico",
-    "/var/log/calico",
-    "/etc/containerd",
-    "/var/lib/containerd",
-    "/opt/calicoctl",
-    "/opt/cni",
-    "/opt/containerd",
-    "/etc/keepalived"
-]
-
-for directory in directories:
-    subprocess.run(["rm", "-rf", directory])
--- a/containerd.charm
+++ b/containerd.charm
--- a/easyrsa.charm
+++ b/easyrsa.charm
--- a/etcd.charm
+++ b/etcd.charm
--- a/keepalived.charm
+++ b/keepalived.charm
--- a/kubeapi-load-balancer/.build.manifest
+++ b/kubeapi-load-balancer/.build.manifest
@ -0,0 +1,976 @@
+{
+  "layers": [
+    {
+      "branch": "refs/heads/main\nrefs/heads/release_1.24",
+      "rev": "f491ebe32b503c9712d2f8cd602dcce18f4aab46",
+      "url": "layer:metrics"
+    },
+    {
+      "branch": "refs/heads/main\nrefs/heads/release_1.24",
+      "rev": "fcdcea4e5de3e1556c24e6704607862d0ba00a56",
+      "url": "layer:options"
+    },
+    {
+      "branch": "refs/heads/main\nrefs/heads/release_1.24",
+      "rev": "fb767dcf0786d1d5364199bb3b40bdc86518b45b",
+      "url": "layer:basic"
+    },
+    {
+      "branch": "refs/heads/main\nrefs/heads/release_1.24",
+      "rev": "47dfcd4920ef6317850a4837ef0057ab0092a18e",
+      "url": "layer:nagios"
+    },
+    {
+      "branch": "refs/heads/main\nrefs/heads/release_1.24",
+      "rev": "a7d7b6423db37a47611310039e6ed1929c0a2eab",
+      "url": "layer:status"
+    },
+    {
+      "branch": "refs/heads/main\nrefs/heads/release_1.24",
+      "rev": "76bddfb640ab8767fc7e4a4b73a4a4e781948f34",
+      "url": "layer:apt"
+    },
+    {
+      "branch": "refs/heads/main\nrefs/heads/release_1.24",
+      "rev": "672d27695b512e50f51777b1eb63c5ff157b3d9e",
+      "url": "layer:nginx"
+    },
+    {
+      "branch": "refs/heads/main\nrefs/heads/release_1.24",
+      "rev": "527dd64fc4b9a6b0f8d80a3c2c0b865155050275",
+      "url": "layer:debug"
+    },
+    {
+      "branch": "refs/heads/main\nrefs/heads/release_1.24",
+      "rev": "fb46dec78d390571753d21876bbba689bbbca9e4",
+      "url": "layer:tls-client"
+    },
+    {
+      "branch": "refs/heads/release_1.24",
+      "rev": "b93fae0e73bb48074deb0062db204b621caa9f1f",
+      "url": "layer:kubernetes-common"
+    },
+    {
+      "branch": "refs/heads/main\nrefs/heads/release_1.24",
+      "rev": "5b0926cdc45f511a0040b0b26f89bd174d5c81eb",
+      "url": "layer:hacluster"
+    },
+    {
+      "branch": "refs/heads/main\nrefs/heads/release_1.24",
+      "rev": "cc5bd3f49b2fa5e6c3ab2336763c313ec8bf083f",
+      "url": "layer:leadership"
+    },
+    {
+      "branch": "refs/heads/release_1.24",
+      "rev": "4db88333338916dac097568ad2610c3024320b05",
+      "url": "kubeapi-load-balancer"
+    },
+    {
+      "branch": "refs/heads/main\nrefs/heads/release_1.24",
+      "rev": "95d744d1dbc4d86fb0462283c9371619bf5bbc24",
+      "url": "interface:nrpe-external-master"
+    },
+    {
+      "branch": "refs/heads/main\nrefs/heads/release_1.24",
+      "rev": "632131b1f122daf6fb601fd4c9f1e4dbb1a92e09",
+      "url": "interface:http"
+    },
+    {
+      "branch": "refs/heads/main\nrefs/heads/release_1.24",
+      "rev": "d9850016d930a6d507b9fd45e2598d327922b140",
+      "url": "interface:tls-certificates"
+    },
+    {
+      "branch": "refs/heads/main\nrefs/heads/release_1.24",
+      "rev": "2b714e90b1b8845ce7390bb1dad5a56a65437907",
+      "url": "interface:hacluster"
+    },
+    {
+      "branch": "refs/heads/main\nrefs/heads/release_1.24",
+      "rev": "5021f8a23f6e6e4cc449d2d02f2d8cb99763ec27",
+      "url": "interface:public-address"
+    }
+  ],
+  "signatures": {
+    ".build.manifest": [
+      "build",
+      "dynamic",
+      "unchecked"
+    ],
+    ".github/workflows/main.yml": [
+      "kubeapi-load-balancer",
+      "static",
+      "c457e9ca89018f53bd3b4e637bb8a7b5599e8748fd514547d4afd4137b908b0e"
+    ],
+    ".gitignore": [
+      "kubeapi-load-balancer",
+      "static",
+      "58e67f82f991b0c2d359d93622964c7c4f963aff3f8e2b7224b69810606c6c42"
+    ],
+    "AUTHORS": [
+      "layer:nginx",
+      "static",
+      "5e460cc5d7fe5ce6dc5c4e8eefc13159ee58874667baf9af3b5fa9b597a10fa2"
+    ],
+    "CONTRIBUTING.md": [
+      "kubeapi-load-balancer",
+      "static",
+      "7155516596ae597b0b7065f0463ff69031d689c0fc565998b51c06d999129d5a"
+    ],
+    "LICENSE": [
+      "kubeapi-load-balancer",
+      "static",
+      "58d1e17ffe5109a7ae296caafcadfdbe6a7d176f0bc4ab01e12a689b0499d8bd"
+    ],
+    "Makefile": [
+      "layer:basic",
+      "static",
+      "b7ab3a34e5faf79b96a8632039a0ad0aa87f2a9b5f0ba604e007cafb22190301"
+    ],
+    "README.md": [
+      "kubeapi-load-balancer",
+      "static",
+      "9efc40856c08af5871a051144d8c3bb518983a3bca118defbb81ad849b3f3c8d"
+    ],
+    "actions.yaml": [
+      "layer:debug",
+      "dynamic",
+      "cea290e28bc78458ea4a56dcad39b9a880c67e4ba53b774ac46bd8778618c7b9"
+    ],
+    "actions/debug": [
+      "layer:debug",
+      "static",
+      "db0a42dae4c5045b2c06385bf22209dfe0e2ded55822ef847d84b01d9ff2b046"
+    ],
+    "bin/charm-env": [
+      "layer:basic",
+      "static",
+      "fb6a20fac4102a6a4b6ffe903fcf666998f9a95a3647e6f9af7a1eeb44e58fd5"
+    ],
+    "bin/layer_option": [
+      "layer:options",
+      "static",
+      "e959bf29da4c5edff28b2602c24113c4df9e25cdc9f2aa3b5d46c8577b2a40cc"
+    ],
+    "config.yaml": [
+      "kubeapi-load-balancer",
+      "dynamic",
+      "586a155cd5fb93090f379e3c1ec9d350b89d73c58ebad447b03e36a886010ba7"
+    ],
+    "copyright": [
+      "kubeapi-load-balancer",
+      "static",
+      "badd4492d214890abd07b615f9e1a7a5ff3339b6c44655a826c746a9263ff00d"
+    ],
+    "copyright.layer-apt": [
+      "layer:apt",
+      "static",
+      "5123b2d0220fefb4424a463216fb41a6dd7cfad49c9799ba7037f1e74a2fd6bc"
+    ],
+    "copyright.layer-basic": [
+      "layer:basic",
+      "static",
+      "f6740d66fd60b60f2533d9fcb53907078d1e20920a0219afce7182e2a1c97629"
+    ],
+    "copyright.layer-leadership": [
+      "layer:leadership",
+      "static",
+      "8ce407829378fc0f72ce44c7f624e4951c7ccb3db1cfb949bee026b701728cc9"
+    ],
+    "copyright.layer-metrics": [
+      "layer:metrics",
+      "static",
+      "08509dcbade4c20761ba4382ef23c831744dbab1d4a8dd94a1c2b4d4e913334c"
+    ],
+    "copyright.layer-nagios": [
+      "layer:nagios",
+      "static",
+      "47b2363574909e748bcc471d9004780ac084b301c154905654b5b6f088474749"
+    ],
+    "copyright.layer-nginx": [
+      "layer:nginx",
+      "static",
+      "66b7d69f452f9203cbf702c57c58b16b359be9970781deb0e21893620dd52516"
+    ],
+    "copyright.layer-options": [
+      "layer:options",
+      "static",
+      "f6740d66fd60b60f2533d9fcb53907078d1e20920a0219afce7182e2a1c97629"
+    ],
+    "copyright.layer-status": [
+      "layer:status",
+      "static",
+      "7c0e36e618a8544faaaa3f8e0533c2f1f4a18bcacbdd8b99b537742e6b587d58"
+    ],
+    "debug-scripts/charm-unitdata": [
+      "layer:debug",
+      "static",
+      "c952b9d31f3942e4e722cb3e70f5119707b69b8e76cc44e2e906bc6d9aef49b7"
+    ],
+    "debug-scripts/filesystem": [
+      "layer:debug",
+      "static",
+      "d29cc8687f4422d024001c91b1ac756ee6bf8a2a125bc98db1199ba775eb8fd7"
+    ],
+    "debug-scripts/juju-logs": [
+      "layer:debug",
+      "static",
+      "d260b35753a917368cb8c64c1312546a0a40ef49cba84c75bc6369549807c55e"
+    ],
+    "debug-scripts/juju-network-get": [
+      "layer:debug",
+      "static",
+      "6d849a1f8e6569bd0d5ea38299f7937cb8b36a5f505e3532f6c756eabeb8b6c5"
+    ],
+    "debug-scripts/network": [
+      "layer:debug",
+      "static",
+      "714afae5dcb45554ff1f05285501e3b7fcc656c8de51217e263b93dab25a9d2e"
+    ],
+    "debug-scripts/packages": [
+      "layer:debug",
+      "static",
+      "e8177102dc2ca853cb9272c1257cf2cfd5253d2a074e602d07c8bc4ea8e27c75"
+    ],
+    "debug-scripts/sysctl": [
+      "layer:debug",
+      "static",
+      "990035b320e09cc2228e1f2f880e795d51118b2959339eacddff9cbb74349c6a"
+    ],
+    "debug-scripts/systemd": [
+      "layer:debug",
+      "static",
+      "23ddf533198bf5b1ce723acde31ada806aab8539292b514c721d8ec08af74106"
+    ],
+    "debug-scripts/tls-certs": [
+      "layer:tls-client",
+      "static",
+      "ebf7f23ef6e39fb8e664bac2e9429e32aaeb673b4a51751724b835c007e85d3b"
+    ],
+    "docs/status.md": [
+      "layer:status",
+      "static",
+      "975dec9f8c938196e102e954a80226bda293407c4e5ae857c118bf692154702a"
+    ],
+    "hooks/apiserver-relation-broken": [
+      "layer:basic",
+      "dynamic",
+      "2b693cb2a11594a80cc91235c2dc219a0a6303ae62bee8aa87eb35781f7158f7"
+    ],
+    "hooks/apiserver-relation-changed": [
+      "layer:basic",
+      "dynamic",
+      "2b693cb2a11594a80cc91235c2dc219a0a6303ae62bee8aa87eb35781f7158f7"
+    ],
+    "hooks/apiserver-relation-departed": [
+      "layer:basic",
+      "dynamic",
+      "2b693cb2a11594a80cc91235c2dc219a0a6303ae62bee8aa87eb35781f7158f7"
+    ],
+    "hooks/apiserver-relation-joined": [
+      "layer:basic",
+      "dynamic",
+      "2b693cb2a11594a80cc91235c2dc219a0a6303ae62bee8aa87eb35781f7158f7"
+    ],
+    "hooks/certificates-relation-broken": [
+      "layer:basic",
+      "dynamic",
+      "2b693cb2a11594a80cc91235c2dc219a0a6303ae62bee8aa87eb35781f7158f7"
+    ],
+    "hooks/certificates-relation-changed": [
+      "layer:basic",
+      "dynamic",
+      "2b693cb2a11594a80cc91235c2dc219a0a6303ae62bee8aa87eb35781f7158f7"
+    ],
+    "hooks/certificates-relation-departed": [
+      "layer:basic",
+      "dynamic",
+      "2b693cb2a11594a80cc91235c2dc219a0a6303ae62bee8aa87eb35781f7158f7"
+    ],
+    "hooks/certificates-relation-joined": [
+      "layer:basic",
+      "dynamic",
+      "2b693cb2a11594a80cc91235c2dc219a0a6303ae62bee8aa87eb35781f7158f7"
+    ],
+    "hooks/collect-metrics": [
+      "layer:metrics",
+      "static",
+      "139fe18ce4cf2bed2155d3d0fce1c3b4cf1bc2598242cda42b3d772ec9bf8558"
+    ],
+    "hooks/config-changed": [
+      "layer:basic",
+      "dynamic",
+      "2b693cb2a11594a80cc91235c2dc219a0a6303ae62bee8aa87eb35781f7158f7"
+    ],
+    "hooks/ha-relation-broken": [
+      "layer:basic",
+      "dynamic",
+      "2b693cb2a11594a80cc91235c2dc219a0a6303ae62bee8aa87eb35781f7158f7"
+    ],
+    "hooks/ha-relation-changed": [
+      "layer:basic",
+      "dynamic",
+      "2b693cb2a11594a80cc91235c2dc219a0a6303ae62bee8aa87eb35781f7158f7"
+    ],
+    "hooks/ha-relation-departed": [
+      "layer:basic",
+      "dynamic",
+      "2b693cb2a11594a80cc91235c2dc219a0a6303ae62bee8aa87eb35781f7158f7"
+    ],
+    "hooks/ha-relation-joined": [
+      "layer:basic",
+      "dynamic",
+      "2b693cb2a11594a80cc91235c2dc219a0a6303ae62bee8aa87eb35781f7158f7"
+    ],
+    "hooks/hook.template": [
+      "layer:basic",
+      "static",
+      "2b693cb2a11594a80cc91235c2dc219a0a6303ae62bee8aa87eb35781f7158f7"
+    ],
+    "hooks/install": [
+      "layer:basic",
+      "dynamic",
+      "2b693cb2a11594a80cc91235c2dc219a0a6303ae62bee8aa87eb35781f7158f7"
+    ],
+    "hooks/lb-consumers-relation-broken": [
+      "layer:basic",
+      "dynamic",
+      "2b693cb2a11594a80cc91235c2dc219a0a6303ae62bee8aa87eb35781f7158f7"
+    ],
+    "hooks/lb-consumers-relation-changed": [
+      "layer:basic",
+      "dynamic",
+      "2b693cb2a11594a80cc91235c2dc219a0a6303ae62bee8aa87eb35781f7158f7"
+    ],
+    "hooks/lb-consumers-relation-departed": [
+      "layer:basic",
+      "dynamic",
+      "2b693cb2a11594a80cc91235c2dc219a0a6303ae62bee8aa87eb35781f7158f7"
+    ],
+    "hooks/lb-consumers-relation-joined": [
+      "layer:basic",
+      "dynamic",
+      "2b693cb2a11594a80cc91235c2dc219a0a6303ae62bee8aa87eb35781f7158f7"
+    ],
+    "hooks/leader-elected": [
+      "layer:basic",
+      "dynamic",
+      "2b693cb2a11594a80cc91235c2dc219a0a6303ae62bee8aa87eb35781f7158f7"
+    ],
+    "hooks/leader-settings-changed": [
+      "layer:basic",
+      "dynamic",
+      "2b693cb2a11594a80cc91235c2dc219a0a6303ae62bee8aa87eb35781f7158f7"
+    ],
+    "hooks/loadbalancer-relation-broken": [
+      "layer:basic",
+      "dynamic",
+      "2b693cb2a11594a80cc91235c2dc219a0a6303ae62bee8aa87eb35781f7158f7"
+    ],
+    "hooks/loadbalancer-relation-changed": [
+      "layer:basic",
+      "dynamic",
+      "2b693cb2a11594a80cc91235c2dc219a0a6303ae62bee8aa87eb35781f7158f7"
+    ],
+    "hooks/loadbalancer-relation-departed": [
+      "layer:basic",
+      "dynamic",
+      "2b693cb2a11594a80cc91235c2dc219a0a6303ae62bee8aa87eb35781f7158f7"
+    ],
+    "hooks/loadbalancer-relation-joined": [
+      "layer:basic",
+      "dynamic",
+      "2b693cb2a11594a80cc91235c2dc219a0a6303ae62bee8aa87eb35781f7158f7"
+    ],
+    "hooks/nrpe-external-master-relation-broken": [
+      "layer:basic",
+      "dynamic",
+      "2b693cb2a11594a80cc91235c2dc219a0a6303ae62bee8aa87eb35781f7158f7"
+    ],
+    "hooks/nrpe-external-master-relation-changed": [
+      "layer:basic",
+      "dynamic",
+      "2b693cb2a11594a80cc91235c2dc219a0a6303ae62bee8aa87eb35781f7158f7"
+    ],
+    "hooks/nrpe-external-master-relation-departed": [
+      "layer:basic",
+      "dynamic",
+      "2b693cb2a11594a80cc91235c2dc219a0a6303ae62bee8aa87eb35781f7158f7"
+    ],
+    "hooks/nrpe-external-master-relation-joined": [
+      "layer:basic",
+      "dynamic",
+      "2b693cb2a11594a80cc91235c2dc219a0a6303ae62bee8aa87eb35781f7158f7"
+    ],
+    "hooks/post-series-upgrade": [
+      "layer:basic",
+      "dynamic",
+      "2b693cb2a11594a80cc91235c2dc219a0a6303ae62bee8aa87eb35781f7158f7"
+    ],
+    "hooks/pre-series-upgrade": [
+      "layer:basic",
+      "dynamic",
+      "2b693cb2a11594a80cc91235c2dc219a0a6303ae62bee8aa87eb35781f7158f7"
+    ],
+    "hooks/relations/hacluster/.stestr.conf": [
+      "interface:hacluster",
+      "static",
+      "46965969e6df6ac729b7dac68d57bc4e677e9f4d79d445be77f54ca3b9e58774"
+    ],
+    "hooks/relations/hacluster/README.md": [
+      "interface:hacluster",
+      "static",
+      "7fad91e409c6e559cdb76d11c89c325531adc25679049a629a28c4f890755f1f"
+    ],
+    "hooks/relations/hacluster/__init__.py": [
+      "interface:hacluster",
+      "static",
+      "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
+    ],
+    "hooks/relations/hacluster/copyright": [
+      "interface:hacluster",
+      "static",
+      "7a296596102da98cecee289a195e00d6af44241911321699b3d4d4af93f11893"
+    ],
+    "hooks/relations/hacluster/interface.yaml": [
+      "interface:hacluster",
+      "static",
+      "5f4e6c8d7b2884bdceeee422821f4db7163dbfa7994d86cb405ffef2c3dea43c"
+    ],
+    "hooks/relations/hacluster/interface_hacluster/__init__.py": [
+      "interface:hacluster",
+      "static",
+      "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
+    ],
+    "hooks/relations/hacluster/interface_hacluster/common.py": [
+      "interface:hacluster",
+      "static",
+      "eabe164702e7a98dd7e05e1ed34e556cfad4f43b37b015c8e21b51c84a316a2c"
+    ],
+    "hooks/relations/hacluster/requires.py": [
+      "interface:hacluster",
+      "static",
+      "68cf3ed22af30e42f34fc70ca484e8e4eeaedac6410bd3f228677cc791e6f46c"
+    ],
+    "hooks/relations/hacluster/test-requirements.txt": [
+      "interface:hacluster",
+      "static",
+      "63756e4b1c67bc161cee0d30d460dbb83911b2c064dc1c55454a30c1ab877616"
+    ],
+    "hooks/relations/http/.gitignore": [
+      "interface:http",
+      "static",
+      "83b4ca18cc39800b1d260b5633cd0252e21501b21e7c33e718db44f1a68a09b8"
+    ],
+    "hooks/relations/http/README.md": [
+      "interface:http",
+      "static",
+      "9c95320ad040745374fc03e972077f52c27e07eb0386ec93ae19bd50dca24c0d"
+    ],
+    "hooks/relations/http/__init__.py": [
+      "interface:http",
+      "static",
+      "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
+    ],
+    "hooks/relations/http/interface.yaml": [
+      "interface:http",
+      "static",
+      "d0b64038b85b7791ee4f3a42d73ffc8c208f206f73f899cbf33a519d12f9ad13"
+    ],
+    "hooks/relations/http/provides.py": [
+      "interface:http",
+      "static",
+      "8c72cd8a5a6ea24f53b6dba11f4353c75265bfa7d3ecc2dd096c8963eab8c877"
+    ],
+    "hooks/relations/http/requires.py": [
+      "interface:http",
+      "static",
+      "76cc886368eaf9c2403a6dc46b40531c3f4eaf67b08829f890c57cb645430abd"
+    ],
+    "hooks/relations/nrpe-external-master/README.md": [
+      "interface:nrpe-external-master",
+      "static",
+      "d8ed3bc7334f6581b12b6091923f58e6f5ef62075a095a4e78fb8f434a948636"
+    ],
+    "hooks/relations/nrpe-external-master/__init__.py": [
+      "interface:nrpe-external-master",
+      "static",
+      "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
+    ],
+    "hooks/relations/nrpe-external-master/interface.yaml": [
+      "interface:nrpe-external-master",
+      "static",
+      "894f24ba56148044dae5b7febf874b427d199239bcbe1f2f55c3db06bb77b5f0"
+    ],
+    "hooks/relations/nrpe-external-master/provides.py": [
+      "interface:nrpe-external-master",
+      "static",
+      "54e5400de99c051ecf6453776ad416b1cb8c6b73b34cbe2f41b617a8ed7b9daa"
+    ],
+    "hooks/relations/public-address/README.md": [
+      "interface:public-address",
+      "static",
+      "7225effe61bfd8571447b8b685a2ecb52be17431b3066a5306330954c4cb064d"
+    ],
+    "hooks/relations/public-address/__init__.py": [
+      "interface:public-address",
+      "static",
+      "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
+    ],
+    "hooks/relations/public-address/interface.yaml": [
+      "interface:public-address",
+      "static",
+      "49d6777a54aa84c7d3be8d531be237564e90f2e4cb2be05ef5617a372a382340"
+    ],
+    "hooks/relations/public-address/provides.py": [
+      "interface:public-address",
+      "static",
+      "7c99b0fe987d38773ed3e67c0378fdb78748c04d6895489cd4bca40aaeb051b2"
+    ],
+    "hooks/relations/public-address/requires.py": [
+      "interface:public-address",
+      "static",
+      "d6a7c6c0762d29a5db19afb4cf82af50812988d5e19a3a48fcbe8b0f6fec12a5"
+    ],
+    "hooks/relations/tls-certificates/.gitignore": [
+      "interface:tls-certificates",
+      "static",
+      "b485e74def213c534676224e655e9276b62d401ebc643508ddc545dd335cb6dc"
+    ],
+    "hooks/relations/tls-certificates/README.md": [
+      "interface:tls-certificates",
+      "static",
+      "6851227de8fcca7edfd504159dbe3e3af31080af64df46f3d3b345da7630827a"
+    ],
+    "hooks/relations/tls-certificates/__init__.py": [
+      "interface:tls-certificates",
+      "static",
+      "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
+    ],
+    "hooks/relations/tls-certificates/docs/common.md": [
+      "interface:tls-certificates",
+      "static",
+      "5e91d6637fc0ccc50af2776de9e59a0f8098244b627816b2e18fabb266e980ff"
+    ],
+    "hooks/relations/tls-certificates/docs/provides.md": [
+      "interface:tls-certificates",
+      "static",
+      "5c12dfca99b5c15ba10b4e7f7cff4cb4c9b621b198deba5f2397d3c837d035fe"
+    ],
+    "hooks/relations/tls-certificates/docs/requires.md": [
+      "interface:tls-certificates",
+      "static",
+      "148dd1de163d75253f0a9d3c35e108dcaacbc9bdf97e47186743e6c82a67b62e"
+    ],
+    "hooks/relations/tls-certificates/interface.yaml": [
+      "interface:tls-certificates",
+      "static",
+      "e412e54b1d327bad15a882f7f0bf996212090db576b863cc9cff7a68afc0e4fa"
+    ],
+    "hooks/relations/tls-certificates/make_docs": [
+      "interface:tls-certificates",
+      "static",
+      "3671543bddc9d277171263310e404df3f11660429582cb27b39b7e7ec8757a37"
+    ],
+    "hooks/relations/tls-certificates/provides.py": [
+      "interface:tls-certificates",
+      "static",
+      "be2a4b9a411c770989c529fd887070ad91649481a13f5239cfd8751f234b637c"
+    ],
+    "hooks/relations/tls-certificates/pydocmd.yml": [
+      "interface:tls-certificates",
+      "static",
+      "48a233f60a89f87d56e9bc715e05766f5d39bbea2bc8741ed31f67b30c8cfcb8"
+    ],
+    "hooks/relations/tls-certificates/requires.py": [
+      "interface:tls-certificates",
+      "static",
+      "442d773112079bc674d3e6be75b00323fcad7efd2f03613a1972b575dd438dba"
+    ],
+    "hooks/relations/tls-certificates/tls_certificates_common.py": [
+      "interface:tls-certificates",
+      "static",
+      "068bd32ba69bfa514e1da386919d18b348ee678b40c372f275c9110f2cc4677c"
+    ],
+    "hooks/relations/tls-certificates/tox.ini": [
+      "interface:tls-certificates",
+      "static",
+      "7ab8ab53e5ed98cfa7fb5c1d5009f84077a4bb76640ba64f561ef7ea3a702eab"
+    ],
+    "hooks/start": [
+      "layer:basic",
+      "dynamic",
+      "2b693cb2a11594a80cc91235c2dc219a0a6303ae62bee8aa87eb35781f7158f7"
+    ],
+    "hooks/stop": [
+      "layer:basic",
+      "dynamic",
+      "2b693cb2a11594a80cc91235c2dc219a0a6303ae62bee8aa87eb35781f7158f7"
+    ],
+    "hooks/update-status": [
+      "layer:basic",
+      "dynamic",
+      "2b693cb2a11594a80cc91235c2dc219a0a6303ae62bee8aa87eb35781f7158f7"
+    ],
+    "hooks/upgrade-charm": [
+      "layer:basic",
+      "dynamic",
+      "2b693cb2a11594a80cc91235c2dc219a0a6303ae62bee8aa87eb35781f7158f7"
+    ],
+    "hooks/website-relation-broken": [
+      "layer:basic",
+      "dynamic",
+      "2b693cb2a11594a80cc91235c2dc219a0a6303ae62bee8aa87eb35781f7158f7"
+    ],
+    "hooks/website-relation-changed": [
+      "layer:basic",
+      "dynamic",
+      "2b693cb2a11594a80cc91235c2dc219a0a6303ae62bee8aa87eb35781f7158f7"
+    ],
+    "hooks/website-relation-departed": [
+      "layer:basic",
+      "dynamic",
+      "2b693cb2a11594a80cc91235c2dc219a0a6303ae62bee8aa87eb35781f7158f7"
+    ],
+    "hooks/website-relation-joined": [
+      "layer:basic",
+      "dynamic",
+      "2b693cb2a11594a80cc91235c2dc219a0a6303ae62bee8aa87eb35781f7158f7"
+    ],
+    "icon.svg": [
+      "kubeapi-load-balancer",
+      "static",
+      "92271bf7063cc3a85a6d0fe2841250cf9bf8cd72697f3655f03ada39f8aee029"
+    ],
+    "layer.yaml": [
+      "kubeapi-load-balancer",
+      "dynamic",
+      "d7bac049bb8874aaab83bbe0339f1c1a4e726f27e548fa9705a0c890db70d5b2"
+    ],
+    "lib/.gitkeep": [
+      "layer:nginx",
+      "static",
+      "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
+    ],
+    "lib/charms/apt.py": [
+      "layer:apt",
+      "static",
+      "c7613992eb33ac94d83fbf02f467b614ea5112eaf561c4715def90989cefa531"
+    ],
+    "lib/charms/layer/__init__.py": [
+      "layer:basic",
+      "static",
+      "dfe0d26c6bf409767de6e2546bc648f150e1b396243619bad3aa0553ab7e0e6f"
+    ],
+    "lib/charms/layer/basic.py": [
+      "layer:basic",
+      "static",
+      "d120158e0c305a3b4529426a1a63a2f59af4f5730dccf3a59a9ffe1988494cee"
+    ],
+    "lib/charms/layer/execd.py": [
+      "layer:basic",
+      "static",
+      "fda8bd491032db1db8ddaf4e99e7cc878c6fb5432efe1f91cadb5b34765d076d"
+    ],
+    "lib/charms/layer/hacluster.py": [
+      "layer:hacluster",
+      "static",
+      "f58e0c1503187247f858ff3c9a1166d59107afd1557ba89e4878ec2e79304f8a"
+    ],
+    "lib/charms/layer/kubernetes_common.py": [
+      "layer:kubernetes-common",
+      "static",
+      "bc89bd609a8e94102e00a192b7ae3caa813cca5e356536330494742bfdb6c4cb"
+    ],
+    "lib/charms/layer/nagios.py": [
+      "layer:nagios",
+      "static",
+      "0246710bdbea844356007a64409907d93e6e94a289d83266e8b7c5d921fb3a6c"
+    ],
+    "lib/charms/layer/nginx.py": [
+      "layer:nginx",
+      "static",
+      "5fea9e756b8e9ad09d0256d9f2a1e8e2169a97741af256653ca85b4412e40174"
+    ],
+    "lib/charms/layer/options.py": [
+      "layer:options",
+      "static",
+      "8ae7a07d22542fc964f2d2bee8219d1c78a68dace70a1b38d36d4aea47b1c3b2"
+    ],
+    "lib/charms/layer/status.py": [
+      "layer:status",
+      "static",
+      "d560a5e07b2e5f2b0f25f30e1f0278b06f3f90c01e4dbad5c83d71efc79018c6"
+    ],
+    "lib/charms/layer/tls_client.py": [
+      "layer:tls-client",
+      "static",
+      "34531c3980777b661b913d77c432fc371ed10425473c2eb365b1dd5540c2ec6e"
+    ],
+    "lib/charms/leadership.py": [
+      "layer:leadership",
+      "static",
+      "20ffcbbc08147506759726ad51567420659ffb8a2e0121079240b8706658e332"
+    ],
+    "lib/debug_script.py": [
+      "layer:debug",
+      "static",
+      "a4d56f2d3e712b1b5cadb657c7195c6268d0aac6d228991049fd769e0ddaf453"
+    ],
+    "lib/nginxlib.py": [
+      "layer:nginx",
+      "static",
+      "bae474acba0fbf9da21f1372dcda1dba848757c5e7cebb6fb22c29f04a67c0aa"
+    ],
+    "make_docs": [
+      "layer:status",
+      "static",
+      "c990f55c8e879793a62ed8464ee3d7e0d7d2225fdecaf17af24b0df0e2daa8c1"
+    ],
+    "manifest.yaml": [
+      "kubeapi-load-balancer",
+      "static",
+      "06bd2f274e54dccb127626d61d42fc2ac0b5a0d15da0713c3c36cd4363389d87"
+    ],
+    "metadata.yaml": [
+      "kubeapi-load-balancer",
+      "dynamic",
+      "6861cfdcfbeead1cbb165aabbb34a7a5ec726ebe4862209af70fcba55a283caa"
+    ],
+    "metrics.yaml": [
+      "kubeapi-load-balancer",
+      "static",
+      "94a5eb0b0966f8ba434d91ff1e9b99b1b4c3b3044657b236d4e742d3e0d57c47"
+    ],
+    "pydocmd.yml": [
+      "layer:status",
+      "static",
+      "11d9293901f32f75f4256ae4ac2073b92ce1d7ef7b6c892ba9fbb98690a0b330"
+    ],
+    "pyproject.toml": [
+      "layer:apt",
+      "static",
+      "19689509a5fb9bfc90ed1e873122ac0a90f22533b7f40055c38fdd587fe297de"
+    ],
+    "reactive/__init__.py": [
+      "layer:leadership",
+      "static",
+      "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
+    ],
+    "reactive/apt.py": [
+      "layer:apt",
+      "static",
+      "6fe40f18eb84a910a71a4acb7ec74856128de846de6029b4fc297a875692c837"
+    ],
+    "reactive/hacluster.py": [
+      "layer:hacluster",
+      "static",
+      "7b56e9efc95ace190694e439eff210f0981811f89dc46a026a400e114f3f833d"
+    ],
+    "reactive/leadership.py": [
+      "layer:leadership",
+      "static",
+      "e2b233cf861adc3b2d9e9c062134ce2f104953f03283cdddd88f49efee652e8f"
+    ],
+    "reactive/load_balancer.py": [
+      "kubeapi-load-balancer",
+      "static",
+      "bca19a310482a2ebb5b5887341998b116a6fb6c63506bea25fcd3eabd3bc1574"
+    ],
+    "reactive/nginx.py": [
+      "layer:nginx",
+      "static",
+      "046769111b72a5a5aa7bfd6362db988361719586bee4e9b40a472f33c0cf09a8"
+    ],
+    "reactive/status.py": [
+      "layer:status",
+      "static",
+      "30207fc206f24e91def5252f1c7f7c8e23c0aed0e93076babf5e03c05296d207"
+    ],
+    "reactive/tls_client.py": [
+      "layer:tls-client",
+      "static",
+      "08e850e401d2004523dca6b5e6bc47c33d558bf575dd55969491e11cd3ed98c8"
+    ],
+    "requirements.txt": [
+      "layer:basic",
+      "static",
+      "a00f75d80849e5b4fc5ad2e7536f947c25b1a4044b341caa8ee87a92d3a4c804"
+    ],
+    "templates/.gitkeep": [
+      "layer:nginx",
+      "static",
+      "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
+    ],
+    "templates/apilb.conf": [
+      "kubeapi-load-balancer",
+      "static",
+      "2f84c6592f300bba3e197d22c0b43c24320ca5510e6d53cc8d5750d0061e1de6"
+    ],
+    "templates/cdk.auth-webhook-secret.yaml": [
+      "layer:kubernetes-common",
+      "static",
+      "efaf34c12a5c961fa7843199070945ba05717b3656a0f3acc3327f45334bcaec"
+    ],
+    "templates/vhost.conf.ex": [
+      "layer:nginx",
+      "static",
+      "f68c366c35a8487acb78da6f1086eeee33a3eccdbe5a524509039c0c41ad5d5a"
+    ],
+    "tests/data/charm.yaml": [
+      "kubeapi-load-balancer",
+      "static",
+      "c20a3e6b0422cf2607ecbfcbf747e876e86abaa16381cbca3c987c4f65611bd4"
+    ],
+    "tests/data/ip_addr_json": [
+      "layer:kubernetes-common",
+      "static",
+      "f129576a9e2c7738aca8669c642f123534eda63121ae450cec4cbda787b1eb06"
+    ],
+    "tests/functional/conftest.py": [
+      "layer:kubernetes-common",
+      "static",
+      "fd53e0c38b4dda0c18096167889cd0d85b98b0a13225f9f8853261241e94078c"
+    ],
+    "tests/functional/test_k8s_common.py": [
+      "layer:kubernetes-common",
+      "static",
+      "680a53724154771dd78422bbaf24b151788d86dd07960712c5d9e0d758499b50"
+    ],
+    "tests/integration/test_kubeapi-load-balancer_integration.py": [
+      "kubeapi-load-balancer",
+      "static",
+      "ad9da65fe5b129dbf24adf0c2892f2405add076357273a8473602eb020540914"
+    ],
+    "tests/unit/conftest.py": [
+      "kubeapi-load-balancer",
+      "static",
+      "b38bf2bc23b57be1345143e07e361e41f20cff55848e7ba3ce86400d61e16081"
+    ],
+    "tests/unit/test_k8s_common.py": [
+      "layer:kubernetes-common",
+      "static",
+      "23e097e7f21e4f4f062caac0146bb85373e895a30be1be5667b90d0e84435882"
+    ],
+    "tests/unit/test_kubeapi_load_balancer.py": [
+      "kubeapi-load-balancer",
+      "static",
+      "8c31c2541800259eab3461d0295ed0c76d763596b2a99a5ecdd683d65402517f"
+    ],
+    "tests/validate-wheelhouse.sh": [
+      "kubeapi-load-balancer",
+      "static",
+      "1c74bea041866cf4bd75763190d3c512e1d63a19b04e35178a64b8c517bb3231"
+    ],
+    "tox.ini": [
+      "kubeapi-load-balancer",
+      "static",
+      "db04c740dd6f024f68c7f24a1e7d4cf3c9d92332475e003ce91a24bc4c7e1002"
+    ],
+    "version": [
+      "kubeapi-load-balancer",
+      "dynamic",
+      "d9a4b742c183b4dd1b9fbf1e74567e06d0d4aa5538814dfde937416b3ac1bcc9"
+    ],
+    "wheelhouse.txt": [
+      "kubeapi-load-balancer",
+      "dynamic",
+      "9cabe75bb18bd698c311a37ba23e8b874196f29469fe6898f87800e7f16fc8fd"
+    ],
+    "wheelhouse/Jinja2-3.0.3.tar.gz": [
+      "layer:basic",
+      "dynamic",
+      "611bb273cd68f3b993fabdc4064fc858c5b47a973cb5aa7999ec1ba405c87cd7"
+    ],
+    "wheelhouse/MarkupSafe-2.0.1.tar.gz": [
+      "layer:basic",
+      "dynamic",
+      "594c67807fb16238b30c44bdf74f36c02cdf22d1c8cda91ef8a0ed8dabf5620a"
+    ],
+    "wheelhouse/PyYAML-5.3.1.tar.gz": [
+      "layer:basic",
+      "dynamic",
+      "b8eac752c5e14d3eca0e6dd9199cd627518cb5ec06add0de9d32baeee6fe645d"
+    ],
+    "wheelhouse/cached-property-1.5.2.tar.gz": [
+      "__pip__",
+      "dynamic",
+      "9fa5755838eecbb2d234c3aa390bd80fbd3ac6b6869109bfc1b499f7bd89a130"
+    ],
+    "wheelhouse/charmhelpers-1.2.1.tar.gz": [
+      "layer:basic",
+      "dynamic",
+      "298bb9e90d9392e2b66d10a5199b1b2d459dc8d5434b897913325904989dd2d7"
+    ],
+    "wheelhouse/charms.reactive-1.5.2.tar.gz": [
+      "layer:basic",
+      "dynamic",
+      "4cb67e15402b95e766877666f985d157b7e917dc6170ec6d922d79928aefa6b8"
+    ],
+    "wheelhouse/loadbalancer_interface-1.2.0.tar.gz": [
+      "kubeapi-load-balancer",
+      "dynamic",
+      "f2b31a5bf25b0435eee696685af78082c8a93fbe85336755bea5b17392a584bd"
+    ],
+    "wheelhouse/marshmallow-3.14.1.tar.gz": [
+      "__pip__",
+      "dynamic",
+      "4c05c1684e0e97fe779c62b91878f173b937fe097b356cd82f793464f5bc6138"
+    ],
+    "wheelhouse/marshmallow-enum-1.5.1.tar.gz": [
+      "__pip__",
+      "dynamic",
+      "38e697e11f45a8e64b4a1e664000897c659b60aa57bfa18d44e226a9920b6e58"
+    ],
+    "wheelhouse/netaddr-0.7.19.tar.gz": [
+      "layer:basic",
+      "dynamic",
+      "38aeec7cdd035081d3a4c306394b19d677623bf76fa0913f6695127c7753aefd"
+    ],
+    "wheelhouse/ops-1.5.5.tar.gz": [
+      "__pip__",
+      "dynamic",
+      "07210019819daf35693585619d01b620f89640f8a9b4d50061f4d84f9fcf31e5"
+    ],
+    "wheelhouse/ops_reactive_interface-1.0.1.tar.gz": [
+      "__pip__",
+      "dynamic",
+      "9ed351c42fc187299c23125975aa3dfee9f6aaae0c9d49bce8904ac079255dba"
+    ],
+    "wheelhouse/pbr-5.11.1.tar.gz": [
+      "__pip__",
+      "dynamic",
+      "aefc51675b0b533d56bb5fd1c8c6c0522fe31896679882e1c4c63d5e4a0fccb3"
+    ],
+    "wheelhouse/pip-18.1.tar.gz": [
+      "layer:basic",
+      "dynamic",
+      "c0a292bd977ef590379a3f05d7b7f65135487b67470f6281289a94e015650ea1"
+    ],
+    "wheelhouse/pyaml-21.10.1.tar.gz": [
+      "__pip__",
+      "dynamic",
+      "c6519fee13bf06e3bb3f20cacdea8eba9140385a7c2546df5dbae4887f768383"
+    ],
+    "wheelhouse/setuptools-41.6.0.zip": [
+      "layer:basic",
+      "dynamic",
+      "6afa61b391dcd16cb8890ec9f66cc4015a8a31a6e1c2b4e0c464514be1a3d722"
+    ],
+    "wheelhouse/setuptools_scm-1.17.0.tar.gz": [
+      "layer:basic",
+      "dynamic",
+      "70a4cf5584e966ae92f54a764e6437af992ba42ac4bca7eb37cc5d02b98ec40a"
+    ],
+    "wheelhouse/toml-0.10.2.tar.gz": [
+      "layer:nginx",
+      "dynamic",
+      "b3bda1d108d5dd99f4a20d24d9c348e91c4db7ab1b749200bded2f839ccbe68f"
+    ],
+    "wheelhouse/wheel-0.33.6.tar.gz": [
+      "layer:basic",
+      "dynamic",
+      "10c9da68765315ed98850f8e048347c3eb06dd81822dc2ab1d4fde9dc9702646"
+    ]
+  }
+}
--- a/kubeapi-load-balancer/.github/workflows/main.yml
+++ b/kubeapi-load-balancer/.github/workflows/main.yml
@ -0,0 +1,44 @@
+name: Test Suite
+on: [pull_request]
+
+jobs:
+  call-inclusive-naming-check:
+    name: Inclusive naming
+    uses: canonical-web-and-design/Inclusive-naming/.github/workflows/woke.yaml@main
+    with:
+      fail-on-error: "true"
+
+  validate-wheelhouse:
+    name: Validate Wheelhouse
+    uses: charmed-kubernetes/workflows/.github/workflows/validate-wheelhouse.yaml@main
+
+  lint-unit:
+    name: Lint Unit
+    uses: charmed-kubernetes/workflows/.github/workflows/lint-unit.yaml@main
+
+  integration-test:
+    name: Integration test with VMWare
+    runs-on: self-hosted
+    timeout-minutes: 360
+    needs:
+      - call-inclusive-naming-check
+      - validate-wheelhouse
+      - lint-unit
+    steps:
+      - name: Check out code
+        uses: actions/checkout@v2
+      - name: Setup Python
+        uses: actions/setup-python@v2
+        with:
+          python-version: 3.8
+      - name: Setup operator environment
+        uses: charmed-kubernetes/actions-operator@main
+        with:
+          provider: vsphere
+          credentials-yaml: ${{ secrets.CREDENTIALS_YAML }}
+          clouds-yaml: ${{ secrets.CLOUDS_YAML }}
+          juju-channel: 2.9/stable
+          bootstrap-constraints: "arch=amd64 cores=2 mem=4G"
+          bootstrap-options: "${{ secrets.FOCAL_BOOTSTRAP_OPTIONS }} --model-default datastore=vsanDatastore --model-default primary-network=VLAN_2764"
+      - name: Run test
+        run: tox -e integration -- --basetemp=/home/ubuntu/pytest
--- a/kubeapi-load-balancer/.gitignore
+++ b/kubeapi-load-balancer/.gitignore
@ -0,0 +1,4 @@
+.tox/
+__pycache__/
+*.pyc
+*.charm
--- a/kubeapi-load-balancer/AUTHORS
+++ b/kubeapi-load-balancer/AUTHORS
@ -0,0 +1,2 @@
+Adam Stokes <adam.stokes@ubuntu.com>
+Marco Ceppi <marco@ceppi.net>
--- a/kubeapi-load-balancer/CONTRIBUTING.md
+++ b/kubeapi-load-balancer/CONTRIBUTING.md
@ -0,0 +1,37 @@
+# Contributor Guide
+
+This Juju charm is open source ([Apache License 2.0](./LICENSE)) and we actively seek any community contibutions
+for code, suggestions and documentation.
+This page details a few notes, workflows and suggestions for how to make contributions most effective and help us
+all build a better charm - please give them a read before working on any contributions.
+
+## Licensing
+
+This charm has been created under the [Apache License 2.0](./LICENSE), which will cover any contributions you may
+make to this project. Please familiarise yourself with the terms of the license.
+
+Additionally, this charm uses the Harmony CLA agreement.  It’s the easiest way for you to give us permission to
+use your contributions.
+In effect, you’re giving us a license, but you still own the copyright — so you retain the right to modify your
+code and use it in other projects. Please [sign the CLA here](https://ubuntu.com/legal/contributors/agreement) before
+making any contributions.
+
+## Code of conduct
+
+We have adopted the Ubuntu code of Conduct. You can read this in full [here](https://ubuntu.com/community/code-of-conduct).
+
+## Contributing code
+
+To contribute code to this project, please use the following workflow:
+
+1. [Submit a bug](https://bugs.launchpad.net/charm-kubeapi-load-balancer/+filebug) to explain the need for and track the change.
+2. Create a branch on your fork of the repo with your changes, including a unit test covering the new or modified code.
+3. Submit a PR. The PR description should include a link to the bug on Launchpad.
+4. Update the Launchpad bug to include a link to the PR and the `review-needed` tag.
+5. Once reviewed and merged, the change will become available on the edge channel and assigned to an appropriate milestone
+   for further release according to priority.
+
+## Documentation
+
+Documentation for this charm is currently maintained as part of the Charmed Kubernetes docs.
+See [this page](https://github.com/charmed-kubernetes/kubernetes-docs/blob/master/pages/k8s/charm-kubeapi-load-balancer.md)
--- a/kubeapi-load-balancer/LICENSE
+++ b/kubeapi-load-balancer/LICENSE
@ -0,0 +1,202 @@
+
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright [yyyy] [name of copyright owner]
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.
--- a/kubeapi-load-balancer/Makefile
+++ b/kubeapi-load-balancer/Makefile
@ -0,0 +1,24 @@
+#!/usr/bin/make
+
+all: lint unit_test
+
+
+.PHONY: clean
+clean:
+	@rm -rf .tox
+
+.PHONY: apt_prereqs
+apt_prereqs:
+	@# Need tox, but don't install the apt version unless we have to (don't want to conflict with pip)
+	@which tox >/dev/null || (sudo apt-get install -y python-pip && sudo pip install tox)
+
+.PHONY: lint
+lint: apt_prereqs
+	@tox --notest
+	@PATH=.tox/py34/bin:.tox/py35/bin flake8 $(wildcard hooks reactive lib unit_tests tests)
+	@charm proof
+
+.PHONY: unit_test
+unit_test: apt_prereqs
+	@echo Starting tests...
+	tox
--- a/kubeapi-load-balancer/README.md
+++ b/kubeapi-load-balancer/README.md
@ -0,0 +1,15 @@
+# kubeapi-load-balancer
+
+Simple NGINX reverse proxy to lend a hand in HA kubernetes-control-plane deployments.
+
+
+This charm is a component of Charmed Kubernetes. For full information,
+please visit the [official Charmed Kubernetes docs](https://www.ubuntu.com/kubernetes/docs/charm-kubeapi-load-balancer).
+
+# Developers
+
+## Building the charm
+
+```
+make charm
+```
--- a/kubeapi-load-balancer/actions.yaml
+++ b/kubeapi-load-balancer/actions.yaml
@ -0,0 +1,2 @@
+"debug":
+  "description": "Collect debug data"
--- a/kubeapi-load-balancer/actions/debug
+++ b/kubeapi-load-balancer/actions/debug
@ -0,0 +1,102 @@
+#!/usr/local/sbin/charm-env python3
+
+import os
+import subprocess
+import tarfile
+import tempfile
+import traceback
+from contextlib import contextmanager
+from datetime import datetime
+from charmhelpers.core.hookenv import action_set, local_unit
+
+archive_dir = None
+log_file = None
+
+
+@contextmanager
+def archive_context():
+    """ Open a context with a new temporary directory.
+
+    When the context closes, the directory is archived, and the archive
+    location is added to Juju action output. """
+    global archive_dir
+    global log_file
+    with tempfile.TemporaryDirectory() as temp_dir:
+        name = "debug-" + datetime.now().strftime("%Y%m%d%H%M%S")
+        archive_dir = os.path.join(temp_dir, name)
+        os.makedirs(archive_dir)
+        with open("%s/debug.log" % archive_dir, "w") as log_file:
+            yield
+        os.chdir(temp_dir)
+        tar_path = "/home/ubuntu/%s.tar.gz" % name
+        with tarfile.open(tar_path, "w:gz") as f:
+            f.add(name)
+        action_set({
+            "path": tar_path,
+            "command": "juju scp %s:%s ." % (local_unit(), tar_path),
+            "message": " ".join([
+                "Archive has been created on unit %s." % local_unit(),
+                "Use the juju scp command to copy it to your local machine."
+            ])
+        })
+
+
+def log(msg):
+    """ Log a message that will be included in the debug archive.
+
+    Must be run within archive_context """
+    timestamp = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
+    for line in str(msg).splitlines():
+        log_file.write(timestamp + " | " + line.rstrip() + "\n")
+
+
+def run_script(script):
+    """ Run a single script. Must be run within archive_context """
+    log("Running script: " + script)
+    script_dir = os.path.join(archive_dir, script)
+    os.makedirs(script_dir)
+    env = os.environ.copy()
+    env["PYTHONPATH"] = "lib"  # allow same imports as reactive code
+    env["DEBUG_SCRIPT_DIR"] = script_dir
+    with open(script_dir + "/stdout", "w") as stdout:
+        with open(script_dir + "/stderr", "w") as stderr:
+            process = subprocess.Popen(
+                "debug-scripts/" + script,
+                stdout=stdout, stderr=stderr, env=env
+            )
+            try:
+                exit_code = process.wait(timeout=300)
+            except subprocess.TimeoutExpired:
+                log("ERROR: still running, terminating")
+                process.terminate()
+                try:
+                    exit_code = process.wait(timeout=10)
+                except subprocess.TimeoutExpired:
+                    log("ERROR: still running, killing")
+                    process.kill()
+                    exit_code = process.wait(timeout=10)
+    if exit_code != 0:
+        log("ERROR: %s failed with exit code %d" % (script, exit_code))
+
+
+def run_all_scripts():
+    """ Run all scripts. For the sake of robustness, log and ignore any
+    exceptions that occur.
+
+    Must be run within archive_context """
+    scripts = os.listdir("debug-scripts")
+    for script in scripts:
+        try:
+            run_script(script)
+        except:
+            log(traceback.format_exc())
+
+
+def main():
+    """ Open an archive context and run all scripts. """
+    with archive_context():
+        run_all_scripts()
+
+
+if __name__ == "__main__":
+    main()
--- a/kubeapi-load-balancer/bin/charm-env
+++ b/kubeapi-load-balancer/bin/charm-env
@ -0,0 +1,107 @@
+#!/bin/bash
+
+VERSION="1.0.0"
+
+
+find_charm_dirs() {
+    # Hopefully, $JUJU_CHARM_DIR is set so which venv to use in unambiguous.
+    if [[ -n "$JUJU_CHARM_DIR" || -n "$CHARM_DIR" ]]; then
+        if [[ -z "$JUJU_CHARM_DIR" ]]; then
+            # accept $CHARM_DIR to be more forgiving
+            export JUJU_CHARM_DIR="$CHARM_DIR"
+        fi
+        if [[ -z "$CHARM_DIR" ]]; then
+            # set CHARM_DIR as well to help with backwards compatibility
+            export CHARM_DIR="$JUJU_CHARM_DIR"
+        fi
+        return
+    fi
+    # Try to guess the value for JUJU_CHARM_DIR by looking for a non-subordinate
+    # (because there's got to be at least one principle) charm directory;
+    # if there are several, pick the first by alpha order.
+    agents_dir="/var/lib/juju/agents"
+    if [[ -d "$agents_dir" ]]; then
+        desired_charm="$1"
+        found_charm_dir=""
+        if [[ -n "$desired_charm" ]]; then
+            for charm_dir in $(/bin/ls -d "$agents_dir"/unit-*/charm); do
+                charm_name="$(grep -o '^['\''"]\?name['\''"]\?:.*' $charm_dir/metadata.yaml 2> /dev/null | sed -e 's/.*: *//' -e 's/['\''"]//g')"
+                if [[ "$charm_name" == "$desired_charm" ]]; then
+                    if [[ -n "$found_charm_dir" ]]; then
+                        >&2 echo "Ambiguous possibilities for JUJU_CHARM_DIR matching '$desired_charm'; please run within a Juju hook context"
+                        exit 1
+                    fi
+                    found_charm_dir="$charm_dir"
+                fi
+            done
+            if [[ -z "$found_charm_dir" ]]; then
+                >&2 echo "Unable to determine JUJU_CHARM_DIR matching '$desired_charm'; please run within a Juju hook context"
+                exit 1
+            fi
+            export JUJU_CHARM_DIR="$found_charm_dir"
+            export CHARM_DIR="$found_charm_dir"
+            return
+        fi
+        # shellcheck disable=SC2126
+        non_subordinates="$(grep -L 'subordinate"\?:.*true' "$agents_dir"/unit-*/charm/metadata.yaml | wc -l)"
+        if [[ "$non_subordinates" -gt 1 ]]; then
+            >&2 echo 'Ambiguous possibilities for JUJU_CHARM_DIR; please use --charm or run within a Juju hook context'
+            exit 1
+        elif [[ "$non_subordinates" -eq 1 ]]; then
+            for charm_dir in $(/bin/ls -d "$agents_dir"/unit-*/charm); do
+                if grep -q 'subordinate"\?:.*true' "$charm_dir/metadata.yaml"; then
+                    continue
+                fi
+                export JUJU_CHARM_DIR="$charm_dir"
+                export CHARM_DIR="$charm_dir"
+                return
+            done
+        fi
+    fi
+    >&2 echo 'Unable to determine JUJU_CHARM_DIR; please run within a Juju hook context'
+    exit 1
+}
+
+try_activate_venv() {
+    if [[ -d "$JUJU_CHARM_DIR/../.venv" ]]; then
+        . "$JUJU_CHARM_DIR/../.venv/bin/activate"
+    fi
+}
+
+find_wrapped() {
+    PATH="${PATH/\/usr\/local\/sbin:}" which "$(basename "$0")"
+}
+
+
+if [[ "$1" == "--version" || "$1" == "-v" ]]; then
+    echo "$VERSION"
+    exit 0
+fi
+
+
+# allow --charm option to hint which JUJU_CHARM_DIR to choose when ambiguous
+# NB: --charm option must come first
+# NB: option must be processed outside find_charm_dirs to modify $@
+charm_name=""
+if [[ "$1" == "--charm" ]]; then
+    charm_name="$2"
+    shift; shift
+fi
+
+find_charm_dirs "$charm_name"
+try_activate_venv
+export PYTHONPATH="$JUJU_CHARM_DIR/lib:$PYTHONPATH"
+
+if [[ "$(basename "$0")" == "charm-env" ]]; then
+    # being used as a shebang
+    exec "$@"
+elif [[ "$0" == "$BASH_SOURCE" ]]; then
+    # being invoked as a symlink wrapping something to find in the venv
+    exec "$(find_wrapped)" "$@"
+elif [[ "$(basename "$BASH_SOURCE")" == "charm-env" ]]; then
+    # being sourced directly; do nothing
+    /bin/true
+else
+    # being sourced for wrapped bash helpers
+    . "$(find_wrapped)"
+fi
--- a/kubeapi-load-balancer/bin/layer_option
+++ b/kubeapi-load-balancer/bin/layer_option
@ -0,0 +1,22 @@
+#!/usr/bin/env python3
+
+import sys
+import argparse
+from charms import layer
+
+
+parser = argparse.ArgumentParser(description='Access layer options.')
+parser.add_argument('section',
+                    help='the section, or layer, the option is from')
+parser.add_argument('option',
+                    help='the option to access')
+
+args = parser.parse_args()
+value = layer.options.get(args.section, args.option)
+if isinstance(value, bool):
+    sys.exit(0 if value else 1)
+elif isinstance(value, list):
+    for val in value:
+        print(val)
+else:
+    print(value)
--- a/kubeapi-load-balancer/config.yaml
+++ b/kubeapi-load-balancer/config.yaml
@ -0,0 +1,91 @@
+"options":
+  "nagios_context":
+    "default": "juju"
+    "type": "string"
+    "description": |
+      Used by the nrpe subordinate charms.
+      A string that will be prepended to instance name to set the host name
+      in nagios. So for instance the hostname would be something like:
+          juju-myservice-0
+      If you're running multiple environments with the same services in them
+      this allows you to differentiate between them.
+  "nagios_servicegroups":
+    "default": ""
+    "type": "string"
+    "description": |
+      A comma-separated list of nagios servicegroups.
+      If left empty, the nagios_context will be used as the servicegroup
+  "extra_packages":
+    "description": >
+      Space separated list of extra deb packages to install.
+    "type": "string"
+    "default": ""
+  "package_status":
+    "default": "install"
+    "type": "string"
+    "description": >
+      The status of service-affecting packages will be set to this
+      value in the dpkg database. Valid values are "install" and "hold".
+  "install_sources":
+    "description": >
+      List of extra apt sources, per charm-helpers standard
+      format (a yaml list of strings encoded as a string). Each source
+      may be either a line that can be added directly to
+      sources.list(5), or in the form ppa:<user>/<ppa-name> for adding
+      Personal Package Archives, or a distribution component to enable.
+    "type": "string"
+    "default": ""
+  "install_keys":
+    "description": >
+      List of signing keys for install_sources package sources, per
+      charmhelpers standard format (a yaml list of strings encoded as
+      a string). The keys should be the full ASCII armoured GPG public
+      keys. While GPG key ids are also supported and looked up on a
+      keyserver, operators should be aware that this mechanism is
+      insecure. null can be used if a standard package signing key is
+      used that will already be installed on the machine, and for PPA
+      sources where the package signing key is securely retrieved from
+      Launchpad.
+    "type": "string"
+    "default": ""
+  "port":
+    "type": "int"
+    "default": !!int "443"
+    "description": |-
+      The port to run the loadbalancer
+  "host":
+    "type": "string"
+    "default": "127.0.0.1"
+    "description": "listen address"
+  "ha-cluster-vip":
+    "type": "string"
+    "description": |
+      Virtual IP for the charm to use with the HA Cluster subordinate charm
+      Mutually exclusive with ha-cluster-dns. Multiple virtual IPs are
+      separated by spaces.
+    "default": ""
+  "ha-cluster-dns":
+    "type": "string"
+    "description": |
+      DNS entry to use with the HA Cluster subordinate charm.
+      Mutually exclusive with ha-cluster-vip.
+    "default": ""
+  "extra_sans":
+    "type": "string"
+    "default": ""
+    "description": |
+      Space-separated list of extra SAN entries to add to the x509 certificate
+      created for the load balancers.
+  "proxy_read_timeout":
+    "type": "int"
+    "default": !!int "600"
+    "description": "Timeout in seconds for reading a response from proxy server."
+  "loadbalancer-ips":
+    "type": "string"
+    "description": |
+      Space seperated list of IP addresses of loadbalancers in front of control plane.
+      A common case for this is virtual IP addresses that are floated in front of the
+      kubeapi-load-balancer charm. The workers will alternate IP addresses from this
+      list to distribute load. If you have 2 IPs and 4 workers, each IP will be used
+      by 2 workers.
+    "default": ""
--- a/kubeapi-load-balancer/copyright
+++ b/kubeapi-load-balancer/copyright
@ -0,0 +1,13 @@
+Copyright 2016 The Kubernetes Authors.
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+     http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
--- a/kubeapi-load-balancer/copyright.layer-apt
+++ b/kubeapi-load-balancer/copyright.layer-apt
@ -0,0 +1,15 @@
+Copyright 2015-2016 Canonical Ltd.
+
+This file is part of the Apt layer for Juju.
+
+This program is free software: you can redistribute it and/or modify
+it under the terms of the GNU General Public License version 3, as
+published by the Free Software Foundation.
+
+This program is distributed in the hope that it will be useful, but
+WITHOUT ANY WARRANTY; without even the implied warranties of
+MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
+PURPOSE.  See the GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program.  If not, see <http://www.gnu.org/licenses/>.
--- a/kubeapi-load-balancer/copyright.layer-basic
+++ b/kubeapi-load-balancer/copyright.layer-basic
@ -0,0 +1,16 @@
+Format: http://dep.debian.net/deps/dep5/
+
+Files: *
+Copyright: Copyright 2015-2017, Canonical Ltd., All Rights Reserved.
+License: Apache License 2.0
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+ .
+     http://www.apache.org/licenses/LICENSE-2.0
+ .
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
--- a/kubeapi-load-balancer/copyright.layer-leadership
+++ b/kubeapi-load-balancer/copyright.layer-leadership
@ -0,0 +1,15 @@
+Copyright 2015-2016 Canonical Ltd.
+
+This file is part of the Leadership Layer for Juju.
+
+This program is free software: you can redistribute it and/or modify
+it under the terms of the GNU General Public License version 3, as
+published by the Free Software Foundation.
+
+This program is distributed in the hope that it will be useful, but
+WITHOUT ANY WARRANTY; without even the implied warranties of
+MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
+PURPOSE.  See the GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program.  If not, see <http://www.gnu.org/licenses/>.
--- a/kubeapi-load-balancer/copyright.layer-metrics
+++ b/kubeapi-load-balancer/copyright.layer-metrics
@ -0,0 +1,13 @@
+Copyright 2016 Canonical Ltd
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+    Unless required by applicable law or agreed to in writing, software
+    distributed under the License is distributed on an "AS IS" BASIS,
+    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+    See the License for the specific language governing permissions and
+    limitations under the License.
--- a/kubeapi-load-balancer/copyright.layer-nagios
+++ b/kubeapi-load-balancer/copyright.layer-nagios
@ -0,0 +1,16 @@
+Format: http://dep.debian.net/deps/dep5/
+
+Files: *
+Copyright: Copyright 2016, Canonical Ltd.
+License: GPL-3
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License version 3, as
+ published by the Free Software Foundation.
+ .
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranties of
+ MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
+ PURPOSE.  See the GNU General Public License for more details.
+ .
+ You should have received a copy of the GNU General Public License
+ along with this program.  If not, see <http://www.gnu.org/licenses/>.
--- a/kubeapi-load-balancer/copyright.layer-nginx
+++ b/kubeapi-load-balancer/copyright.layer-nginx
@ -0,0 +1,21 @@
+The MIT License (MIT)
+
+Copyright (c) 2015 Adam Stokes <adam.stokes@ubuntu.com>
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.
--- a/kubeapi-load-balancer/copyright.layer-options
+++ b/kubeapi-load-balancer/copyright.layer-options
@ -0,0 +1,16 @@
+Format: http://dep.debian.net/deps/dep5/
+
+Files: *
+Copyright: Copyright 2015-2017, Canonical Ltd., All Rights Reserved.
+License: Apache License 2.0
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+ .
+     http://www.apache.org/licenses/LICENSE-2.0
+ .
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
--- a/kubeapi-load-balancer/copyright.layer-status
+++ b/kubeapi-load-balancer/copyright.layer-status
@ -0,0 +1,16 @@
+Format: http://dep.debian.net/deps/dep5/
+
+Files: *
+Copyright: Copyright 2018, Canonical Ltd., All Rights Reserved.
+License: Apache License 2.0
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+ .
+     http://www.apache.org/licenses/LICENSE-2.0
+ .
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
--- a/kubeapi-load-balancer/debug-scripts/charm-unitdata
+++ b/kubeapi-load-balancer/debug-scripts/charm-unitdata
@ -0,0 +1,12 @@
+#!/usr/local/sbin/charm-env python3
+
+import debug_script
+import json
+from charmhelpers.core import unitdata
+
+kv = unitdata.kv()
+data = kv.getrange("")
+
+with debug_script.open_file("unitdata.json", "w") as f:
+  json.dump(data, f, indent=2)
+  f.write("\n")
--- a/kubeapi-load-balancer/debug-scripts/filesystem
+++ b/kubeapi-load-balancer/debug-scripts/filesystem
@ -0,0 +1,17 @@
+#!/bin/sh
+set -ux
+
+# report file system disk space usage
+df -hT > $DEBUG_SCRIPT_DIR/df-hT
+# estimate file space usage
+du -h / 2>&1 > $DEBUG_SCRIPT_DIR/du-h
+# list the mounted filesystems
+mount > $DEBUG_SCRIPT_DIR/mount
+# list the mounted systems with ascii trees
+findmnt -A > $DEBUG_SCRIPT_DIR/findmnt
+# list block devices
+lsblk > $DEBUG_SCRIPT_DIR/lsblk
+# list open files
+lsof 2>&1 > $DEBUG_SCRIPT_DIR/lsof
+# list local system locks
+lslocks > $DEBUG_SCRIPT_DIR/lslocks
--- a/kubeapi-load-balancer/debug-scripts/juju-logs
+++ b/kubeapi-load-balancer/debug-scripts/juju-logs
@ -0,0 +1,4 @@
+#!/bin/sh
+set -ux
+
+cp -v /var/log/juju/* $DEBUG_SCRIPT_DIR
--- a/kubeapi-load-balancer/debug-scripts/juju-network-get
+++ b/kubeapi-load-balancer/debug-scripts/juju-network-get
@ -0,0 +1,21 @@
+#!/usr/local/sbin/charm-env python3
+
+import os
+import subprocess
+import yaml
+import debug_script
+
+with open('metadata.yaml') as f:
+    metadata = yaml.load(f)
+
+relations = []
+for key in ['requires', 'provides', 'peers']:
+    relations += list(metadata.get(key, {}).keys())
+
+os.mkdir(os.path.join(debug_script.dir, 'relations'))
+
+for relation in relations:
+    path = 'relations/' + relation
+    with debug_script.open_file(path, 'w') as f:
+        cmd = ['network-get', relation]
+        subprocess.call(cmd, stdout=f, stderr=subprocess.STDOUT)
--- a/kubeapi-load-balancer/debug-scripts/network
+++ b/kubeapi-load-balancer/debug-scripts/network
@ -0,0 +1,11 @@
+#!/bin/sh
+set -ux
+
+ifconfig -a > $DEBUG_SCRIPT_DIR/ifconfig
+cp -v /etc/resolv.conf $DEBUG_SCRIPT_DIR/resolv.conf
+cp -v /etc/network/interfaces $DEBUG_SCRIPT_DIR/interfaces
+netstat -planut > $DEBUG_SCRIPT_DIR/netstat
+route -n > $DEBUG_SCRIPT_DIR/route
+iptables-save > $DEBUG_SCRIPT_DIR/iptables-save
+dig google.com > $DEBUG_SCRIPT_DIR/dig-google
+ping -w 2 -i 0.1 google.com > $DEBUG_SCRIPT_DIR/ping-google
--- a/kubeapi-load-balancer/debug-scripts/packages
+++ b/kubeapi-load-balancer/debug-scripts/packages
@ -0,0 +1,7 @@
+#!/bin/sh
+set -ux
+
+dpkg --list > $DEBUG_SCRIPT_DIR/dpkg-list
+snap list > $DEBUG_SCRIPT_DIR/snap-list
+pip2 list > $DEBUG_SCRIPT_DIR/pip2-list
+pip3 list > $DEBUG_SCRIPT_DIR/pip3-list
--- a/kubeapi-load-balancer/debug-scripts/sysctl
+++ b/kubeapi-load-balancer/debug-scripts/sysctl
@ -0,0 +1,4 @@
+#!/bin/sh
+set -ux
+
+sysctl -a > $DEBUG_SCRIPT_DIR/sysctl
--- a/kubeapi-load-balancer/debug-scripts/systemd
+++ b/kubeapi-load-balancer/debug-scripts/systemd
@ -0,0 +1,9 @@
+#!/bin/sh
+set -ux
+
+systemctl --all > $DEBUG_SCRIPT_DIR/systemctl
+journalctl > $DEBUG_SCRIPT_DIR/journalctl
+systemd-analyze time > $DEBUG_SCRIPT_DIR/systemd-analyze-time
+systemd-analyze blame > $DEBUG_SCRIPT_DIR/systemd-analyze-blame
+systemd-analyze critical-chain > $DEBUG_SCRIPT_DIR/systemd-analyze-critical-chain
+systemd-analyze dump > $DEBUG_SCRIPT_DIR/systemd-analyze-dump
--- a/kubeapi-load-balancer/debug-scripts/tls-certs
+++ b/kubeapi-load-balancer/debug-scripts/tls-certs
@ -0,0 +1,21 @@
+#!/usr/local/sbin/charm-env python3
+
+import os
+import shutil
+import traceback
+import debug_script
+from charms import layer
+
+options = layer.options.get('tls-client')
+
+def copy_cert(source_key, name):
+    try:
+        source = options[source_key]
+        dest = os.path.join(debug_script.dir, name)
+        shutil.copy(source, dest)
+    except Exception:
+        traceback.print_exc()
+
+copy_cert('client_certificate_path', 'client.crt')
+copy_cert('server_certificate_path', 'server.crt')
+copy_cert('ca_certificate_path', 'ca.crt')
--- a/kubeapi-load-balancer/docs/status.md
+++ b/kubeapi-load-balancer/docs/status.md
@ -0,0 +1,91 @@
+<h1 id="charms.layer.status.WorkloadState">WorkloadState</h1>
+
+```python
+WorkloadState(self, /, *args, **kwargs)
+```
+
+Enum of the valid workload states.
+
+Valid options are:
+
+  * `WorkloadState.MAINTENANCE`
+  * `WorkloadState.BLOCKED`
+  * `WorkloadState.WAITING`
+  * `WorkloadState.ACTIVE`
+
+<h1 id="charms.layer.status.maintenance">maintenance</h1>
+
+```python
+maintenance(message)
+```
+
+Set the status to the `MAINTENANCE` state with the given operator message.
+
+__Parameters__
+
+- __`message` (str)__: Message to convey to the operator.
+
+<h1 id="charms.layer.status.maint">maint</h1>
+
+```python
+maint(message)
+```
+
+Shorthand alias for
+[maintenance](status.md#charms.layer.status.maintenance).
+
+__Parameters__
+
+- __`message` (str)__: Message to convey to the operator.
+
+<h1 id="charms.layer.status.blocked">blocked</h1>
+
+```python
+blocked(message)
+```
+
+Set the status to the `BLOCKED` state with the given operator message.
+
+__Parameters__
+
+- __`message` (str)__: Message to convey to the operator.
+
+<h1 id="charms.layer.status.waiting">waiting</h1>
+
+```python
+waiting(message)
+```
+
+Set the status to the `WAITING` state with the given operator message.
+
+__Parameters__
+
+- __`message` (str)__: Message to convey to the operator.
+
+<h1 id="charms.layer.status.active">active</h1>
+
+```python
+active(message)
+```
+
+Set the status to the `ACTIVE` state with the given operator message.
+
+__Parameters__
+
+- __`message` (str)__: Message to convey to the operator.
+
+<h1 id="charms.layer.status.status_set">status_set</h1>
+
+```python
+status_set(workload_state, message)
+```
+
+Set the status to the given workload state with a message.
+
+__Parameters__
+
+- __`workload_state` (WorkloadState or str)__: State of the workload.  Should be
+    a [WorkloadState](status.md#charms.layer.status.WorkloadState) enum
+    member, or the string value of one of those members.
+- __`message` (str)__: Message to convey to the operator.
+
--- a/kubeapi-load-balancer/hooks/apiserver-relation-broken
+++ b/kubeapi-load-balancer/hooks/apiserver-relation-broken
@ -0,0 +1,22 @@
+#!/usr/bin/env python3
+
+# Load modules from $JUJU_CHARM_DIR/lib
+import sys
+sys.path.append('lib')
+
+from charms.layer import basic  # noqa
+basic.bootstrap_charm_deps()
+
+from charmhelpers.core import hookenv  # noqa
+hookenv.atstart(basic.init_config_states)
+hookenv.atexit(basic.clear_config_states)
+
+
+# This will load and run the appropriate @hook and other decorated
+# handlers from $JUJU_CHARM_DIR/reactive, $JUJU_CHARM_DIR/hooks/reactive,
+# and $JUJU_CHARM_DIR/hooks/relations.
+#
+# See https://jujucharms.com/docs/stable/authors-charm-building
+# for more information on this pattern.
+from charms.reactive import main  # noqa
+main()
--- a/kubeapi-load-balancer/hooks/apiserver-relation-changed
+++ b/kubeapi-load-balancer/hooks/apiserver-relation-changed
@ -0,0 +1,22 @@
+#!/usr/bin/env python3
+
+# Load modules from $JUJU_CHARM_DIR/lib
+import sys
+sys.path.append('lib')
+
+from charms.layer import basic  # noqa
+basic.bootstrap_charm_deps()
+
+from charmhelpers.core import hookenv  # noqa
+hookenv.atstart(basic.init_config_states)
+hookenv.atexit(basic.clear_config_states)
+
+
+# This will load and run the appropriate @hook and other decorated
+# handlers from $JUJU_CHARM_DIR/reactive, $JUJU_CHARM_DIR/hooks/reactive,
+# and $JUJU_CHARM_DIR/hooks/relations.
+#
+# See https://jujucharms.com/docs/stable/authors-charm-building
+# for more information on this pattern.
+from charms.reactive import main  # noqa
+main()
--- a/kubeapi-load-balancer/hooks/apiserver-relation-departed
+++ b/kubeapi-load-balancer/hooks/apiserver-relation-departed
@ -0,0 +1,22 @@
+#!/usr/bin/env python3
+
+# Load modules from $JUJU_CHARM_DIR/lib
+import sys
+sys.path.append('lib')
+
+from charms.layer import basic  # noqa
+basic.bootstrap_charm_deps()
+
+from charmhelpers.core import hookenv  # noqa
+hookenv.atstart(basic.init_config_states)
+hookenv.atexit(basic.clear_config_states)
+
+
+# This will load and run the appropriate @hook and other decorated
+# handlers from $JUJU_CHARM_DIR/reactive, $JUJU_CHARM_DIR/hooks/reactive,
+# and $JUJU_CHARM_DIR/hooks/relations.
+#
+# See https://jujucharms.com/docs/stable/authors-charm-building
+# for more information on this pattern.
+from charms.reactive import main  # noqa
+main()
--- a/kubeapi-load-balancer/hooks/apiserver-relation-joined
+++ b/kubeapi-load-balancer/hooks/apiserver-relation-joined
@ -0,0 +1,22 @@
+#!/usr/bin/env python3
+
+# Load modules from $JUJU_CHARM_DIR/lib
+import sys
+sys.path.append('lib')
+
+from charms.layer import basic  # noqa
+basic.bootstrap_charm_deps()
+
+from charmhelpers.core import hookenv  # noqa
+hookenv.atstart(basic.init_config_states)
+hookenv.atexit(basic.clear_config_states)
+
+
+# This will load and run the appropriate @hook and other decorated
+# handlers from $JUJU_CHARM_DIR/reactive, $JUJU_CHARM_DIR/hooks/reactive,
+# and $JUJU_CHARM_DIR/hooks/relations.
+#
+# See https://jujucharms.com/docs/stable/authors-charm-building
+# for more information on this pattern.
+from charms.reactive import main  # noqa
+main()
--- a/kubeapi-load-balancer/hooks/certificates-relation-broken
+++ b/kubeapi-load-balancer/hooks/certificates-relation-broken
@ -0,0 +1,22 @@
+#!/usr/bin/env python3
+
+# Load modules from $JUJU_CHARM_DIR/lib
+import sys
+sys.path.append('lib')
+
+from charms.layer import basic  # noqa
+basic.bootstrap_charm_deps()
+
+from charmhelpers.core import hookenv  # noqa
+hookenv.atstart(basic.init_config_states)
+hookenv.atexit(basic.clear_config_states)
+
+
+# This will load and run the appropriate @hook and other decorated
+# handlers from $JUJU_CHARM_DIR/reactive, $JUJU_CHARM_DIR/hooks/reactive,
+# and $JUJU_CHARM_DIR/hooks/relations.
+#
+# See https://jujucharms.com/docs/stable/authors-charm-building
+# for more information on this pattern.
+from charms.reactive import main  # noqa
+main()
--- a/kubeapi-load-balancer/hooks/certificates-relation-changed
+++ b/kubeapi-load-balancer/hooks/certificates-relation-changed
@ -0,0 +1,22 @@
+#!/usr/bin/env python3
+
+# Load modules from $JUJU_CHARM_DIR/lib
+import sys
+sys.path.append('lib')
+
+from charms.layer import basic  # noqa
+basic.bootstrap_charm_deps()
+
+from charmhelpers.core import hookenv  # noqa
+hookenv.atstart(basic.init_config_states)
+hookenv.atexit(basic.clear_config_states)
+
+
+# This will load and run the appropriate @hook and other decorated
+# handlers from $JUJU_CHARM_DIR/reactive, $JUJU_CHARM_DIR/hooks/reactive,
+# and $JUJU_CHARM_DIR/hooks/relations.
+#
+# See https://jujucharms.com/docs/stable/authors-charm-building
+# for more information on this pattern.
+from charms.reactive import main  # noqa
+main()
--- a/kubeapi-load-balancer/hooks/certificates-relation-departed
+++ b/kubeapi-load-balancer/hooks/certificates-relation-departed
@ -0,0 +1,22 @@
+#!/usr/bin/env python3
+
+# Load modules from $JUJU_CHARM_DIR/lib
+import sys
+sys.path.append('lib')
+
+from charms.layer import basic  # noqa
+basic.bootstrap_charm_deps()
+
+from charmhelpers.core import hookenv  # noqa
+hookenv.atstart(basic.init_config_states)
+hookenv.atexit(basic.clear_config_states)
+
+
+# This will load and run the appropriate @hook and other decorated
+# handlers from $JUJU_CHARM_DIR/reactive, $JUJU_CHARM_DIR/hooks/reactive,
+# and $JUJU_CHARM_DIR/hooks/relations.
+#
+# See https://jujucharms.com/docs/stable/authors-charm-building
+# for more information on this pattern.
+from charms.reactive import main  # noqa
+main()
--- a/kubeapi-load-balancer/hooks/certificates-relation-joined
+++ b/kubeapi-load-balancer/hooks/certificates-relation-joined
@ -0,0 +1,22 @@
+#!/usr/bin/env python3
+
+# Load modules from $JUJU_CHARM_DIR/lib
+import sys
+sys.path.append('lib')
+
+from charms.layer import basic  # noqa
+basic.bootstrap_charm_deps()
+
+from charmhelpers.core import hookenv  # noqa
+hookenv.atstart(basic.init_config_states)
+hookenv.atexit(basic.clear_config_states)
+
+
+# This will load and run the appropriate @hook and other decorated
+# handlers from $JUJU_CHARM_DIR/reactive, $JUJU_CHARM_DIR/hooks/reactive,
+# and $JUJU_CHARM_DIR/hooks/relations.
+#
+# See https://jujucharms.com/docs/stable/authors-charm-building
+# for more information on this pattern.
+from charms.reactive import main  # noqa
+main()
--- a/kubeapi-load-balancer/hooks/collect-metrics
+++ b/kubeapi-load-balancer/hooks/collect-metrics
@ -0,0 +1,46 @@
+#!/usr/bin/env python3
+
+# Load modules from $CHARM_DIR/lib
+import sys
+sys.path.append('lib')
+
+import yaml
+import os
+from subprocess import check_output, check_call, CalledProcessError
+
+
+def build_command(doc):
+    values = {}
+    metrics = doc.get("metrics", {})
+    for metric, mdoc in metrics.items():
+        if not mdoc:
+            continue
+        cmd = mdoc.get("command")
+        if cmd:
+            try:
+                value = check_output(cmd, shell=True, universal_newlines=True)
+            except CalledProcessError as e:
+                check_call(['juju-log', '-lERROR',
+                            'Error collecting metric {}:\n{}'.format(
+                                metric, e.output)])
+                continue
+            value = value.strip()
+            if value:
+                values[metric] = value
+
+    if not values:
+        return None
+    command = ["add-metric"]
+    for metric, value in values.items():
+        command.append("%s=%s" % (metric, value))
+    return command
+
+
+if __name__ == '__main__':
+    charm_dir = os.path.dirname(os.path.abspath(os.path.join(__file__, "..")))
+    metrics_yaml = os.path.join(charm_dir, "metrics.yaml")
+    with open(metrics_yaml) as f:
+        doc = yaml.load(f)
+        command = build_command(doc)
+        if command:
+            check_call(command)
--- a/kubeapi-load-balancer/hooks/config-changed
+++ b/kubeapi-load-balancer/hooks/config-changed
@ -0,0 +1,22 @@
+#!/usr/bin/env python3
+
+# Load modules from $JUJU_CHARM_DIR/lib
+import sys
+sys.path.append('lib')
+
+from charms.layer import basic  # noqa
+basic.bootstrap_charm_deps()
+
+from charmhelpers.core import hookenv  # noqa
+hookenv.atstart(basic.init_config_states)
+hookenv.atexit(basic.clear_config_states)
+
+
+# This will load and run the appropriate @hook and other decorated
+# handlers from $JUJU_CHARM_DIR/reactive, $JUJU_CHARM_DIR/hooks/reactive,
+# and $JUJU_CHARM_DIR/hooks/relations.
+#
+# See https://jujucharms.com/docs/stable/authors-charm-building
+# for more information on this pattern.
+from charms.reactive import main  # noqa
+main()
--- a/kubeapi-load-balancer/hooks/ha-relation-broken
+++ b/kubeapi-load-balancer/hooks/ha-relation-broken
@ -0,0 +1,22 @@
+#!/usr/bin/env python3
+
+# Load modules from $JUJU_CHARM_DIR/lib
+import sys
+sys.path.append('lib')
+
+from charms.layer import basic  # noqa
+basic.bootstrap_charm_deps()
+
+from charmhelpers.core import hookenv  # noqa
+hookenv.atstart(basic.init_config_states)
+hookenv.atexit(basic.clear_config_states)
+
+
+# This will load and run the appropriate @hook and other decorated
+# handlers from $JUJU_CHARM_DIR/reactive, $JUJU_CHARM_DIR/hooks/reactive,
+# and $JUJU_CHARM_DIR/hooks/relations.
+#
+# See https://jujucharms.com/docs/stable/authors-charm-building
+# for more information on this pattern.
+from charms.reactive import main  # noqa
+main()
--- a/kubeapi-load-balancer/hooks/ha-relation-changed
+++ b/kubeapi-load-balancer/hooks/ha-relation-changed
@ -0,0 +1,22 @@
+#!/usr/bin/env python3
+
+# Load modules from $JUJU_CHARM_DIR/lib
+import sys
+sys.path.append('lib')
+
+from charms.layer import basic  # noqa
+basic.bootstrap_charm_deps()
+
+from charmhelpers.core import hookenv  # noqa
+hookenv.atstart(basic.init_config_states)
+hookenv.atexit(basic.clear_config_states)
+
+
+# This will load and run the appropriate @hook and other decorated
+# handlers from $JUJU_CHARM_DIR/reactive, $JUJU_CHARM_DIR/hooks/reactive,
+# and $JUJU_CHARM_DIR/hooks/relations.
+#
+# See https://jujucharms.com/docs/stable/authors-charm-building
+# for more information on this pattern.
+from charms.reactive import main  # noqa
+main()
--- a/kubeapi-load-balancer/hooks/ha-relation-departed
+++ b/kubeapi-load-balancer/hooks/ha-relation-departed
@ -0,0 +1,22 @@
+#!/usr/bin/env python3
+
+# Load modules from $JUJU_CHARM_DIR/lib
+import sys
+sys.path.append('lib')
+
+from charms.layer import basic  # noqa
+basic.bootstrap_charm_deps()
+
+from charmhelpers.core import hookenv  # noqa
+hookenv.atstart(basic.init_config_states)
+hookenv.atexit(basic.clear_config_states)
+
+
+# This will load and run the appropriate @hook and other decorated
+# handlers from $JUJU_CHARM_DIR/reactive, $JUJU_CHARM_DIR/hooks/reactive,
+# and $JUJU_CHARM_DIR/hooks/relations.
+#
+# See https://jujucharms.com/docs/stable/authors-charm-building
+# for more information on this pattern.
+from charms.reactive import main  # noqa
+main()
--- a/kubeapi-load-balancer/hooks/ha-relation-joined
+++ b/kubeapi-load-balancer/hooks/ha-relation-joined
@ -0,0 +1,22 @@
+#!/usr/bin/env python3
+
+# Load modules from $JUJU_CHARM_DIR/lib
+import sys
+sys.path.append('lib')
+
+from charms.layer import basic  # noqa
+basic.bootstrap_charm_deps()
+
+from charmhelpers.core import hookenv  # noqa
+hookenv.atstart(basic.init_config_states)
+hookenv.atexit(basic.clear_config_states)
+
+
+# This will load and run the appropriate @hook and other decorated
+# handlers from $JUJU_CHARM_DIR/reactive, $JUJU_CHARM_DIR/hooks/reactive,
+# and $JUJU_CHARM_DIR/hooks/relations.
+#
+# See https://jujucharms.com/docs/stable/authors-charm-building
+# for more information on this pattern.
+from charms.reactive import main  # noqa
+main()
--- a/kubeapi-load-balancer/hooks/hook.template
+++ b/kubeapi-load-balancer/hooks/hook.template
@ -0,0 +1,22 @@
+#!/usr/bin/env python3
+
+# Load modules from $JUJU_CHARM_DIR/lib
+import sys
+sys.path.append('lib')
+
+from charms.layer import basic  # noqa
+basic.bootstrap_charm_deps()
+
+from charmhelpers.core import hookenv  # noqa
+hookenv.atstart(basic.init_config_states)
+hookenv.atexit(basic.clear_config_states)
+
+
+# This will load and run the appropriate @hook and other decorated
+# handlers from $JUJU_CHARM_DIR/reactive, $JUJU_CHARM_DIR/hooks/reactive,
+# and $JUJU_CHARM_DIR/hooks/relations.
+#
+# See https://jujucharms.com/docs/stable/authors-charm-building
+# for more information on this pattern.
+from charms.reactive import main  # noqa
+main()
--- a/kubeapi-load-balancer/hooks/install
+++ b/kubeapi-load-balancer/hooks/install
@ -0,0 +1,22 @@
+#!/usr/bin/env python3
+
+# Load modules from $JUJU_CHARM_DIR/lib
+import sys
+sys.path.append('lib')
+
+from charms.layer import basic  # noqa
+basic.bootstrap_charm_deps()
+
+from charmhelpers.core import hookenv  # noqa
+hookenv.atstart(basic.init_config_states)
+hookenv.atexit(basic.clear_config_states)
+
+
+# This will load and run the appropriate @hook and other decorated
+# handlers from $JUJU_CHARM_DIR/reactive, $JUJU_CHARM_DIR/hooks/reactive,
+# and $JUJU_CHARM_DIR/hooks/relations.
+#
+# See https://jujucharms.com/docs/stable/authors-charm-building
+# for more information on this pattern.
+from charms.reactive import main  # noqa
+main()
--- a/kubeapi-load-balancer/hooks/lb-consumers-relation-broken
+++ b/kubeapi-load-balancer/hooks/lb-consumers-relation-broken
@ -0,0 +1,22 @@
+#!/usr/bin/env python3
+
+# Load modules from $JUJU_CHARM_DIR/lib
+import sys
+sys.path.append('lib')
+
+from charms.layer import basic  # noqa
+basic.bootstrap_charm_deps()
+
+from charmhelpers.core import hookenv  # noqa
+hookenv.atstart(basic.init_config_states)
+hookenv.atexit(basic.clear_config_states)
+
+
+# This will load and run the appropriate @hook and other decorated
+# handlers from $JUJU_CHARM_DIR/reactive, $JUJU_CHARM_DIR/hooks/reactive,
+# and $JUJU_CHARM_DIR/hooks/relations.
+#
+# See https://jujucharms.com/docs/stable/authors-charm-building
+# for more information on this pattern.
+from charms.reactive import main  # noqa
+main()
--- a/kubeapi-load-balancer/hooks/lb-consumers-relation-changed
+++ b/kubeapi-load-balancer/hooks/lb-consumers-relation-changed
@ -0,0 +1,22 @@
+#!/usr/bin/env python3
+
+# Load modules from $JUJU_CHARM_DIR/lib
+import sys
+sys.path.append('lib')
+
+from charms.layer import basic  # noqa
+basic.bootstrap_charm_deps()
+
+from charmhelpers.core import hookenv  # noqa
+hookenv.atstart(basic.init_config_states)
+hookenv.atexit(basic.clear_config_states)
+
+
+# This will load and run the appropriate @hook and other decorated
+# handlers from $JUJU_CHARM_DIR/reactive, $JUJU_CHARM_DIR/hooks/reactive,
+# and $JUJU_CHARM_DIR/hooks/relations.
+#
+# See https://jujucharms.com/docs/stable/authors-charm-building
+# for more information on this pattern.
+from charms.reactive import main  # noqa
+main()
--- a/kubeapi-load-balancer/hooks/lb-consumers-relation-departed
+++ b/kubeapi-load-balancer/hooks/lb-consumers-relation-departed
@ -0,0 +1,22 @@
+#!/usr/bin/env python3
+
+# Load modules from $JUJU_CHARM_DIR/lib
+import sys
+sys.path.append('lib')
+
+from charms.layer import basic  # noqa
+basic.bootstrap_charm_deps()
+
+from charmhelpers.core import hookenv  # noqa
+hookenv.atstart(basic.init_config_states)
+hookenv.atexit(basic.clear_config_states)
+
+
+# This will load and run the appropriate @hook and other decorated
+# handlers from $JUJU_CHARM_DIR/reactive, $JUJU_CHARM_DIR/hooks/reactive,
+# and $JUJU_CHARM_DIR/hooks/relations.
+#
+# See https://jujucharms.com/docs/stable/authors-charm-building
+# for more information on this pattern.
+from charms.reactive import main  # noqa
+main()
--- a/kubeapi-load-balancer/hooks/lb-consumers-relation-joined
+++ b/kubeapi-load-balancer/hooks/lb-consumers-relation-joined
@ -0,0 +1,22 @@
+#!/usr/bin/env python3
+
+# Load modules from $JUJU_CHARM_DIR/lib
+import sys
+sys.path.append('lib')
+
+from charms.layer import basic  # noqa
+basic.bootstrap_charm_deps()
+
+from charmhelpers.core import hookenv  # noqa
+hookenv.atstart(basic.init_config_states)
+hookenv.atexit(basic.clear_config_states)
+
+
+# This will load and run the appropriate @hook and other decorated
+# handlers from $JUJU_CHARM_DIR/reactive, $JUJU_CHARM_DIR/hooks/reactive,
+# and $JUJU_CHARM_DIR/hooks/relations.
+#
+# See https://jujucharms.com/docs/stable/authors-charm-building
+# for more information on this pattern.
+from charms.reactive import main  # noqa
+main()
--- a/kubeapi-load-balancer/hooks/leader-elected
+++ b/kubeapi-load-balancer/hooks/leader-elected
@ -0,0 +1,22 @@
+#!/usr/bin/env python3
+
+# Load modules from $JUJU_CHARM_DIR/lib
+import sys
+sys.path.append('lib')
+
+from charms.layer import basic  # noqa
+basic.bootstrap_charm_deps()
+
+from charmhelpers.core import hookenv  # noqa
+hookenv.atstart(basic.init_config_states)
+hookenv.atexit(basic.clear_config_states)
+
+
+# This will load and run the appropriate @hook and other decorated
+# handlers from $JUJU_CHARM_DIR/reactive, $JUJU_CHARM_DIR/hooks/reactive,
+# and $JUJU_CHARM_DIR/hooks/relations.
+#
+# See https://jujucharms.com/docs/stable/authors-charm-building
+# for more information on this pattern.
+from charms.reactive import main  # noqa
+main()
--- a/kubeapi-load-balancer/hooks/leader-settings-changed
+++ b/kubeapi-load-balancer/hooks/leader-settings-changed
@ -0,0 +1,22 @@
+#!/usr/bin/env python3
+
+# Load modules from $JUJU_CHARM_DIR/lib
+import sys
+sys.path.append('lib')
+
+from charms.layer import basic  # noqa
+basic.bootstrap_charm_deps()
+
+from charmhelpers.core import hookenv  # noqa
+hookenv.atstart(basic.init_config_states)
+hookenv.atexit(basic.clear_config_states)
+
+
+# This will load and run the appropriate @hook and other decorated
+# handlers from $JUJU_CHARM_DIR/reactive, $JUJU_CHARM_DIR/hooks/reactive,
+# and $JUJU_CHARM_DIR/hooks/relations.
+#
+# See https://jujucharms.com/docs/stable/authors-charm-building
+# for more information on this pattern.
+from charms.reactive import main  # noqa
+main()
--- a/kubeapi-load-balancer/hooks/loadbalancer-relation-broken
+++ b/kubeapi-load-balancer/hooks/loadbalancer-relation-broken
@ -0,0 +1,22 @@
+#!/usr/bin/env python3
+
+# Load modules from $JUJU_CHARM_DIR/lib
+import sys
+sys.path.append('lib')
+
+from charms.layer import basic  # noqa
+basic.bootstrap_charm_deps()
+
+from charmhelpers.core import hookenv  # noqa
+hookenv.atstart(basic.init_config_states)
+hookenv.atexit(basic.clear_config_states)
+
+
+# This will load and run the appropriate @hook and other decorated
+# handlers from $JUJU_CHARM_DIR/reactive, $JUJU_CHARM_DIR/hooks/reactive,
+# and $JUJU_CHARM_DIR/hooks/relations.
+#
+# See https://jujucharms.com/docs/stable/authors-charm-building
+# for more information on this pattern.
+from charms.reactive import main  # noqa
+main()
--- a/kubeapi-load-balancer/hooks/loadbalancer-relation-changed
+++ b/kubeapi-load-balancer/hooks/loadbalancer-relation-changed
@ -0,0 +1,22 @@
+#!/usr/bin/env python3
+
+# Load modules from $JUJU_CHARM_DIR/lib
+import sys
+sys.path.append('lib')
+
+from charms.layer import basic  # noqa
+basic.bootstrap_charm_deps()
+
+from charmhelpers.core import hookenv  # noqa
+hookenv.atstart(basic.init_config_states)
+hookenv.atexit(basic.clear_config_states)
+
+
+# This will load and run the appropriate @hook and other decorated
+# handlers from $JUJU_CHARM_DIR/reactive, $JUJU_CHARM_DIR/hooks/reactive,
+# and $JUJU_CHARM_DIR/hooks/relations.
+#
+# See https://jujucharms.com/docs/stable/authors-charm-building
+# for more information on this pattern.
+from charms.reactive import main  # noqa
+main()
--- a/kubeapi-load-balancer/hooks/loadbalancer-relation-departed
+++ b/kubeapi-load-balancer/hooks/loadbalancer-relation-departed
@ -0,0 +1,22 @@
+#!/usr/bin/env python3
+
+# Load modules from $JUJU_CHARM_DIR/lib
+import sys
+sys.path.append('lib')
+
+from charms.layer import basic  # noqa
+basic.bootstrap_charm_deps()
+
+from charmhelpers.core import hookenv  # noqa
+hookenv.atstart(basic.init_config_states)
+hookenv.atexit(basic.clear_config_states)
+
+
+# This will load and run the appropriate @hook and other decorated
+# handlers from $JUJU_CHARM_DIR/reactive, $JUJU_CHARM_DIR/hooks/reactive,
+# and $JUJU_CHARM_DIR/hooks/relations.
+#
+# See https://jujucharms.com/docs/stable/authors-charm-building
+# for more information on this pattern.
+from charms.reactive import main  # noqa
+main()
--- a/kubeapi-load-balancer/hooks/loadbalancer-relation-joined
+++ b/kubeapi-load-balancer/hooks/loadbalancer-relation-joined
@ -0,0 +1,22 @@
+#!/usr/bin/env python3
+
+# Load modules from $JUJU_CHARM_DIR/lib
+import sys
+sys.path.append('lib')
+
+from charms.layer import basic  # noqa
+basic.bootstrap_charm_deps()
+
+from charmhelpers.core import hookenv  # noqa
+hookenv.atstart(basic.init_config_states)
+hookenv.atexit(basic.clear_config_states)
+
+
+# This will load and run the appropriate @hook and other decorated
+# handlers from $JUJU_CHARM_DIR/reactive, $JUJU_CHARM_DIR/hooks/reactive,
+# and $JUJU_CHARM_DIR/hooks/relations.
+#
+# See https://jujucharms.com/docs/stable/authors-charm-building
+# for more information on this pattern.
+from charms.reactive import main  # noqa
+main()
--- a/kubeapi-load-balancer/hooks/nrpe-external-master-relation-broken
+++ b/kubeapi-load-balancer/hooks/nrpe-external-master-relation-broken
@ -0,0 +1,22 @@
+#!/usr/bin/env python3
+
+# Load modules from $JUJU_CHARM_DIR/lib
+import sys
+sys.path.append('lib')
+
+from charms.layer import basic  # noqa
+basic.bootstrap_charm_deps()
+
+from charmhelpers.core import hookenv  # noqa
+hookenv.atstart(basic.init_config_states)
+hookenv.atexit(basic.clear_config_states)
+
+
+# This will load and run the appropriate @hook and other decorated
+# handlers from $JUJU_CHARM_DIR/reactive, $JUJU_CHARM_DIR/hooks/reactive,
+# and $JUJU_CHARM_DIR/hooks/relations.
+#
+# See https://jujucharms.com/docs/stable/authors-charm-building
+# for more information on this pattern.
+from charms.reactive import main  # noqa
+main()
--- a/kubeapi-load-balancer/hooks/nrpe-external-master-relation-changed
+++ b/kubeapi-load-balancer/hooks/nrpe-external-master-relation-changed
@ -0,0 +1,22 @@
+#!/usr/bin/env python3
+
+# Load modules from $JUJU_CHARM_DIR/lib
+import sys
+sys.path.append('lib')
+
+from charms.layer import basic  # noqa
+basic.bootstrap_charm_deps()
+
+from charmhelpers.core import hookenv  # noqa
+hookenv.atstart(basic.init_config_states)
+hookenv.atexit(basic.clear_config_states)
+
+
+# This will load and run the appropriate @hook and other decorated
+# handlers from $JUJU_CHARM_DIR/reactive, $JUJU_CHARM_DIR/hooks/reactive,
+# and $JUJU_CHARM_DIR/hooks/relations.
+#
+# See https://jujucharms.com/docs/stable/authors-charm-building
+# for more information on this pattern.
+from charms.reactive import main  # noqa
+main()
--- a/kubeapi-load-balancer/hooks/nrpe-external-master-relation-departed
+++ b/kubeapi-load-balancer/hooks/nrpe-external-master-relation-departed
@ -0,0 +1,22 @@
+#!/usr/bin/env python3
+
+# Load modules from $JUJU_CHARM_DIR/lib
+import sys
+sys.path.append('lib')
+
+from charms.layer import basic  # noqa
+basic.bootstrap_charm_deps()
+
+from charmhelpers.core import hookenv  # noqa
+hookenv.atstart(basic.init_config_states)
+hookenv.atexit(basic.clear_config_states)
+
+
+# This will load and run the appropriate @hook and other decorated
+# handlers from $JUJU_CHARM_DIR/reactive, $JUJU_CHARM_DIR/hooks/reactive,
+# and $JUJU_CHARM_DIR/hooks/relations.
+#
+# See https://jujucharms.com/docs/stable/authors-charm-building
+# for more information on this pattern.
+from charms.reactive import main  # noqa
+main()
--- a/kubeapi-load-balancer/hooks/nrpe-external-master-relation-joined
+++ b/kubeapi-load-balancer/hooks/nrpe-external-master-relation-joined
@ -0,0 +1,22 @@
+#!/usr/bin/env python3
+
+# Load modules from $JUJU_CHARM_DIR/lib
+import sys
+sys.path.append('lib')
+
+from charms.layer import basic  # noqa
+basic.bootstrap_charm_deps()
+
+from charmhelpers.core import hookenv  # noqa
+hookenv.atstart(basic.init_config_states)
+hookenv.atexit(basic.clear_config_states)
+
+
+# This will load and run the appropriate @hook and other decorated
+# handlers from $JUJU_CHARM_DIR/reactive, $JUJU_CHARM_DIR/hooks/reactive,
+# and $JUJU_CHARM_DIR/hooks/relations.
+#
+# See https://jujucharms.com/docs/stable/authors-charm-building
+# for more information on this pattern.
+from charms.reactive import main  # noqa
+main()
--- a/kubeapi-load-balancer/hooks/post-series-upgrade
+++ b/kubeapi-load-balancer/hooks/post-series-upgrade
@ -0,0 +1,22 @@
+#!/usr/bin/env python3
+
+# Load modules from $JUJU_CHARM_DIR/lib
+import sys
+sys.path.append('lib')
+
+from charms.layer import basic  # noqa
+basic.bootstrap_charm_deps()
+
+from charmhelpers.core import hookenv  # noqa
+hookenv.atstart(basic.init_config_states)
+hookenv.atexit(basic.clear_config_states)
+
+
+# This will load and run the appropriate @hook and other decorated
+# handlers from $JUJU_CHARM_DIR/reactive, $JUJU_CHARM_DIR/hooks/reactive,
+# and $JUJU_CHARM_DIR/hooks/relations.
+#
+# See https://jujucharms.com/docs/stable/authors-charm-building
+# for more information on this pattern.
+from charms.reactive import main  # noqa
+main()
--- a/kubeapi-load-balancer/hooks/pre-series-upgrade
+++ b/kubeapi-load-balancer/hooks/pre-series-upgrade
@ -0,0 +1,22 @@
+#!/usr/bin/env python3
+
+# Load modules from $JUJU_CHARM_DIR/lib
+import sys
+sys.path.append('lib')
+
+from charms.layer import basic  # noqa
+basic.bootstrap_charm_deps()
+
+from charmhelpers.core import hookenv  # noqa
+hookenv.atstart(basic.init_config_states)
+hookenv.atexit(basic.clear_config_states)
+
+
+# This will load and run the appropriate @hook and other decorated
+# handlers from $JUJU_CHARM_DIR/reactive, $JUJU_CHARM_DIR/hooks/reactive,
+# and $JUJU_CHARM_DIR/hooks/relations.
+#
+# See https://jujucharms.com/docs/stable/authors-charm-building
+# for more information on this pattern.
+from charms.reactive import main  # noqa
+main()
--- a/kubeapi-load-balancer/hooks/relations/hacluster/.stestr.conf
+++ b/kubeapi-load-balancer/hooks/relations/hacluster/.stestr.conf
@ -0,0 +1,3 @@
+[DEFAULT]
+test_path=./unit_tests
+top_dir=./
--- a/kubeapi-load-balancer/hooks/relations/hacluster/README.md
+++ b/kubeapi-load-balancer/hooks/relations/hacluster/README.md
@ -0,0 +1,90 @@
+# Overview
+
+This interface handles the communication with the hacluster subordinate
+charm using the `ha` interface protocol.
+
+# Usage
+
+## Requires
+
+The interface layer will set the following reactive states, as appropriate:
+
+  * `{relation_name}.connected` The relation is established and ready for
+    the local charm to configure the hacluster subordinate charm. The
+    configuration of the resources to manage for the hacluster charm
+    can be managed via one of the following methods:
+
+    * `manage_resources` method
+    * `bind_on` method
+
+    Configuration of the managed resources within the hacluster can be
+    managed by passing `common.CRM` object definitions to the
+    `manage_resources` method.
+
+  * `{relation_name}.available` The hacluster is up and ready.
+
+For example:
+```python
+from charms.reactive import when, when_not
+from charms.reactive import set_state, remove_state
+
+from relations.hacluster.common import CRM
+
+
+@when('ha.connected')
+def cluster_connected(hacluster):
+
+    resources = CRM()
+    resources.primitive('res_vip', 'ocf:IPAddr2',
+                        params='ip=10.0.3.100 nic=eth0',
+                        op='monitor interval="10s"')
+    resources.clone('cl_res_vip', 'res_vip')
+
+    hacluster.bind_on(iface='eth0', mcastport=4430)
+    hacluster.manage_resources(resources)
+```
+
+Additionally, for more code clarity a custom object implements the interface
+defined in common.ResourceDescriptor can be used to simplify the code for
+reuse.
+
+For example:
+```python
+import ipaddress
+
+from relation.hacluster.common import CRM
+from relation.hacluster.common import ResourceDescriptor
+
+class VirtualIP(ResourceDescriptor):
+    def __init__(self, vip, nic='eth0'):
+        self.vip = vip
+        self.nic = 'eth0'
+
+    def configure_resource(self, crm):
+        ipaddr = ipaddress.ip_address(self.vip)
+        if isinstance(ipaddr, ipaddress.IPv4Address):
+            res_type = 'ocf:heartbeat:IPAddr2'
+            res_parms = 'ip={ip} nic={nic}'.format(ip=self.vip,
+                                                   nic=self.nic)
+        else:
+            res_type = 'ocf:heartbeat:IPv6addr'
+            res_params = 'ipv6addr={ip} nic={nic}'.format(ip=self.vip,
+                                                          nic=self.nic)
+
+        crm.primitive('res_vip', res_type, params=res_params,
+                      op='monitor interval="10s"')
+        crm.clone('cl_res_vip', 'res_vip')
+```
+
+Once the VirtualIP class above has been defined in charm code, it can make
+the code a bit cleaner. The example above can thusly be written as:
+
+```python
+@when('ha.connected')
+def cluster_connected(hacluster):
+    resources = CRM()
+    resources.add(VirtualIP('10.0.3.100'))
+
+    hacluster.bind_on(iface='eth0', mcastport=4430)
+    hacluster.manage_resources(resources)
+```
--- a/kubeapi-load-balancer/hooks/relations/hacluster/init.py
+++ b/kubeapi-load-balancer/hooks/relations/hacluster/init.py
--- a/kubeapi-load-balancer/hooks/relations/hacluster/copyright
+++ b/kubeapi-load-balancer/hooks/relations/hacluster/copyright
@ -0,0 +1,21 @@
+Format: http://www.debian.org/doc/packaging-manuals/copyright-format/1.0
+
+Files: *
+Copyright: 2015, Canonical Ltd.
+License: Apache-2.0
+
+License: Apache-2.0
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+ .
+    http://www.apache.org/licenses/LICENSE-2.0
+ .
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+ .
+ On Debian-based systems the full text of the Apache version 2.0 license
+ can be found in `/usr/share/common-licenses/Apache-2.0'.
--- a/kubeapi-load-balancer/hooks/relations/hacluster/interface.yaml
+++ b/kubeapi-load-balancer/hooks/relations/hacluster/interface.yaml
@ -0,0 +1,16 @@
+name: hacluster
+summary: |
+ Provides the hacluster interface used for configuring Corosync
+ and Pacemaker services.
+maintainer: OpenStack Charmers <openstack-charmers@lists.ubuntu.com>
+ignore:
+  - '.gitignore'
+  - '.gitreview'
+  - '.testr.conf'
+  - 'test-requirements'
+  - 'tox.ini'
+  - 'unit_tests'
+  - '.zuul.yaml'
+  - 'setup.cfg'
+  - 'setup.py'
+  - '**/ops_ha_interface.py'
--- a/kubeapi-load-balancer/hooks/relations/hacluster/interface_hacluster/init.py
+++ b/kubeapi-load-balancer/hooks/relations/hacluster/interface_hacluster/init.py
--- a/kubeapi-load-balancer/hooks/relations/hacluster/interface_hacluster/common.py
+++ b/kubeapi-load-balancer/hooks/relations/hacluster/interface_hacluster/common.py
--- a/kubeapi-load-balancer/hooks/relations/hacluster/requires.py
+++ b/kubeapi-load-balancer/hooks/relations/hacluster/requires.py
@ -0,0 +1,58 @@
+#!/usr/bin/python
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import relations.hacluster.interface_hacluster.common as common
+from charms.reactive import hook
+from charms.reactive import RelationBase
+from charms.reactive import scopes
+from charms.reactive.helpers import data_changed as rh_data_changed
+from charmhelpers.core import hookenv
+
+
+class HAClusterRequires(RelationBase, common.ResourceManagement):
+    # The hacluster charm is a subordinate charm and really only works
+    # for a single service to the HA Cluster relation, therefore set the
+    # expected scope to be GLOBAL.
+    scope = scopes.GLOBAL
+
+    @hook('{requires:hacluster}-relation-joined')
+    def joined(self):
+        self.set_state('{relation_name}.connected')
+
+    @hook('{requires:hacluster}-relation-changed')
+    def changed(self):
+        if self.is_clustered():
+            self.set_state('{relation_name}.available')
+        else:
+            self.remove_state('{relation_name}.available')
+
+    @hook('{requires:hacluster}-relation-{broken,departed}')
+    def departed(self):
+        self.remove_state('{relation_name}.available')
+        self.remove_state('{relation_name}.connected')
+
+    def data_changed(self, data_id, data, hash_type='md5'):
+        return rh_data_changed(data_id, data, hash_type)
+
+    def get_remote_all(self, key, default=None):
+        """Return a list of all values presented by remote units for key"""
+        values = []
+        for conversation in self.conversations():
+            for relation_id in conversation.relation_ids:
+                for unit in hookenv.related_units(relation_id):
+                    value = hookenv.relation_get(key,
+                                                 unit,
+                                                 relation_id) or default
+                    if value:
+                        values.append(value)
+        return list(set(values))
--- a/kubeapi-load-balancer/hooks/relations/hacluster/test-requirements.txt
+++ b/kubeapi-load-balancer/hooks/relations/hacluster/test-requirements.txt
@ -0,0 +1,7 @@
+# Lint and unit test requirements
+flake8
+stestr>=2.2.0
+charms.reactive
+coverage>=3.6
+netifaces
+git+https://github.com/canonical/operator.git#egg=ops
--- a/kubeapi-load-balancer/hooks/relations/http/.gitignore
+++ b/kubeapi-load-balancer/hooks/relations/http/.gitignore
@ -0,0 +1,5 @@
+# Emacs save files
+*~
+\#*\#
+.\#*
+
--- a/kubeapi-load-balancer/hooks/relations/http/README.md
+++ b/kubeapi-load-balancer/hooks/relations/http/README.md
@ -0,0 +1,68 @@
+# Overview
+
+This interface layer implements the basic form of the `http` interface protocol,
+which is used for things such as reverse-proxies, load-balanced servers, REST
+service discovery, et cetera.
+
+# Usage
+
+## Provides
+
+By providing the `http` interface, your charm is providing an HTTP server that
+can be load-balanced, reverse-proxied, used as a REST endpoint, etc.
+
+Your charm need only provide the port on which it is serving its content, as
+soon as the `{relation_name}.available` state is set:
+
+```python
+@when('website.available')
+def configure_website(website):
+    website.configure(port=hookenv.config('port'))
+```
+
+## Requires
+
+By requiring the `http` interface, your charm is consuming one or more HTTP
+servers, as a REST endpoint, to load-balance a set of servers, etc.
+
+Your charm should respond to the `{relation_name}.available` state, which
+indicates that there is at least one HTTP server connected.
+
+The `services()` method returns a list of available HTTP services and their
+associated hosts and ports.
+
+The return value is a list of dicts of the following form:
+
+```python
+[
+    {
+        'service_name': name_of_service,
+        'hosts': [
+            {
+                'hostname': address_of_host,
+                'port': port_for_host,
+            },
+            # ...
+        ],
+    },
+    # ...
+]
+```
+
+A trivial example of handling this interface would be:
+
+```python
+from charms.reactive.helpers import data_changed
+
+@when('reverseproxy.available')
+def update_reverse_proxy_config(reverseproxy):
+    services = reverseproxy.services()
+    if not data_changed('reverseproxy.services', services):
+        return
+    for service in services:
+        for host in service['hosts']:
+            hookenv.log('{} has a unit {}:{}'.format(
+                services['service_name'],
+                host['hostname'],
+                host['port']))
+```
--- a/kubeapi-load-balancer/hooks/relations/http/init.py
+++ b/kubeapi-load-balancer/hooks/relations/http/init.py
--- a/kubeapi-load-balancer/hooks/relations/http/interface.yaml
+++ b/kubeapi-load-balancer/hooks/relations/http/interface.yaml
@ -0,0 +1,4 @@
+name: http
+summary: Basic HTTP interface
+version: 1
+repo: https://git.launchpad.net/~bcsaller/charms/+source/http
--- a/kubeapi-load-balancer/hooks/relations/http/provides.py
+++ b/kubeapi-load-balancer/hooks/relations/http/provides.py
@ -0,0 +1,67 @@
+import json
+
+from charmhelpers.core import hookenv
+from charms.reactive import when, when_not
+from charms.reactive import set_flag, clear_flag
+from charms.reactive import Endpoint
+
+
+class HttpProvides(Endpoint):
+
+    @when('endpoint.{endpoint_name}.joined')
+    def joined(self):
+        set_flag(self.expand_name('{endpoint_name}.available'))
+
+    @when_not('endpoint.{endpoint_name}.joined')
+    def broken(self):
+        clear_flag(self.expand_name('{endpoint_name}.available'))
+
+    def get_ingress_address(self, rel_id=None):
+        # If no rel_id is provided, we fallback to the first one
+        if rel_id is None:
+            rel_id = self.relations[0].relation_id
+        return hookenv.ingress_address(rel_id, hookenv.local_unit())
+
+    def configure(self, port, private_address=None, hostname=None):
+        ''' configure the address(es). private_address and hostname can
+        be None, a single string address/hostname, or a list of addresses
+        and hostnames. Note that if a list is passed, it is assumed both
+        private_address and hostname are either lists or None '''
+        for relation in self.relations:
+            ingress_address = self.get_ingress_address(relation.relation_id)
+            if type(private_address) is list or type(hostname) is list:
+                # build 3 lists to zip together that are the same length
+                length = max(len(private_address), len(hostname))
+                p = [port] * length
+                a = private_address + [ingress_address] *\
+                    (length - len(private_address))
+                h = hostname + [ingress_address] * (length - len(hostname))
+                zipped_list = zip(p, a, h)
+                # now build an array of dictionaries from that in the desired
+                # format for the interface
+                data_list = [{'hostname': h, 'port': p, 'private-address': a}
+                             for p, a, h in zipped_list]
+                # for backwards compatibility, we just send a single entry
+                # and have an array of dictionaries in a field of that
+                # entry for the other entries.
+                data = data_list.pop(0)
+                data['extended_data'] = json.dumps(data_list)
+
+                relation.to_publish_raw.update(data)
+            else:
+                relation.to_publish_raw.update({
+                    'hostname': hostname or ingress_address,
+                    'private-address': private_address or ingress_address,
+                    'port': port,
+                })
+
+    def set_remote(self, **kwargs):
+        # NB: This method provides backwards compatibility for charms that
+        # called RelationBase.set_remote. Most commonly, this was done by
+        # charms that needed to pass reverse proxy stanzas to http proxies.
+        # This type of interaction with base relation classes is discouraged,
+        # and should be handled with logic encapsulated in appropriate
+        # interfaces. Eventually, this method will be deprecated in favor of
+        # that behavior.
+        for relation in self.relations:
+            relation.to_publish_raw.update(kwargs)
--- a/kubeapi-load-balancer/hooks/relations/http/requires.py
+++ b/kubeapi-load-balancer/hooks/relations/http/requires.py
@ -0,0 +1,76 @@
+import json
+
+from charms.reactive import when, when_not
+from charms.reactive import set_flag, clear_flag
+from charms.reactive import Endpoint
+
+
+class HttpRequires(Endpoint):
+
+    @when('endpoint.{endpoint_name}.changed')
+    def changed(self):
+        if any(unit.received_raw['port'] for unit in self.all_joined_units):
+            set_flag(self.expand_name('{endpoint_name}.available'))
+
+    @when_not('endpoint.{endpoint_name}.joined')
+    def broken(self):
+        clear_flag(self.expand_name('{endpoint_name}.available'))
+
+    def services(self):
+        """
+        Returns a list of available HTTP services and their associated hosts
+        and ports.
+
+        The return value is a list of dicts of the following form::
+
+            [
+                {
+                    'service_name': name_of_service,
+                    'hosts': [
+                        {
+                            'hostname': address_of_host,
+                            'private-address': private_address_of_host,
+                            'port': port_for_host,
+                        },
+                        # ...
+                    ],
+                },
+                # ...
+            ]
+        """
+        def build_service_host(data):
+            private_address = data['private-address']
+            host = data['hostname'] or private_address
+            if host and data['port']:
+                return (host, private_address, data['port'])
+            else:
+                return None
+
+        services = {}
+        for relation in self.relations:
+            service_name = relation.application_name
+            service = services.setdefault(service_name, {
+                'service_name': service_name,
+                'hosts': [],
+            })
+            host_set = set()
+            for unit in relation.joined_units:
+                data = unit.received_raw
+                host = build_service_host(data)
+                if host:
+                    host_set.add(host)
+
+                # if we have extended data, add it
+                if 'extended_data' in data:
+                    for ed in json.loads(data['extended_data']):
+                        host = build_service_host(ed)
+                        if host:
+                            host_set.add(host)
+
+            service['hosts'] = [
+                {'hostname': h, 'private-address': pa, 'port': p}
+                for h, pa, p in sorted(host_set)
+            ]
+
+        ret = [s for s in services.values() if s['hosts']]
+        return ret
--- a/kubeapi-load-balancer/hooks/relations/nrpe-external-master/README.md
+++ b/kubeapi-load-balancer/hooks/relations/nrpe-external-master/README.md
@ -0,0 +1,66 @@
+# nrpe-external-master interface
+
+Use this interface to register nagios checks in your charm layers.
+
+## Purpose
+
+This interface is designed to interoperate with the
+[nrpe-external-master](https://jujucharms.com/nrpe-external-master) subordinate charm.
+
+## How to use in your layers
+
+The event handler for `nrpe-external-master.available` is called with an object
+through which you can register your own custom nagios checks, when a relation
+is established with `nrpe-external-master:nrpe-external-master`.
+
+This object provides a method,
+
+_add_check_(args, name=_check_name_, description=_description_, context=_context_, unit=_unit_)
+
+which is called to register a nagios plugin check for your service.
+
+All arguments are required.
+
+*args* is a list of nagios plugin command line arguments, starting with the path to the plugin executable.
+
+*name* is the name of the check registered in nagios
+
+*description* is some text that describes what the check is for and what it does
+
+*context* is the nagios context name, something that identifies your application
+
+*unit* is `hookenv.local_unit()`
+
+The nrpe subordinate installs `check_http`, so you can use it like this:
+
+```
+@when('nrpe-external-master.available')
+def setup_nagios(nagios):
+    config = hookenv.config()
+    unit_name = hookenv.local_unit()
+    nagios.add_check(['/usr/lib/nagios/plugins/check_http',
+            '-I', '127.0.0.1', '-p', str(config['port']),
+            '-e', " 200 OK", '-u', '/publickey'],
+        name="check_http",
+        description="Verify my awesome service is responding",
+        context=config["nagios_context"],
+        unit=unit_name,
+    )
+```
+If your `nagios.add_check` defines a custom plugin, you will also need to restart the `nagios-nrpe-server` service. 
+
+Consult the nagios documentation for more information on [how to write your own
+plugins](https://assets.nagios.com/downloads/nagioscore/docs/nagioscore/4/en/pluginapi.html)
+or [find one](https://www.nagios.org/projects/nagios-plugins/) that does what you need.
+
+## Example deployment
+
+```
+$ juju deploy your-awesome-charm
+$ juju deploy nrpe-external-master --config site-nagios.yaml
+$ juju add-relation your-awesome-charm nrpe-external-master
+```
+
+where `site-nagios.yaml` has the necessary configuration settings for the
+subordinate to connect to nagios.
+
--- a/kubeapi-load-balancer/hooks/relations/nrpe-external-master/init.py
+++ b/kubeapi-load-balancer/hooks/relations/nrpe-external-master/init.py
--- a/kubeapi-load-balancer/hooks/relations/nrpe-external-master/interface.yaml
+++ b/kubeapi-load-balancer/hooks/relations/nrpe-external-master/interface.yaml
@ -0,0 +1,3 @@
+name: nrpe-external-master
+summary: Nagios interface
+version: 1
--- a/kubeapi-load-balancer/hooks/relations/nrpe-external-master/provides.py
+++ b/kubeapi-load-balancer/hooks/relations/nrpe-external-master/provides.py
@ -0,0 +1,62 @@
+import datetime
+
+from charms.reactive import hook
+from charms.reactive import RelationBase
+from charms.reactive import scopes
+
+
+class NrpeExternalMasterProvides(RelationBase):
+    scope = scopes.GLOBAL
+
+    @hook('{provides:nrpe-external-master}-relation-{joined,changed}')
+    def changed_nrpe(self):
+        self.set_state('{relation_name}.available')
+
+    @hook('{provides:nrpe-external-master}-relation-{broken,departed}')
+    def broken_nrpe(self):
+        self.remove_state('{relation_name}.available')
+
+    def add_check(self, args, name=None, description=None, context=None,
+                  servicegroups=None, unit=None):
+        unit = unit.replace('/', '-')
+        check_tmpl = """
+#---------------------------------------------------
+# This file is Juju managed
+#---------------------------------------------------
+command[%(check_name)s]=%(check_args)s
+"""
+        service_tmpl = """
+#---------------------------------------------------
+# This file is Juju managed
+#---------------------------------------------------
+define service {
+    use                             active-service
+    host_name                       %(context)s-%(unit_name)s
+    service_description             %(description)s
+    check_command                   check_nrpe!%(check_name)s
+    servicegroups                   %(servicegroups)s
+}
+"""
+        check_filename = "/etc/nagios/nrpe.d/check_%s.cfg" % (name)
+        with open(check_filename, "w") as fh:
+            fh.write(check_tmpl % {
+                'check_args': ' '.join(args),
+                'check_name': name,
+            })
+        service_filename = "/var/lib/nagios/export/service__%s_%s.cfg" % (
+                           unit, name)
+        with open(service_filename, "w") as fh:
+            fh.write(service_tmpl % {
+                'servicegroups': servicegroups or context,
+                'context': context,
+                'description': description,
+                'check_name': name,
+                'unit_name': unit,
+            })
+
+    def updated(self):
+        relation_info = {
+            'timestamp': datetime.datetime.now().isoformat(),
+        }
+        self.set_remote(**relation_info)
+        self.remove_state('{relation_name}.available')
--- a/kubeapi-load-balancer/hooks/relations/public-address/README.md
+++ b/kubeapi-load-balancer/hooks/relations/public-address/README.md
@ -0,0 +1,59 @@
+# Overview
+
+This interface layer implements a public address protocol useful for load 
+balancers and their subordinates. The load balancers (providers) set their 
+own public address and port, which is then available to the subordinates 
+(requirers).
+
+# Usage
+
+## Provides
+
+By providing the `public-address` interface, your charm is providing an HTTP 
+server that can load-balance for another HTTP based service.
+
+Your charm need only provide the address and port on which it is serving its 
+content, as soon as the `{relation_name}.available` state is set:
+
+```python
+from charmhelpers.core import hookenv
+@when('website.available')
+def configure_website(website):
+    website.set_address_port(hookenv.unit_get('public-address'), hookenv.config('port'))
+```
+
+## Requires
+
+By requiring the `public-address` interface, your charm is consuming one or 
+more HTTP servers, to load-balance a set of servers, etc.
+
+Your charm should respond to the `{relation_name}.available` state, which
+indicates that there is at least one HTTP server connected.
+
+The `get_addresses_ports()` method returns a list of available addresses and
+ports.
+
+The return value is a list of dicts of the following form:
+
+```python
+[
+    {
+        'public-address': address_of_host,
+        'port': port_for_host,
+    },
+    # ...
+]
+```
+
+A trivial example of handling this interface would be:
+
+```python
+from charmhelpers.core import hookenv
+@when('loadbalancer.available')
+def update_reverse_proxy_config(loadbalancer):
+    hosts = loadbalancer.get_addresses_ports()
+    for host in hosts:
+        hookenv.log('The loadbalancer for this unit is {}:{}'.format(
+                host['public-address'],
+                host['port']))
+```
--- a/kubeapi-load-balancer/hooks/relations/public-address/init.py
+++ b/kubeapi-load-balancer/hooks/relations/public-address/init.py
--- a/kubeapi-load-balancer/hooks/relations/public-address/interface.yaml
+++ b/kubeapi-load-balancer/hooks/relations/public-address/interface.yaml
@ -0,0 +1,4 @@
+name: public-address
+summary: A basic interface to provide the public address for load balancers.
+version: 1
+repo: https://githb.com/juju-solutions/interface-public-address.git
--- a/kubeapi-load-balancer/hooks/relations/public-address/provides.py
+++ b/kubeapi-load-balancer/hooks/relations/public-address/provides.py
@ -0,0 +1,60 @@
+import json
+
+from charms.reactive import toggle_flag
+from charms.reactive import Endpoint
+
+
+class PublicAdddressProvides(Endpoint):
+
+    def manage_flags(self):
+        toggle_flag(self.expand_name('{endpoint_name}.available'),
+                    self.is_joined)
+
+    def set_address_port(self, address, port, relation=None):
+        if relation is None:
+            # no relation specified, so send the same data to everyone
+            relations = self.relations
+        else:
+            # specific relation given, so only send the data to that one
+            relations = [relation]
+        if type(address) is list:
+            # build 2 lists to zip together that are the same length
+            length = len(address)
+            p = [port] * length
+            combined = zip(address, p)
+            clients = [{'public-address': a, 'port': p}
+                       for a, p in combined]
+            # for backwards compatibility, we just send a single entry
+            # and have an array of dictionaries in a field of that
+            # entry for the other entries.
+            first = clients.pop(0)
+            first['extended_data'] = json.dumps(clients)
+            for relation in relations:
+                relation.to_publish_raw.update(first)
+        else:
+            for relation in relations:
+                relation.to_publish_raw.update({'public-address': address,
+                                                'port': port})
+
+    @property
+    def requests(self):
+        return [Request(rel) for rel in self.relations]
+
+
+class Request:
+    def __init__(self, rel):
+        self.rel = rel
+
+    @property
+    def application_name(self):
+        return self.rel.application_name
+
+    @property
+    def members(self):
+        return [(u.received_raw.get('ingress-address',
+                                    u.received_raw['private-address']),
+                 u.received_raw.get('port', '6443'))
+                for u in self.rel.joined_units]
+
+    def set_address_port(self, address, port):
+        self.rel.endpoint.set_address_port(address, port, self.rel)
--- a/kubeapi-load-balancer/hooks/relations/public-address/requires.py
+++ b/kubeapi-load-balancer/hooks/relations/public-address/requires.py
@ -0,0 +1,44 @@
+import json
+
+from charms.reactive import toggle_flag, Endpoint
+
+
+class PublicAddressRequires(Endpoint):
+    def manage_flags(self):
+        toggle_flag(self.expand_name('{endpoint_name}.available'),
+                    len(self.get_addresses_ports()) > 0)
+
+    def set_backend_port(self, port):
+        """
+        Set the port that the backend service is listening on.
+
+        Defaults to 6443 if not set.
+        """
+        for rel in self.relations:
+            rel.to_publish_raw['port'] = str(port)
+
+    def get_addresses_ports(self):
+        '''Returns a list of available HTTP providers and their associated
+        public addresses and ports.
+
+        The return value is a list of dicts of the following form::
+            [
+                {
+                    'public-address': address_for_frontend,
+                    'port': port_for_frontend,
+                },
+                # ...
+            ]
+        '''
+        hosts = set()
+        for relation in self.relations:
+            for unit in relation.joined_units:
+                data = unit.received_raw
+                hosts.add((data['public-address'], data['port']))
+                if 'extended_data' in data:
+                    for ed in json.loads(data['extended_data']):
+                        hosts.add((ed['public-address'], ed['port']))
+
+        return [{'public-address': pa, 'port': p}
+                for pa, p in sorted(host for host in hosts
+                                    if None not in host)]
--- a/kubeapi-load-balancer/hooks/relations/tls-certificates/.gitignore
+++ b/kubeapi-load-balancer/hooks/relations/tls-certificates/.gitignore
@ -0,0 +1,4 @@
+.tox
+__pycache__
+*.pyc
+_build
--- a/kubeapi-load-balancer/hooks/relations/tls-certificates/README.md
+++ b/kubeapi-load-balancer/hooks/relations/tls-certificates/README.md
@ -0,0 +1,90 @@
+# Interface tls-certificates
+
+This is a [Juju][] interface layer that enables a charm which requires TLS
+certificates to relate to a charm which can provide them, such as [Vault][] or
+[EasyRSA][]
+
+To get started please read the [Introduction to PKI][] which defines some PKI
+terms, concepts and processes used in this document.
+
+# Example Usage
+
+Let's say you have a charm which needs a server certificate for a service it
+provides to other charms and a client certificate for a database it consumes
+from another charm.  The charm provides its own service on the `clients`
+relation endpoint, and it consumes the database on the `db` relation endpoint.
+
+First, you must define the relation endpoint in your charm's `metadata.yaml`:
+
+```yaml
+requires:
+  cert-provider:
+    interface: tls-certificates
+```
+
+Next, you must ensure the interface layer is included in your `layer.yaml`:
+
+```yaml
+includes:
+  - interface:tls-certificates
+```
+
+Then, in your reactive code, add the following, changing `update_certs` to
+handle the certificates however your charm needs:
+
+```python
+from charmhelpers.core import hookenv, host
+from charms.reactive import endpoint_from_flag
+
+
+@when('cert-provider.ca.changed')
+def install_root_ca_cert():
+    cert_provider = endpoint_from_flag('cert-provider.ca.available')
+    host.install_ca_cert(cert_provider.root_ca_cert)
+    clear_flag('cert-provider.ca.changed')
+
+
+@when('cert-provider.available')
+def request_certificates():
+    cert_provider = endpoint_from_flag('cert-provider.available')
+
+    # get ingress info
+    ingress_for_clients = hookenv.network_get('clients')['ingress-addresses']
+    ingress_for_db = hookenv.network_get('db')['ingress-addresses']
+
+    # use first ingress address as primary and any additional as SANs
+    server_cn, server_sans = ingress_for_clients[0], ingress_for_clients[:1]
+    client_cn, client_sans = ingress_for_db[0], ingress_for_db[:1]
+
+    # request a single server and single client cert; note that multiple certs
+    # of either type can be requested as long as they have unique common names
+    cert_provider.request_server_cert(server_cn, server_sans)
+    cert_provider.request_client_cert(client_cn, client_sans)
+
+
+@when('cert-provider.certs.changed')
+def update_certs():
+    cert_provider = endpoint_from_flag('cert-provider.available')
+    server_cert = cert_provider.server_certs[0]  # only requested one
+    myserver.update_server_cert(server_cert.cert, server_cert.key)
+
+    client_cert = cert_provider.client_certs[0]  # only requested one
+    myclient.update_client_cert(client_cert.cert, client_cert.key)
+    clear_flag('cert-provider.certs.changed')
+```
+
+
+# Reference
+
+  * [Requires](docs/requires.md)
+  * [Provides](docs/provides.md)
+
+# Contact Information
+
+Maintainer: Cory Johns &lt;Cory.Johns@canonical.com&gt;
+
+
+[Juju]: https://jujucharms.com
+[Vault]: https://jujucharms.com/u/openstack-charmers/vault
+[EasyRSA]: https://jujucharms.com/u/containers/easyrsa
+[Introduction to PKI]: https://github.com/OpenVPN/easy-rsa/blob/master/doc/Intro-To-PKI.md
--- a/kubeapi-load-balancer/hooks/relations/tls-certificates/init.py
+++ b/kubeapi-load-balancer/hooks/relations/tls-certificates/init.py
--- a/kubeapi-load-balancer/hooks/relations/tls-certificates/docs/common.md
+++ b/kubeapi-load-balancer/hooks/relations/tls-certificates/docs/common.md
@ -0,0 +1,51 @@
+<h1 id="tls_certificates_common.CertificateRequest">CertificateRequest</h1>
+
+```python
+CertificateRequest(self, unit, cert_type, cert_name, common_name, sans)
+```
+
+<h2 id="tls_certificates_common.CertificateRequest.application_name">application_name</h2>
+
+Name of the application which the request came from.
+
+:returns: Name of application
+:rtype: str
+
+<h2 id="tls_certificates_common.CertificateRequest.cert">cert</h2>
+
+
+The cert published for this request, if any.
+
+<h2 id="tls_certificates_common.CertificateRequest.cert_type">cert_type</h2>
+
+
+Type of certificate, 'server' or 'client', being requested.
+
+<h2 id="tls_certificates_common.CertificateRequest.resolve_unit_name">resolve_unit_name</h2>
+
+```python
+CertificateRequest.resolve_unit_name(unit)
+```
+Return name of unit associated with this request.
+
+unit_name should be provided in the relation data to ensure
+compatability with cross-model relations. If the unit name
+is absent then fall back to unit_name attribute of the
+unit associated with this request.
+
+:param unit: Unit to extract name from
+:type unit: charms.reactive.endpoints.RelatedUnit
+:returns: Name of unit
+:rtype: str
+
+<h1 id="tls_certificates_common.Certificate">Certificate</h1>
+
+```python
+Certificate(self, cert_type, common_name, cert, key)
+```
+
+Represents a created certificate and key.
+
+The ``cert_type``, ``common_name``, ``cert``, and ``key`` values can
+be accessed either as properties or as the contents of the dict.
+
--- a/kubeapi-load-balancer/hooks/relations/tls-certificates/docs/provides.md
+++ b/kubeapi-load-balancer/hooks/relations/tls-certificates/docs/provides.md
@ -0,0 +1,212 @@
+<h1 id="provides">provides</h1>
+
+
+<h1 id="provides.TlsProvides">TlsProvides</h1>
+
+```python
+TlsProvides(self, endpoint_name, relation_ids=None)
+```
+
+The provider's side of the interface protocol.
+
+The following flags may be set:
+
+  * `{endpoint_name}.available`
+    Whenever any clients are joined.
+
+  * `{endpoint_name}.certs.requested`
+    When there are new certificate requests of any kind to be processed.
+    The requests can be accessed via [new_requests][].
+
+  * `{endpoint_name}.server.certs.requested`
+    When there are new server certificate requests to be processed.
+    The requests can be accessed via [new_server_requests][].
+
+  * `{endpoint_name}.client.certs.requested`
+    When there are new client certificate requests to be processed.
+    The requests can be accessed via [new_client_requests][].
+
+[Certificate]: common.md#tls_certificates_common.Certificate
+[CertificateRequest]: common.md#tls_certificates_common.CertificateRequest
+[all_requests]: provides.md#provides.TlsProvides.all_requests
+[new_requests]: provides.md#provides.TlsProvides.new_requests
+[new_server_requests]: provides.md#provides.TlsProvides.new_server_requests
+[new_client_requests]: provides.md#provides.TlsProvides.new_client_requests
+
+<h2 id="provides.TlsProvides.all_published_certs">all_published_certs</h2>
+
+
+List of all [Certificate][] instances that this provider has published
+for all related applications.
+
+<h2 id="provides.TlsProvides.all_requests">all_requests</h2>
+
+
+List of all requests that have been made.
+
+Each will be an instance of [CertificateRequest][].
+
+Example usage:
+
+```python
+@when('certs.regen',
+      'tls.certs.available')
+def regen_all_certs():
+    tls = endpoint_from_flag('tls.certs.available')
+    for request in tls.all_requests:
+        cert, key = generate_cert(request.cert_type,
+                                  request.common_name,
+                                  request.sans)
+        request.set_cert(cert, key)
+```
+
+<h2 id="provides.TlsProvides.new_application_requests">new_application_requests</h2>
+
+
+Filtered view of [new_requests][] that only includes application cert
+requests.
+
+Each will be an instance of [ApplicationCertificateRequest][].
+
+Example usage:
+
+```python
+@when('tls.application.certs.requested')
+def gen_application_certs():
+    tls = endpoint_from_flag('tls.application.certs.requested')
+    for request in tls.new_application_requests:
+        cert, key = generate_application_cert(request.common_name,
+                                              request.sans)
+        request.set_cert(cert, key)
+```
+
+<h2 id="provides.TlsProvides.new_client_requests">new_client_requests</h2>
+
+
+Filtered view of [new_requests][] that only includes client cert
+requests.
+
+Each will be an instance of [CertificateRequest][].
+
+Example usage:
+
+```python
+@when('tls.client.certs.requested')
+def gen_client_certs():
+    tls = endpoint_from_flag('tls.client.certs.requested')
+    for request in tls.new_client_requests:
+        cert, key = generate_client_cert(request.common_name,
+                                         request.sans)
+        request.set_cert(cert, key)
+```
+
+<h2 id="provides.TlsProvides.new_requests">new_requests</h2>
+
+
+Filtered view of [all_requests][] that only includes requests that
+haven't been handled.
+
+Each will be an instance of [CertificateRequest][].
+
+This collection can also be further filtered by request type using
+[new_server_requests][] or [new_client_requests][].
+
+Example usage:
+
+```python
+@when('tls.certs.requested')
+def gen_certs():
+    tls = endpoint_from_flag('tls.certs.requested')
+    for request in tls.new_requests:
+        cert, key = generate_cert(request.cert_type,
+                                  request.common_name,
+                                  request.sans)
+        request.set_cert(cert, key)
+```
+
+<h2 id="provides.TlsProvides.new_server_requests">new_server_requests</h2>
+
+
+Filtered view of [new_requests][] that only includes server cert
+requests.
+
+Each will be an instance of [CertificateRequest][].
+
+Example usage:
+
+```python
+@when('tls.server.certs.requested')
+def gen_server_certs():
+    tls = endpoint_from_flag('tls.server.certs.requested')
+    for request in tls.new_server_requests:
+        cert, key = generate_server_cert(request.common_name,
+                                         request.sans)
+        request.set_cert(cert, key)
+```
+
+<h2 id="provides.TlsProvides.set_ca">set_ca</h2>
+
+```python
+TlsProvides.set_ca(certificate_authority)
+```
+
+Publish the CA to all related applications.
+
+<h2 id="provides.TlsProvides.set_chain">set_chain</h2>
+
+```python
+TlsProvides.set_chain(chain)
+```
+
+Publish the chain of trust to all related applications.
+
+<h2 id="provides.TlsProvides.set_client_cert">set_client_cert</h2>
+
+```python
+TlsProvides.set_client_cert(cert, key)
+```
+
+Deprecated.  This is only for backwards compatibility.
+
+Publish a globally shared client cert and key.
+
+<h2 id="provides.TlsProvides.set_server_cert">set_server_cert</h2>
+
+```python
+TlsProvides.set_server_cert(scope, cert, key)
+```
+
+Deprecated.  Use one of the [new_requests][] collections and
+`request.set_cert()` instead.
+
+Set the server cert and key for the request identified by `scope`.
+
+<h2 id="provides.TlsProvides.set_server_multicerts">set_server_multicerts</h2>
+
+```python
+TlsProvides.set_server_multicerts(scope)
+```
+
+Deprecated.  Done automatically.
+
+<h2 id="provides.TlsProvides.add_server_cert">add_server_cert</h2>
+
+```python
+TlsProvides.add_server_cert(scope, cn, cert, key)
+```
+
+Deprecated.  Use `request.set_cert()` instead.
+
+<h2 id="provides.TlsProvides.get_server_requests">get_server_requests</h2>
+
+```python
+TlsProvides.get_server_requests()
+```
+
+Deprecated.  Use the [new_requests][] or [server_requests][]
+collections instead.
+
+One provider can have many requests to generate server certificates.
+Return a map of all server request objects indexed by a unique
+identifier.
+
--- a/Show More
+++ b/Show More
Author	SHA1	Message	Date
Ares	17b26ae910	update bugfix/1.24-GA	2024-03-13 23:33:49 +08:00
Ares	e2678ea2f3	update bugfix/1.24-GA	2024-03-13 15:02:21 +00:00
Ares	54a60deff2	update bugfix/1.24-GA	2024-03-13 11:54:15 +00:00
Ares	504cd003e6	Revert "update bugfix/1.24-GA" This reverts commit `8d1fdc331d`.	2024-03-13 11:52:02 +00:00
Ares	8d1fdc331d	update bugfix/1.24-GA	2024-03-13 11:51:19 +00:00
Ares	6d5242594f	update bugfix/1.24-GA	2024-03-13 10:05:53 +08:00
Ares	6bf11cb615	update release/1.24-GA	2024-03-12 16:31:12 +08:00