update
This commit is contained in:
havelight-ee
5
kubespray/contrib/terraform/openstack/.gitignore
vendored
Normal file
5
kubespray/contrib/terraform/openstack/.gitignore
vendored
Normal file
@@ -0,0 +1,5 @@
|
||||
.terraform
|
||||
*.tfvars
|
||||
!sample-inventory\/cluster.tfvars
|
||||
*.tfstate
|
||||
*.tfstate.backup
|
||||
763
kubespray/contrib/terraform/openstack/README.md
Normal file
763
kubespray/contrib/terraform/openstack/README.md
Normal file
@@ -0,0 +1,763 @@
|
||||
# Kubernetes on OpenStack with Terraform
|
||||
|
||||
Provision a Kubernetes cluster with [Terraform](https://www.terraform.io) on
|
||||
OpenStack.
|
||||
|
||||
## Status
|
||||
|
||||
This will install a Kubernetes cluster on an OpenStack Cloud. It should work on
|
||||
most modern installs of OpenStack that support the basic services.
|
||||
|
||||
### Known compatible public clouds
|
||||
|
||||
- [Auro](https://auro.io/)
|
||||
- [Betacloud](https://www.betacloud.io/)
|
||||
- [CityCloud](https://www.citycloud.com/)
|
||||
- [DreamHost](https://www.dreamhost.com/cloud/computing/)
|
||||
- [ELASTX](https://elastx.se/)
|
||||
- [EnterCloudSuite](https://www.entercloudsuite.com/)
|
||||
- [FugaCloud](https://fuga.cloud/)
|
||||
- [Open Telekom Cloud](https://cloud.telekom.de/)
|
||||
- [OVH](https://www.ovh.com/)
|
||||
- [Rackspace](https://www.rackspace.com/)
|
||||
- [Safespring](https://www.safespring.com)
|
||||
- [Ultimum](https://ultimum.io/)
|
||||
- [VexxHost](https://vexxhost.com/)
|
||||
- [Zetta](https://www.zetta.io/)
|
||||
|
||||
## Approach
|
||||
|
||||
The terraform configuration inspects variables found in
|
||||
[variables.tf](variables.tf) to create resources in your OpenStack cluster.
|
||||
There is a [python script](../terraform.py) that reads the generated`.tfstate`
|
||||
file to generate a dynamic inventory that is consumed by the main ansible script
|
||||
to actually install kubernetes and stand up the cluster.
|
||||
|
||||
### Networking
|
||||
|
||||
The configuration includes creating a private subnet with a router to the
|
||||
external net. It will allocate floating IPs from a pool and assign them to the
|
||||
hosts where that makes sense. You have the option of creating bastion hosts
|
||||
inside the private subnet to access the nodes there. Alternatively, a node with
|
||||
a floating IP can be used as a jump host to nodes without.
|
||||
|
||||
#### Using an existing router
|
||||
|
||||
It is possible to use an existing router instead of creating one. To use an
|
||||
existing router set the router\_id variable to the uuid of the router you wish
|
||||
to use.
|
||||
|
||||
For example:
|
||||
|
||||
```ShellSession
|
||||
router_id = "00c542e7-6f46-4535-ae95-984c7f0391a3"
|
||||
```
|
||||
|
||||
### Kubernetes Nodes
|
||||
|
||||
You can create many different kubernetes topologies by setting the number of
|
||||
different classes of hosts. For each class there are options for allocating
|
||||
floating IP addresses or not.
|
||||
|
||||
- Master nodes with etcd
|
||||
- Master nodes without etcd
|
||||
- Standalone etcd hosts
|
||||
- Kubernetes worker nodes
|
||||
|
||||
Note that the Ansible script will report an invalid configuration if you wind up
|
||||
with an even number of etcd instances since that is not a valid configuration. This
|
||||
restriction includes standalone etcd nodes that are deployed in a cluster along with
|
||||
master nodes with etcd replicas. As an example, if you have three master nodes with
|
||||
etcd replicas and three standalone etcd nodes, the script will fail since there are
|
||||
now six total etcd replicas.
|
||||
|
||||
### GlusterFS shared file system
|
||||
|
||||
The Terraform configuration supports provisioning of an optional GlusterFS
|
||||
shared file system based on a separate set of VMs. To enable this, you need to
|
||||
specify:
|
||||
|
||||
- the number of Gluster hosts (minimum 2)
|
||||
- Size of the non-ephemeral volumes to be attached to store the GlusterFS bricks
|
||||
- Other properties related to provisioning the hosts
|
||||
|
||||
Even if you are using Flatcar Container Linux by Kinvolk for your cluster, you will still
|
||||
need the GlusterFS VMs to be based on either Debian or RedHat based images.
|
||||
Flatcar Container Linux by Kinvolk cannot serve GlusterFS, but can connect to it through
|
||||
binaries available on hyperkube v1.4.3_coreos.0 or higher.
|
||||
|
||||
## Requirements
|
||||
|
||||
- [Install Terraform](https://www.terraform.io/intro/getting-started/install.html) 0.12 or later
|
||||
- [Install Ansible](http://docs.ansible.com/ansible/latest/intro_installation.html)
|
||||
- you already have a suitable OS image in Glance
|
||||
- you already have a floating IP pool created
|
||||
- you have security groups enabled
|
||||
- you have a pair of keys generated that can be used to secure the new hosts
|
||||
|
||||
## Module Architecture
|
||||
|
||||
The configuration is divided into three modules:
|
||||
|
||||
- Network
|
||||
- IPs
|
||||
- Compute
|
||||
|
||||
The main reason for splitting the configuration up in this way is to easily
|
||||
accommodate situations where floating IPs are limited by a quota or if you have
|
||||
any external references to the floating IP (e.g. DNS) that would otherwise have
|
||||
to be updated.
|
||||
|
||||
You can force your existing IPs by modifying the compute variables in
|
||||
`kubespray.tf` as follows:
|
||||
|
||||
```ini
|
||||
k8s_master_fips = ["151.101.129.67"]
|
||||
k8s_node_fips = ["151.101.129.68"]
|
||||
```
|
||||
|
||||
## Terraform
|
||||
|
||||
Terraform will be used to provision all of the OpenStack resources with base software as appropriate.
|
||||
|
||||
### Configuration
|
||||
|
||||
#### Inventory files
|
||||
|
||||
Create an inventory directory for your cluster by copying the existing sample and linking the `hosts` script (used to build the inventory based on Terraform state):
|
||||
|
||||
```ShellSession
|
||||
cp -LRp contrib/terraform/openstack/sample-inventory inventory/$CLUSTER
|
||||
cd inventory/$CLUSTER
|
||||
ln -s ../../contrib/terraform/openstack/hosts
|
||||
ln -s ../../contrib
|
||||
```
|
||||
|
||||
This will be the base for subsequent Terraform commands.
|
||||
|
||||
#### OpenStack access and credentials
|
||||
|
||||
No provider variables are hardcoded inside `variables.tf` because Terraform
|
||||
supports various authentication methods for OpenStack: the older script and
|
||||
environment method (using `openrc`) as well as a newer declarative method, and
|
||||
different OpenStack environments may support Identity API version 2 or 3.
|
||||
|
||||
These are examples and may vary depending on your OpenStack cloud provider,
|
||||
for an exhaustive list on how to authenticate on OpenStack with Terraform
|
||||
please read the [OpenStack provider documentation](https://www.terraform.io/docs/providers/openstack/).
|
||||
|
||||
##### Declarative method (recommended)
|
||||
|
||||
The recommended authentication method is to describe credentials in a YAML file `clouds.yaml` that can be stored in:
|
||||
|
||||
- the current directory
|
||||
- `~/.config/openstack`
|
||||
- `/etc/openstack`
|
||||
|
||||
`clouds.yaml`:
|
||||
|
||||
```yaml
|
||||
clouds:
|
||||
mycloud:
|
||||
auth:
|
||||
auth_url: https://openstack:5000/v3
|
||||
username: "username"
|
||||
project_name: "projectname"
|
||||
project_id: projectid
|
||||
user_domain_name: "Default"
|
||||
password: "password"
|
||||
region_name: "RegionOne"
|
||||
interface: "public"
|
||||
identity_api_version: 3
|
||||
```
|
||||
|
||||
If you have multiple clouds defined in your `clouds.yaml` file you can choose
|
||||
the one you want to use with the environment variable `OS_CLOUD`:
|
||||
|
||||
```ShellSession
|
||||
export OS_CLOUD=mycloud
|
||||
```
|
||||
|
||||
##### Openrc method
|
||||
|
||||
When using classic environment variables, Terraform uses default `OS_*`
|
||||
environment variables. A script suitable for your environment may be available
|
||||
from Horizon under *Project* -> *Compute* -> *Access & Security* -> *API Access*.
|
||||
|
||||
With identity v2:
|
||||
|
||||
```ShellSession
|
||||
source openrc
|
||||
|
||||
env | grep OS
|
||||
|
||||
OS_AUTH_URL=https://openstack:5000/v2.0
|
||||
OS_PROJECT_ID=projectid
|
||||
OS_PROJECT_NAME=projectname
|
||||
OS_USERNAME=username
|
||||
OS_PASSWORD=password
|
||||
OS_REGION_NAME=RegionOne
|
||||
OS_INTERFACE=public
|
||||
OS_IDENTITY_API_VERSION=2
|
||||
```
|
||||
|
||||
With identity v3:
|
||||
|
||||
```ShellSession
|
||||
source openrc
|
||||
|
||||
env | grep OS
|
||||
|
||||
OS_AUTH_URL=https://openstack:5000/v3
|
||||
OS_PROJECT_ID=projectid
|
||||
OS_PROJECT_NAME=username
|
||||
OS_PROJECT_DOMAIN_ID=default
|
||||
OS_USERNAME=username
|
||||
OS_PASSWORD=password
|
||||
OS_REGION_NAME=RegionOne
|
||||
OS_INTERFACE=public
|
||||
OS_IDENTITY_API_VERSION=3
|
||||
OS_USER_DOMAIN_NAME=Default
|
||||
```
|
||||
|
||||
Terraform does not support a mix of DomainName and DomainID, choose one or the other:
|
||||
|
||||
- provider.openstack: You must provide exactly one of DomainID or DomainName to authenticate by Username
|
||||
|
||||
```ShellSession
|
||||
unset OS_USER_DOMAIN_NAME
|
||||
export OS_USER_DOMAIN_ID=default
|
||||
```
|
||||
|
||||
or
|
||||
|
||||
```ShellSession
|
||||
unset OS_PROJECT_DOMAIN_ID
|
||||
set OS_PROJECT_DOMAIN_NAME=Default
|
||||
```
|
||||
|
||||
#### Cluster variables
|
||||
|
||||
The construction of the cluster is driven by values found in
|
||||
[variables.tf](variables.tf).
|
||||
|
||||
For your cluster, edit `inventory/$CLUSTER/cluster.tfvars`.
|
||||
|
||||
|Variable | Description |
|
||||
|---------|-------------|
|
||||
|`cluster_name` | All OpenStack resources will use the Terraform variable`cluster_name` (default`example`) in their name to make it easier to track. For example the first compute resource will be named`example-kubernetes-1`. |
|
||||
|`az_list` | List of Availability Zones available in your OpenStack cluster. |
|
||||
|`network_name` | The name to be given to the internal network that will be generated |
|
||||
|`use_existing_network`| Use an existing network with the name of `network_name`. `false` by default |
|
||||
|`network_dns_domain` | (Optional) The dns_domain for the internal network that will be generated |
|
||||
|`dns_nameservers`| An array of DNS name server names to be used by hosts in the internal subnet. |
|
||||
|`floatingip_pool` | Name of the pool from which floating IPs will be allocated |
|
||||
|`k8s_master_fips` | A list of floating IPs that you have already pre-allocated; they will be attached to master nodes instead of creating new random floating IPs. |
|
||||
|`bastion_fips` | A list of floating IPs that you have already pre-allocated; they will be attached to bastion node instead of creating new random floating IPs. |
|
||||
|`external_net` | UUID of the external network that will be routed to |
|
||||
|`flavor_k8s_master`,`flavor_k8s_node`,`flavor_etcd`, `flavor_bastion`,`flavor_gfs_node` | Flavor depends on your openstack installation, you can get available flavor IDs through `openstack flavor list` |
|
||||
|`image`,`image_gfs` | Name of the image to use in provisioning the compute resources. Should already be loaded into glance. |
|
||||
|`ssh_user`,`ssh_user_gfs` | The username to ssh into the image with. This usually depends on the image you have selected |
|
||||
|`public_key_path` | Path on your local workstation to the public key file you wish to use in creating the key pairs |
|
||||
|`number_of_k8s_masters`, `number_of_k8s_masters_no_floating_ip` | Number of nodes that serve as both master and etcd. These can be provisioned with or without floating IP addresses|
|
||||
|`number_of_k8s_masters_no_etcd`, `number_of_k8s_masters_no_floating_ip_no_etcd` | Number of nodes that serve as just master with no etcd. These can be provisioned with or without floating IP addresses |
|
||||
|`number_of_etcd` | Number of pure etcd nodes |
|
||||
|`number_of_k8s_nodes`, `number_of_k8s_nodes_no_floating_ip` | Kubernetes worker nodes. These can be provisioned with or without floating ip addresses. |
|
||||
|`number_of_bastions` | Number of bastion hosts to create. Scripts assume this is really just zero or one |
|
||||
|`number_of_gfs_nodes_no_floating_ip` | Number of gluster servers to provision. |
|
||||
| `gfs_volume_size_in_gb` | Size of the non-ephemeral volumes to be attached to store the GlusterFS bricks |
|
||||
|`supplementary_master_groups` | To add ansible groups to the masters, such as `kube_node` for tainting them as nodes, empty by default. |
|
||||
|`supplementary_node_groups` | To add ansible groups to the nodes, such as `kube_ingress` for running ingress controller pods, empty by default. |
|
||||
|`bastion_allowed_remote_ips` | List of CIDR allowed to initiate a SSH connection, `["0.0.0.0/0"]` by default |
|
||||
|`master_allowed_remote_ips` | List of CIDR blocks allowed to initiate an API connection, `["0.0.0.0/0"]` by default |
|
||||
|`bastion_allowed_ports` | List of ports to open on bastion node, `[]` by default |
|
||||
|`k8s_allowed_remote_ips` | List of CIDR allowed to initiate a SSH connection, empty by default |
|
||||
|`worker_allowed_ports` | List of ports to open on worker nodes, `[{ "protocol" = "tcp", "port_range_min" = 30000, "port_range_max" = 32767, "remote_ip_prefix" = "0.0.0.0/0"}]` by default |
|
||||
|`master_allowed_ports` | List of ports to open on master nodes, expected format is `[{ "protocol" = "tcp", "port_range_min" = 443, "port_range_max" = 443, "remote_ip_prefix" = "0.0.0.0/0"}]`, empty by default |
|
||||
|`node_root_volume_size_in_gb` | Size of the root volume for nodes, 0 to use ephemeral storage |
|
||||
|`master_root_volume_size_in_gb` | Size of the root volume for masters, 0 to use ephemeral storage |
|
||||
|`master_volume_type` | Volume type of the root volume for control_plane, 'Default' by default |
|
||||
|`node_volume_type` | Volume type of the root volume for nodes, 'Default' by default |
|
||||
|`gfs_root_volume_size_in_gb` | Size of the root volume for gluster, 0 to use ephemeral storage |
|
||||
|`etcd_root_volume_size_in_gb` | Size of the root volume for etcd nodes, 0 to use ephemeral storage |
|
||||
|`bastion_root_volume_size_in_gb` | Size of the root volume for bastions, 0 to use ephemeral storage |
|
||||
|`master_server_group_policy` | Enable and use openstack nova servergroups for masters with set policy, default: "" (disabled) |
|
||||
|`node_server_group_policy` | Enable and use openstack nova servergroups for nodes with set policy, default: "" (disabled) |
|
||||
|`etcd_server_group_policy` | Enable and use openstack nova servergroups for etcd with set policy, default: "" (disabled) |
|
||||
|`use_access_ip` | If 1, nodes with floating IPs will transmit internal cluster traffic via floating IPs; if 0 private IPs will be used instead. Default value is 1. |
|
||||
|`port_security_enabled` | Allow to disable port security by setting this to `false`. `true` by default |
|
||||
|`force_null_port_security` | Set `null` instead of `true` or `false` for `port_security`. `false` by default |
|
||||
|`k8s_nodes` | Map containing worker node definition, see explanation below |
|
||||
|`k8s_masters` | Map containing master node definition, see explanation for k8s_nodes and `sample-inventory/cluster.tfvars` |
|
||||
|
||||
##### k8s_nodes
|
||||
|
||||
Allows a custom definition of worker nodes giving the operator full control over individual node flavor and
|
||||
availability zone placement. To enable the use of this mode set the `number_of_k8s_nodes` and
|
||||
`number_of_k8s_nodes_no_floating_ip` variables to 0. Then define your desired worker node configuration
|
||||
using the `k8s_nodes` variable. The `az`, `flavor` and `floating_ip` parameters are mandatory.
|
||||
The optional parameter `extra_groups` (a comma-delimited string) can be used to define extra inventory group memberships for specific nodes.
|
||||
|
||||
For example:
|
||||
|
||||
```ini
|
||||
k8s_nodes = {
|
||||
"1" = {
|
||||
"az" = "sto1"
|
||||
"flavor" = "83d8b44a-26a0-4f02-a981-079446926445"
|
||||
"floating_ip" = true
|
||||
},
|
||||
"2" = {
|
||||
"az" = "sto2"
|
||||
"flavor" = "83d8b44a-26a0-4f02-a981-079446926445"
|
||||
"floating_ip" = true
|
||||
},
|
||||
"3" = {
|
||||
"az" = "sto3"
|
||||
"flavor" = "83d8b44a-26a0-4f02-a981-079446926445"
|
||||
"floating_ip" = true
|
||||
"extra_groups" = "calico_rr"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Would result in the same configuration as:
|
||||
|
||||
```ini
|
||||
number_of_k8s_nodes = 3
|
||||
flavor_k8s_node = "83d8b44a-26a0-4f02-a981-079446926445"
|
||||
az_list = ["sto1", "sto2", "sto3"]
|
||||
```
|
||||
|
||||
And:
|
||||
|
||||
```ini
|
||||
k8s_nodes = {
|
||||
"ing-1" = {
|
||||
"az" = "sto1"
|
||||
"flavor" = "83d8b44a-26a0-4f02-a981-079446926445"
|
||||
"floating_ip" = true
|
||||
},
|
||||
"ing-2" = {
|
||||
"az" = "sto2"
|
||||
"flavor" = "83d8b44a-26a0-4f02-a981-079446926445"
|
||||
"floating_ip" = true
|
||||
},
|
||||
"ing-3" = {
|
||||
"az" = "sto3"
|
||||
"flavor" = "83d8b44a-26a0-4f02-a981-079446926445"
|
||||
"floating_ip" = true
|
||||
},
|
||||
"big-1" = {
|
||||
"az" = "sto1"
|
||||
"flavor" = "3f73fc93-ec61-4808-88df-2580d94c1a9b"
|
||||
"floating_ip" = false
|
||||
},
|
||||
"big-2" = {
|
||||
"az" = "sto2"
|
||||
"flavor" = "3f73fc93-ec61-4808-88df-2580d94c1a9b"
|
||||
"floating_ip" = false
|
||||
},
|
||||
"big-3" = {
|
||||
"az" = "sto3"
|
||||
"flavor" = "3f73fc93-ec61-4808-88df-2580d94c1a9b"
|
||||
"floating_ip" = false
|
||||
},
|
||||
"small-1" = {
|
||||
"az" = "sto1"
|
||||
"flavor" = "7a6a998f-ac7f-4fb8-a534-2175b254f75e"
|
||||
"floating_ip" = false
|
||||
},
|
||||
"small-2" = {
|
||||
"az" = "sto2"
|
||||
"flavor" = "7a6a998f-ac7f-4fb8-a534-2175b254f75e"
|
||||
"floating_ip" = false
|
||||
},
|
||||
"small-3" = {
|
||||
"az" = "sto3"
|
||||
"flavor" = "7a6a998f-ac7f-4fb8-a534-2175b254f75e"
|
||||
"floating_ip" = false
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Would result in three nodes in each availability zone each with their own separate naming,
|
||||
flavor and floating ip configuration.
|
||||
|
||||
The "schema":
|
||||
|
||||
```ini
|
||||
k8s_nodes = {
|
||||
"key | node name suffix, must be unique" = {
|
||||
"az" = string
|
||||
"flavor" = string
|
||||
"floating_ip" = bool
|
||||
},
|
||||
}
|
||||
```
|
||||
|
||||
All values are required.
|
||||
|
||||
#### Terraform state files
|
||||
|
||||
In the cluster's inventory folder, the following files might be created (either by Terraform
|
||||
or manually), to prevent you from pushing them accidentally they are in a
|
||||
`.gitignore` file in the `terraform/openstack` directory :
|
||||
|
||||
- `.terraform`
|
||||
- `.tfvars`
|
||||
- `.tfstate`
|
||||
- `.tfstate.backup`
|
||||
|
||||
You can still add them manually if you want to.
|
||||
|
||||
### Initialization
|
||||
|
||||
Before Terraform can operate on your cluster you need to install the required
|
||||
plugins. This is accomplished as follows:
|
||||
|
||||
```ShellSession
|
||||
cd inventory/$CLUSTER
|
||||
terraform -chdir="../../contrib/terraform/openstack" init
|
||||
```
|
||||
|
||||
This should finish fairly quickly telling you Terraform has successfully initialized and loaded necessary modules.
|
||||
|
||||
### Customizing with cloud-init
|
||||
|
||||
You can apply cloud-init based customization for the openstack instances before provisioning your cluster.
|
||||
One common template is used for all instances. Adjust the file shown below:
|
||||
`contrib/terraform/openstack/modules/compute/templates/cloudinit.yaml`
|
||||
For example, to enable openstack novnc access and ansible_user=root SSH access:
|
||||
|
||||
```ShellSession
|
||||
#cloud-config
|
||||
## in some cases novnc console access is required
|
||||
## it requires ssh password to be set
|
||||
ssh_pwauth: yes
|
||||
chpasswd:
|
||||
list: |
|
||||
root:secret
|
||||
expire: False
|
||||
|
||||
## in some cases direct root ssh access via ssh key is required
|
||||
disable_root: false
|
||||
```
|
||||
|
||||
### Provisioning cluster
|
||||
|
||||
You can apply the Terraform configuration to your cluster with the following command
|
||||
issued from your cluster's inventory directory (`inventory/$CLUSTER`):
|
||||
|
||||
```ShellSession
|
||||
terraform -chdir="../../contrib/terraform/openstack" apply -var-file=cluster.tfvars
|
||||
```
|
||||
|
||||
if you chose to create a bastion host, this script will create
|
||||
`contrib/terraform/openstack/k8s_cluster.yml` with an ssh command for Ansible to
|
||||
be able to access your machines tunneling through the bastion's IP address. If
|
||||
you want to manually handle the ssh tunneling to these machines, please delete
|
||||
or move that file. If you want to use this, just leave it there, as ansible will
|
||||
pick it up automatically.
|
||||
|
||||
### Destroying cluster
|
||||
|
||||
You can destroy your new cluster with the following command issued from the cluster's inventory directory:
|
||||
|
||||
```ShellSession
|
||||
terraform -chdir="../../contrib/terraform/openstack" destroy -var-file=cluster.tfvars
|
||||
```
|
||||
|
||||
If you've started the Ansible run, it may also be a good idea to do some manual cleanup:
|
||||
|
||||
- remove SSH keys from the destroyed cluster from your `~/.ssh/known_hosts` file
|
||||
- clean up any temporary cache files: `rm /tmp/$CLUSTER-*`
|
||||
|
||||
### Debugging
|
||||
|
||||
You can enable debugging output from Terraform by setting
|
||||
`OS_DEBUG` to 1 and`TF_LOG` to`DEBUG` before running the Terraform command.
|
||||
|
||||
### Terraform output
|
||||
|
||||
Terraform can output values that are useful for configure Neutron/Octavia LBaaS or Cinder persistent volume provisioning as part of your Kubernetes deployment:
|
||||
|
||||
- `private_subnet_id`: the subnet where your instances are running is used for `openstack_lbaas_subnet_id`
|
||||
- `floating_network_id`: the network_id where the floating IP are provisioned is used for `openstack_lbaas_floating_network_id`
|
||||
|
||||
## Ansible
|
||||
|
||||
### Node access
|
||||
|
||||
#### SSH
|
||||
|
||||
Ensure your local ssh-agent is running and your ssh key has been added. This
|
||||
step is required by the terraform provisioner:
|
||||
|
||||
```ShellSession
|
||||
eval $(ssh-agent -s)
|
||||
ssh-add ~/.ssh/id_rsa
|
||||
```
|
||||
|
||||
If you have deployed and destroyed a previous iteration of your cluster, you will need to clear out any stale keys from your SSH "known hosts" file ( `~/.ssh/known_hosts`).
|
||||
|
||||
#### Metadata variables
|
||||
|
||||
The [python script](../terraform.py) that reads the
|
||||
generated`.tfstate` file to generate a dynamic inventory recognizes
|
||||
some variables within a "metadata" block, defined in a "resource"
|
||||
block (example):
|
||||
|
||||
```ini
|
||||
resource "openstack_compute_instance_v2" "example" {
|
||||
...
|
||||
metadata {
|
||||
ssh_user = "ubuntu"
|
||||
prefer_ipv6 = true
|
||||
python_bin = "/usr/bin/python3"
|
||||
}
|
||||
...
|
||||
}
|
||||
```
|
||||
|
||||
As the example shows, these let you define the SSH username for
|
||||
Ansible, a Python binary which is needed by Ansible if
|
||||
`/usr/bin/python` doesn't exist, and whether the IPv6 address of the
|
||||
instance should be preferred over IPv4.
|
||||
|
||||
#### Bastion host
|
||||
|
||||
Bastion access will be determined by:
|
||||
|
||||
- Your choice on the amount of bastion hosts (set by `number_of_bastions` terraform variable).
|
||||
- The existence of nodes/masters with floating IPs (set by `number_of_k8s_masters`, `number_of_k8s_nodes`, `number_of_k8s_masters_no_etcd` terraform variables).
|
||||
|
||||
If you have a bastion host, your ssh traffic will be directly routed through it. This is regardless of whether you have masters/nodes with a floating IP assigned.
|
||||
If you don't have a bastion host, but at least one of your masters/nodes have a floating IP, then ssh traffic will be tunneled by one of these machines.
|
||||
|
||||
So, either a bastion host, or at least master/node with a floating IP are required.
|
||||
|
||||
#### Test access
|
||||
|
||||
Make sure you can connect to the hosts. Note that Flatcar Container Linux by Kinvolk will have a state `FAILED` due to Python not being present. This is okay, because Python will be installed during bootstrapping, so long as the hosts are not `UNREACHABLE`.
|
||||
|
||||
```ShellSession
|
||||
$ ansible -i inventory/$CLUSTER/hosts -m ping all
|
||||
example-k8s_node-1 | SUCCESS => {
|
||||
"changed": false,
|
||||
"ping": "pong"
|
||||
}
|
||||
example-etcd-1 | SUCCESS => {
|
||||
"changed": false,
|
||||
"ping": "pong"
|
||||
}
|
||||
example-k8s-master-1 | SUCCESS => {
|
||||
"changed": false,
|
||||
"ping": "pong"
|
||||
}
|
||||
```
|
||||
|
||||
If it fails try to connect manually via SSH. It could be something as simple as a stale host key.
|
||||
|
||||
### Configure cluster variables
|
||||
|
||||
Edit `inventory/$CLUSTER/group_vars/all/all.yml`:
|
||||
|
||||
- **bin_dir**:
|
||||
|
||||
```yml
|
||||
# Directory where the binaries will be installed
|
||||
# Default:
|
||||
# bin_dir: /usr/local/bin
|
||||
# For Flatcar Container Linux by Kinvolk:
|
||||
bin_dir: /opt/bin
|
||||
```
|
||||
|
||||
- and **cloud_provider**:
|
||||
|
||||
```yml
|
||||
cloud_provider: openstack
|
||||
```
|
||||
|
||||
Edit `inventory/$CLUSTER/group_vars/k8s_cluster/k8s_cluster.yml`:
|
||||
|
||||
- Set variable **kube_network_plugin** to your desired networking plugin.
|
||||
- **flannel** works out-of-the-box
|
||||
- **calico** requires [configuring OpenStack Neutron ports](/docs/openstack.md) to allow service and pod subnets
|
||||
|
||||
```yml
|
||||
# Choose network plugin (calico, weave or flannel)
|
||||
# Can also be set to 'cloud', which lets the cloud provider setup appropriate routing
|
||||
kube_network_plugin: flannel
|
||||
```
|
||||
|
||||
- Set variable **resolvconf_mode**
|
||||
|
||||
```yml
|
||||
# Can be docker_dns, host_resolvconf or none
|
||||
# Default:
|
||||
# resolvconf_mode: docker_dns
|
||||
# For Flatcar Container Linux by Kinvolk:
|
||||
resolvconf_mode: host_resolvconf
|
||||
```
|
||||
|
||||
- Set max amount of attached cinder volume per host (default 256)
|
||||
|
||||
```yml
|
||||
node_volume_attach_limit: 26
|
||||
```
|
||||
|
||||
### Deploy Kubernetes
|
||||
|
||||
```ShellSession
|
||||
ansible-playbook --become -i inventory/$CLUSTER/hosts cluster.yml
|
||||
```
|
||||
|
||||
This will take some time as there are many tasks to run.
|
||||
|
||||
## Kubernetes
|
||||
|
||||
### Set up kubectl
|
||||
|
||||
1. [Install kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/) on your workstation
|
||||
2. Add a route to the internal IP of a master node (if needed):
|
||||
|
||||
```ShellSession
|
||||
sudo route add [master-internal-ip] gw [router-ip]
|
||||
```
|
||||
|
||||
or
|
||||
|
||||
```ShellSession
|
||||
sudo route add -net [internal-subnet]/24 gw [router-ip]
|
||||
```
|
||||
|
||||
1. List Kubernetes certificates & keys:
|
||||
|
||||
```ShellSession
|
||||
ssh [os-user]@[master-ip] sudo ls /etc/kubernetes/ssl/
|
||||
```
|
||||
|
||||
1. Get `admin`'s certificates and keys:
|
||||
|
||||
```ShellSession
|
||||
ssh [os-user]@[master-ip] sudo cat /etc/kubernetes/ssl/admin-kube-master-1-key.pem > admin-key.pem
|
||||
ssh [os-user]@[master-ip] sudo cat /etc/kubernetes/ssl/admin-kube-master-1.pem > admin.pem
|
||||
ssh [os-user]@[master-ip] sudo cat /etc/kubernetes/ssl/ca.pem > ca.pem
|
||||
```
|
||||
|
||||
1. Configure kubectl:
|
||||
|
||||
```ShellSession
|
||||
$ kubectl config set-cluster default-cluster --server=https://[master-internal-ip]:6443 \
|
||||
--certificate-authority=ca.pem
|
||||
|
||||
$ kubectl config set-credentials default-admin \
|
||||
--certificate-authority=ca.pem \
|
||||
--client-key=admin-key.pem \
|
||||
--client-certificate=admin.pem
|
||||
|
||||
$ kubectl config set-context default-system --cluster=default-cluster --user=default-admin
|
||||
$ kubectl config use-context default-system
|
||||
```
|
||||
|
||||
1. Check it:
|
||||
|
||||
```ShellSession
|
||||
kubectl version
|
||||
```
|
||||
|
||||
## GlusterFS
|
||||
|
||||
GlusterFS is not deployed by the standard `cluster.yml` playbook, see the
|
||||
[GlusterFS playbook documentation](../../network-storage/glusterfs/README.md)
|
||||
for instructions.
|
||||
|
||||
Basically you will install Gluster as
|
||||
|
||||
```ShellSession
|
||||
ansible-playbook --become -i inventory/$CLUSTER/hosts ./contrib/network-storage/glusterfs/glusterfs.yml
|
||||
```
|
||||
|
||||
## What's next
|
||||
|
||||
Try out your new Kubernetes cluster with the [Hello Kubernetes service](https://kubernetes.io/docs/tasks/access-application-cluster/service-access-application-cluster/).
|
||||
|
||||
## Appendix
|
||||
|
||||
### Migration from `number_of_k8s_nodes*` to `k8s_nodes`
|
||||
|
||||
If you currently have a cluster defined using the `number_of_k8s_nodes*` variables and wish
|
||||
to migrate to the `k8s_nodes` style you can do it like so:
|
||||
|
||||
```ShellSession
|
||||
$ terraform state list
|
||||
module.compute.data.openstack_images_image_v2.gfs_image
|
||||
module.compute.data.openstack_images_image_v2.vm_image
|
||||
module.compute.openstack_compute_floatingip_associate_v2.k8s_master[0]
|
||||
module.compute.openstack_compute_floatingip_associate_v2.k8s_node[0]
|
||||
module.compute.openstack_compute_floatingip_associate_v2.k8s_node[1]
|
||||
module.compute.openstack_compute_floatingip_associate_v2.k8s_node[2]
|
||||
module.compute.openstack_compute_instance_v2.k8s_master[0]
|
||||
module.compute.openstack_compute_instance_v2.k8s_node[0]
|
||||
module.compute.openstack_compute_instance_v2.k8s_node[1]
|
||||
module.compute.openstack_compute_instance_v2.k8s_node[2]
|
||||
module.compute.openstack_compute_keypair_v2.k8s
|
||||
module.compute.openstack_compute_servergroup_v2.k8s_etcd[0]
|
||||
module.compute.openstack_compute_servergroup_v2.k8s_master[0]
|
||||
module.compute.openstack_compute_servergroup_v2.k8s_node[0]
|
||||
module.compute.openstack_networking_secgroup_rule_v2.bastion[0]
|
||||
module.compute.openstack_networking_secgroup_rule_v2.egress[0]
|
||||
module.compute.openstack_networking_secgroup_rule_v2.k8s
|
||||
module.compute.openstack_networking_secgroup_rule_v2.k8s_allowed_remote_ips[0]
|
||||
module.compute.openstack_networking_secgroup_rule_v2.k8s_allowed_remote_ips[1]
|
||||
module.compute.openstack_networking_secgroup_rule_v2.k8s_allowed_remote_ips[2]
|
||||
module.compute.openstack_networking_secgroup_rule_v2.k8s_master[0]
|
||||
module.compute.openstack_networking_secgroup_rule_v2.worker[0]
|
||||
module.compute.openstack_networking_secgroup_rule_v2.worker[1]
|
||||
module.compute.openstack_networking_secgroup_rule_v2.worker[2]
|
||||
module.compute.openstack_networking_secgroup_rule_v2.worker[3]
|
||||
module.compute.openstack_networking_secgroup_rule_v2.worker[4]
|
||||
module.compute.openstack_networking_secgroup_v2.bastion[0]
|
||||
module.compute.openstack_networking_secgroup_v2.k8s
|
||||
module.compute.openstack_networking_secgroup_v2.k8s_master
|
||||
module.compute.openstack_networking_secgroup_v2.worker
|
||||
module.ips.null_resource.dummy_dependency
|
||||
module.ips.openstack_networking_floatingip_v2.k8s_master[0]
|
||||
module.ips.openstack_networking_floatingip_v2.k8s_node[0]
|
||||
module.ips.openstack_networking_floatingip_v2.k8s_node[1]
|
||||
module.ips.openstack_networking_floatingip_v2.k8s_node[2]
|
||||
module.network.openstack_networking_network_v2.k8s[0]
|
||||
module.network.openstack_networking_router_interface_v2.k8s[0]
|
||||
module.network.openstack_networking_router_v2.k8s[0]
|
||||
module.network.openstack_networking_subnet_v2.k8s[0]
|
||||
$ terraform state mv 'module.compute.openstack_compute_floatingip_associate_v2.k8s_node[0]' 'module.compute.openstack_compute_floatingip_associate_v2.k8s_nodes["1"]'
|
||||
Move "module.compute.openstack_compute_floatingip_associate_v2.k8s_node[0]" to "module.compute.openstack_compute_floatingip_associate_v2.k8s_nodes[\"1\"]"
|
||||
Successfully moved 1 object(s).
|
||||
$ terraform state mv 'module.compute.openstack_compute_floatingip_associate_v2.k8s_node[1]' 'module.compute.openstack_compute_floatingip_associate_v2.k8s_nodes["2"]'
|
||||
Move "module.compute.openstack_compute_floatingip_associate_v2.k8s_node[1]" to "module.compute.openstack_compute_floatingip_associate_v2.k8s_nodes[\"2\"]"
|
||||
Successfully moved 1 object(s).
|
||||
$ terraform state mv 'module.compute.openstack_compute_floatingip_associate_v2.k8s_node[2]' 'module.compute.openstack_compute_floatingip_associate_v2.k8s_nodes["3"]'
|
||||
Move "module.compute.openstack_compute_floatingip_associate_v2.k8s_node[2]" to "module.compute.openstack_compute_floatingip_associate_v2.k8s_nodes[\"3\"]"
|
||||
Successfully moved 1 object(s).
|
||||
$ terraform state mv 'module.compute.openstack_compute_instance_v2.k8s_node[0]' 'module.compute.openstack_compute_instance_v2.k8s_node["1"]'
|
||||
Move "module.compute.openstack_compute_instance_v2.k8s_node[0]" to "module.compute.openstack_compute_instance_v2.k8s_node[\"1\"]"
|
||||
Successfully moved 1 object(s).
|
||||
$ terraform state mv 'module.compute.openstack_compute_instance_v2.k8s_node[1]' 'module.compute.openstack_compute_instance_v2.k8s_node["2"]'
|
||||
Move "module.compute.openstack_compute_instance_v2.k8s_node[1]" to "module.compute.openstack_compute_instance_v2.k8s_node[\"2\"]"
|
||||
Successfully moved 1 object(s).
|
||||
$ terraform state mv 'module.compute.openstack_compute_instance_v2.k8s_node[2]' 'module.compute.openstack_compute_instance_v2.k8s_node["3"]'
|
||||
Move "module.compute.openstack_compute_instance_v2.k8s_node[2]" to "module.compute.openstack_compute_instance_v2.k8s_node[\"3\"]"
|
||||
Successfully moved 1 object(s).
|
||||
$ terraform state mv 'module.ips.openstack_networking_floatingip_v2.k8s_node[0]' 'module.ips.openstack_networking_floatingip_v2.k8s_node["1"]'
|
||||
Move "module.ips.openstack_networking_floatingip_v2.k8s_node[0]" to "module.ips.openstack_networking_floatingip_v2.k8s_node[\"1\"]"
|
||||
Successfully moved 1 object(s).
|
||||
$ terraform state mv 'module.ips.openstack_networking_floatingip_v2.k8s_node[1]' 'module.ips.openstack_networking_floatingip_v2.k8s_node["2"]'
|
||||
Move "module.ips.openstack_networking_floatingip_v2.k8s_node[1]" to "module.ips.openstack_networking_floatingip_v2.k8s_node[\"2\"]"
|
||||
Successfully moved 1 object(s).
|
||||
$ terraform state mv 'module.ips.openstack_networking_floatingip_v2.k8s_node[2]' 'module.ips.openstack_networking_floatingip_v2.k8s_node["3"]'
|
||||
Move "module.ips.openstack_networking_floatingip_v2.k8s_node[2]" to "module.ips.openstack_networking_floatingip_v2.k8s_node[\"3\"]"
|
||||
Successfully moved 1 object(s).
|
||||
```
|
||||
|
||||
Of course for nodes without floating ips those steps can be omitted.
|
||||
458
kubespray/contrib/terraform/openstack/hosts
Executable file
458
kubespray/contrib/terraform/openstack/hosts
Executable file
@@ -0,0 +1,458 @@
|
||||
#!/usr/bin/env python3
|
||||
#
|
||||
# Copyright 2015 Cisco Systems, Inc.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
#
|
||||
# original: https://github.com/CiscoCloud/terraform.py
|
||||
|
||||
"""\
|
||||
Dynamic inventory for Terraform - finds all `.tfstate` files below the working
|
||||
directory and generates an inventory based on them.
|
||||
"""
|
||||
import argparse
|
||||
from collections import defaultdict
|
||||
import random
|
||||
from functools import wraps
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
|
||||
VERSION = '0.4.0pre'
|
||||
|
||||
|
||||
def tfstates(root=None):
|
||||
root = root or os.getcwd()
|
||||
for dirpath, _, filenames in os.walk(root):
|
||||
for name in filenames:
|
||||
if os.path.splitext(name)[-1] == '.tfstate':
|
||||
yield os.path.join(dirpath, name)
|
||||
|
||||
def convert_to_v3_structure(attributes, prefix=''):
|
||||
""" Convert the attributes from v4 to v3
|
||||
Receives a dict and return a dictionary """
|
||||
result = {}
|
||||
if isinstance(attributes, str):
|
||||
# In the case when we receive a string (e.g. values for security_groups)
|
||||
return {'{}{}'.format(prefix, random.randint(1,10**10)): attributes}
|
||||
for key, value in attributes.items():
|
||||
if isinstance(value, list):
|
||||
if len(value):
|
||||
result['{}{}.#'.format(prefix, key, hash)] = len(value)
|
||||
for i, v in enumerate(value):
|
||||
result.update(convert_to_v3_structure(v, '{}{}.{}.'.format(prefix, key, i)))
|
||||
elif isinstance(value, dict):
|
||||
result['{}{}.%'.format(prefix, key)] = len(value)
|
||||
for k, v in value.items():
|
||||
result['{}{}.{}'.format(prefix, key, k)] = v
|
||||
else:
|
||||
result['{}{}'.format(prefix, key)] = value
|
||||
return result
|
||||
|
||||
def iterresources(filenames):
|
||||
for filename in filenames:
|
||||
with open(filename, 'r') as json_file:
|
||||
state = json.load(json_file)
|
||||
tf_version = state['version']
|
||||
if tf_version == 3:
|
||||
for module in state['modules']:
|
||||
name = module['path'][-1]
|
||||
for key, resource in module['resources'].items():
|
||||
yield name, key, resource
|
||||
elif tf_version == 4:
|
||||
# In version 4 the structure changes so we need to iterate
|
||||
# each instance inside the resource branch.
|
||||
for resource in state['resources']:
|
||||
name = resource['provider'].split('.')[-1]
|
||||
for instance in resource['instances']:
|
||||
key = "{}.{}".format(resource['type'], resource['name'])
|
||||
if 'index_key' in instance:
|
||||
key = "{}.{}".format(key, instance['index_key'])
|
||||
data = {}
|
||||
data['type'] = resource['type']
|
||||
data['provider'] = resource['provider']
|
||||
data['depends_on'] = instance.get('depends_on', [])
|
||||
data['primary'] = {'attributes': convert_to_v3_structure(instance['attributes'])}
|
||||
if 'id' in instance['attributes']:
|
||||
data['primary']['id'] = instance['attributes']['id']
|
||||
data['primary']['meta'] = instance['attributes'].get('meta',{})
|
||||
yield name, key, data
|
||||
else:
|
||||
raise KeyError('tfstate version %d not supported' % tf_version)
|
||||
|
||||
|
||||
## READ RESOURCES
|
||||
PARSERS = {}
|
||||
|
||||
|
||||
def _clean_dc(dcname):
|
||||
# Consul DCs are strictly alphanumeric with underscores and hyphens -
|
||||
# ensure that the consul_dc attribute meets these requirements.
|
||||
return re.sub('[^\w_\-]', '-', dcname)
|
||||
|
||||
|
||||
def iterhosts(resources):
|
||||
'''yield host tuples of (name, attributes, groups)'''
|
||||
for module_name, key, resource in resources:
|
||||
resource_type, name = key.split('.', 1)
|
||||
try:
|
||||
parser = PARSERS[resource_type]
|
||||
except KeyError:
|
||||
continue
|
||||
|
||||
yield parser(resource, module_name)
|
||||
|
||||
|
||||
def iterips(resources):
|
||||
'''yield ip tuples of (port_id, ip)'''
|
||||
for module_name, key, resource in resources:
|
||||
resource_type, name = key.split('.', 1)
|
||||
if resource_type == 'openstack_networking_floatingip_associate_v2':
|
||||
yield openstack_floating_ips(resource)
|
||||
|
||||
|
||||
def parses(prefix):
|
||||
def inner(func):
|
||||
PARSERS[prefix] = func
|
||||
return func
|
||||
|
||||
return inner
|
||||
|
||||
|
||||
def calculate_mantl_vars(func):
|
||||
"""calculate Mantl vars"""
|
||||
|
||||
@wraps(func)
|
||||
def inner(*args, **kwargs):
|
||||
name, attrs, groups = func(*args, **kwargs)
|
||||
|
||||
# attrs
|
||||
if attrs.get('role', '') == 'control':
|
||||
attrs['consul_is_server'] = True
|
||||
else:
|
||||
attrs['consul_is_server'] = False
|
||||
|
||||
# groups
|
||||
if attrs.get('publicly_routable', False):
|
||||
groups.append('publicly_routable')
|
||||
|
||||
return name, attrs, groups
|
||||
|
||||
return inner
|
||||
|
||||
|
||||
def _parse_prefix(source, prefix, sep='.'):
|
||||
for compkey, value in list(source.items()):
|
||||
try:
|
||||
curprefix, rest = compkey.split(sep, 1)
|
||||
except ValueError:
|
||||
continue
|
||||
|
||||
if curprefix != prefix or rest == '#':
|
||||
continue
|
||||
|
||||
yield rest, value
|
||||
|
||||
|
||||
def parse_attr_list(source, prefix, sep='.'):
|
||||
attrs = defaultdict(dict)
|
||||
for compkey, value in _parse_prefix(source, prefix, sep):
|
||||
idx, key = compkey.split(sep, 1)
|
||||
attrs[idx][key] = value
|
||||
|
||||
return list(attrs.values())
|
||||
|
||||
|
||||
def parse_dict(source, prefix, sep='.'):
|
||||
return dict(_parse_prefix(source, prefix, sep))
|
||||
|
||||
|
||||
def parse_list(source, prefix, sep='.'):
|
||||
return [value for _, value in _parse_prefix(source, prefix, sep)]
|
||||
|
||||
|
||||
def parse_bool(string_form):
|
||||
if type(string_form) is bool:
|
||||
return string_form
|
||||
|
||||
token = string_form.lower()[0]
|
||||
|
||||
if token == 't':
|
||||
return True
|
||||
elif token == 'f':
|
||||
return False
|
||||
else:
|
||||
raise ValueError('could not convert %r to a bool' % string_form)
|
||||
|
||||
|
||||
@parses('metal_device')
|
||||
def metal_device(resource, tfvars=None):
|
||||
raw_attrs = resource['primary']['attributes']
|
||||
name = raw_attrs['hostname']
|
||||
groups = []
|
||||
|
||||
attrs = {
|
||||
'id': raw_attrs['id'],
|
||||
'facilities': parse_list(raw_attrs, 'facilities'),
|
||||
'hostname': raw_attrs['hostname'],
|
||||
'operating_system': raw_attrs['operating_system'],
|
||||
'locked': parse_bool(raw_attrs['locked']),
|
||||
'tags': parse_list(raw_attrs, 'tags'),
|
||||
'plan': raw_attrs['plan'],
|
||||
'project_id': raw_attrs['project_id'],
|
||||
'state': raw_attrs['state'],
|
||||
# ansible
|
||||
'ansible_host': raw_attrs['network.0.address'],
|
||||
'ansible_ssh_user': 'root', # Use root by default in metal
|
||||
# generic
|
||||
'ipv4_address': raw_attrs['network.0.address'],
|
||||
'public_ipv4': raw_attrs['network.0.address'],
|
||||
'ipv6_address': raw_attrs['network.1.address'],
|
||||
'public_ipv6': raw_attrs['network.1.address'],
|
||||
'private_ipv4': raw_attrs['network.2.address'],
|
||||
'provider': 'metal',
|
||||
}
|
||||
|
||||
if raw_attrs['operating_system'] == 'flatcar_stable':
|
||||
# For Flatcar set the ssh_user to core
|
||||
attrs.update({'ansible_ssh_user': 'core'})
|
||||
|
||||
# add groups based on attrs
|
||||
groups.append('metal_operating_system=' + attrs['operating_system'])
|
||||
groups.append('metal_locked=%s' % attrs['locked'])
|
||||
groups.append('metal_state=' + attrs['state'])
|
||||
groups.append('metal_plan=' + attrs['plan'])
|
||||
|
||||
# groups specific to kubespray
|
||||
groups = groups + attrs['tags']
|
||||
|
||||
return name, attrs, groups
|
||||
|
||||
|
||||
def openstack_floating_ips(resource):
|
||||
raw_attrs = resource['primary']['attributes']
|
||||
attrs = {
|
||||
'ip': raw_attrs['floating_ip'],
|
||||
'port_id': raw_attrs['port_id'],
|
||||
}
|
||||
return attrs
|
||||
|
||||
def openstack_floating_ips(resource):
|
||||
raw_attrs = resource['primary']['attributes']
|
||||
return raw_attrs['port_id'], raw_attrs['floating_ip']
|
||||
|
||||
@parses('openstack_compute_instance_v2')
|
||||
@calculate_mantl_vars
|
||||
def openstack_host(resource, module_name):
|
||||
raw_attrs = resource['primary']['attributes']
|
||||
name = raw_attrs['name']
|
||||
groups = []
|
||||
|
||||
attrs = {
|
||||
'access_ip_v4': raw_attrs['access_ip_v4'],
|
||||
'access_ip_v6': raw_attrs['access_ip_v6'],
|
||||
'access_ip': raw_attrs['access_ip_v4'],
|
||||
'ip': raw_attrs['network.0.fixed_ip_v4'],
|
||||
'flavor': parse_dict(raw_attrs, 'flavor',
|
||||
sep='_'),
|
||||
'id': raw_attrs['id'],
|
||||
'image': parse_dict(raw_attrs, 'image',
|
||||
sep='_'),
|
||||
'key_pair': raw_attrs['key_pair'],
|
||||
'metadata': parse_dict(raw_attrs, 'metadata'),
|
||||
'network': parse_attr_list(raw_attrs, 'network'),
|
||||
'region': raw_attrs.get('region', ''),
|
||||
'security_groups': parse_list(raw_attrs, 'security_groups'),
|
||||
# ansible
|
||||
'ansible_ssh_port': 22,
|
||||
# workaround for an OpenStack bug where hosts have a different domain
|
||||
# after they're restarted
|
||||
'host_domain': 'novalocal',
|
||||
'use_host_domain': True,
|
||||
# generic
|
||||
'public_ipv4': raw_attrs['access_ip_v4'],
|
||||
'private_ipv4': raw_attrs['access_ip_v4'],
|
||||
'port_id' : raw_attrs['network.0.port'],
|
||||
'provider': 'openstack',
|
||||
}
|
||||
|
||||
if 'floating_ip' in raw_attrs:
|
||||
attrs['private_ipv4'] = raw_attrs['network.0.fixed_ip_v4']
|
||||
|
||||
try:
|
||||
if 'metadata.prefer_ipv6' in raw_attrs and raw_attrs['metadata.prefer_ipv6'] == "1":
|
||||
attrs.update({
|
||||
'ansible_host': re.sub("[\[\]]", "", raw_attrs['access_ip_v6']),
|
||||
'publicly_routable': True,
|
||||
})
|
||||
else:
|
||||
attrs.update({
|
||||
'ansible_host': raw_attrs['access_ip_v4'],
|
||||
'publicly_routable': True,
|
||||
})
|
||||
except (KeyError, ValueError):
|
||||
attrs.update({'ansible_host': '', 'publicly_routable': False})
|
||||
|
||||
# Handling of floating IPs has changed: https://github.com/terraform-providers/terraform-provider-openstack/blob/master/CHANGELOG.md#010-june-21-2017
|
||||
|
||||
# attrs specific to Ansible
|
||||
if 'metadata.ssh_user' in raw_attrs:
|
||||
attrs['ansible_ssh_user'] = raw_attrs['metadata.ssh_user']
|
||||
|
||||
if 'volume.#' in list(raw_attrs.keys()) and int(raw_attrs['volume.#']) > 0:
|
||||
device_index = 1
|
||||
for key, value in list(raw_attrs.items()):
|
||||
match = re.search("^volume.*.device$", key)
|
||||
if match:
|
||||
attrs['disk_volume_device_'+str(device_index)] = value
|
||||
device_index += 1
|
||||
|
||||
|
||||
# attrs specific to Mantl
|
||||
attrs.update({
|
||||
'role': attrs['metadata'].get('role', 'none')
|
||||
})
|
||||
|
||||
# add groups based on attrs
|
||||
groups.append('os_image=' + str(attrs['image']['id']))
|
||||
groups.append('os_flavor=' + str(attrs['flavor']['name']))
|
||||
groups.extend('os_metadata_%s=%s' % item
|
||||
for item in list(attrs['metadata'].items()))
|
||||
groups.append('os_region=' + str(attrs['region']))
|
||||
|
||||
# groups specific to kubespray
|
||||
for group in attrs['metadata'].get('kubespray_groups', "").split(","):
|
||||
groups.append(group)
|
||||
|
||||
return name, attrs, groups
|
||||
|
||||
|
||||
def iter_host_ips(hosts, ips):
|
||||
'''Update hosts that have an entry in the floating IP list'''
|
||||
for host in hosts:
|
||||
port_id = host[1]['port_id']
|
||||
|
||||
if port_id in ips:
|
||||
ip = ips[port_id]
|
||||
|
||||
host[1].update({
|
||||
'access_ip_v4': ip,
|
||||
'access_ip': ip,
|
||||
'public_ipv4': ip,
|
||||
'ansible_host': ip,
|
||||
})
|
||||
|
||||
if 'use_access_ip' in host[1]['metadata'] and host[1]['metadata']['use_access_ip'] == "0":
|
||||
host[1].pop('access_ip')
|
||||
|
||||
yield host
|
||||
|
||||
|
||||
## QUERY TYPES
|
||||
def query_host(hosts, target):
|
||||
for name, attrs, _ in hosts:
|
||||
if name == target:
|
||||
return attrs
|
||||
|
||||
return {}
|
||||
|
||||
|
||||
def query_list(hosts):
|
||||
groups = defaultdict(dict)
|
||||
meta = {}
|
||||
|
||||
for name, attrs, hostgroups in hosts:
|
||||
for group in set(hostgroups):
|
||||
# Ansible 2.6.2 stopped supporting empty group names: https://github.com/ansible/ansible/pull/42584/commits/d4cd474b42ed23d8f8aabb2a7f84699673852eaf
|
||||
# Empty group name defaults to "all" in Ansible < 2.6.2 so we alter empty group names to "all"
|
||||
if not group: group = "all"
|
||||
|
||||
groups[group].setdefault('hosts', [])
|
||||
groups[group]['hosts'].append(name)
|
||||
|
||||
meta[name] = attrs
|
||||
|
||||
groups['_meta'] = {'hostvars': meta}
|
||||
return groups
|
||||
|
||||
|
||||
def query_hostfile(hosts):
|
||||
out = ['## begin hosts generated by terraform.py ##']
|
||||
out.extend(
|
||||
'{}\t{}'.format(attrs['ansible_host'].ljust(16), name)
|
||||
for name, attrs, _ in hosts
|
||||
)
|
||||
|
||||
out.append('## end hosts generated by terraform.py ##')
|
||||
return '\n'.join(out)
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(
|
||||
__file__, __doc__,
|
||||
formatter_class=argparse.ArgumentDefaultsHelpFormatter, )
|
||||
modes = parser.add_mutually_exclusive_group(required=True)
|
||||
modes.add_argument('--list',
|
||||
action='store_true',
|
||||
help='list all variables')
|
||||
modes.add_argument('--host', help='list variables for a single host')
|
||||
modes.add_argument('--version',
|
||||
action='store_true',
|
||||
help='print version and exit')
|
||||
modes.add_argument('--hostfile',
|
||||
action='store_true',
|
||||
help='print hosts as a /etc/hosts snippet')
|
||||
parser.add_argument('--pretty',
|
||||
action='store_true',
|
||||
help='pretty-print output JSON')
|
||||
parser.add_argument('--nometa',
|
||||
action='store_true',
|
||||
help='with --list, exclude hostvars')
|
||||
default_root = os.environ.get('TERRAFORM_STATE_ROOT',
|
||||
os.path.abspath(os.path.join(os.path.dirname(__file__),
|
||||
'..', '..', )))
|
||||
parser.add_argument('--root',
|
||||
default=default_root,
|
||||
help='custom root to search for `.tfstate`s in')
|
||||
|
||||
args = parser.parse_args()
|
||||
|
||||
if args.version:
|
||||
print('%s %s' % (__file__, VERSION))
|
||||
parser.exit()
|
||||
|
||||
hosts = iterhosts(iterresources(tfstates(args.root)))
|
||||
|
||||
# Perform a second pass on the file to pick up floating_ip entries to update the ip address of referenced hosts
|
||||
ips = dict(iterips(iterresources(tfstates(args.root))))
|
||||
|
||||
if ips:
|
||||
hosts = iter_host_ips(hosts, ips)
|
||||
|
||||
if args.list:
|
||||
output = query_list(hosts)
|
||||
if args.nometa:
|
||||
del output['_meta']
|
||||
print(json.dumps(output, indent=4 if args.pretty else None))
|
||||
elif args.host:
|
||||
output = query_host(hosts, args.host)
|
||||
print(json.dumps(output, indent=4 if args.pretty else None))
|
||||
elif args.hostfile:
|
||||
output = query_hostfile(hosts)
|
||||
print(output)
|
||||
|
||||
parser.exit()
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
129
kubespray/contrib/terraform/openstack/kubespray.tf
Normal file
129
kubespray/contrib/terraform/openstack/kubespray.tf
Normal file
@@ -0,0 +1,129 @@
|
||||
module "network" {
|
||||
source = "./modules/network"
|
||||
|
||||
external_net = var.external_net
|
||||
network_name = var.network_name
|
||||
subnet_cidr = var.subnet_cidr
|
||||
cluster_name = var.cluster_name
|
||||
dns_nameservers = var.dns_nameservers
|
||||
network_dns_domain = var.network_dns_domain
|
||||
use_neutron = var.use_neutron
|
||||
port_security_enabled = var.port_security_enabled
|
||||
router_id = var.router_id
|
||||
}
|
||||
|
||||
module "ips" {
|
||||
source = "./modules/ips"
|
||||
|
||||
number_of_k8s_masters = var.number_of_k8s_masters
|
||||
number_of_k8s_masters_no_etcd = var.number_of_k8s_masters_no_etcd
|
||||
number_of_k8s_nodes = var.number_of_k8s_nodes
|
||||
floatingip_pool = var.floatingip_pool
|
||||
number_of_bastions = var.number_of_bastions
|
||||
external_net = var.external_net
|
||||
network_name = var.network_name
|
||||
router_id = module.network.router_id
|
||||
k8s_nodes = var.k8s_nodes
|
||||
k8s_masters = var.k8s_masters
|
||||
k8s_master_fips = var.k8s_master_fips
|
||||
bastion_fips = var.bastion_fips
|
||||
router_internal_port_id = module.network.router_internal_port_id
|
||||
}
|
||||
|
||||
module "compute" {
|
||||
source = "./modules/compute"
|
||||
|
||||
cluster_name = var.cluster_name
|
||||
az_list = var.az_list
|
||||
az_list_node = var.az_list_node
|
||||
number_of_k8s_masters = var.number_of_k8s_masters
|
||||
number_of_k8s_masters_no_etcd = var.number_of_k8s_masters_no_etcd
|
||||
number_of_etcd = var.number_of_etcd
|
||||
number_of_k8s_masters_no_floating_ip = var.number_of_k8s_masters_no_floating_ip
|
||||
number_of_k8s_masters_no_floating_ip_no_etcd = var.number_of_k8s_masters_no_floating_ip_no_etcd
|
||||
number_of_k8s_nodes = var.number_of_k8s_nodes
|
||||
number_of_bastions = var.number_of_bastions
|
||||
number_of_k8s_nodes_no_floating_ip = var.number_of_k8s_nodes_no_floating_ip
|
||||
number_of_gfs_nodes_no_floating_ip = var.number_of_gfs_nodes_no_floating_ip
|
||||
k8s_masters = var.k8s_masters
|
||||
k8s_nodes = var.k8s_nodes
|
||||
bastion_root_volume_size_in_gb = var.bastion_root_volume_size_in_gb
|
||||
etcd_root_volume_size_in_gb = var.etcd_root_volume_size_in_gb
|
||||
master_root_volume_size_in_gb = var.master_root_volume_size_in_gb
|
||||
node_root_volume_size_in_gb = var.node_root_volume_size_in_gb
|
||||
gfs_root_volume_size_in_gb = var.gfs_root_volume_size_in_gb
|
||||
gfs_volume_size_in_gb = var.gfs_volume_size_in_gb
|
||||
master_volume_type = var.master_volume_type
|
||||
node_volume_type = var.node_volume_type
|
||||
public_key_path = var.public_key_path
|
||||
image = var.image
|
||||
image_uuid = var.image_uuid
|
||||
image_gfs = var.image_gfs
|
||||
image_master = var.image_master
|
||||
image_master_uuid = var.image_master_uuid
|
||||
image_gfs_uuid = var.image_gfs_uuid
|
||||
ssh_user = var.ssh_user
|
||||
ssh_user_gfs = var.ssh_user_gfs
|
||||
flavor_k8s_master = var.flavor_k8s_master
|
||||
flavor_k8s_node = var.flavor_k8s_node
|
||||
flavor_etcd = var.flavor_etcd
|
||||
flavor_gfs_node = var.flavor_gfs_node
|
||||
network_name = var.network_name
|
||||
flavor_bastion = var.flavor_bastion
|
||||
k8s_master_fips = module.ips.k8s_master_fips
|
||||
k8s_master_no_etcd_fips = module.ips.k8s_master_no_etcd_fips
|
||||
k8s_masters_fips = module.ips.k8s_masters_fips
|
||||
k8s_node_fips = module.ips.k8s_node_fips
|
||||
k8s_nodes_fips = module.ips.k8s_nodes_fips
|
||||
bastion_fips = module.ips.bastion_fips
|
||||
bastion_allowed_remote_ips = var.bastion_allowed_remote_ips
|
||||
master_allowed_remote_ips = var.master_allowed_remote_ips
|
||||
k8s_allowed_remote_ips = var.k8s_allowed_remote_ips
|
||||
k8s_allowed_egress_ips = var.k8s_allowed_egress_ips
|
||||
supplementary_master_groups = var.supplementary_master_groups
|
||||
supplementary_node_groups = var.supplementary_node_groups
|
||||
master_allowed_ports = var.master_allowed_ports
|
||||
worker_allowed_ports = var.worker_allowed_ports
|
||||
bastion_allowed_ports = var.bastion_allowed_ports
|
||||
use_access_ip = var.use_access_ip
|
||||
master_server_group_policy = var.master_server_group_policy
|
||||
node_server_group_policy = var.node_server_group_policy
|
||||
etcd_server_group_policy = var.etcd_server_group_policy
|
||||
extra_sec_groups = var.extra_sec_groups
|
||||
extra_sec_groups_name = var.extra_sec_groups_name
|
||||
group_vars_path = var.group_vars_path
|
||||
port_security_enabled = var.port_security_enabled
|
||||
force_null_port_security = var.force_null_port_security
|
||||
network_router_id = module.network.router_id
|
||||
network_id = module.network.network_id
|
||||
use_existing_network = var.use_existing_network
|
||||
private_subnet_id = module.network.subnet_id
|
||||
|
||||
depends_on = [
|
||||
module.network.subnet_id
|
||||
]
|
||||
}
|
||||
|
||||
output "private_subnet_id" {
|
||||
value = module.network.subnet_id
|
||||
}
|
||||
|
||||
output "floating_network_id" {
|
||||
value = var.external_net
|
||||
}
|
||||
|
||||
output "router_id" {
|
||||
value = module.network.router_id
|
||||
}
|
||||
|
||||
output "k8s_master_fips" {
|
||||
value = var.number_of_k8s_masters + var.number_of_k8s_masters_no_etcd > 0 ? concat(module.ips.k8s_master_fips, module.ips.k8s_master_no_etcd_fips) : [for key, value in module.ips.k8s_masters_fips : value.address]
|
||||
}
|
||||
|
||||
output "k8s_node_fips" {
|
||||
value = var.number_of_k8s_nodes > 0 ? module.ips.k8s_node_fips : [for key, value in module.ips.k8s_nodes_fips : value.address]
|
||||
}
|
||||
|
||||
output "bastion_fips" {
|
||||
value = module.ips.bastion_fips
|
||||
}
|
||||
@@ -0,0 +1 @@
|
||||
ansible_ssh_common_args: "-o ProxyCommand='ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -W %h:%p -q USER@BASTION_ADDRESS {% if ansible_ssh_private_key_file is defined %}-i {{ ansible_ssh_private_key_file }}{% endif %}'"
|
||||
903
kubespray/contrib/terraform/openstack/modules/compute/main.tf
Normal file
903
kubespray/contrib/terraform/openstack/modules/compute/main.tf
Normal file
@@ -0,0 +1,903 @@
|
||||
data "openstack_images_image_v2" "vm_image" {
|
||||
count = var.image_uuid == "" ? 1 : 0
|
||||
most_recent = true
|
||||
name = var.image
|
||||
}
|
||||
|
||||
data "openstack_images_image_v2" "gfs_image" {
|
||||
count = var.image_gfs_uuid == "" ? var.image_uuid == "" ? 1 : 0 : 0
|
||||
most_recent = true
|
||||
name = var.image_gfs == "" ? var.image : var.image_gfs
|
||||
}
|
||||
|
||||
data "openstack_images_image_v2" "image_master" {
|
||||
count = var.image_master_uuid == "" ? var.image_uuid == "" ? 1 : 0 : 0
|
||||
name = var.image_master == "" ? var.image : var.image_master
|
||||
}
|
||||
|
||||
data "cloudinit_config" "cloudinit" {
|
||||
part {
|
||||
content_type = "text/cloud-config"
|
||||
content = file("${path.module}/templates/cloudinit.yaml")
|
||||
}
|
||||
}
|
||||
|
||||
data "openstack_networking_network_v2" "k8s_network" {
|
||||
count = var.use_existing_network ? 1 : 0
|
||||
name = var.network_name
|
||||
}
|
||||
|
||||
resource "openstack_compute_keypair_v2" "k8s" {
|
||||
name = "kubernetes-${var.cluster_name}"
|
||||
public_key = chomp(file(var.public_key_path))
|
||||
}
|
||||
|
||||
resource "openstack_networking_secgroup_v2" "k8s_master" {
|
||||
name = "${var.cluster_name}-k8s-master"
|
||||
description = "${var.cluster_name} - Kubernetes Master"
|
||||
delete_default_rules = true
|
||||
}
|
||||
|
||||
resource "openstack_networking_secgroup_v2" "k8s_master_extra" {
|
||||
count = "%{if var.extra_sec_groups}1%{else}0%{endif}"
|
||||
name = "${var.cluster_name}-k8s-master-${var.extra_sec_groups_name}"
|
||||
description = "${var.cluster_name} - Kubernetes Master nodes - rules not managed by terraform"
|
||||
delete_default_rules = true
|
||||
}
|
||||
|
||||
resource "openstack_networking_secgroup_rule_v2" "k8s_master" {
|
||||
count = length(var.master_allowed_remote_ips)
|
||||
direction = "ingress"
|
||||
ethertype = "IPv4"
|
||||
protocol = "tcp"
|
||||
port_range_min = "6443"
|
||||
port_range_max = "6443"
|
||||
remote_ip_prefix = var.master_allowed_remote_ips[count.index]
|
||||
security_group_id = openstack_networking_secgroup_v2.k8s_master.id
|
||||
}
|
||||
|
||||
resource "openstack_networking_secgroup_rule_v2" "k8s_master_ports" {
|
||||
count = length(var.master_allowed_ports)
|
||||
direction = "ingress"
|
||||
ethertype = "IPv4"
|
||||
protocol = lookup(var.master_allowed_ports[count.index], "protocol", "tcp")
|
||||
port_range_min = lookup(var.master_allowed_ports[count.index], "port_range_min")
|
||||
port_range_max = lookup(var.master_allowed_ports[count.index], "port_range_max")
|
||||
remote_ip_prefix = lookup(var.master_allowed_ports[count.index], "remote_ip_prefix", "0.0.0.0/0")
|
||||
security_group_id = openstack_networking_secgroup_v2.k8s_master.id
|
||||
}
|
||||
|
||||
resource "openstack_networking_secgroup_v2" "bastion" {
|
||||
name = "${var.cluster_name}-bastion"
|
||||
count = var.number_of_bastions != "" ? 1 : 0
|
||||
description = "${var.cluster_name} - Bastion Server"
|
||||
delete_default_rules = true
|
||||
}
|
||||
|
||||
resource "openstack_networking_secgroup_rule_v2" "bastion" {
|
||||
count = var.number_of_bastions != "" ? length(var.bastion_allowed_remote_ips) : 0
|
||||
direction = "ingress"
|
||||
ethertype = "IPv4"
|
||||
protocol = "tcp"
|
||||
port_range_min = "22"
|
||||
port_range_max = "22"
|
||||
remote_ip_prefix = var.bastion_allowed_remote_ips[count.index]
|
||||
security_group_id = openstack_networking_secgroup_v2.bastion[0].id
|
||||
}
|
||||
|
||||
resource "openstack_networking_secgroup_rule_v2" "k8s_bastion_ports" {
|
||||
count = length(var.bastion_allowed_ports)
|
||||
direction = "ingress"
|
||||
ethertype = "IPv4"
|
||||
protocol = lookup(var.bastion_allowed_ports[count.index], "protocol", "tcp")
|
||||
port_range_min = lookup(var.bastion_allowed_ports[count.index], "port_range_min")
|
||||
port_range_max = lookup(var.bastion_allowed_ports[count.index], "port_range_max")
|
||||
remote_ip_prefix = lookup(var.bastion_allowed_ports[count.index], "remote_ip_prefix", "0.0.0.0/0")
|
||||
security_group_id = openstack_networking_secgroup_v2.bastion[0].id
|
||||
}
|
||||
|
||||
resource "openstack_networking_secgroup_v2" "k8s" {
|
||||
name = "${var.cluster_name}-k8s"
|
||||
description = "${var.cluster_name} - Kubernetes"
|
||||
delete_default_rules = true
|
||||
}
|
||||
|
||||
resource "openstack_networking_secgroup_rule_v2" "k8s" {
|
||||
direction = "ingress"
|
||||
ethertype = "IPv4"
|
||||
remote_group_id = openstack_networking_secgroup_v2.k8s.id
|
||||
security_group_id = openstack_networking_secgroup_v2.k8s.id
|
||||
}
|
||||
|
||||
resource "openstack_networking_secgroup_rule_v2" "k8s_allowed_remote_ips" {
|
||||
count = length(var.k8s_allowed_remote_ips)
|
||||
direction = "ingress"
|
||||
ethertype = "IPv4"
|
||||
protocol = "tcp"
|
||||
port_range_min = "22"
|
||||
port_range_max = "22"
|
||||
remote_ip_prefix = var.k8s_allowed_remote_ips[count.index]
|
||||
security_group_id = openstack_networking_secgroup_v2.k8s.id
|
||||
}
|
||||
|
||||
resource "openstack_networking_secgroup_rule_v2" "egress" {
|
||||
count = length(var.k8s_allowed_egress_ips)
|
||||
direction = "egress"
|
||||
ethertype = "IPv4"
|
||||
remote_ip_prefix = var.k8s_allowed_egress_ips[count.index]
|
||||
security_group_id = openstack_networking_secgroup_v2.k8s.id
|
||||
}
|
||||
|
||||
resource "openstack_networking_secgroup_v2" "worker" {
|
||||
name = "${var.cluster_name}-k8s-worker"
|
||||
description = "${var.cluster_name} - Kubernetes worker nodes"
|
||||
delete_default_rules = true
|
||||
}
|
||||
|
||||
resource "openstack_networking_secgroup_v2" "worker_extra" {
|
||||
count = "%{if var.extra_sec_groups}1%{else}0%{endif}"
|
||||
name = "${var.cluster_name}-k8s-worker-${var.extra_sec_groups_name}"
|
||||
description = "${var.cluster_name} - Kubernetes worker nodes - rules not managed by terraform"
|
||||
delete_default_rules = true
|
||||
}
|
||||
|
||||
resource "openstack_networking_secgroup_rule_v2" "worker" {
|
||||
count = length(var.worker_allowed_ports)
|
||||
direction = "ingress"
|
||||
ethertype = "IPv4"
|
||||
protocol = lookup(var.worker_allowed_ports[count.index], "protocol", "tcp")
|
||||
port_range_min = lookup(var.worker_allowed_ports[count.index], "port_range_min")
|
||||
port_range_max = lookup(var.worker_allowed_ports[count.index], "port_range_max")
|
||||
remote_ip_prefix = lookup(var.worker_allowed_ports[count.index], "remote_ip_prefix", "0.0.0.0/0")
|
||||
security_group_id = openstack_networking_secgroup_v2.worker.id
|
||||
}
|
||||
|
||||
resource "openstack_compute_servergroup_v2" "k8s_master" {
|
||||
count = var.master_server_group_policy != "" ? 1 : 0
|
||||
name = "k8s-master-srvgrp"
|
||||
policies = [var.master_server_group_policy]
|
||||
}
|
||||
|
||||
resource "openstack_compute_servergroup_v2" "k8s_node" {
|
||||
count = var.node_server_group_policy != "" ? 1 : 0
|
||||
name = "k8s-node-srvgrp"
|
||||
policies = [var.node_server_group_policy]
|
||||
}
|
||||
|
||||
resource "openstack_compute_servergroup_v2" "k8s_etcd" {
|
||||
count = var.etcd_server_group_policy != "" ? 1 : 0
|
||||
name = "k8s-etcd-srvgrp"
|
||||
policies = [var.etcd_server_group_policy]
|
||||
}
|
||||
|
||||
locals {
|
||||
# master groups
|
||||
master_sec_groups = compact([
|
||||
openstack_networking_secgroup_v2.k8s_master.id,
|
||||
openstack_networking_secgroup_v2.k8s.id,
|
||||
var.extra_sec_groups ?openstack_networking_secgroup_v2.k8s_master_extra[0].id : "",
|
||||
])
|
||||
# worker groups
|
||||
worker_sec_groups = compact([
|
||||
openstack_networking_secgroup_v2.k8s.id,
|
||||
openstack_networking_secgroup_v2.worker.id,
|
||||
var.extra_sec_groups ? openstack_networking_secgroup_v2.worker_extra[0].id : "",
|
||||
])
|
||||
# bastion groups
|
||||
bastion_sec_groups = compact(concat([
|
||||
openstack_networking_secgroup_v2.k8s.id,
|
||||
openstack_networking_secgroup_v2.bastion[0].id,
|
||||
]))
|
||||
# etcd groups
|
||||
etcd_sec_groups = compact([openstack_networking_secgroup_v2.k8s.id])
|
||||
# glusterfs groups
|
||||
gfs_sec_groups = compact([openstack_networking_secgroup_v2.k8s.id])
|
||||
|
||||
# Image uuid
|
||||
image_to_use_node = var.image_uuid != "" ? var.image_uuid : data.openstack_images_image_v2.vm_image[0].id
|
||||
# Image_gfs uuid
|
||||
image_to_use_gfs = var.image_gfs_uuid != "" ? var.image_gfs_uuid : var.image_uuid != "" ? var.image_uuid : data.openstack_images_image_v2.gfs_image[0].id
|
||||
# image_master uuidimage_gfs_uuid
|
||||
image_to_use_master = var.image_master_uuid != "" ? var.image_master_uuid : var.image_uuid != "" ? var.image_uuid : data.openstack_images_image_v2.image_master[0].id
|
||||
}
|
||||
|
||||
resource "openstack_networking_port_v2" "bastion_port" {
|
||||
count = var.number_of_bastions
|
||||
name = "${var.cluster_name}-bastion-${count.index + 1}"
|
||||
network_id = var.use_existing_network ? data.openstack_networking_network_v2.k8s_network[0].id : var.network_id
|
||||
admin_state_up = "true"
|
||||
port_security_enabled = var.force_null_port_security ? null : var.port_security_enabled
|
||||
security_group_ids = var.port_security_enabled ? local.bastion_sec_groups : null
|
||||
no_security_groups = var.port_security_enabled ? null : false
|
||||
fixed_ip {
|
||||
subnet_id = var.private_subnet_id
|
||||
}
|
||||
|
||||
depends_on = [
|
||||
var.network_router_id
|
||||
]
|
||||
}
|
||||
|
||||
resource "openstack_compute_instance_v2" "bastion" {
|
||||
name = "${var.cluster_name}-bastion-${count.index + 1}"
|
||||
count = var.number_of_bastions
|
||||
image_id = var.bastion_root_volume_size_in_gb == 0 ? local.image_to_use_node : null
|
||||
flavor_id = var.flavor_bastion
|
||||
key_pair = openstack_compute_keypair_v2.k8s.name
|
||||
user_data = data.cloudinit_config.cloudinit.rendered
|
||||
|
||||
dynamic "block_device" {
|
||||
for_each = var.bastion_root_volume_size_in_gb > 0 ? [local.image_to_use_node] : []
|
||||
content {
|
||||
uuid = local.image_to_use_node
|
||||
source_type = "image"
|
||||
volume_size = var.bastion_root_volume_size_in_gb
|
||||
boot_index = 0
|
||||
destination_type = "volume"
|
||||
delete_on_termination = true
|
||||
}
|
||||
}
|
||||
|
||||
network {
|
||||
port = element(openstack_networking_port_v2.bastion_port.*.id, count.index)
|
||||
}
|
||||
|
||||
metadata = {
|
||||
ssh_user = var.ssh_user
|
||||
kubespray_groups = "bastion"
|
||||
depends_on = var.network_router_id
|
||||
use_access_ip = var.use_access_ip
|
||||
}
|
||||
|
||||
provisioner "local-exec" {
|
||||
command = "sed -e s/USER/${var.ssh_user}/ -e s/BASTION_ADDRESS/${var.bastion_fips[0]}/ ${path.module}/ansible_bastion_template.txt > ${var.group_vars_path}/no_floating.yml"
|
||||
}
|
||||
}
|
||||
|
||||
resource "openstack_networking_port_v2" "k8s_master_port" {
|
||||
count = var.number_of_k8s_masters
|
||||
name = "${var.cluster_name}-k8s-master-${count.index + 1}"
|
||||
network_id = var.use_existing_network ? data.openstack_networking_network_v2.k8s_network[0].id : var.network_id
|
||||
admin_state_up = "true"
|
||||
port_security_enabled = var.force_null_port_security ? null : var.port_security_enabled
|
||||
security_group_ids = var.port_security_enabled ? local.master_sec_groups : null
|
||||
no_security_groups = var.port_security_enabled ? null : false
|
||||
fixed_ip {
|
||||
subnet_id = var.private_subnet_id
|
||||
}
|
||||
|
||||
depends_on = [
|
||||
var.network_router_id
|
||||
]
|
||||
}
|
||||
|
||||
resource "openstack_compute_instance_v2" "k8s_master" {
|
||||
name = "${var.cluster_name}-k8s-master-${count.index + 1}"
|
||||
count = var.number_of_k8s_masters
|
||||
availability_zone = element(var.az_list, count.index)
|
||||
image_id = var.master_root_volume_size_in_gb == 0 ? local.image_to_use_master : null
|
||||
flavor_id = var.flavor_k8s_master
|
||||
key_pair = openstack_compute_keypair_v2.k8s.name
|
||||
user_data = data.cloudinit_config.cloudinit.rendered
|
||||
|
||||
|
||||
dynamic "block_device" {
|
||||
for_each = var.master_root_volume_size_in_gb > 0 ? [local.image_to_use_master] : []
|
||||
content {
|
||||
uuid = local.image_to_use_master
|
||||
source_type = "image"
|
||||
volume_size = var.master_root_volume_size_in_gb
|
||||
volume_type = var.master_volume_type
|
||||
boot_index = 0
|
||||
destination_type = "volume"
|
||||
delete_on_termination = true
|
||||
}
|
||||
}
|
||||
|
||||
network {
|
||||
port = element(openstack_networking_port_v2.k8s_master_port.*.id, count.index)
|
||||
}
|
||||
|
||||
dynamic "scheduler_hints" {
|
||||
for_each = var.master_server_group_policy != "" ? [openstack_compute_servergroup_v2.k8s_master[0]] : []
|
||||
content {
|
||||
group = openstack_compute_servergroup_v2.k8s_master[0].id
|
||||
}
|
||||
}
|
||||
|
||||
metadata = {
|
||||
ssh_user = var.ssh_user
|
||||
kubespray_groups = "etcd,kube_control_plane,${var.supplementary_master_groups},k8s_cluster"
|
||||
depends_on = var.network_router_id
|
||||
use_access_ip = var.use_access_ip
|
||||
}
|
||||
|
||||
provisioner "local-exec" {
|
||||
command = "sed -e s/USER/${var.ssh_user}/ -e s/BASTION_ADDRESS/${element(concat(var.bastion_fips, var.k8s_master_fips), 0)}/ ${path.module}/ansible_bastion_template.txt > ${var.group_vars_path}/no_floating.yml"
|
||||
}
|
||||
}
|
||||
|
||||
resource "openstack_networking_port_v2" "k8s_masters_port" {
|
||||
for_each = var.number_of_k8s_masters == 0 && var.number_of_k8s_masters_no_etcd == 0 && var.number_of_k8s_masters_no_floating_ip == 0 && var.number_of_k8s_masters_no_floating_ip_no_etcd == 0 ? var.k8s_masters : {}
|
||||
name = "${var.cluster_name}-k8s-${each.key}"
|
||||
network_id = var.use_existing_network ? data.openstack_networking_network_v2.k8s_network[0].id : var.network_id
|
||||
admin_state_up = "true"
|
||||
port_security_enabled = var.force_null_port_security ? null : var.port_security_enabled
|
||||
security_group_ids = var.port_security_enabled ? local.master_sec_groups : null
|
||||
no_security_groups = var.port_security_enabled ? null : false
|
||||
fixed_ip {
|
||||
subnet_id = var.private_subnet_id
|
||||
}
|
||||
|
||||
depends_on = [
|
||||
var.network_router_id
|
||||
]
|
||||
}
|
||||
|
||||
resource "openstack_compute_instance_v2" "k8s_masters" {
|
||||
for_each = var.number_of_k8s_masters == 0 && var.number_of_k8s_masters_no_etcd == 0 && var.number_of_k8s_masters_no_floating_ip == 0 && var.number_of_k8s_masters_no_floating_ip_no_etcd == 0 ? var.k8s_masters : {}
|
||||
name = "${var.cluster_name}-k8s-${each.key}"
|
||||
availability_zone = each.value.az
|
||||
image_id = var.master_root_volume_size_in_gb == 0 ? local.image_to_use_master : null
|
||||
flavor_id = each.value.flavor
|
||||
key_pair = openstack_compute_keypair_v2.k8s.name
|
||||
|
||||
dynamic "block_device" {
|
||||
for_each = var.master_root_volume_size_in_gb > 0 ? [local.image_to_use_master] : []
|
||||
content {
|
||||
uuid = local.image_to_use_master
|
||||
source_type = "image"
|
||||
volume_size = var.master_root_volume_size_in_gb
|
||||
volume_type = var.master_volume_type
|
||||
boot_index = 0
|
||||
destination_type = "volume"
|
||||
delete_on_termination = true
|
||||
}
|
||||
}
|
||||
|
||||
network {
|
||||
port = openstack_networking_port_v2.k8s_masters_port[each.key].id
|
||||
}
|
||||
|
||||
dynamic "scheduler_hints" {
|
||||
for_each = var.master_server_group_policy != "" ? [openstack_compute_servergroup_v2.k8s_master[0]] : []
|
||||
content {
|
||||
group = openstack_compute_servergroup_v2.k8s_master[0].id
|
||||
}
|
||||
}
|
||||
|
||||
metadata = {
|
||||
ssh_user = var.ssh_user
|
||||
kubespray_groups = "%{if each.value.etcd == true}etcd,%{endif}kube_control_plane,${var.supplementary_master_groups},k8s_cluster%{if each.value.floating_ip == false},no_floating%{endif}"
|
||||
depends_on = var.network_router_id
|
||||
use_access_ip = var.use_access_ip
|
||||
}
|
||||
|
||||
provisioner "local-exec" {
|
||||
command = "%{if each.value.floating_ip}sed s/USER/${var.ssh_user}/ ${path.root}/ansible_bastion_template.txt | sed s/BASTION_ADDRESS/${element(concat(var.bastion_fips, [for key, value in var.k8s_masters_fips : value.address]), 0)}/ > ${var.group_vars_path}/no_floating.yml%{else}true%{endif}"
|
||||
}
|
||||
}
|
||||
|
||||
resource "openstack_networking_port_v2" "k8s_master_no_etcd_port" {
|
||||
count = var.number_of_k8s_masters_no_etcd
|
||||
name = "${var.cluster_name}-k8s-master-ne-${count.index + 1}"
|
||||
network_id = var.use_existing_network ? data.openstack_networking_network_v2.k8s_network[0].id : var.network_id
|
||||
admin_state_up = "true"
|
||||
port_security_enabled = var.force_null_port_security ? null : var.port_security_enabled
|
||||
security_group_ids = var.port_security_enabled ? local.master_sec_groups : null
|
||||
no_security_groups = var.port_security_enabled ? null : false
|
||||
fixed_ip {
|
||||
subnet_id = var.private_subnet_id
|
||||
}
|
||||
|
||||
depends_on = [
|
||||
var.network_router_id
|
||||
]
|
||||
}
|
||||
|
||||
resource "openstack_compute_instance_v2" "k8s_master_no_etcd" {
|
||||
name = "${var.cluster_name}-k8s-master-ne-${count.index + 1}"
|
||||
count = var.number_of_k8s_masters_no_etcd
|
||||
availability_zone = element(var.az_list, count.index)
|
||||
image_id = var.master_root_volume_size_in_gb == 0 ? local.image_to_use_master : null
|
||||
flavor_id = var.flavor_k8s_master
|
||||
key_pair = openstack_compute_keypair_v2.k8s.name
|
||||
user_data = data.cloudinit_config.cloudinit.rendered
|
||||
|
||||
|
||||
dynamic "block_device" {
|
||||
for_each = var.master_root_volume_size_in_gb > 0 ? [local.image_to_use_master] : []
|
||||
content {
|
||||
uuid = local.image_to_use_master
|
||||
source_type = "image"
|
||||
volume_size = var.master_root_volume_size_in_gb
|
||||
volume_type = var.master_volume_type
|
||||
boot_index = 0
|
||||
destination_type = "volume"
|
||||
delete_on_termination = true
|
||||
}
|
||||
}
|
||||
|
||||
network {
|
||||
port = element(openstack_networking_port_v2.k8s_master_no_etcd_port.*.id, count.index)
|
||||
}
|
||||
|
||||
dynamic "scheduler_hints" {
|
||||
for_each = var.master_server_group_policy != "" ? [openstack_compute_servergroup_v2.k8s_master[0]] : []
|
||||
content {
|
||||
group = openstack_compute_servergroup_v2.k8s_master[0].id
|
||||
}
|
||||
}
|
||||
|
||||
metadata = {
|
||||
ssh_user = var.ssh_user
|
||||
kubespray_groups = "kube_control_plane,${var.supplementary_master_groups},k8s_cluster"
|
||||
depends_on = var.network_router_id
|
||||
use_access_ip = var.use_access_ip
|
||||
}
|
||||
|
||||
provisioner "local-exec" {
|
||||
command = "sed -e s/USER/${var.ssh_user}/ -e s/BASTION_ADDRESS/${element(concat(var.bastion_fips, var.k8s_master_fips), 0)}/ ${path.module}/ansible_bastion_template.txt > ${var.group_vars_path}/no_floating.yml"
|
||||
}
|
||||
}
|
||||
|
||||
resource "openstack_networking_port_v2" "etcd_port" {
|
||||
count = var.number_of_etcd
|
||||
name = "${var.cluster_name}-etcd-${count.index + 1}"
|
||||
network_id = var.use_existing_network ? data.openstack_networking_network_v2.k8s_network[0].id : var.network_id
|
||||
admin_state_up = "true"
|
||||
port_security_enabled = var.force_null_port_security ? null : var.port_security_enabled
|
||||
security_group_ids = var.port_security_enabled ? local.etcd_sec_groups : null
|
||||
no_security_groups = var.port_security_enabled ? null : false
|
||||
fixed_ip {
|
||||
subnet_id = var.private_subnet_id
|
||||
}
|
||||
|
||||
depends_on = [
|
||||
var.network_router_id
|
||||
]
|
||||
}
|
||||
|
||||
resource "openstack_compute_instance_v2" "etcd" {
|
||||
name = "${var.cluster_name}-etcd-${count.index + 1}"
|
||||
count = var.number_of_etcd
|
||||
availability_zone = element(var.az_list, count.index)
|
||||
image_id = var.etcd_root_volume_size_in_gb == 0 ? local.image_to_use_master : null
|
||||
flavor_id = var.flavor_etcd
|
||||
key_pair = openstack_compute_keypair_v2.k8s.name
|
||||
user_data = data.cloudinit_config.cloudinit.rendered
|
||||
|
||||
dynamic "block_device" {
|
||||
for_each = var.etcd_root_volume_size_in_gb > 0 ? [local.image_to_use_master] : []
|
||||
content {
|
||||
uuid = local.image_to_use_master
|
||||
source_type = "image"
|
||||
volume_size = var.etcd_root_volume_size_in_gb
|
||||
boot_index = 0
|
||||
destination_type = "volume"
|
||||
delete_on_termination = true
|
||||
}
|
||||
}
|
||||
|
||||
network {
|
||||
port = element(openstack_networking_port_v2.etcd_port.*.id, count.index)
|
||||
}
|
||||
|
||||
dynamic "scheduler_hints" {
|
||||
for_each = var.etcd_server_group_policy != "" ? [openstack_compute_servergroup_v2.k8s_etcd[0]] : []
|
||||
content {
|
||||
group = openstack_compute_servergroup_v2.k8s_etcd[0].id
|
||||
}
|
||||
}
|
||||
|
||||
metadata = {
|
||||
ssh_user = var.ssh_user
|
||||
kubespray_groups = "etcd,no_floating"
|
||||
depends_on = var.network_router_id
|
||||
use_access_ip = var.use_access_ip
|
||||
}
|
||||
}
|
||||
|
||||
resource "openstack_networking_port_v2" "k8s_master_no_floating_ip_port" {
|
||||
count = var.number_of_k8s_masters_no_floating_ip
|
||||
name = "${var.cluster_name}-k8s-master-nf-${count.index + 1}"
|
||||
network_id = var.use_existing_network ? data.openstack_networking_network_v2.k8s_network[0].id : var.network_id
|
||||
admin_state_up = "true"
|
||||
port_security_enabled = var.force_null_port_security ? null : var.port_security_enabled
|
||||
security_group_ids = var.port_security_enabled ? local.master_sec_groups : null
|
||||
no_security_groups = var.port_security_enabled ? null : false
|
||||
fixed_ip {
|
||||
subnet_id = var.private_subnet_id
|
||||
}
|
||||
|
||||
depends_on = [
|
||||
var.network_router_id
|
||||
]
|
||||
}
|
||||
|
||||
resource "openstack_compute_instance_v2" "k8s_master_no_floating_ip" {
|
||||
name = "${var.cluster_name}-k8s-master-nf-${count.index + 1}"
|
||||
count = var.number_of_k8s_masters_no_floating_ip
|
||||
availability_zone = element(var.az_list, count.index)
|
||||
image_id = var.master_root_volume_size_in_gb == 0 ? local.image_to_use_master : null
|
||||
flavor_id = var.flavor_k8s_master
|
||||
key_pair = openstack_compute_keypair_v2.k8s.name
|
||||
|
||||
dynamic "block_device" {
|
||||
for_each = var.master_root_volume_size_in_gb > 0 ? [local.image_to_use_master] : []
|
||||
content {
|
||||
uuid = local.image_to_use_master
|
||||
source_type = "image"
|
||||
volume_size = var.master_root_volume_size_in_gb
|
||||
volume_type = var.master_volume_type
|
||||
boot_index = 0
|
||||
destination_type = "volume"
|
||||
delete_on_termination = true
|
||||
}
|
||||
}
|
||||
|
||||
network {
|
||||
port = element(openstack_networking_port_v2.k8s_master_no_floating_ip_port.*.id, count.index)
|
||||
}
|
||||
|
||||
dynamic "scheduler_hints" {
|
||||
for_each = var.master_server_group_policy != "" ? [openstack_compute_servergroup_v2.k8s_master[0]] : []
|
||||
content {
|
||||
group = openstack_compute_servergroup_v2.k8s_master[0].id
|
||||
}
|
||||
}
|
||||
|
||||
metadata = {
|
||||
ssh_user = var.ssh_user
|
||||
kubespray_groups = "etcd,kube_control_plane,${var.supplementary_master_groups},k8s_cluster,no_floating"
|
||||
depends_on = var.network_router_id
|
||||
use_access_ip = var.use_access_ip
|
||||
}
|
||||
}
|
||||
|
||||
resource "openstack_networking_port_v2" "k8s_master_no_floating_ip_no_etcd_port" {
|
||||
count = var.number_of_k8s_masters_no_floating_ip_no_etcd
|
||||
name = "${var.cluster_name}-k8s-master-ne-nf-${count.index + 1}"
|
||||
network_id = var.use_existing_network ? data.openstack_networking_network_v2.k8s_network[0].id : var.network_id
|
||||
admin_state_up = "true"
|
||||
port_security_enabled = var.force_null_port_security ? null : var.port_security_enabled
|
||||
security_group_ids = var.port_security_enabled ? local.master_sec_groups : null
|
||||
no_security_groups = var.port_security_enabled ? null : false
|
||||
fixed_ip {
|
||||
subnet_id = var.private_subnet_id
|
||||
}
|
||||
|
||||
depends_on = [
|
||||
var.network_router_id
|
||||
]
|
||||
}
|
||||
|
||||
resource "openstack_compute_instance_v2" "k8s_master_no_floating_ip_no_etcd" {
|
||||
name = "${var.cluster_name}-k8s-master-ne-nf-${count.index + 1}"
|
||||
count = var.number_of_k8s_masters_no_floating_ip_no_etcd
|
||||
availability_zone = element(var.az_list, count.index)
|
||||
image_id = var.master_root_volume_size_in_gb == 0 ? local.image_to_use_master : null
|
||||
flavor_id = var.flavor_k8s_master
|
||||
key_pair = openstack_compute_keypair_v2.k8s.name
|
||||
user_data = data.cloudinit_config.cloudinit.rendered
|
||||
|
||||
dynamic "block_device" {
|
||||
for_each = var.master_root_volume_size_in_gb > 0 ? [local.image_to_use_master] : []
|
||||
content {
|
||||
uuid = local.image_to_use_master
|
||||
source_type = "image"
|
||||
volume_size = var.master_root_volume_size_in_gb
|
||||
volume_type = var.master_volume_type
|
||||
boot_index = 0
|
||||
destination_type = "volume"
|
||||
delete_on_termination = true
|
||||
}
|
||||
}
|
||||
|
||||
network {
|
||||
port = element(openstack_networking_port_v2.k8s_master_no_floating_ip_no_etcd_port.*.id, count.index)
|
||||
}
|
||||
|
||||
dynamic "scheduler_hints" {
|
||||
for_each = var.master_server_group_policy != "" ? [openstack_compute_servergroup_v2.k8s_master[0]] : []
|
||||
content {
|
||||
group = openstack_compute_servergroup_v2.k8s_master[0].id
|
||||
}
|
||||
}
|
||||
|
||||
metadata = {
|
||||
ssh_user = var.ssh_user
|
||||
kubespray_groups = "kube_control_plane,${var.supplementary_master_groups},k8s_cluster,no_floating"
|
||||
depends_on = var.network_router_id
|
||||
use_access_ip = var.use_access_ip
|
||||
}
|
||||
}
|
||||
|
||||
resource "openstack_networking_port_v2" "k8s_node_port" {
|
||||
count = var.number_of_k8s_nodes
|
||||
name = "${var.cluster_name}-k8s-node-${count.index + 1}"
|
||||
network_id = var.use_existing_network ? data.openstack_networking_network_v2.k8s_network[0].id : var.network_id
|
||||
admin_state_up = "true"
|
||||
port_security_enabled = var.force_null_port_security ? null : var.port_security_enabled
|
||||
security_group_ids = var.port_security_enabled ? local.worker_sec_groups : null
|
||||
no_security_groups = var.port_security_enabled ? null : false
|
||||
fixed_ip {
|
||||
subnet_id = var.private_subnet_id
|
||||
}
|
||||
|
||||
depends_on = [
|
||||
var.network_router_id
|
||||
]
|
||||
}
|
||||
|
||||
resource "openstack_compute_instance_v2" "k8s_node" {
|
||||
name = "${var.cluster_name}-k8s-node-${count.index + 1}"
|
||||
count = var.number_of_k8s_nodes
|
||||
availability_zone = element(var.az_list_node, count.index)
|
||||
image_id = var.node_root_volume_size_in_gb == 0 ? local.image_to_use_node : null
|
||||
flavor_id = var.flavor_k8s_node
|
||||
key_pair = openstack_compute_keypair_v2.k8s.name
|
||||
user_data = data.cloudinit_config.cloudinit.rendered
|
||||
|
||||
dynamic "block_device" {
|
||||
for_each = var.node_root_volume_size_in_gb > 0 ? [local.image_to_use_node] : []
|
||||
content {
|
||||
uuid = local.image_to_use_node
|
||||
source_type = "image"
|
||||
volume_size = var.node_root_volume_size_in_gb
|
||||
volume_type = var.node_volume_type
|
||||
boot_index = 0
|
||||
destination_type = "volume"
|
||||
delete_on_termination = true
|
||||
}
|
||||
}
|
||||
|
||||
network {
|
||||
port = element(openstack_networking_port_v2.k8s_node_port.*.id, count.index)
|
||||
}
|
||||
|
||||
|
||||
dynamic "scheduler_hints" {
|
||||
for_each = var.node_server_group_policy != "" ? [openstack_compute_servergroup_v2.k8s_node[0]] : []
|
||||
content {
|
||||
group = openstack_compute_servergroup_v2.k8s_node[0].id
|
||||
}
|
||||
}
|
||||
|
||||
metadata = {
|
||||
ssh_user = var.ssh_user
|
||||
kubespray_groups = "kube_node,k8s_cluster,${var.supplementary_node_groups}"
|
||||
depends_on = var.network_router_id
|
||||
use_access_ip = var.use_access_ip
|
||||
}
|
||||
|
||||
provisioner "local-exec" {
|
||||
command = "sed -e s/USER/${var.ssh_user}/ -e s/BASTION_ADDRESS/${element(concat(var.bastion_fips, var.k8s_node_fips), 0)}/ ${path.module}/ansible_bastion_template.txt > ${var.group_vars_path}/no_floating.yml"
|
||||
}
|
||||
}
|
||||
|
||||
resource "openstack_networking_port_v2" "k8s_node_no_floating_ip_port" {
|
||||
count = var.number_of_k8s_nodes_no_floating_ip
|
||||
name = "${var.cluster_name}-k8s-node-nf-${count.index + 1}"
|
||||
network_id = var.use_existing_network ? data.openstack_networking_network_v2.k8s_network[0].id : var.network_id
|
||||
admin_state_up = "true"
|
||||
port_security_enabled = var.force_null_port_security ? null : var.port_security_enabled
|
||||
security_group_ids = var.port_security_enabled ? local.worker_sec_groups : null
|
||||
no_security_groups = var.port_security_enabled ? null : false
|
||||
fixed_ip {
|
||||
subnet_id = var.private_subnet_id
|
||||
}
|
||||
|
||||
depends_on = [
|
||||
var.network_router_id
|
||||
]
|
||||
}
|
||||
|
||||
resource "openstack_compute_instance_v2" "k8s_node_no_floating_ip" {
|
||||
name = "${var.cluster_name}-k8s-node-nf-${count.index + 1}"
|
||||
count = var.number_of_k8s_nodes_no_floating_ip
|
||||
availability_zone = element(var.az_list_node, count.index)
|
||||
image_id = var.node_root_volume_size_in_gb == 0 ? local.image_to_use_node : null
|
||||
flavor_id = var.flavor_k8s_node
|
||||
key_pair = openstack_compute_keypair_v2.k8s.name
|
||||
user_data = data.cloudinit_config.cloudinit.rendered
|
||||
|
||||
dynamic "block_device" {
|
||||
for_each = var.node_root_volume_size_in_gb > 0 ? [local.image_to_use_node] : []
|
||||
content {
|
||||
uuid = local.image_to_use_node
|
||||
source_type = "image"
|
||||
volume_size = var.node_root_volume_size_in_gb
|
||||
volume_type = var.node_volume_type
|
||||
boot_index = 0
|
||||
destination_type = "volume"
|
||||
delete_on_termination = true
|
||||
}
|
||||
}
|
||||
|
||||
network {
|
||||
port = element(openstack_networking_port_v2.k8s_node_no_floating_ip_port.*.id, count.index)
|
||||
}
|
||||
|
||||
dynamic "scheduler_hints" {
|
||||
for_each = var.node_server_group_policy != "" ? [openstack_compute_servergroup_v2.k8s_node[0]] : []
|
||||
content {
|
||||
group = openstack_compute_servergroup_v2.k8s_node[0].id
|
||||
}
|
||||
}
|
||||
|
||||
metadata = {
|
||||
ssh_user = var.ssh_user
|
||||
kubespray_groups = "kube_node,k8s_cluster,no_floating,${var.supplementary_node_groups}"
|
||||
depends_on = var.network_router_id
|
||||
use_access_ip = var.use_access_ip
|
||||
}
|
||||
}
|
||||
|
||||
resource "openstack_networking_port_v2" "k8s_nodes_port" {
|
||||
for_each = var.number_of_k8s_nodes == 0 && var.number_of_k8s_nodes_no_floating_ip == 0 ? var.k8s_nodes : {}
|
||||
name = "${var.cluster_name}-k8s-node-${each.key}"
|
||||
network_id = var.use_existing_network ? data.openstack_networking_network_v2.k8s_network[0].id : var.network_id
|
||||
admin_state_up = "true"
|
||||
port_security_enabled = var.force_null_port_security ? null : var.port_security_enabled
|
||||
security_group_ids = var.port_security_enabled ? local.worker_sec_groups : null
|
||||
no_security_groups = var.port_security_enabled ? null : false
|
||||
fixed_ip {
|
||||
subnet_id = var.private_subnet_id
|
||||
}
|
||||
|
||||
depends_on = [
|
||||
var.network_router_id
|
||||
]
|
||||
}
|
||||
|
||||
resource "openstack_compute_instance_v2" "k8s_nodes" {
|
||||
for_each = var.number_of_k8s_nodes == 0 && var.number_of_k8s_nodes_no_floating_ip == 0 ? var.k8s_nodes : {}
|
||||
name = "${var.cluster_name}-k8s-node-${each.key}"
|
||||
availability_zone = each.value.az
|
||||
image_id = var.node_root_volume_size_in_gb == 0 ? local.image_to_use_node : null
|
||||
flavor_id = each.value.flavor
|
||||
key_pair = openstack_compute_keypair_v2.k8s.name
|
||||
user_data = data.cloudinit_config.cloudinit.rendered
|
||||
|
||||
dynamic "block_device" {
|
||||
for_each = var.node_root_volume_size_in_gb > 0 ? [local.image_to_use_node] : []
|
||||
content {
|
||||
uuid = local.image_to_use_node
|
||||
source_type = "image"
|
||||
volume_size = var.node_root_volume_size_in_gb
|
||||
volume_type = var.node_volume_type
|
||||
boot_index = 0
|
||||
destination_type = "volume"
|
||||
delete_on_termination = true
|
||||
}
|
||||
}
|
||||
|
||||
network {
|
||||
port = openstack_networking_port_v2.k8s_nodes_port[each.key].id
|
||||
}
|
||||
|
||||
dynamic "scheduler_hints" {
|
||||
for_each = var.node_server_group_policy != "" ? [openstack_compute_servergroup_v2.k8s_node[0]] : []
|
||||
content {
|
||||
group = openstack_compute_servergroup_v2.k8s_node[0].id
|
||||
}
|
||||
}
|
||||
|
||||
metadata = {
|
||||
ssh_user = var.ssh_user
|
||||
kubespray_groups = "kube_node,k8s_cluster,%{if each.value.floating_ip == false}no_floating,%{endif}${var.supplementary_node_groups},${try(each.value.extra_groups, "")}"
|
||||
depends_on = var.network_router_id
|
||||
use_access_ip = var.use_access_ip
|
||||
}
|
||||
|
||||
provisioner "local-exec" {
|
||||
command = "%{if each.value.floating_ip}sed -e s/USER/${var.ssh_user}/ -e s/BASTION_ADDRESS/${element(concat(var.bastion_fips, [for key, value in var.k8s_nodes_fips : value.address]), 0)}/ ${path.module}/ansible_bastion_template.txt > ${var.group_vars_path}/no_floating.yml%{else}true%{endif}"
|
||||
}
|
||||
}
|
||||
|
||||
resource "openstack_networking_port_v2" "glusterfs_node_no_floating_ip_port" {
|
||||
count = var.number_of_gfs_nodes_no_floating_ip
|
||||
name = "${var.cluster_name}-gfs-node-nf-${count.index + 1}"
|
||||
network_id = var.use_existing_network ? data.openstack_networking_network_v2.k8s_network[0].id : var.network_id
|
||||
admin_state_up = "true"
|
||||
port_security_enabled = var.force_null_port_security ? null : var.port_security_enabled
|
||||
security_group_ids = var.port_security_enabled ? local.gfs_sec_groups : null
|
||||
no_security_groups = var.port_security_enabled ? null : false
|
||||
fixed_ip {
|
||||
subnet_id = var.private_subnet_id
|
||||
}
|
||||
|
||||
depends_on = [
|
||||
var.network_router_id
|
||||
]
|
||||
}
|
||||
|
||||
resource "openstack_compute_instance_v2" "glusterfs_node_no_floating_ip" {
|
||||
name = "${var.cluster_name}-gfs-node-nf-${count.index + 1}"
|
||||
count = var.number_of_gfs_nodes_no_floating_ip
|
||||
availability_zone = element(var.az_list, count.index)
|
||||
image_name = var.gfs_root_volume_size_in_gb == 0 ? local.image_to_use_gfs : null
|
||||
flavor_id = var.flavor_gfs_node
|
||||
key_pair = openstack_compute_keypair_v2.k8s.name
|
||||
|
||||
dynamic "block_device" {
|
||||
for_each = var.gfs_root_volume_size_in_gb > 0 ? [local.image_to_use_gfs] : []
|
||||
content {
|
||||
uuid = local.image_to_use_gfs
|
||||
source_type = "image"
|
||||
volume_size = var.gfs_root_volume_size_in_gb
|
||||
boot_index = 0
|
||||
destination_type = "volume"
|
||||
delete_on_termination = true
|
||||
}
|
||||
}
|
||||
|
||||
network {
|
||||
port = element(openstack_networking_port_v2.glusterfs_node_no_floating_ip_port.*.id, count.index)
|
||||
}
|
||||
|
||||
dynamic "scheduler_hints" {
|
||||
for_each = var.node_server_group_policy != "" ? [openstack_compute_servergroup_v2.k8s_node[0]] : []
|
||||
content {
|
||||
group = openstack_compute_servergroup_v2.k8s_node[0].id
|
||||
}
|
||||
}
|
||||
|
||||
metadata = {
|
||||
ssh_user = var.ssh_user_gfs
|
||||
kubespray_groups = "gfs-cluster,network-storage,no_floating"
|
||||
depends_on = var.network_router_id
|
||||
use_access_ip = var.use_access_ip
|
||||
}
|
||||
}
|
||||
|
||||
resource "openstack_networking_floatingip_associate_v2" "bastion" {
|
||||
count = var.number_of_bastions
|
||||
floating_ip = var.bastion_fips[count.index]
|
||||
port_id = element(openstack_networking_port_v2.bastion_port.*.id, count.index)
|
||||
}
|
||||
|
||||
|
||||
resource "openstack_networking_floatingip_associate_v2" "k8s_master" {
|
||||
count = var.number_of_k8s_masters
|
||||
floating_ip = var.k8s_master_fips[count.index]
|
||||
port_id = element(openstack_networking_port_v2.k8s_master_port.*.id, count.index)
|
||||
}
|
||||
|
||||
resource "openstack_networking_floatingip_associate_v2" "k8s_masters" {
|
||||
for_each = var.number_of_k8s_masters == 0 && var.number_of_k8s_masters_no_etcd == 0 && var.number_of_k8s_masters_no_floating_ip == 0 && var.number_of_k8s_masters_no_floating_ip_no_etcd == 0 ? { for key, value in var.k8s_masters : key => value if value.floating_ip } : {}
|
||||
floating_ip = var.k8s_masters_fips[each.key].address
|
||||
port_id = openstack_networking_port_v2.k8s_masters_port[each.key].id
|
||||
}
|
||||
|
||||
resource "openstack_networking_floatingip_associate_v2" "k8s_master_no_etcd" {
|
||||
count = var.master_root_volume_size_in_gb == 0 ? var.number_of_k8s_masters_no_etcd : 0
|
||||
floating_ip = var.k8s_master_no_etcd_fips[count.index]
|
||||
port_id = element(openstack_networking_port_v2.k8s_master_no_etcd_port.*.id, count.index)
|
||||
}
|
||||
|
||||
resource "openstack_networking_floatingip_associate_v2" "k8s_node" {
|
||||
count = var.node_root_volume_size_in_gb == 0 ? var.number_of_k8s_nodes : 0
|
||||
floating_ip = var.k8s_node_fips[count.index]
|
||||
port_id = element(openstack_networking_port_v2.k8s_node_port.*.id, count.index)
|
||||
}
|
||||
|
||||
resource "openstack_networking_floatingip_associate_v2" "k8s_nodes" {
|
||||
for_each = var.number_of_k8s_nodes == 0 && var.number_of_k8s_nodes_no_floating_ip == 0 ? { for key, value in var.k8s_nodes : key => value if value.floating_ip } : {}
|
||||
floating_ip = var.k8s_nodes_fips[each.key].address
|
||||
port_id = openstack_networking_port_v2.k8s_nodes_port[each.key].id
|
||||
}
|
||||
|
||||
resource "openstack_blockstorage_volume_v2" "glusterfs_volume" {
|
||||
name = "${var.cluster_name}-glusterfs_volume-${count.index + 1}"
|
||||
count = var.gfs_root_volume_size_in_gb == 0 ? var.number_of_gfs_nodes_no_floating_ip : 0
|
||||
description = "Non-ephemeral volume for GlusterFS"
|
||||
size = var.gfs_volume_size_in_gb
|
||||
}
|
||||
|
||||
resource "openstack_compute_volume_attach_v2" "glusterfs_volume" {
|
||||
count = var.gfs_root_volume_size_in_gb == 0 ? var.number_of_gfs_nodes_no_floating_ip : 0
|
||||
instance_id = element(openstack_compute_instance_v2.glusterfs_node_no_floating_ip.*.id, count.index)
|
||||
volume_id = element(openstack_blockstorage_volume_v2.glusterfs_volume.*.id, count.index)
|
||||
}
|
||||
@@ -0,0 +1,17 @@
|
||||
# yamllint disable rule:comments
|
||||
#cloud-config
|
||||
## in some cases novnc console access is required
|
||||
## it requires ssh password to be set
|
||||
#ssh_pwauth: yes
|
||||
#chpasswd:
|
||||
# list: |
|
||||
# root:secret
|
||||
# expire: False
|
||||
|
||||
## in some cases direct root ssh access via ssh key is required
|
||||
#disable_root: false
|
||||
|
||||
## in some cases additional CA certs are required
|
||||
#ca-certs:
|
||||
# trusted: |
|
||||
# -----BEGIN CERTIFICATE-----
|
||||
@@ -0,0 +1,195 @@
|
||||
variable "cluster_name" {}
|
||||
|
||||
variable "az_list" {
|
||||
type = list(string)
|
||||
}
|
||||
|
||||
variable "az_list_node" {
|
||||
type = list(string)
|
||||
}
|
||||
|
||||
variable "number_of_k8s_masters" {}
|
||||
|
||||
variable "number_of_k8s_masters_no_etcd" {}
|
||||
|
||||
variable "number_of_etcd" {}
|
||||
|
||||
variable "number_of_k8s_masters_no_floating_ip" {}
|
||||
|
||||
variable "number_of_k8s_masters_no_floating_ip_no_etcd" {}
|
||||
|
||||
variable "number_of_k8s_nodes" {}
|
||||
|
||||
variable "number_of_k8s_nodes_no_floating_ip" {}
|
||||
|
||||
variable "number_of_bastions" {}
|
||||
|
||||
variable "number_of_gfs_nodes_no_floating_ip" {}
|
||||
|
||||
variable "bastion_root_volume_size_in_gb" {}
|
||||
|
||||
variable "etcd_root_volume_size_in_gb" {}
|
||||
|
||||
variable "master_root_volume_size_in_gb" {}
|
||||
|
||||
variable "node_root_volume_size_in_gb" {}
|
||||
|
||||
variable "gfs_root_volume_size_in_gb" {}
|
||||
|
||||
variable "gfs_volume_size_in_gb" {}
|
||||
|
||||
variable "master_volume_type" {}
|
||||
|
||||
variable "node_volume_type" {}
|
||||
|
||||
variable "public_key_path" {}
|
||||
|
||||
variable "image" {}
|
||||
|
||||
variable "image_gfs" {}
|
||||
|
||||
variable "ssh_user" {}
|
||||
|
||||
variable "ssh_user_gfs" {}
|
||||
|
||||
variable "flavor_k8s_master" {}
|
||||
|
||||
variable "flavor_k8s_node" {}
|
||||
|
||||
variable "flavor_etcd" {}
|
||||
|
||||
variable "flavor_gfs_node" {}
|
||||
|
||||
variable "network_name" {}
|
||||
|
||||
variable "flavor_bastion" {}
|
||||
|
||||
variable "network_id" {
|
||||
default = ""
|
||||
}
|
||||
|
||||
variable "use_existing_network" {
|
||||
type = bool
|
||||
}
|
||||
|
||||
variable "network_router_id" {
|
||||
default = ""
|
||||
}
|
||||
|
||||
variable "k8s_master_fips" {
|
||||
type = list
|
||||
}
|
||||
|
||||
variable "k8s_master_no_etcd_fips" {
|
||||
type = list
|
||||
}
|
||||
|
||||
variable "k8s_node_fips" {
|
||||
type = list
|
||||
}
|
||||
|
||||
variable "k8s_masters_fips" {
|
||||
type = map
|
||||
}
|
||||
|
||||
variable "k8s_nodes_fips" {
|
||||
type = map
|
||||
}
|
||||
|
||||
variable "bastion_fips" {
|
||||
type = list
|
||||
}
|
||||
|
||||
variable "bastion_allowed_remote_ips" {
|
||||
type = list
|
||||
}
|
||||
|
||||
variable "master_allowed_remote_ips" {
|
||||
type = list
|
||||
}
|
||||
|
||||
variable "k8s_allowed_remote_ips" {
|
||||
type = list
|
||||
}
|
||||
|
||||
variable "k8s_allowed_egress_ips" {
|
||||
type = list
|
||||
}
|
||||
|
||||
variable "k8s_masters" {}
|
||||
|
||||
variable "k8s_nodes" {}
|
||||
|
||||
variable "supplementary_master_groups" {
|
||||
default = ""
|
||||
}
|
||||
|
||||
variable "supplementary_node_groups" {
|
||||
default = ""
|
||||
}
|
||||
|
||||
variable "master_allowed_ports" {
|
||||
type = list
|
||||
}
|
||||
|
||||
variable "worker_allowed_ports" {
|
||||
type = list
|
||||
}
|
||||
|
||||
variable "bastion_allowed_ports" {
|
||||
type = list
|
||||
}
|
||||
|
||||
variable "use_access_ip" {}
|
||||
|
||||
variable "master_server_group_policy" {
|
||||
type = string
|
||||
}
|
||||
|
||||
variable "node_server_group_policy" {
|
||||
type = string
|
||||
}
|
||||
|
||||
variable "etcd_server_group_policy" {
|
||||
type = string
|
||||
}
|
||||
|
||||
variable "extra_sec_groups" {
|
||||
type = bool
|
||||
}
|
||||
|
||||
variable "extra_sec_groups_name" {
|
||||
type = string
|
||||
}
|
||||
|
||||
variable "image_uuid" {
|
||||
type = string
|
||||
}
|
||||
|
||||
variable "image_gfs_uuid" {
|
||||
type = string
|
||||
}
|
||||
|
||||
variable "image_master" {
|
||||
type = string
|
||||
}
|
||||
|
||||
variable "image_master_uuid" {
|
||||
type = string
|
||||
}
|
||||
|
||||
variable "group_vars_path" {
|
||||
type = string
|
||||
}
|
||||
|
||||
variable "port_security_enabled" {
|
||||
type = bool
|
||||
}
|
||||
|
||||
variable "force_null_port_security" {
|
||||
type = bool
|
||||
}
|
||||
|
||||
variable "private_subnet_id" {
|
||||
type = string
|
||||
}
|
||||
@@ -0,0 +1,8 @@
|
||||
terraform {
|
||||
required_providers {
|
||||
openstack = {
|
||||
source = "terraform-provider-openstack/openstack"
|
||||
}
|
||||
}
|
||||
required_version = ">= 0.12.26"
|
||||
}
|
||||
47
kubespray/contrib/terraform/openstack/modules/ips/main.tf
Normal file
47
kubespray/contrib/terraform/openstack/modules/ips/main.tf
Normal file
@@ -0,0 +1,47 @@
|
||||
resource "null_resource" "dummy_dependency" {
|
||||
triggers = {
|
||||
dependency_id = var.router_id
|
||||
}
|
||||
depends_on = [
|
||||
var.router_internal_port_id
|
||||
]
|
||||
}
|
||||
|
||||
# If user specifies pre-existing IPs to use in k8s_master_fips, do not create new ones.
|
||||
resource "openstack_networking_floatingip_v2" "k8s_master" {
|
||||
count = length(var.k8s_master_fips) > 0 ? 0 : var.number_of_k8s_masters
|
||||
pool = var.floatingip_pool
|
||||
depends_on = [null_resource.dummy_dependency]
|
||||
}
|
||||
|
||||
resource "openstack_networking_floatingip_v2" "k8s_masters" {
|
||||
for_each = var.number_of_k8s_masters == 0 && var.number_of_k8s_masters_no_etcd == 0 ? { for key, value in var.k8s_masters : key => value if value.floating_ip } : {}
|
||||
pool = var.floatingip_pool
|
||||
depends_on = [null_resource.dummy_dependency]
|
||||
}
|
||||
|
||||
# If user specifies pre-existing IPs to use in k8s_master_fips, do not create new ones.
|
||||
resource "openstack_networking_floatingip_v2" "k8s_master_no_etcd" {
|
||||
count = length(var.k8s_master_fips) > 0 ? 0 : var.number_of_k8s_masters_no_etcd
|
||||
pool = var.floatingip_pool
|
||||
depends_on = [null_resource.dummy_dependency]
|
||||
}
|
||||
|
||||
resource "openstack_networking_floatingip_v2" "k8s_node" {
|
||||
count = var.number_of_k8s_nodes
|
||||
pool = var.floatingip_pool
|
||||
depends_on = [null_resource.dummy_dependency]
|
||||
}
|
||||
|
||||
resource "openstack_networking_floatingip_v2" "bastion" {
|
||||
count = length(var.bastion_fips) > 0 ? 0 : var.number_of_bastions
|
||||
pool = var.floatingip_pool
|
||||
depends_on = [null_resource.dummy_dependency]
|
||||
}
|
||||
|
||||
resource "openstack_networking_floatingip_v2" "k8s_nodes" {
|
||||
for_each = var.number_of_k8s_nodes == 0 ? { for key, value in var.k8s_nodes : key => value if value.floating_ip } : {}
|
||||
pool = var.floatingip_pool
|
||||
depends_on = [null_resource.dummy_dependency]
|
||||
}
|
||||
|
||||
25
kubespray/contrib/terraform/openstack/modules/ips/outputs.tf
Normal file
25
kubespray/contrib/terraform/openstack/modules/ips/outputs.tf
Normal file
@@ -0,0 +1,25 @@
|
||||
# If k8s_master_fips is already defined as input, keep the same value since new FIPs have not been created.
|
||||
output "k8s_master_fips" {
|
||||
value = length(var.k8s_master_fips) > 0 ? var.k8s_master_fips : openstack_networking_floatingip_v2.k8s_master[*].address
|
||||
}
|
||||
|
||||
output "k8s_masters_fips" {
|
||||
value = openstack_networking_floatingip_v2.k8s_masters
|
||||
}
|
||||
|
||||
# If k8s_master_fips is already defined as input, keep the same value since new FIPs have not been created.
|
||||
output "k8s_master_no_etcd_fips" {
|
||||
value = length(var.k8s_master_fips) > 0 ? var.k8s_master_fips : openstack_networking_floatingip_v2.k8s_master_no_etcd[*].address
|
||||
}
|
||||
|
||||
output "k8s_node_fips" {
|
||||
value = openstack_networking_floatingip_v2.k8s_node[*].address
|
||||
}
|
||||
|
||||
output "k8s_nodes_fips" {
|
||||
value = openstack_networking_floatingip_v2.k8s_nodes
|
||||
}
|
||||
|
||||
output "bastion_fips" {
|
||||
value = length(var.bastion_fips) > 0 ? var.bastion_fips : openstack_networking_floatingip_v2.bastion[*].address
|
||||
}
|
||||
@@ -0,0 +1,27 @@
|
||||
variable "number_of_k8s_masters" {}
|
||||
|
||||
variable "number_of_k8s_masters_no_etcd" {}
|
||||
|
||||
variable "number_of_k8s_nodes" {}
|
||||
|
||||
variable "floatingip_pool" {}
|
||||
|
||||
variable "number_of_bastions" {}
|
||||
|
||||
variable "external_net" {}
|
||||
|
||||
variable "network_name" {}
|
||||
|
||||
variable "router_id" {
|
||||
default = ""
|
||||
}
|
||||
|
||||
variable "k8s_masters" {}
|
||||
|
||||
variable "k8s_nodes" {}
|
||||
|
||||
variable "k8s_master_fips" {}
|
||||
|
||||
variable "bastion_fips" {}
|
||||
|
||||
variable "router_internal_port_id" {}
|
||||
@@ -0,0 +1,11 @@
|
||||
terraform {
|
||||
required_providers {
|
||||
null = {
|
||||
source = "hashicorp/null"
|
||||
}
|
||||
openstack = {
|
||||
source = "terraform-provider-openstack/openstack"
|
||||
}
|
||||
}
|
||||
required_version = ">= 0.12.26"
|
||||
}
|
||||
@@ -0,0 +1,34 @@
|
||||
resource "openstack_networking_router_v2" "k8s" {
|
||||
name = "${var.cluster_name}-router"
|
||||
count = var.use_neutron == 1 && var.router_id == null ? 1 : 0
|
||||
admin_state_up = "true"
|
||||
external_network_id = var.external_net
|
||||
}
|
||||
|
||||
data "openstack_networking_router_v2" "k8s" {
|
||||
router_id = var.router_id
|
||||
count = var.use_neutron == 1 && var.router_id != null ? 1 : 0
|
||||
}
|
||||
|
||||
resource "openstack_networking_network_v2" "k8s" {
|
||||
name = var.network_name
|
||||
count = var.use_neutron
|
||||
dns_domain = var.network_dns_domain != null ? var.network_dns_domain : null
|
||||
admin_state_up = "true"
|
||||
port_security_enabled = var.port_security_enabled
|
||||
}
|
||||
|
||||
resource "openstack_networking_subnet_v2" "k8s" {
|
||||
name = "${var.cluster_name}-internal-network"
|
||||
count = var.use_neutron
|
||||
network_id = openstack_networking_network_v2.k8s[count.index].id
|
||||
cidr = var.subnet_cidr
|
||||
ip_version = 4
|
||||
dns_nameservers = var.dns_nameservers
|
||||
}
|
||||
|
||||
resource "openstack_networking_router_interface_v2" "k8s" {
|
||||
count = var.use_neutron
|
||||
router_id = "%{if openstack_networking_router_v2.k8s != []}${openstack_networking_router_v2.k8s[count.index].id}%{else}${var.router_id}%{endif}"
|
||||
subnet_id = openstack_networking_subnet_v2.k8s[count.index].id
|
||||
}
|
||||
@@ -0,0 +1,15 @@
|
||||
output "router_id" {
|
||||
value = "%{if var.use_neutron == 1} ${var.router_id == null ? element(concat(openstack_networking_router_v2.k8s.*.id, [""]), 0) : var.router_id} %{else} %{endif}"
|
||||
}
|
||||
|
||||
output "network_id" {
|
||||
value = element(concat(openstack_networking_network_v2.k8s.*.id, [""]),0)
|
||||
}
|
||||
|
||||
output "router_internal_port_id" {
|
||||
value = element(concat(openstack_networking_router_interface_v2.k8s.*.id, [""]), 0)
|
||||
}
|
||||
|
||||
output "subnet_id" {
|
||||
value = element(concat(openstack_networking_subnet_v2.k8s.*.id, [""]), 0)
|
||||
}
|
||||
@@ -0,0 +1,21 @@
|
||||
variable "external_net" {}
|
||||
|
||||
variable "network_name" {}
|
||||
|
||||
variable "network_dns_domain" {}
|
||||
|
||||
variable "cluster_name" {}
|
||||
|
||||
variable "dns_nameservers" {
|
||||
type = list
|
||||
}
|
||||
|
||||
variable "port_security_enabled" {
|
||||
type = bool
|
||||
}
|
||||
|
||||
variable "subnet_cidr" {}
|
||||
|
||||
variable "use_neutron" {}
|
||||
|
||||
variable "router_id" {}
|
||||
@@ -0,0 +1,8 @@
|
||||
terraform {
|
||||
required_providers {
|
||||
openstack = {
|
||||
source = "terraform-provider-openstack/openstack"
|
||||
}
|
||||
}
|
||||
required_version = ">= 0.12.26"
|
||||
}
|
||||
@@ -0,0 +1,89 @@
|
||||
# your Kubernetes cluster name here
|
||||
cluster_name = "i-didnt-read-the-docs"
|
||||
|
||||
# list of availability zones available in your OpenStack cluster
|
||||
#az_list = ["nova"]
|
||||
|
||||
# SSH key to use for access to nodes
|
||||
public_key_path = "~/.ssh/id_rsa.pub"
|
||||
|
||||
# image to use for bastion, masters, standalone etcd instances, and nodes
|
||||
image = "<image name>"
|
||||
|
||||
# user on the node (ex. core on Container Linux, ubuntu on Ubuntu, etc.)
|
||||
ssh_user = "<cloud-provisioned user>"
|
||||
|
||||
# 0|1 bastion nodes
|
||||
number_of_bastions = 0
|
||||
|
||||
#flavor_bastion = "<UUID>"
|
||||
|
||||
# standalone etcds
|
||||
number_of_etcd = 0
|
||||
|
||||
# masters
|
||||
number_of_k8s_masters = 1
|
||||
|
||||
number_of_k8s_masters_no_etcd = 0
|
||||
|
||||
number_of_k8s_masters_no_floating_ip = 0
|
||||
|
||||
number_of_k8s_masters_no_floating_ip_no_etcd = 0
|
||||
|
||||
flavor_k8s_master = "<UUID>"
|
||||
|
||||
k8s_masters = {
|
||||
# "master-1" = {
|
||||
# "az" = "nova"
|
||||
# "flavor" = "<UUID>"
|
||||
# "floating_ip" = true
|
||||
# "etcd" = true
|
||||
# },
|
||||
# "master-2" = {
|
||||
# "az" = "nova"
|
||||
# "flavor" = "<UUID>"
|
||||
# "floating_ip" = false
|
||||
# "etcd" = true
|
||||
# },
|
||||
# "master-3" = {
|
||||
# "az" = "nova"
|
||||
# "flavor" = "<UUID>"
|
||||
# "floating_ip" = true
|
||||
# "etcd" = true
|
||||
# },
|
||||
}
|
||||
|
||||
|
||||
# nodes
|
||||
number_of_k8s_nodes = 2
|
||||
|
||||
number_of_k8s_nodes_no_floating_ip = 4
|
||||
|
||||
#flavor_k8s_node = "<UUID>"
|
||||
|
||||
# GlusterFS
|
||||
# either 0 or more than one
|
||||
#number_of_gfs_nodes_no_floating_ip = 0
|
||||
#gfs_volume_size_in_gb = 150
|
||||
# Container Linux does not support GlusterFS
|
||||
#image_gfs = "<image name>"
|
||||
# May be different from other nodes
|
||||
#ssh_user_gfs = "ubuntu"
|
||||
#flavor_gfs_node = "<UUID>"
|
||||
|
||||
# networking
|
||||
network_name = "<network>"
|
||||
|
||||
# Use a existing network with the name of network_name. Set to false to create a network with name of network_name.
|
||||
# use_existing_network = true
|
||||
|
||||
external_net = "<UUID>"
|
||||
|
||||
subnet_cidr = "<cidr>"
|
||||
|
||||
floatingip_pool = "<pool>"
|
||||
|
||||
bastion_allowed_remote_ips = ["0.0.0.0/0"]
|
||||
|
||||
# Force port security to be null. Some cloud providers do not allow to set port security.
|
||||
# force_null_port_security = false
|
||||
@@ -0,0 +1,140 @@
|
||||
---
|
||||
## Directory where the binaries will be installed
|
||||
bin_dir: /usr/local/bin
|
||||
|
||||
## The access_ip variable is used to define how other nodes should access
|
||||
## the node. This is used in flannel to allow other flannel nodes to see
|
||||
## this node for example. The access_ip is really useful AWS and Google
|
||||
## environments where the nodes are accessed remotely by the "public" ip,
|
||||
## but don't know about that address themselves.
|
||||
# access_ip: 1.1.1.1
|
||||
|
||||
|
||||
## External LB example config
|
||||
## apiserver_loadbalancer_domain_name: "elb.some.domain"
|
||||
# loadbalancer_apiserver:
|
||||
# address: 1.2.3.4
|
||||
# port: 1234
|
||||
|
||||
## Internal loadbalancers for apiservers
|
||||
# loadbalancer_apiserver_localhost: true
|
||||
# valid options are "nginx" or "haproxy"
|
||||
# loadbalancer_apiserver_type: nginx # valid values "nginx" or "haproxy"
|
||||
|
||||
## If the cilium is going to be used in strict mode, we can use the
|
||||
## localhost connection and not use the external LB. If this parameter is
|
||||
## not specified, the first node to connect to kubeapi will be used.
|
||||
# use_localhost_as_kubeapi_loadbalancer: true
|
||||
|
||||
## Local loadbalancer should use this port
|
||||
## And must be set port 6443
|
||||
loadbalancer_apiserver_port: 6443
|
||||
|
||||
## If loadbalancer_apiserver_healthcheck_port variable defined, enables proxy liveness check for nginx.
|
||||
loadbalancer_apiserver_healthcheck_port: 8081
|
||||
|
||||
### OTHER OPTIONAL VARIABLES
|
||||
|
||||
## By default, Kubespray collects nameservers on the host. It then adds the previously collected nameservers in nameserverentries.
|
||||
## If true, Kubespray does not include host nameservers in nameserverentries in dns_late stage. However, It uses the nameserver to make sure cluster installed safely in dns_early stage.
|
||||
## Use this option with caution, you may need to define your dns servers. Otherwise, the outbound queries such as www.google.com may fail.
|
||||
# disable_host_nameservers: false
|
||||
|
||||
## Upstream dns servers
|
||||
# upstream_dns_servers:
|
||||
# - 8.8.8.8
|
||||
# - 8.8.4.4
|
||||
|
||||
## There are some changes specific to the cloud providers
|
||||
## for instance we need to encapsulate packets with some network plugins
|
||||
## If set the possible values are either 'gce', 'aws', 'azure', 'openstack', 'vsphere', 'oci', or 'external'
|
||||
## When openstack is used make sure to source in the openstack credentials
|
||||
## like you would do when using openstack-client before starting the playbook.
|
||||
# cloud_provider:
|
||||
|
||||
## When cloud_provider is set to 'external', you can set the cloud controller to deploy
|
||||
## Supported cloud controllers are: 'openstack', 'vsphere' and 'hcloud'
|
||||
## When openstack or vsphere are used make sure to source in the required fields
|
||||
# external_cloud_provider:
|
||||
|
||||
## Set these proxy values in order to update package manager and docker daemon to use proxies
|
||||
# http_proxy: ""
|
||||
# https_proxy: ""
|
||||
|
||||
## Refer to roles/kubespray-defaults/defaults/main.yml before modifying no_proxy
|
||||
# no_proxy: ""
|
||||
|
||||
## Some problems may occur when downloading files over https proxy due to ansible bug
|
||||
## https://github.com/ansible/ansible/issues/32750. Set this variable to False to disable
|
||||
## SSL validation of get_url module. Note that kubespray will still be performing checksum validation.
|
||||
# download_validate_certs: False
|
||||
|
||||
## If you need exclude all cluster nodes from proxy and other resources, add other resources here.
|
||||
# additional_no_proxy: ""
|
||||
|
||||
## If you need to disable proxying of os package repositories but are still behind an http_proxy set
|
||||
## skip_http_proxy_on_os_packages to true
|
||||
## This will cause kubespray not to set proxy environment in /etc/yum.conf for centos and in /etc/apt/apt.conf for debian/ubuntu
|
||||
## Special information for debian/ubuntu - you have to set the no_proxy variable, then apt package will install from your source of wish
|
||||
# skip_http_proxy_on_os_packages: false
|
||||
|
||||
## Since workers are included in the no_proxy variable by default, docker engine will be restarted on all nodes (all
|
||||
## pods will restart) when adding or removing workers. To override this behaviour by only including master nodes in the
|
||||
## no_proxy variable, set below to true:
|
||||
no_proxy_exclude_workers: false
|
||||
|
||||
## Certificate Management
|
||||
## This setting determines whether certs are generated via scripts.
|
||||
## Chose 'none' if you provide your own certificates.
|
||||
## Option is "script", "none"
|
||||
# cert_management: script
|
||||
|
||||
## Set to true to allow pre-checks to fail and continue deployment
|
||||
# ignore_assert_errors: false
|
||||
|
||||
## The read-only port for the Kubelet to serve on with no authentication/authorization. Uncomment to enable.
|
||||
# kube_read_only_port: 10255
|
||||
|
||||
## Set true to download and cache container
|
||||
# download_container: true
|
||||
|
||||
## Deploy container engine
|
||||
# Set false if you want to deploy container engine manually.
|
||||
# deploy_container_engine: true
|
||||
|
||||
## Red Hat Enterprise Linux subscription registration
|
||||
## Add either RHEL subscription Username/Password or Organization ID/Activation Key combination
|
||||
## Update RHEL subscription purpose usage, role and SLA if necessary
|
||||
# rh_subscription_username: ""
|
||||
# rh_subscription_password: ""
|
||||
# rh_subscription_org_id: ""
|
||||
# rh_subscription_activation_key: ""
|
||||
# rh_subscription_usage: "Development"
|
||||
# rh_subscription_role: "Red Hat Enterprise Server"
|
||||
# rh_subscription_sla: "Self-Support"
|
||||
|
||||
## Check if access_ip responds to ping. Set false if your firewall blocks ICMP.
|
||||
# ping_access_ip: true
|
||||
|
||||
# sysctl_file_path to add sysctl conf to
|
||||
# sysctl_file_path: "/etc/sysctl.d/99-sysctl.conf"
|
||||
|
||||
## Variables for webhook token auth https://kubernetes.io/docs/reference/access-authn-authz/authentication/#webhook-token-authentication
|
||||
kube_webhook_token_auth: false
|
||||
kube_webhook_token_auth_url_skip_tls_verify: false
|
||||
# kube_webhook_token_auth_url: https://...
|
||||
## base64-encoded string of the webhook's CA certificate
|
||||
# kube_webhook_token_auth_ca_data: "LS0t..."
|
||||
|
||||
## NTP Settings
|
||||
# Start the ntpd or chrony service and enable it at system boot.
|
||||
ntp_enabled: false
|
||||
ntp_manage_config: false
|
||||
ntp_servers:
|
||||
- "0.pool.ntp.org iburst"
|
||||
- "1.pool.ntp.org iburst"
|
||||
- "2.pool.ntp.org iburst"
|
||||
- "3.pool.ntp.org iburst"
|
||||
|
||||
## Used to control no_log attribute
|
||||
unsafe_show_logs: false
|
||||
@@ -0,0 +1,9 @@
|
||||
## To use AWS EBS CSI Driver to provision volumes, uncomment the first value
|
||||
## and configure the parameters below
|
||||
# aws_ebs_csi_enabled: true
|
||||
# aws_ebs_csi_enable_volume_scheduling: true
|
||||
# aws_ebs_csi_enable_volume_snapshot: false
|
||||
# aws_ebs_csi_enable_volume_resizing: false
|
||||
# aws_ebs_csi_controller_replicas: 1
|
||||
# aws_ebs_csi_plugin_image_tag: latest
|
||||
# aws_ebs_csi_extra_volume_tags: "Owner=owner,Team=team,Environment=environment'
|
||||
@@ -0,0 +1,40 @@
|
||||
## When azure is used, you need to also set the following variables.
|
||||
## see docs/azure.md for details on how to get these values
|
||||
|
||||
# azure_cloud:
|
||||
# azure_tenant_id:
|
||||
# azure_subscription_id:
|
||||
# azure_aad_client_id:
|
||||
# azure_aad_client_secret:
|
||||
# azure_resource_group:
|
||||
# azure_location:
|
||||
# azure_subnet_name:
|
||||
# azure_security_group_name:
|
||||
# azure_security_group_resource_group:
|
||||
# azure_vnet_name:
|
||||
# azure_vnet_resource_group:
|
||||
# azure_route_table_name:
|
||||
# azure_route_table_resource_group:
|
||||
# supported values are 'standard' or 'vmss'
|
||||
# azure_vmtype: standard
|
||||
|
||||
## Azure Disk CSI credentials and parameters
|
||||
## see docs/azure-csi.md for details on how to get these values
|
||||
|
||||
# azure_csi_tenant_id:
|
||||
# azure_csi_subscription_id:
|
||||
# azure_csi_aad_client_id:
|
||||
# azure_csi_aad_client_secret:
|
||||
# azure_csi_location:
|
||||
# azure_csi_resource_group:
|
||||
# azure_csi_vnet_name:
|
||||
# azure_csi_vnet_resource_group:
|
||||
# azure_csi_subnet_name:
|
||||
# azure_csi_security_group_name:
|
||||
# azure_csi_use_instance_metadata:
|
||||
# azure_csi_tags: "Owner=owner,Team=team,Environment=environment'
|
||||
|
||||
## To enable Azure Disk CSI, uncomment below
|
||||
# azure_csi_enabled: true
|
||||
# azure_csi_controller_replicas: 1
|
||||
# azure_csi_plugin_image_tag: latest
|
||||
@@ -0,0 +1,50 @@
|
||||
---
|
||||
# Please see roles/container-engine/containerd/defaults/main.yml for more configuration options
|
||||
|
||||
# containerd_storage_dir: "/var/lib/containerd"
|
||||
# containerd_state_dir: "/run/containerd"
|
||||
# containerd_oom_score: 0
|
||||
|
||||
# containerd_default_runtime: "runc"
|
||||
# containerd_snapshotter: "native"
|
||||
|
||||
# containerd_runc_runtime:
|
||||
# name: runc
|
||||
# type: "io.containerd.runc.v2"
|
||||
# engine: ""
|
||||
# root: ""
|
||||
|
||||
# containerd_additional_runtimes:
|
||||
# Example for Kata Containers as additional runtime:
|
||||
# - name: kata
|
||||
# type: "io.containerd.kata.v2"
|
||||
# engine: ""
|
||||
# root: ""
|
||||
|
||||
# containerd_grpc_max_recv_message_size: 16777216
|
||||
# containerd_grpc_max_send_message_size: 16777216
|
||||
|
||||
# containerd_debug_level: "info"
|
||||
|
||||
# containerd_metrics_address: ""
|
||||
|
||||
# containerd_metrics_grpc_histogram: false
|
||||
|
||||
## An obvious use case is allowing insecure-registry access to self hosted registries.
|
||||
## Can be ipaddress and domain_name.
|
||||
## example define mirror.registry.io or 172.19.16.11:5000
|
||||
## set "name": "url". insecure url must be started http://
|
||||
## Port number is also needed if the default HTTPS port is not used.
|
||||
# containerd_insecure_registries:
|
||||
# "localhost": "http://127.0.0.1"
|
||||
# "172.19.16.11:5000": "http://172.19.16.11:5000"
|
||||
|
||||
# containerd_registries:
|
||||
# "docker.io": "https://registry-1.docker.io"
|
||||
|
||||
# containerd_max_container_log_line_size: -1
|
||||
|
||||
# containerd_registry_auth:
|
||||
# - registry: 10.0.0.2:5000
|
||||
# username: user
|
||||
# password: pass
|
||||
@@ -0,0 +1,2 @@
|
||||
## Does coreos need auto upgrade, default is true
|
||||
# coreos_auto_upgrade: true
|
||||
@@ -0,0 +1,6 @@
|
||||
# crio_insecure_registries:
|
||||
# - 10.0.0.2:5000
|
||||
# crio_registry_auth:
|
||||
# - registry: 10.0.0.2:5000
|
||||
# username: user
|
||||
# password: pass
|
||||
@@ -0,0 +1,59 @@
|
||||
---
|
||||
## Uncomment this if you want to force overlay/overlay2 as docker storage driver
|
||||
## Please note that overlay2 is only supported on newer kernels
|
||||
# docker_storage_options: -s overlay2
|
||||
|
||||
## Enable docker_container_storage_setup, it will configure devicemapper driver on Centos7 or RedHat7.
|
||||
docker_container_storage_setup: false
|
||||
|
||||
## It must be define a disk path for docker_container_storage_setup_devs.
|
||||
## Otherwise docker-storage-setup will be executed incorrectly.
|
||||
# docker_container_storage_setup_devs: /dev/vdb
|
||||
|
||||
## Uncomment this if you want to change the Docker Cgroup driver (native.cgroupdriver)
|
||||
## Valid options are systemd or cgroupfs, default is systemd
|
||||
# docker_cgroup_driver: systemd
|
||||
|
||||
## Only set this if you have more than 3 nameservers:
|
||||
## If true Kubespray will only use the first 3, otherwise it will fail
|
||||
docker_dns_servers_strict: false
|
||||
|
||||
# Path used to store Docker data
|
||||
docker_daemon_graph: "/var/lib/docker"
|
||||
|
||||
## Used to set docker daemon iptables options to true
|
||||
docker_iptables_enabled: "false"
|
||||
|
||||
# Docker log options
|
||||
# Rotate container stderr/stdout logs at 50m and keep last 5
|
||||
docker_log_opts: "--log-opt max-size=50m --log-opt max-file=5"
|
||||
|
||||
# define docker bin_dir
|
||||
docker_bin_dir: "/usr/bin"
|
||||
|
||||
# keep docker packages after installation; speeds up repeated ansible provisioning runs when '1'
|
||||
# kubespray deletes the docker package on each run, so caching the package makes sense
|
||||
docker_rpm_keepcache: 1
|
||||
|
||||
## An obvious use case is allowing insecure-registry access to self hosted registries.
|
||||
## Can be ipaddress and domain_name.
|
||||
## example define 172.19.16.11 or mirror.registry.io
|
||||
# docker_insecure_registries:
|
||||
# - mirror.registry.io
|
||||
# - 172.19.16.11
|
||||
|
||||
## Add other registry,example China registry mirror.
|
||||
# docker_registry_mirrors:
|
||||
# - https://registry.docker-cn.com
|
||||
# - https://mirror.aliyuncs.com
|
||||
|
||||
## If non-empty will override default system MountFlags value.
|
||||
## This option takes a mount propagation flag: shared, slave
|
||||
## or private, which control whether mounts in the file system
|
||||
## namespace set up for docker will receive or propagate mounts
|
||||
## and unmounts. Leave empty for system default
|
||||
# docker_mount_flags:
|
||||
|
||||
## A string of extra options to pass to the docker daemon.
|
||||
## This string should be exactly as you wish it to appear.
|
||||
# docker_options: ""
|
||||
@@ -0,0 +1,16 @@
|
||||
---
|
||||
## Directory where etcd data stored
|
||||
etcd_data_dir: /var/lib/etcd
|
||||
|
||||
## Container runtime
|
||||
## docker for docker, crio for cri-o and containerd for containerd.
|
||||
## Additionally you can set this to kubeadm if you want to install etcd using kubeadm
|
||||
## Kubeadm etcd deployment is experimental and only available for new deployments
|
||||
## If this is not set, container manager will be inherited from the Kubespray defaults
|
||||
## and not from k8s_cluster/k8s-cluster.yml, which might not be what you want.
|
||||
## Also this makes possible to use different container manager for etcd nodes.
|
||||
# container_manager: containerd
|
||||
|
||||
## Settings for etcd deployment type
|
||||
# Set this to docker if you are using container_manager: docker
|
||||
etcd_deployment_type: host
|
||||
@@ -0,0 +1,10 @@
|
||||
## GCP compute Persistent Disk CSI Driver credentials and parameters
|
||||
## See docs/gcp-pd-csi.md for information about the implementation
|
||||
|
||||
## Specify the path to the file containing the service account credentials
|
||||
# gcp_pd_csi_sa_cred_file: "/my/safe/credentials/directory/cloud-sa.json"
|
||||
|
||||
## To enable GCP Persistent Disk CSI driver, uncomment below
|
||||
# gcp_pd_csi_enabled: true
|
||||
# gcp_pd_csi_controller_replicas: 1
|
||||
# gcp_pd_csi_driver_image_tag: "v0.7.0-gke.0"
|
||||
@@ -0,0 +1,14 @@
|
||||
## Values for the external Hcloud Cloud Controller
|
||||
# external_hcloud_cloud:
|
||||
# hcloud_api_token: ""
|
||||
# token_secret_name: hcloud
|
||||
# with_networks: false # Use the hcloud controller-manager with networks support https://github.com/hetznercloud/hcloud-cloud-controller-manager#networks-support
|
||||
# service_account_name: cloud-controller-manager
|
||||
#
|
||||
# controller_image_tag: "latest"
|
||||
# ## A dictionary of extra arguments to add to the openstack cloud controller manager daemonset
|
||||
# ## Format:
|
||||
# ## external_hcloud_cloud.controller_extra_args:
|
||||
# ## arg1: "value1"
|
||||
# ## arg2: "value2"
|
||||
# controller_extra_args: {}
|
||||
@@ -0,0 +1,28 @@
|
||||
## When Oracle Cloud Infrastructure is used, set these variables
|
||||
# oci_private_key:
|
||||
# oci_region_id:
|
||||
# oci_tenancy_id:
|
||||
# oci_user_id:
|
||||
# oci_user_fingerprint:
|
||||
# oci_compartment_id:
|
||||
# oci_vnc_id:
|
||||
# oci_subnet1_id:
|
||||
# oci_subnet2_id:
|
||||
## Override these default/optional behaviors if you wish
|
||||
# oci_security_list_management: All
|
||||
## If you would like the controller to manage specific lists per subnet. This is a mapping of subnet ocids to security list ocids. Below are examples.
|
||||
# oci_security_lists:
|
||||
# ocid1.subnet.oc1.phx.aaaaaaaasa53hlkzk6nzksqfccegk2qnkxmphkblst3riclzs4rhwg7rg57q: ocid1.securitylist.oc1.iad.aaaaaaaaqti5jsfvyw6ejahh7r4okb2xbtuiuguswhs746mtahn72r7adt7q
|
||||
# ocid1.subnet.oc1.phx.aaaaaaaahuxrgvs65iwdz7ekwgg3l5gyah7ww5klkwjcso74u3e4i64hvtvq: ocid1.securitylist.oc1.iad.aaaaaaaaqti5jsfvyw6ejahh7r4okb2xbtuiuguswhs746mtahn72r7adt7q
|
||||
## If oci_use_instance_principals is true, you do not need to set the region, tenancy, user, key, passphrase, or fingerprint
|
||||
# oci_use_instance_principals: false
|
||||
# oci_cloud_controller_version: 0.6.0
|
||||
## If you would like to control OCI query rate limits for the controller
|
||||
# oci_rate_limit:
|
||||
# rate_limit_qps_read:
|
||||
# rate_limit_qps_write:
|
||||
# rate_limit_bucket_read:
|
||||
# rate_limit_bucket_write:
|
||||
## Other optional variables
|
||||
# oci_cloud_controller_pull_source: (default iad.ocir.io/oracle/cloud-provider-oci)
|
||||
# oci_cloud_controller_pull_secret: (name of pull secret to use if you define your own mirror above)
|
||||
@@ -0,0 +1,103 @@
|
||||
---
|
||||
## Global Offline settings
|
||||
### Private Container Image Registry
|
||||
# registry_host: "myprivateregisry.com"
|
||||
# files_repo: "http://myprivatehttpd"
|
||||
### If using CentOS, RedHat, AlmaLinux or Fedora
|
||||
# yum_repo: "http://myinternalyumrepo"
|
||||
### If using Debian
|
||||
# debian_repo: "http://myinternaldebianrepo"
|
||||
### If using Ubuntu
|
||||
# ubuntu_repo: "http://myinternalubunturepo"
|
||||
|
||||
## Container Registry overrides
|
||||
# kube_image_repo: "{{ registry_host }}"
|
||||
# gcr_image_repo: "{{ registry_host }}"
|
||||
# github_image_repo: "{{ registry_host }}"
|
||||
# docker_image_repo: "{{ registry_host }}"
|
||||
# quay_image_repo: "{{ registry_host }}"
|
||||
|
||||
## Kubernetes components
|
||||
# kubeadm_download_url: "{{ files_repo }}/storage.googleapis.com/kubernetes-release/release/{{ kubeadm_version }}/bin/linux/{{ image_arch }}/kubeadm"
|
||||
# kubectl_download_url: "{{ files_repo }}/storage.googleapis.com/kubernetes-release/release/{{ kube_version }}/bin/linux/{{ image_arch }}/kubectl"
|
||||
# kubelet_download_url: "{{ files_repo }}/storage.googleapis.com/kubernetes-release/release/{{ kube_version }}/bin/linux/{{ image_arch }}/kubelet"
|
||||
|
||||
## CNI Plugins
|
||||
# cni_download_url: "{{ files_repo }}/github.com/containernetworking/plugins/releases/download/{{ cni_version }}/cni-plugins-linux-{{ image_arch }}-{{ cni_version }}.tgz"
|
||||
|
||||
## cri-tools
|
||||
# crictl_download_url: "{{ files_repo }}/github.com/kubernetes-sigs/cri-tools/releases/download/{{ crictl_version }}/crictl-{{ crictl_version }}-{{ ansible_system | lower }}-{{ image_arch }}.tar.gz"
|
||||
|
||||
## [Optional] etcd: only if you **DON'T** use etcd_deployment=host
|
||||
# etcd_download_url: "{{ files_repo }}/github.com/etcd-io/etcd/releases/download/{{ etcd_version }}/etcd-{{ etcd_version }}-linux-{{ image_arch }}.tar.gz"
|
||||
|
||||
# [Optional] Calico: If using Calico network plugin
|
||||
# calicoctl_download_url: "{{ files_repo }}/github.com/projectcalico/calico/releases/download/{{ calico_ctl_version }}/calicoctl-linux-{{ image_arch }}"
|
||||
# calicoctl_alternate_download_url: "{{ files_repo }}/github.com/projectcalico/calicoctl/releases/download/{{ calico_ctl_version }}/calicoctl-linux-{{ image_arch }}"
|
||||
# [Optional] Calico with kdd: If using Calico network plugin with kdd datastore
|
||||
# calico_crds_download_url: "{{ files_repo }}/github.com/projectcalico/calico/archive/{{ calico_version }}.tar.gz"
|
||||
|
||||
# [Optional] Cilium: If using Cilium network plugin
|
||||
# ciliumcli_download_url: "{{ files_repo }}/github.com/cilium/cilium-cli/releases/download/{{ cilium_cli_version }}/cilium-linux-{{ image_arch }}.tar.gz"
|
||||
|
||||
# [Optional] Flannel: If using Falnnel network plugin
|
||||
# flannel_cni_download_url: "{{ files_repo }}/kubernetes/flannel/{{ flannel_cni_version }}/flannel-{{ image_arch }}"
|
||||
|
||||
# [Optional] helm: only if you set helm_enabled: true
|
||||
# helm_download_url: "{{ files_repo }}/get.helm.sh/helm-{{ helm_version }}-linux-{{ image_arch }}.tar.gz"
|
||||
|
||||
# [Optional] crun: only if you set crun_enabled: true
|
||||
# crun_download_url: "{{ files_repo }}/github.com/containers/crun/releases/download/{{ crun_version }}/crun-{{ crun_version }}-linux-{{ image_arch }}"
|
||||
|
||||
# [Optional] kata: only if you set kata_containers_enabled: true
|
||||
# kata_containers_download_url: "{{ files_repo }}/github.com/kata-containers/kata-containers/releases/download/{{ kata_containers_version }}/kata-static-{{ kata_containers_version }}-{{ ansible_architecture }}.tar.xz"
|
||||
|
||||
# [Optional] cri-dockerd: only if you set container_manager: docker
|
||||
# cri_dockerd_download_url: "{{ files_repo }}/github.com/Mirantis/cri-dockerd/releases/download/v{{ cri_dockerd_version }}/cri-dockerd-{{ cri_dockerd_version }}.{{ image_arch }}.tgz"
|
||||
|
||||
# [Optional] cri-o: only if you set container_manager: crio
|
||||
# crio_download_base: "download.opensuse.org/repositories/devel:kubic:libcontainers:stable"
|
||||
# crio_download_crio: "http://{{ crio_download_base }}:/cri-o:/"
|
||||
|
||||
# [Optional] runc,containerd: only if you set container_runtime: containerd
|
||||
# runc_download_url: "{{ files_repo }}/github.com/opencontainers/runc/releases/download/{{ runc_version }}/runc.{{ image_arch }}"
|
||||
# containerd_download_url: "{{ files_repo }}/github.com/containerd/containerd/releases/download/v{{ containerd_version }}/containerd-{{ containerd_version }}-linux-{{ image_arch }}.tar.gz"
|
||||
# nerdctl_download_url: "{{ files_repo }}/github.com/containerd/nerdctl/releases/download/v{{ nerdctl_version }}/nerdctl-{{ nerdctl_version }}-{{ ansible_system | lower }}-{{ image_arch }}.tar.gz"
|
||||
|
||||
# [Optional] runsc,containerd-shim-runsc: only if you set gvisor_enabled: true
|
||||
# gvisor_runsc_download_url: "{{ files_repo }}/storage.googleapis.com/gvisor/releases/release/{{ gvisor_version }}/{{ ansible_architecture }}/runsc"
|
||||
# gvisor_containerd_shim_runsc_download_url: "{{ files_repo }}/storage.googleapis.com/gvisor/releases/release/{{ gvisor_version }}/{{ ansible_architecture }}/containerd-shim-runsc-v1"
|
||||
|
||||
## CentOS/Redhat/AlmaLinux
|
||||
### For EL7, base and extras repo must be available, for EL8, baseos and appstream
|
||||
### By default we enable those repo automatically
|
||||
# rhel_enable_repos: false
|
||||
### Docker / Containerd
|
||||
# docker_rh_repo_base_url: "{{ yum_repo }}/docker-ce/$releasever/$basearch"
|
||||
# docker_rh_repo_gpgkey: "{{ yum_repo }}/docker-ce/gpg"
|
||||
|
||||
## Fedora
|
||||
### Docker
|
||||
# docker_fedora_repo_base_url: "{{ yum_repo }}/docker-ce/{{ ansible_distribution_major_version }}/{{ ansible_architecture }}"
|
||||
# docker_fedora_repo_gpgkey: "{{ yum_repo }}/docker-ce/gpg"
|
||||
### Containerd
|
||||
# containerd_fedora_repo_base_url: "{{ yum_repo }}/containerd"
|
||||
# containerd_fedora_repo_gpgkey: "{{ yum_repo }}/docker-ce/gpg"
|
||||
|
||||
## Debian
|
||||
### Docker
|
||||
# docker_debian_repo_base_url: "{{ debian_repo }}/docker-ce"
|
||||
# docker_debian_repo_gpgkey: "{{ debian_repo }}/docker-ce/gpg"
|
||||
### Containerd
|
||||
# containerd_debian_repo_base_url: "{{ debian_repo }}/containerd"
|
||||
# containerd_debian_repo_gpgkey: "{{ debian_repo }}/containerd/gpg"
|
||||
# containerd_debian_repo_repokey: 'YOURREPOKEY'
|
||||
|
||||
## Ubuntu
|
||||
### Docker
|
||||
# docker_ubuntu_repo_base_url: "{{ ubuntu_repo }}/docker-ce"
|
||||
# docker_ubuntu_repo_gpgkey: "{{ ubuntu_repo }}/docker-ce/gpg"
|
||||
### Containerd
|
||||
# containerd_ubuntu_repo_base_url: "{{ ubuntu_repo }}/containerd"
|
||||
# containerd_ubuntu_repo_gpgkey: "{{ ubuntu_repo }}/containerd/gpg"
|
||||
# containerd_ubuntu_repo_repokey: 'YOURREPOKEY'
|
||||
@@ -0,0 +1,49 @@
|
||||
## When OpenStack is used, Cinder version can be explicitly specified if autodetection fails (Fixed in 1.9: https://github.com/kubernetes/kubernetes/issues/50461)
|
||||
# openstack_blockstorage_version: "v1/v2/auto (default)"
|
||||
# openstack_blockstorage_ignore_volume_az: yes
|
||||
## When OpenStack is used, if LBaaSv2 is available you can enable it with the following 2 variables.
|
||||
# openstack_lbaas_enabled: True
|
||||
# openstack_lbaas_subnet_id: "Neutron subnet ID (not network ID) to create LBaaS VIP"
|
||||
## To enable automatic floating ip provisioning, specify a subnet.
|
||||
# openstack_lbaas_floating_network_id: "Neutron network ID (not subnet ID) to get floating IP from, disabled by default"
|
||||
## Override default LBaaS behavior
|
||||
# openstack_lbaas_use_octavia: False
|
||||
# openstack_lbaas_method: "ROUND_ROBIN"
|
||||
# openstack_lbaas_provider: "haproxy"
|
||||
# openstack_lbaas_create_monitor: "yes"
|
||||
# openstack_lbaas_monitor_delay: "1m"
|
||||
# openstack_lbaas_monitor_timeout: "30s"
|
||||
# openstack_lbaas_monitor_max_retries: "3"
|
||||
|
||||
## Values for the external OpenStack Cloud Controller
|
||||
# external_openstack_lbaas_network_id: "Neutron network ID to create LBaaS VIP"
|
||||
# external_openstack_lbaas_subnet_id: "Neutron subnet ID to create LBaaS VIP"
|
||||
# external_openstack_lbaas_floating_network_id: "Neutron network ID to get floating IP from"
|
||||
# external_openstack_lbaas_floating_subnet_id: "Neutron subnet ID to get floating IP from"
|
||||
# external_openstack_lbaas_method: "ROUND_ROBIN"
|
||||
# external_openstack_lbaas_provider: "octavia"
|
||||
# external_openstack_lbaas_create_monitor: false
|
||||
# external_openstack_lbaas_monitor_delay: "1m"
|
||||
# external_openstack_lbaas_monitor_timeout: "30s"
|
||||
# external_openstack_lbaas_monitor_max_retries: "3"
|
||||
# external_openstack_lbaas_manage_security_groups: false
|
||||
# external_openstack_lbaas_internal_lb: false
|
||||
# external_openstack_network_ipv6_disabled: false
|
||||
# external_openstack_network_internal_networks: []
|
||||
# external_openstack_network_public_networks: []
|
||||
# external_openstack_metadata_search_order: "configDrive,metadataService"
|
||||
|
||||
## Application credentials to authenticate against Keystone API
|
||||
## Those settings will take precedence over username and password that might be set your environment
|
||||
## All of them are required
|
||||
# external_openstack_application_credential_name:
|
||||
# external_openstack_application_credential_id:
|
||||
# external_openstack_application_credential_secret:
|
||||
|
||||
## The tag of the external OpenStack Cloud Controller image
|
||||
# external_openstack_cloud_controller_image_tag: "latest"
|
||||
|
||||
## To use Cinder CSI plugin to provision volumes set this value to true
|
||||
## Make sure to source in the openstack credentials
|
||||
# cinder_csi_enabled: true
|
||||
# cinder_csi_controller_replicas: 1
|
||||
@@ -0,0 +1,24 @@
|
||||
## Repo for UpClouds csi-driver: https://github.com/UpCloudLtd/upcloud-csi
|
||||
## To use UpClouds CSI plugin to provision volumes set this value to true
|
||||
## Remember to set UPCLOUD_USERNAME and UPCLOUD_PASSWORD
|
||||
# upcloud_csi_enabled: true
|
||||
# upcloud_csi_controller_replicas: 1
|
||||
## Override used image tags
|
||||
# upcloud_csi_provisioner_image_tag: "v3.1.0"
|
||||
# upcloud_csi_attacher_image_tag: "v3.4.0"
|
||||
# upcloud_csi_resizer_image_tag: "v1.4.0"
|
||||
# upcloud_csi_plugin_image_tag: "v0.3.3"
|
||||
# upcloud_csi_node_image_tag: "v2.5.0"
|
||||
# upcloud_tolerations: []
|
||||
## Storage class options
|
||||
# storage_classes:
|
||||
# - name: standard
|
||||
# is_default: true
|
||||
# expand_persistent_volumes: true
|
||||
# parameters:
|
||||
# tier: maxiops
|
||||
# - name: hdd
|
||||
# is_default: false
|
||||
# expand_persistent_volumes: true
|
||||
# parameters:
|
||||
# tier: hdd
|
||||
@@ -0,0 +1,32 @@
|
||||
## Values for the external vSphere Cloud Provider
|
||||
# external_vsphere_vcenter_ip: "myvcenter.domain.com"
|
||||
# external_vsphere_vcenter_port: "443"
|
||||
# external_vsphere_insecure: "true"
|
||||
# external_vsphere_user: "administrator@vsphere.local" # Can also be set via the `VSPHERE_USER` environment variable
|
||||
# external_vsphere_password: "K8s_admin" # Can also be set via the `VSPHERE_PASSWORD` environment variable
|
||||
# external_vsphere_datacenter: "DATACENTER_name"
|
||||
# external_vsphere_kubernetes_cluster_id: "kubernetes-cluster-id"
|
||||
|
||||
## Vsphere version where located VMs
|
||||
# external_vsphere_version: "6.7u3"
|
||||
|
||||
## Tags for the external vSphere Cloud Provider images
|
||||
## gcr.io/cloud-provider-vsphere/cpi/release/manager
|
||||
# external_vsphere_cloud_controller_image_tag: "latest"
|
||||
## gcr.io/cloud-provider-vsphere/csi/release/syncer
|
||||
# vsphere_syncer_image_tag: "v2.5.1"
|
||||
## registry.k8s.io/sig-storage/csi-attacher
|
||||
# vsphere_csi_attacher_image_tag: "v3.4.0"
|
||||
## gcr.io/cloud-provider-vsphere/csi/release/driver
|
||||
# vsphere_csi_controller: "v2.5.1"
|
||||
## registry.k8s.io/sig-storage/livenessprobe
|
||||
# vsphere_csi_liveness_probe_image_tag: "v2.6.0"
|
||||
## registry.k8s.io/sig-storage/csi-provisioner
|
||||
# vsphere_csi_provisioner_image_tag: "v3.1.0"
|
||||
## registry.k8s.io/sig-storage/csi-resizer
|
||||
## makes sense only for vSphere version >=7.0
|
||||
# vsphere_csi_resizer_tag: "v1.3.0"
|
||||
|
||||
## To use vSphere CSI plugin to provision volumes set this value to true
|
||||
# vsphere_csi_enabled: true
|
||||
# vsphere_csi_controller_replicas: 1
|
||||
@@ -0,0 +1,26 @@
|
||||
---
|
||||
## Etcd auto compaction retention for mvcc key value store in hour
|
||||
# etcd_compaction_retention: 0
|
||||
|
||||
## Set level of detail for etcd exported metrics, specify 'extensive' to include histogram metrics.
|
||||
# etcd_metrics: basic
|
||||
|
||||
## Etcd is restricted by default to 512M on systems under 4GB RAM, 512MB is not enough for much more than testing.
|
||||
## Set this if your etcd nodes have less than 4GB but you want more RAM for etcd. Set to 0 for unrestricted RAM.
|
||||
## This value is only relevant when deploying etcd with `etcd_deployment_type: docker`
|
||||
# etcd_memory_limit: "512M"
|
||||
|
||||
## Etcd has a default of 2G for its space quota. If you put a value in etcd_memory_limit which is less than
|
||||
## etcd_quota_backend_bytes, you may encounter out of memory terminations of the etcd cluster. Please check
|
||||
## etcd documentation for more information.
|
||||
# 8G is a suggested maximum size for normal environments and etcd warns at startup if the configured value exceeds it.
|
||||
# etcd_quota_backend_bytes: "2147483648"
|
||||
|
||||
# Maximum client request size in bytes the server will accept.
|
||||
# etcd is designed to handle small key value pairs typical for metadata.
|
||||
# Larger requests will work, but may increase the latency of other requests
|
||||
# etcd_max_request_bytes: "1572864"
|
||||
|
||||
### ETCD: disable peer client cert authentication.
|
||||
# This affects ETCD_PEER_CLIENT_CERT_AUTH variable
|
||||
# etcd_peer_client_auth: true
|
||||
@@ -0,0 +1,228 @@
|
||||
---
|
||||
# Kubernetes dashboard
|
||||
# RBAC required. see docs/getting-started.md for access details.
|
||||
# dashboard_enabled: false
|
||||
|
||||
# Helm deployment
|
||||
helm_enabled: false
|
||||
|
||||
# Registry deployment
|
||||
registry_enabled: false
|
||||
# registry_namespace: kube-system
|
||||
# registry_storage_class: ""
|
||||
# registry_disk_size: "10Gi"
|
||||
|
||||
# Metrics Server deployment
|
||||
metrics_server_enabled: false
|
||||
# metrics_server_container_port: 4443
|
||||
# metrics_server_kubelet_insecure_tls: true
|
||||
# metrics_server_metric_resolution: 15s
|
||||
# metrics_server_kubelet_preferred_address_types: "InternalIP,ExternalIP,Hostname"
|
||||
# metrics_server_host_network: false
|
||||
# metrics_server_replicas: 1
|
||||
|
||||
# Rancher Local Path Provisioner
|
||||
local_path_provisioner_enabled: false
|
||||
# local_path_provisioner_namespace: "local-path-storage"
|
||||
# local_path_provisioner_storage_class: "local-path"
|
||||
# local_path_provisioner_reclaim_policy: Delete
|
||||
# local_path_provisioner_claim_root: /opt/local-path-provisioner/
|
||||
# local_path_provisioner_debug: false
|
||||
# local_path_provisioner_image_repo: "rancher/local-path-provisioner"
|
||||
# local_path_provisioner_image_tag: "v0.0.22"
|
||||
# local_path_provisioner_helper_image_repo: "busybox"
|
||||
# local_path_provisioner_helper_image_tag: "latest"
|
||||
|
||||
# Local volume provisioner deployment
|
||||
local_volume_provisioner_enabled: false
|
||||
# local_volume_provisioner_namespace: kube-system
|
||||
# local_volume_provisioner_nodelabels:
|
||||
# - kubernetes.io/hostname
|
||||
# - topology.kubernetes.io/region
|
||||
# - topology.kubernetes.io/zone
|
||||
# local_volume_provisioner_storage_classes:
|
||||
# local-storage:
|
||||
# host_dir: /mnt/disks
|
||||
# mount_dir: /mnt/disks
|
||||
# volume_mode: Filesystem
|
||||
# fs_type: ext4
|
||||
# fast-disks:
|
||||
# host_dir: /mnt/fast-disks
|
||||
# mount_dir: /mnt/fast-disks
|
||||
# block_cleaner_command:
|
||||
# - "/scripts/shred.sh"
|
||||
# - "2"
|
||||
# volume_mode: Filesystem
|
||||
# fs_type: ext4
|
||||
# local_volume_provisioner_tolerations:
|
||||
# - effect: NoSchedule
|
||||
# operator: Exists
|
||||
|
||||
# CSI Volume Snapshot Controller deployment, set this to true if your CSI is able to manage snapshots
|
||||
# currently, setting cinder_csi_enabled=true would automatically enable the snapshot controller
|
||||
# Longhorn is an extenal CSI that would also require setting this to true but it is not included in kubespray
|
||||
# csi_snapshot_controller_enabled: false
|
||||
# csi snapshot namespace
|
||||
# snapshot_controller_namespace: kube-system
|
||||
|
||||
# CephFS provisioner deployment
|
||||
cephfs_provisioner_enabled: false
|
||||
# cephfs_provisioner_namespace: "cephfs-provisioner"
|
||||
# cephfs_provisioner_cluster: ceph
|
||||
# cephfs_provisioner_monitors: "172.24.0.1:6789,172.24.0.2:6789,172.24.0.3:6789"
|
||||
# cephfs_provisioner_admin_id: admin
|
||||
# cephfs_provisioner_secret: secret
|
||||
# cephfs_provisioner_storage_class: cephfs
|
||||
# cephfs_provisioner_reclaim_policy: Delete
|
||||
# cephfs_provisioner_claim_root: /volumes
|
||||
# cephfs_provisioner_deterministic_names: true
|
||||
|
||||
# RBD provisioner deployment
|
||||
rbd_provisioner_enabled: false
|
||||
# rbd_provisioner_namespace: rbd-provisioner
|
||||
# rbd_provisioner_replicas: 2
|
||||
# rbd_provisioner_monitors: "172.24.0.1:6789,172.24.0.2:6789,172.24.0.3:6789"
|
||||
# rbd_provisioner_pool: kube
|
||||
# rbd_provisioner_admin_id: admin
|
||||
# rbd_provisioner_secret_name: ceph-secret-admin
|
||||
# rbd_provisioner_secret: ceph-key-admin
|
||||
# rbd_provisioner_user_id: kube
|
||||
# rbd_provisioner_user_secret_name: ceph-secret-user
|
||||
# rbd_provisioner_user_secret: ceph-key-user
|
||||
# rbd_provisioner_user_secret_namespace: rbd-provisioner
|
||||
# rbd_provisioner_fs_type: ext4
|
||||
# rbd_provisioner_image_format: "2"
|
||||
# rbd_provisioner_image_features: layering
|
||||
# rbd_provisioner_storage_class: rbd
|
||||
# rbd_provisioner_reclaim_policy: Delete
|
||||
|
||||
# Nginx ingress controller deployment
|
||||
ingress_nginx_enabled: false
|
||||
# ingress_nginx_host_network: false
|
||||
ingress_publish_status_address: ""
|
||||
# ingress_nginx_nodeselector:
|
||||
# kubernetes.io/os: "linux"
|
||||
# ingress_nginx_tolerations:
|
||||
# - key: "node-role.kubernetes.io/master"
|
||||
# operator: "Equal"
|
||||
# value: ""
|
||||
# effect: "NoSchedule"
|
||||
# - key: "node-role.kubernetes.io/control-plane"
|
||||
# operator: "Equal"
|
||||
# value: ""
|
||||
# effect: "NoSchedule"
|
||||
# ingress_nginx_namespace: "ingress-nginx"
|
||||
# ingress_nginx_insecure_port: 80
|
||||
# ingress_nginx_secure_port: 443
|
||||
# ingress_nginx_configmap:
|
||||
# map-hash-bucket-size: "128"
|
||||
# ssl-protocols: "TLSv1.2 TLSv1.3"
|
||||
# ingress_nginx_configmap_tcp_services:
|
||||
# 9000: "default/example-go:8080"
|
||||
# ingress_nginx_configmap_udp_services:
|
||||
# 53: "kube-system/coredns:53"
|
||||
# ingress_nginx_extra_args:
|
||||
# - --default-ssl-certificate=default/foo-tls
|
||||
# ingress_nginx_termination_grace_period_seconds: 300
|
||||
# ingress_nginx_class: nginx
|
||||
|
||||
# ALB ingress controller deployment
|
||||
ingress_alb_enabled: false
|
||||
# alb_ingress_aws_region: "us-east-1"
|
||||
# alb_ingress_restrict_scheme: "false"
|
||||
# Enables logging on all outbound requests sent to the AWS API.
|
||||
# If logging is desired, set to true.
|
||||
# alb_ingress_aws_debug: "false"
|
||||
|
||||
# Cert manager deployment
|
||||
cert_manager_enabled: false
|
||||
# cert_manager_namespace: "cert-manager"
|
||||
# cert_manager_tolerations:
|
||||
# - key: node-role.kubernetes.io/master
|
||||
# effect: NoSchedule
|
||||
# - key: node-role.kubernetes.io/control-plane
|
||||
# effect: NoSchedule
|
||||
# cert_manager_affinity:
|
||||
# nodeAffinity:
|
||||
# preferredDuringSchedulingIgnoredDuringExecution:
|
||||
# - weight: 100
|
||||
# preference:
|
||||
# matchExpressions:
|
||||
# - key: node-role.kubernetes.io/control-plane
|
||||
# operator: In
|
||||
# values:
|
||||
# - ""
|
||||
# cert_manager_nodeselector:
|
||||
# kubernetes.io/os: "linux"
|
||||
|
||||
# cert_manager_trusted_internal_ca: |
|
||||
# -----BEGIN CERTIFICATE-----
|
||||
# [REPLACE with your CA certificate]
|
||||
# -----END CERTIFICATE-----
|
||||
# cert_manager_leader_election_namespace: kube-system
|
||||
|
||||
# MetalLB deployment
|
||||
metallb_enabled: false
|
||||
metallb_speaker_enabled: "{{ metallb_enabled }}"
|
||||
# metallb_ip_range:
|
||||
# - "10.5.0.50-10.5.0.99"
|
||||
# metallb_pool_name: "loadbalanced"
|
||||
# metallb_auto_assign: true
|
||||
# metallb_avoid_buggy_ips: false
|
||||
# metallb_speaker_nodeselector:
|
||||
# kubernetes.io/os: "linux"
|
||||
# metallb_controller_nodeselector:
|
||||
# kubernetes.io/os: "linux"
|
||||
# metallb_speaker_tolerations:
|
||||
# - key: "node-role.kubernetes.io/master"
|
||||
# operator: "Equal"
|
||||
# value: ""
|
||||
# effect: "NoSchedule"
|
||||
# - key: "node-role.kubernetes.io/control-plane"
|
||||
# operator: "Equal"
|
||||
# value: ""
|
||||
# effect: "NoSchedule"
|
||||
# metallb_controller_tolerations:
|
||||
# - key: "node-role.kubernetes.io/master"
|
||||
# operator: "Equal"
|
||||
# value: ""
|
||||
# effect: "NoSchedule"
|
||||
# - key: "node-role.kubernetes.io/control-plane"
|
||||
# operator: "Equal"
|
||||
# value: ""
|
||||
# effect: "NoSchedule"
|
||||
# metallb_version: v0.12.1
|
||||
# metallb_protocol: "layer2"
|
||||
# metallb_port: "7472"
|
||||
# metallb_memberlist_port: "7946"
|
||||
# metallb_additional_address_pools:
|
||||
# kube_service_pool:
|
||||
# ip_range:
|
||||
# - "10.5.1.50-10.5.1.99"
|
||||
# protocol: "layer2"
|
||||
# auto_assign: false
|
||||
# avoid_buggy_ips: false
|
||||
# metallb_protocol: "bgp"
|
||||
# metallb_peers:
|
||||
# - peer_address: 192.0.2.1
|
||||
# peer_asn: 64512
|
||||
# my_asn: 4200000000
|
||||
# - peer_address: 192.0.2.2
|
||||
# peer_asn: 64513
|
||||
# my_asn: 4200000000
|
||||
|
||||
argocd_enabled: false
|
||||
# argocd_version: v2.5.5
|
||||
# argocd_namespace: argocd
|
||||
# Default password:
|
||||
# - https://argo-cd.readthedocs.io/en/stable/getting_started/#4-login-using-the-cli
|
||||
# ---
|
||||
# The initial password is autogenerated to be the pod name of the Argo CD API server. This can be retrieved with the command:
|
||||
# kubectl get pods -n argocd -l app.kubernetes.io/name=argocd-server -o name | cut -d'/' -f 2
|
||||
# ---
|
||||
# Use the following var to set admin password
|
||||
# argocd_admin_password: "password"
|
||||
|
||||
# The plugin manager for kubectl
|
||||
krew_enabled: false
|
||||
krew_root_dir: "/usr/local/krew"
|
||||
@@ -0,0 +1,350 @@
|
||||
---
|
||||
# Kubernetes configuration dirs and system namespace.
|
||||
# Those are where all the additional config stuff goes
|
||||
# the kubernetes normally puts in /srv/kubernetes.
|
||||
# This puts them in a sane location and namespace.
|
||||
# Editing those values will almost surely break something.
|
||||
kube_config_dir: /etc/kubernetes
|
||||
kube_script_dir: "{{ bin_dir }}/kubernetes-scripts"
|
||||
kube_manifest_dir: "{{ kube_config_dir }}/manifests"
|
||||
|
||||
# This is where all the cert scripts and certs will be located
|
||||
kube_cert_dir: "{{ kube_config_dir }}/ssl"
|
||||
|
||||
# This is where all of the bearer tokens will be stored
|
||||
kube_token_dir: "{{ kube_config_dir }}/tokens"
|
||||
|
||||
kube_api_anonymous_auth: true
|
||||
|
||||
## Change this to use another Kubernetes version, e.g. a current beta release
|
||||
kube_version: v1.25.5
|
||||
|
||||
# Where the binaries will be downloaded.
|
||||
# Note: ensure that you've enough disk space (about 1G)
|
||||
local_release_dir: "/tmp/releases"
|
||||
# Random shifts for retrying failed ops like pushing/downloading
|
||||
retry_stagger: 5
|
||||
|
||||
# This is the user that owns tha cluster installation.
|
||||
kube_owner: kube
|
||||
|
||||
# This is the group that the cert creation scripts chgrp the
|
||||
# cert files to. Not really changeable...
|
||||
kube_cert_group: kube-cert
|
||||
|
||||
# Cluster Loglevel configuration
|
||||
kube_log_level: 2
|
||||
|
||||
# Directory where credentials will be stored
|
||||
credentials_dir: "{{ inventory_dir }}/credentials"
|
||||
|
||||
## It is possible to activate / deactivate selected authentication methods (oidc, static token auth)
|
||||
# kube_oidc_auth: false
|
||||
# kube_token_auth: false
|
||||
|
||||
|
||||
## Variables for OpenID Connect Configuration https://kubernetes.io/docs/admin/authentication/
|
||||
## To use OpenID you have to deploy additional an OpenID Provider (e.g Dex, Keycloak, ...)
|
||||
|
||||
# kube_oidc_url: https:// ...
|
||||
# kube_oidc_client_id: kubernetes
|
||||
## Optional settings for OIDC
|
||||
# kube_oidc_ca_file: "{{ kube_cert_dir }}/ca.pem"
|
||||
# kube_oidc_username_claim: sub
|
||||
# kube_oidc_username_prefix: 'oidc:'
|
||||
# kube_oidc_groups_claim: groups
|
||||
# kube_oidc_groups_prefix: 'oidc:'
|
||||
|
||||
## Variables to control webhook authn/authz
|
||||
# kube_webhook_token_auth: false
|
||||
# kube_webhook_token_auth_url: https://...
|
||||
# kube_webhook_token_auth_url_skip_tls_verify: false
|
||||
|
||||
## For webhook authorization, authorization_modes must include Webhook
|
||||
# kube_webhook_authorization: false
|
||||
# kube_webhook_authorization_url: https://...
|
||||
# kube_webhook_authorization_url_skip_tls_verify: false
|
||||
|
||||
# Choose network plugin (cilium, calico, kube-ovn, weave or flannel. Use cni for generic cni plugin)
|
||||
# Can also be set to 'cloud', which lets the cloud provider setup appropriate routing
|
||||
kube_network_plugin: calico
|
||||
|
||||
# Setting multi_networking to true will install Multus: https://github.com/intel/multus-cni
|
||||
kube_network_plugin_multus: false
|
||||
|
||||
# Kubernetes internal network for services, unused block of space.
|
||||
kube_service_addresses: 10.233.0.0/18
|
||||
|
||||
# internal network. When used, it will assign IP
|
||||
# addresses from this range to individual pods.
|
||||
# This network must be unused in your network infrastructure!
|
||||
kube_pods_subnet: 10.233.64.0/18
|
||||
|
||||
# internal network node size allocation (optional). This is the size allocated
|
||||
# to each node for pod IP address allocation. Note that the number of pods per node is
|
||||
# also limited by the kubelet_max_pods variable which defaults to 110.
|
||||
#
|
||||
# Example:
|
||||
# Up to 64 nodes and up to 254 or kubelet_max_pods (the lowest of the two) pods per node:
|
||||
# - kube_pods_subnet: 10.233.64.0/18
|
||||
# - kube_network_node_prefix: 24
|
||||
# - kubelet_max_pods: 110
|
||||
#
|
||||
# Example:
|
||||
# Up to 128 nodes and up to 126 or kubelet_max_pods (the lowest of the two) pods per node:
|
||||
# - kube_pods_subnet: 10.233.64.0/18
|
||||
# - kube_network_node_prefix: 25
|
||||
# - kubelet_max_pods: 110
|
||||
kube_network_node_prefix: 24
|
||||
|
||||
# Configure Dual Stack networking (i.e. both IPv4 and IPv6)
|
||||
enable_dual_stack_networks: false
|
||||
|
||||
# Kubernetes internal network for IPv6 services, unused block of space.
|
||||
# This is only used if enable_dual_stack_networks is set to true
|
||||
# This provides 4096 IPv6 IPs
|
||||
kube_service_addresses_ipv6: fd85:ee78:d8a6:8607::1000/116
|
||||
|
||||
# Internal network. When used, it will assign IPv6 addresses from this range to individual pods.
|
||||
# This network must not already be in your network infrastructure!
|
||||
# This is only used if enable_dual_stack_networks is set to true.
|
||||
# This provides room for 256 nodes with 254 pods per node.
|
||||
kube_pods_subnet_ipv6: fd85:ee78:d8a6:8607::1:0000/112
|
||||
|
||||
# IPv6 subnet size allocated to each for pods.
|
||||
# This is only used if enable_dual_stack_networks is set to true
|
||||
# This provides room for 254 pods per node.
|
||||
kube_network_node_prefix_ipv6: 120
|
||||
|
||||
# The port the API Server will be listening on.
|
||||
kube_apiserver_ip: "{{ kube_service_addresses|ipaddr('net')|ipaddr(1)|ipaddr('address') }}"
|
||||
kube_apiserver_port: 6443 # (https)
|
||||
|
||||
# Kube-proxy proxyMode configuration.
|
||||
# Can be ipvs, iptables
|
||||
kube_proxy_mode: ipvs
|
||||
|
||||
# configure arp_ignore and arp_announce to avoid answering ARP queries from kube-ipvs0 interface
|
||||
# must be set to true for MetalLB, kube-vip(ARP enabled) to work
|
||||
kube_proxy_strict_arp: false
|
||||
|
||||
# A string slice of values which specify the addresses to use for NodePorts.
|
||||
# Values may be valid IP blocks (e.g. 1.2.3.0/24, 1.2.3.4/32).
|
||||
# The default empty string slice ([]) means to use all local addresses.
|
||||
# kube_proxy_nodeport_addresses_cidr is retained for legacy config
|
||||
kube_proxy_nodeport_addresses: >-
|
||||
{%- if kube_proxy_nodeport_addresses_cidr is defined -%}
|
||||
[{{ kube_proxy_nodeport_addresses_cidr }}]
|
||||
{%- else -%}
|
||||
[]
|
||||
{%- endif -%}
|
||||
|
||||
# If non-empty, will use this string as identification instead of the actual hostname
|
||||
# kube_override_hostname: >-
|
||||
# {%- if cloud_provider is defined and cloud_provider in [ 'aws' ] -%}
|
||||
# {%- else -%}
|
||||
# {{ inventory_hostname }}
|
||||
# {%- endif -%}
|
||||
|
||||
## Encrypting Secret Data at Rest
|
||||
kube_encrypt_secret_data: false
|
||||
|
||||
# Graceful Node Shutdown (Kubernetes >= 1.21.0), see https://kubernetes.io/blog/2021/04/21/graceful-node-shutdown-beta/
|
||||
# kubelet_shutdown_grace_period had to be greater than kubelet_shutdown_grace_period_critical_pods to allow
|
||||
# non-critical podsa to also terminate gracefully
|
||||
# kubelet_shutdown_grace_period: 60s
|
||||
# kubelet_shutdown_grace_period_critical_pods: 20s
|
||||
|
||||
# DNS configuration.
|
||||
# Kubernetes cluster name, also will be used as DNS domain
|
||||
cluster_name: cluster.local
|
||||
# Subdomains of DNS domain to be resolved via /etc/resolv.conf for hostnet pods
|
||||
ndots: 2
|
||||
# dns_timeout: 2
|
||||
# dns_attempts: 2
|
||||
# Custom search domains to be added in addition to the default cluster search domains
|
||||
# searchdomains:
|
||||
# - svc.{{ cluster_name }}
|
||||
# - default.svc.{{ cluster_name }}
|
||||
# Remove default cluster search domains (``default.svc.{{ dns_domain }}, svc.{{ dns_domain }}``).
|
||||
# remove_default_searchdomains: false
|
||||
# Can be coredns, coredns_dual, manual or none
|
||||
dns_mode: coredns
|
||||
# Set manual server if using a custom cluster DNS server
|
||||
# manual_dns_server: 10.x.x.x
|
||||
# Enable nodelocal dns cache
|
||||
enable_nodelocaldns: true
|
||||
enable_nodelocaldns_secondary: false
|
||||
nodelocaldns_ip: 169.254.25.10
|
||||
nodelocaldns_health_port: 9254
|
||||
nodelocaldns_second_health_port: 9256
|
||||
nodelocaldns_bind_metrics_host_ip: false
|
||||
nodelocaldns_secondary_skew_seconds: 5
|
||||
# nodelocaldns_external_zones:
|
||||
# - zones:
|
||||
# - example.com
|
||||
# - example.io:1053
|
||||
# nameservers:
|
||||
# - 1.1.1.1
|
||||
# - 2.2.2.2
|
||||
# cache: 5
|
||||
# - zones:
|
||||
# - https://mycompany.local:4453
|
||||
# nameservers:
|
||||
# - 192.168.0.53
|
||||
# cache: 0
|
||||
# - zones:
|
||||
# - mydomain.tld
|
||||
# nameservers:
|
||||
# - 10.233.0.3
|
||||
# cache: 5
|
||||
# rewrite:
|
||||
# - name website.tld website.namespace.svc.cluster.local
|
||||
# Enable k8s_external plugin for CoreDNS
|
||||
enable_coredns_k8s_external: false
|
||||
coredns_k8s_external_zone: k8s_external.local
|
||||
# Enable endpoint_pod_names option for kubernetes plugin
|
||||
enable_coredns_k8s_endpoint_pod_names: false
|
||||
# Set forward options for upstream DNS servers in coredns (and nodelocaldns) config
|
||||
# dns_upstream_forward_extra_opts:
|
||||
# policy: sequential
|
||||
|
||||
# Can be docker_dns, host_resolvconf or none
|
||||
resolvconf_mode: host_resolvconf
|
||||
# Deploy netchecker app to verify DNS resolve as an HTTP service
|
||||
deploy_netchecker: false
|
||||
# Ip address of the kubernetes skydns service
|
||||
skydns_server: "{{ kube_service_addresses|ipaddr('net')|ipaddr(3)|ipaddr('address') }}"
|
||||
skydns_server_secondary: "{{ kube_service_addresses|ipaddr('net')|ipaddr(4)|ipaddr('address') }}"
|
||||
dns_domain: "{{ cluster_name }}"
|
||||
|
||||
## Container runtime
|
||||
## docker for docker, crio for cri-o and containerd for containerd.
|
||||
## Default: containerd
|
||||
container_manager: containerd
|
||||
|
||||
# Additional container runtimes
|
||||
kata_containers_enabled: false
|
||||
|
||||
kubeadm_certificate_key: "{{ lookup('password', credentials_dir + '/kubeadm_certificate_key.creds length=64 chars=hexdigits') | lower }}"
|
||||
|
||||
# K8s image pull policy (imagePullPolicy)
|
||||
k8s_image_pull_policy: IfNotPresent
|
||||
|
||||
# audit log for kubernetes
|
||||
kubernetes_audit: false
|
||||
|
||||
# define kubelet config dir for dynamic kubelet
|
||||
# kubelet_config_dir:
|
||||
default_kubelet_config_dir: "{{ kube_config_dir }}/dynamic_kubelet_dir"
|
||||
|
||||
# pod security policy (RBAC must be enabled either by having 'RBAC' in authorization_modes or kubeadm enabled)
|
||||
podsecuritypolicy_enabled: false
|
||||
|
||||
# Custom PodSecurityPolicySpec for restricted policy
|
||||
# podsecuritypolicy_restricted_spec: {}
|
||||
|
||||
# Custom PodSecurityPolicySpec for privileged policy
|
||||
# podsecuritypolicy_privileged_spec: {}
|
||||
|
||||
# Make a copy of kubeconfig on the host that runs Ansible in {{ inventory_dir }}/artifacts
|
||||
# kubeconfig_localhost: false
|
||||
# Use ansible_host as external api ip when copying over kubeconfig.
|
||||
# kubeconfig_localhost_ansible_host: false
|
||||
# Download kubectl onto the host that runs Ansible in {{ bin_dir }}
|
||||
# kubectl_localhost: false
|
||||
|
||||
# A comma separated list of levels of node allocatable enforcement to be enforced by kubelet.
|
||||
# Acceptable options are 'pods', 'system-reserved', 'kube-reserved' and ''. Default is "".
|
||||
# kubelet_enforce_node_allocatable: pods
|
||||
|
||||
## Optionally reserve resources for OS system daemons.
|
||||
# system_reserved: true
|
||||
## Uncomment to override default values
|
||||
# system_memory_reserved: 512Mi
|
||||
# system_cpu_reserved: 500m
|
||||
# system_ephemeral_storage_reserved: 2Gi
|
||||
## Reservation for master hosts
|
||||
# system_master_memory_reserved: 256Mi
|
||||
# system_master_cpu_reserved: 250m
|
||||
# system_master_ephemeral_storage_reserved: 2Gi
|
||||
|
||||
## Eviction Thresholds to avoid system OOMs
|
||||
# https://kubernetes.io/docs/tasks/administer-cluster/reserve-compute-resources/#eviction-thresholds
|
||||
# eviction_hard: {}
|
||||
# eviction_hard_control_plane: {}
|
||||
|
||||
# An alternative flexvolume plugin directory
|
||||
# kubelet_flexvolumes_plugins_dir: /usr/libexec/kubernetes/kubelet-plugins/volume/exec
|
||||
|
||||
## Supplementary addresses that can be added in kubernetes ssl keys.
|
||||
## That can be useful for example to setup a keepalived virtual IP
|
||||
# supplementary_addresses_in_ssl_keys: [10.0.0.1, 10.0.0.2, 10.0.0.3]
|
||||
|
||||
## Running on top of openstack vms with cinder enabled may lead to unschedulable pods due to NoVolumeZoneConflict restriction in kube-scheduler.
|
||||
## See https://github.com/kubernetes-sigs/kubespray/issues/2141
|
||||
## Set this variable to true to get rid of this issue
|
||||
volume_cross_zone_attachment: false
|
||||
## Add Persistent Volumes Storage Class for corresponding cloud provider (supported: in-tree OpenStack, Cinder CSI,
|
||||
## AWS EBS CSI, Azure Disk CSI, GCP Persistent Disk CSI)
|
||||
persistent_volumes_enabled: false
|
||||
|
||||
## Container Engine Acceleration
|
||||
## Enable container acceleration feature, for example use gpu acceleration in containers
|
||||
# nvidia_accelerator_enabled: true
|
||||
## Nvidia GPU driver install. Install will by done by a (init) pod running as a daemonset.
|
||||
## Important: if you use Ubuntu then you should set in all.yml 'docker_storage_options: -s overlay2'
|
||||
## Array with nvida_gpu_nodes, leave empty or comment if you don't want to install drivers.
|
||||
## Labels and taints won't be set to nodes if they are not in the array.
|
||||
# nvidia_gpu_nodes:
|
||||
# - kube-gpu-001
|
||||
# nvidia_driver_version: "384.111"
|
||||
## flavor can be tesla or gtx
|
||||
# nvidia_gpu_flavor: gtx
|
||||
## NVIDIA driver installer images. Change them if you have trouble accessing gcr.io.
|
||||
# nvidia_driver_install_centos_container: atzedevries/nvidia-centos-driver-installer:2
|
||||
# nvidia_driver_install_ubuntu_container: gcr.io/google-containers/ubuntu-nvidia-driver-installer@sha256:7df76a0f0a17294e86f691c81de6bbb7c04a1b4b3d4ea4e7e2cccdc42e1f6d63
|
||||
## NVIDIA GPU device plugin image.
|
||||
# nvidia_gpu_device_plugin_container: "registry.k8s.io/nvidia-gpu-device-plugin@sha256:0842734032018be107fa2490c98156992911e3e1f2a21e059ff0105b07dd8e9e"
|
||||
|
||||
## Support tls min version, Possible values: VersionTLS10, VersionTLS11, VersionTLS12, VersionTLS13.
|
||||
# tls_min_version: ""
|
||||
|
||||
## Support tls cipher suites.
|
||||
# tls_cipher_suites: {}
|
||||
# - TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA
|
||||
# - TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256
|
||||
# - TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
|
||||
# - TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA
|
||||
# - TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
|
||||
# - TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305
|
||||
# - TLS_ECDHE_ECDSA_WITH_RC4_128_SHA
|
||||
# - TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA
|
||||
# - TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA
|
||||
# - TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256
|
||||
# - TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
|
||||
# - TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA
|
||||
# - TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
|
||||
# - TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305
|
||||
# - TLS_ECDHE_RSA_WITH_RC4_128_SHA
|
||||
# - TLS_RSA_WITH_3DES_EDE_CBC_SHA
|
||||
# - TLS_RSA_WITH_AES_128_CBC_SHA
|
||||
# - TLS_RSA_WITH_AES_128_CBC_SHA256
|
||||
# - TLS_RSA_WITH_AES_128_GCM_SHA256
|
||||
# - TLS_RSA_WITH_AES_256_CBC_SHA
|
||||
# - TLS_RSA_WITH_AES_256_GCM_SHA384
|
||||
# - TLS_RSA_WITH_RC4_128_SHA
|
||||
|
||||
## Amount of time to retain events. (default 1h0m0s)
|
||||
event_ttl_duration: "1h0m0s"
|
||||
|
||||
## Automatically renew K8S control plane certificates on first Monday of each month
|
||||
auto_renew_certificates: false
|
||||
# First Monday of each month
|
||||
# auto_renew_certificates_systemd_calendar: "Mon *-*-1,2,3,4,5,6,7 03:{{ groups['kube_control_plane'].index(inventory_hostname) }}0:00"
|
||||
|
||||
# kubeadm patches path
|
||||
kubeadm_patches:
|
||||
enabled: false
|
||||
source_dir: "{{ inventory_dir }}/patches"
|
||||
dest_dir: "{{ kube_config_dir }}/patches"
|
||||
@@ -0,0 +1,131 @@
|
||||
---
|
||||
# see roles/network_plugin/calico/defaults/main.yml
|
||||
|
||||
# the default value of name
|
||||
calico_cni_name: k8s-pod-network
|
||||
|
||||
## With calico it is possible to distributed routes with border routers of the datacenter.
|
||||
## Warning : enabling router peering will disable calico's default behavior ('node mesh').
|
||||
## The subnets of each nodes will be distributed by the datacenter router
|
||||
# peer_with_router: false
|
||||
|
||||
# Enables Internet connectivity from containers
|
||||
# nat_outgoing: true
|
||||
|
||||
# Enables Calico CNI "host-local" IPAM plugin
|
||||
# calico_ipam_host_local: true
|
||||
|
||||
# add default ippool name
|
||||
# calico_pool_name: "default-pool"
|
||||
|
||||
# add default ippool blockSize (defaults kube_network_node_prefix)
|
||||
calico_pool_blocksize: 26
|
||||
|
||||
# add default ippool CIDR (must be inside kube_pods_subnet, defaults to kube_pods_subnet otherwise)
|
||||
# calico_pool_cidr: 1.2.3.4/5
|
||||
|
||||
# add default ippool CIDR to CNI config
|
||||
# calico_cni_pool: true
|
||||
|
||||
# Add default IPV6 IPPool CIDR. Must be inside kube_pods_subnet_ipv6. Defaults to kube_pods_subnet_ipv6 if not set.
|
||||
# calico_pool_cidr_ipv6: fd85:ee78:d8a6:8607::1:0000/112
|
||||
|
||||
# Add default IPV6 IPPool CIDR to CNI config
|
||||
# calico_cni_pool_ipv6: true
|
||||
|
||||
# Global as_num (/calico/bgp/v1/global/as_num)
|
||||
# global_as_num: "64512"
|
||||
|
||||
# If doing peering with node-assigned asn where the globas does not match your nodes, you want this
|
||||
# to be true. All other cases, false.
|
||||
# calico_no_global_as_num: false
|
||||
|
||||
# You can set MTU value here. If left undefined or empty, it will
|
||||
# not be specified in calico CNI config, so Calico will use built-in
|
||||
# defaults. The value should be a number, not a string.
|
||||
# calico_mtu: 1500
|
||||
|
||||
# Configure the MTU to use for workload interfaces and tunnels.
|
||||
# - If Wireguard is enabled, subtract 60 from your network MTU (i.e 1500-60=1440)
|
||||
# - Otherwise, if VXLAN or BPF mode is enabled, subtract 50 from your network MTU (i.e. 1500-50=1450)
|
||||
# - Otherwise, if IPIP is enabled, subtract 20 from your network MTU (i.e. 1500-20=1480)
|
||||
# - Otherwise, if not using any encapsulation, set to your network MTU (i.e. 1500)
|
||||
# calico_veth_mtu: 1440
|
||||
|
||||
# Advertise Cluster IPs
|
||||
# calico_advertise_cluster_ips: true
|
||||
|
||||
# Advertise Service External IPs
|
||||
# calico_advertise_service_external_ips:
|
||||
# - x.x.x.x/24
|
||||
# - y.y.y.y/32
|
||||
|
||||
# Advertise Service LoadBalancer IPs
|
||||
# calico_advertise_service_loadbalancer_ips:
|
||||
# - x.x.x.x/24
|
||||
# - y.y.y.y/16
|
||||
|
||||
# Choose data store type for calico: "etcd" or "kdd" (kubernetes datastore)
|
||||
# calico_datastore: "kdd"
|
||||
|
||||
# Choose Calico iptables backend: "Legacy", "Auto" or "NFT"
|
||||
# calico_iptables_backend: "Auto"
|
||||
|
||||
# Use typha (only with kdd)
|
||||
# typha_enabled: false
|
||||
|
||||
# Generate TLS certs for secure typha<->calico-node communication
|
||||
# typha_secure: false
|
||||
|
||||
# Scaling typha: 1 replica per 100 nodes is adequate
|
||||
# Number of typha replicas
|
||||
# typha_replicas: 1
|
||||
|
||||
# Set max typha connections
|
||||
# typha_max_connections_lower_limit: 300
|
||||
|
||||
# Set calico network backend: "bird", "vxlan" or "none"
|
||||
# bird enable BGP routing, required for ipip and no encapsulation modes
|
||||
# calico_network_backend: vxlan
|
||||
|
||||
# IP in IP and VXLAN is mutualy exclusive modes.
|
||||
# set IP in IP encapsulation mode: "Always", "CrossSubnet", "Never"
|
||||
# calico_ipip_mode: 'Never'
|
||||
|
||||
# set VXLAN encapsulation mode: "Always", "CrossSubnet", "Never"
|
||||
# calico_vxlan_mode: 'Always'
|
||||
|
||||
# set VXLAN port and VNI
|
||||
# calico_vxlan_vni: 4096
|
||||
# calico_vxlan_port: 4789
|
||||
|
||||
# Enable eBPF mode
|
||||
# calico_bpf_enabled: false
|
||||
|
||||
# If you want to use non default IP_AUTODETECTION_METHOD, IP6_AUTODETECTION_METHOD for calico node set this option to one of:
|
||||
# * can-reach=DESTINATION
|
||||
# * interface=INTERFACE-REGEX
|
||||
# see https://docs.projectcalico.org/reference/node/configuration
|
||||
# calico_ip_auto_method: "interface=eth.*"
|
||||
# calico_ip6_auto_method: "interface=eth.*"
|
||||
|
||||
# Set FELIX_MTUIFACEPATTERN, Pattern used to discover the host’s interface for MTU auto-detection.
|
||||
# see https://projectcalico.docs.tigera.io/reference/felix/configuration
|
||||
# calico_felix_mtu_iface_pattern: "^((en|wl|ww|sl|ib)[opsx].*|(eth|wlan|wwan).*)"
|
||||
|
||||
# Choose the iptables insert mode for Calico: "Insert" or "Append".
|
||||
# calico_felix_chaininsertmode: Insert
|
||||
|
||||
# If you want use the default route interface when you use multiple interface with dynamique route (iproute2)
|
||||
# see https://docs.projectcalico.org/reference/node/configuration : FELIX_DEVICEROUTESOURCEADDRESS
|
||||
# calico_use_default_route_src_ipaddr: false
|
||||
|
||||
# Enable calico traffic encryption with wireguard
|
||||
# calico_wireguard_enabled: false
|
||||
|
||||
# Under certain situations liveness and readiness probes may need tunning
|
||||
# calico_node_livenessprobe_timeout: 10
|
||||
# calico_node_readinessprobe_timeout: 10
|
||||
|
||||
# Calico apiserver (only with kdd)
|
||||
# calico_apiserver_enabled: false
|
||||
@@ -0,0 +1,10 @@
|
||||
# see roles/network_plugin/canal/defaults/main.yml
|
||||
|
||||
# The interface used by canal for host <-> host communication.
|
||||
# If left blank, then the interface is choosing using the node's
|
||||
# default route.
|
||||
# canal_iface: ""
|
||||
|
||||
# Whether or not to masquerade traffic to destinations not within
|
||||
# the pod network.
|
||||
# canal_masquerade: "true"
|
||||
@@ -0,0 +1,245 @@
|
||||
---
|
||||
# cilium_version: "v1.12.1"
|
||||
|
||||
# Log-level
|
||||
# cilium_debug: false
|
||||
|
||||
# cilium_mtu: ""
|
||||
# cilium_enable_ipv4: true
|
||||
# cilium_enable_ipv6: false
|
||||
|
||||
# Cilium agent health port
|
||||
# cilium_agent_health_port: "9879"
|
||||
|
||||
# Identity allocation mode selects how identities are shared between cilium
|
||||
# nodes by setting how they are stored. The options are "crd" or "kvstore".
|
||||
# - "crd" stores identities in kubernetes as CRDs (custom resource definition).
|
||||
# These can be queried with:
|
||||
# `kubectl get ciliumid`
|
||||
# - "kvstore" stores identities in an etcd kvstore.
|
||||
# - In order to support External Workloads, "crd" is required
|
||||
# - Ref: https://docs.cilium.io/en/stable/gettingstarted/external-workloads/#setting-up-support-for-external-workloads-beta
|
||||
# - KVStore operations are only required when cilium-operator is running with any of the below options:
|
||||
# - --synchronize-k8s-services
|
||||
# - --synchronize-k8s-nodes
|
||||
# - --identity-allocation-mode=kvstore
|
||||
# - Ref: https://docs.cilium.io/en/stable/internals/cilium_operator/#kvstore-operations
|
||||
# cilium_identity_allocation_mode: kvstore
|
||||
|
||||
# Etcd SSL dirs
|
||||
# cilium_cert_dir: /etc/cilium/certs
|
||||
# kube_etcd_cacert_file: ca.pem
|
||||
# kube_etcd_cert_file: cert.pem
|
||||
# kube_etcd_key_file: cert-key.pem
|
||||
|
||||
# Limits for apps
|
||||
# cilium_memory_limit: 500M
|
||||
# cilium_cpu_limit: 500m
|
||||
# cilium_memory_requests: 64M
|
||||
# cilium_cpu_requests: 100m
|
||||
|
||||
# Overlay Network Mode
|
||||
# cilium_tunnel_mode: vxlan
|
||||
# Optional features
|
||||
# cilium_enable_prometheus: false
|
||||
# Enable if you want to make use of hostPort mappings
|
||||
# cilium_enable_portmap: false
|
||||
# Monitor aggregation level (none/low/medium/maximum)
|
||||
# cilium_monitor_aggregation: medium
|
||||
# The monitor aggregation flags determine which TCP flags which, upon the
|
||||
# first observation, cause monitor notifications to be generated.
|
||||
#
|
||||
# Only effective when monitor aggregation is set to "medium" or higher.
|
||||
# cilium_monitor_aggregation_flags: "all"
|
||||
# Kube Proxy Replacement mode (strict/probe/partial)
|
||||
# cilium_kube_proxy_replacement: probe
|
||||
|
||||
# If upgrading from Cilium < 1.5, you may want to override some of these options
|
||||
# to prevent service disruptions. See also:
|
||||
# http://docs.cilium.io/en/stable/install/upgrade/#changes-that-may-require-action
|
||||
# cilium_preallocate_bpf_maps: false
|
||||
|
||||
# `cilium_tofqdns_enable_poller` is deprecated in 1.8, removed in 1.9
|
||||
# cilium_tofqdns_enable_poller: false
|
||||
|
||||
# `cilium_enable_legacy_services` is deprecated in 1.6, removed in 1.9
|
||||
# cilium_enable_legacy_services: false
|
||||
|
||||
# Unique ID of the cluster. Must be unique across all conneted clusters and
|
||||
# in the range of 1 and 255. Only relevant when building a mesh of clusters.
|
||||
# This value is not defined by default
|
||||
# cilium_cluster_id:
|
||||
|
||||
# Deploy cilium even if kube_network_plugin is not cilium.
|
||||
# This enables to deploy cilium alongside another CNI to replace kube-proxy.
|
||||
# cilium_deploy_additionally: false
|
||||
|
||||
# Auto direct nodes routes can be used to advertise pods routes in your cluster
|
||||
# without any tunelling (with `cilium_tunnel_mode` sets to `disabled`).
|
||||
# This works only if you have a L2 connectivity between all your nodes.
|
||||
# You wil also have to specify the variable `cilium_native_routing_cidr` to
|
||||
# make this work. Please refer to the cilium documentation for more
|
||||
# information about this kind of setups.
|
||||
# cilium_auto_direct_node_routes: false
|
||||
|
||||
# Allows to explicitly specify the IPv4 CIDR for native routing.
|
||||
# When specified, Cilium assumes networking for this CIDR is preconfigured and
|
||||
# hands traffic destined for that range to the Linux network stack without
|
||||
# applying any SNAT.
|
||||
# Generally speaking, specifying a native routing CIDR implies that Cilium can
|
||||
# depend on the underlying networking stack to route packets to their
|
||||
# destination. To offer a concrete example, if Cilium is configured to use
|
||||
# direct routing and the Kubernetes CIDR is included in the native routing CIDR,
|
||||
# the user must configure the routes to reach pods, either manually or by
|
||||
# setting the auto-direct-node-routes flag.
|
||||
# cilium_native_routing_cidr: ""
|
||||
|
||||
# Allows to explicitly specify the IPv6 CIDR for native routing.
|
||||
# cilium_native_routing_cidr_ipv6: ""
|
||||
|
||||
# Enable transparent network encryption.
|
||||
# cilium_encryption_enabled: false
|
||||
|
||||
# Encryption method. Can be either ipsec or wireguard.
|
||||
# Only effective when `cilium_encryption_enabled` is set to true.
|
||||
# cilium_encryption_type: "ipsec"
|
||||
|
||||
# Enable encryption for pure node to node traffic.
|
||||
# This option is only effective when `cilium_encryption_type` is set to `ipsec`.
|
||||
# cilium_ipsec_node_encryption: false
|
||||
|
||||
# If your kernel or distribution does not support WireGuard, Cilium agent can be configured to fall back on the user-space implementation.
|
||||
# When this flag is enabled and Cilium detects that the kernel has no native support for WireGuard,
|
||||
# it will fallback on the wireguard-go user-space implementation of WireGuard.
|
||||
# This option is only effective when `cilium_encryption_type` is set to `wireguard`.
|
||||
# cilium_wireguard_userspace_fallback: false
|
||||
|
||||
# IP Masquerade Agent
|
||||
# https://docs.cilium.io/en/stable/concepts/networking/masquerading/
|
||||
# By default, all packets from a pod destined to an IP address outside of the cilium_native_routing_cidr range are masqueraded
|
||||
# cilium_ip_masq_agent_enable: false
|
||||
|
||||
### A packet sent from a pod to a destination which belongs to any CIDR from the nonMasqueradeCIDRs is not going to be masqueraded
|
||||
# cilium_non_masquerade_cidrs:
|
||||
# - 10.0.0.0/8
|
||||
# - 172.16.0.0/12
|
||||
# - 192.168.0.0/16
|
||||
# - 100.64.0.0/10
|
||||
# - 192.0.0.0/24
|
||||
# - 192.0.2.0/24
|
||||
# - 192.88.99.0/24
|
||||
# - 198.18.0.0/15
|
||||
# - 198.51.100.0/24
|
||||
# - 203.0.113.0/24
|
||||
# - 240.0.0.0/4
|
||||
### Indicates whether to masquerade traffic to the link local prefix.
|
||||
### If the masqLinkLocal is not set or set to false, then 169.254.0.0/16 is appended to the non-masquerade CIDRs list.
|
||||
# cilium_masq_link_local: false
|
||||
### A time interval at which the agent attempts to reload config from disk
|
||||
# cilium_ip_masq_resync_interval: 60s
|
||||
|
||||
# Hubble
|
||||
### Enable Hubble without install
|
||||
# cilium_enable_hubble: false
|
||||
### Enable Hubble Metrics
|
||||
# cilium_enable_hubble_metrics: false
|
||||
### if cilium_enable_hubble_metrics: true
|
||||
# cilium_hubble_metrics: {}
|
||||
# - dns
|
||||
# - drop
|
||||
# - tcp
|
||||
# - flow
|
||||
# - icmp
|
||||
# - http
|
||||
### Enable Hubble install
|
||||
# cilium_hubble_install: false
|
||||
### Enable auto generate certs if cilium_hubble_install: true
|
||||
# cilium_hubble_tls_generate: false
|
||||
|
||||
# IP address management mode for v1.9+.
|
||||
# https://docs.cilium.io/en/v1.9/concepts/networking/ipam/
|
||||
# cilium_ipam_mode: kubernetes
|
||||
|
||||
# Extra arguments for the Cilium agent
|
||||
# cilium_agent_custom_args: []
|
||||
|
||||
# For adding and mounting extra volumes to the cilium agent
|
||||
# cilium_agent_extra_volumes: []
|
||||
# cilium_agent_extra_volume_mounts: []
|
||||
|
||||
# cilium_agent_extra_env_vars: []
|
||||
|
||||
# cilium_operator_replicas: 2
|
||||
|
||||
# The address at which the cillium operator bind health check api
|
||||
# cilium_operator_api_serve_addr: "127.0.0.1:9234"
|
||||
|
||||
## A dictionary of extra config variables to add to cilium-config, formatted like:
|
||||
## cilium_config_extra_vars:
|
||||
## var1: "value1"
|
||||
## var2: "value2"
|
||||
# cilium_config_extra_vars: {}
|
||||
|
||||
# For adding and mounting extra volumes to the cilium operator
|
||||
# cilium_operator_extra_volumes: []
|
||||
# cilium_operator_extra_volume_mounts: []
|
||||
|
||||
# Extra arguments for the Cilium Operator
|
||||
# cilium_operator_custom_args: []
|
||||
|
||||
# Name of the cluster. Only relevant when building a mesh of clusters.
|
||||
# cilium_cluster_name: default
|
||||
|
||||
# Make Cilium take ownership over the `/etc/cni/net.d` directory on the node, renaming all non-Cilium CNI configurations to `*.cilium_bak`.
|
||||
# This ensures no Pods can be scheduled using other CNI plugins during Cilium agent downtime.
|
||||
# Available for Cilium v1.10 and up.
|
||||
# cilium_cni_exclusive: true
|
||||
|
||||
# Configure the log file for CNI logging with retention policy of 7 days.
|
||||
# Disable CNI file logging by setting this field to empty explicitly.
|
||||
# Available for Cilium v1.12 and up.
|
||||
# cilium_cni_log_file: "/var/run/cilium/cilium-cni.log"
|
||||
|
||||
# -- Configure cgroup related configuration
|
||||
# -- Enable auto mount of cgroup2 filesystem.
|
||||
# When `cilium_cgroup_auto_mount` is enabled, cgroup2 filesystem is mounted at
|
||||
# `cilium_cgroup_host_root` path on the underlying host and inside the cilium agent pod.
|
||||
# If users disable `cilium_cgroup_auto_mount`, it's expected that users have mounted
|
||||
# cgroup2 filesystem at the specified `cilium_cgroup_auto_mount` volume, and then the
|
||||
# volume will be mounted inside the cilium agent pod at the same path.
|
||||
# Available for Cilium v1.11 and up
|
||||
# cilium_cgroup_auto_mount: true
|
||||
# -- Configure cgroup root where cgroup2 filesystem is mounted on the host
|
||||
# cilium_cgroup_host_root: "/run/cilium/cgroupv2"
|
||||
|
||||
# Specifies the ratio (0.0-1.0) of total system memory to use for dynamic
|
||||
# sizing of the TCP CT, non-TCP CT, NAT and policy BPF maps.
|
||||
# cilium_bpf_map_dynamic_size_ratio: "0.0"
|
||||
|
||||
# -- Enables masquerading of IPv4 traffic leaving the node from endpoints.
|
||||
# Available for Cilium v1.10 and up
|
||||
# cilium_enable_ipv4_masquerade: true
|
||||
# -- Enables masquerading of IPv6 traffic leaving the node from endpoints.
|
||||
# Available for Cilium v1.10 and up
|
||||
# cilium_enable_ipv6_masquerade: true
|
||||
|
||||
# -- Enable native IP masquerade support in eBPF
|
||||
# cilium_enable_bpf_masquerade: false
|
||||
|
||||
# -- Configure whether direct routing mode should route traffic via
|
||||
# host stack (true) or directly and more efficiently out of BPF (false) if
|
||||
# the kernel supports it. The latter has the implication that it will also
|
||||
# bypass netfilter in the host namespace.
|
||||
# cilium_enable_host_legacy_routing: true
|
||||
|
||||
# -- Enable use of the remote node identity.
|
||||
# ref: https://docs.cilium.io/en/v1.7/install/upgrade/#configmap-remote-node-identity
|
||||
# cilium_enable_remote_node_identity: true
|
||||
|
||||
# -- Enable the use of well-known identities.
|
||||
# cilium_enable_well_known_identities: false
|
||||
|
||||
# cilium_enable_bpf_clock_probe: true
|
||||
|
||||
# -- Whether to enable CNP status updates.
|
||||
# cilium_disable_cnp_status_updates: true
|
||||
@@ -0,0 +1,18 @@
|
||||
# see roles/network_plugin/flannel/defaults/main.yml
|
||||
|
||||
## interface that should be used for flannel operations
|
||||
## This is actually an inventory cluster-level item
|
||||
# flannel_interface:
|
||||
|
||||
## Select interface that should be used for flannel operations by regexp on Name or IP
|
||||
## This is actually an inventory cluster-level item
|
||||
## example: select interface with ip from net 10.0.0.0/23
|
||||
## single quote and escape backslashes
|
||||
# flannel_interface_regexp: '10\\.0\\.[0-2]\\.\\d{1,3}'
|
||||
|
||||
# You can choose what type of flannel backend to use: 'vxlan', 'host-gw' or 'wireguard'
|
||||
# please refer to flannel's docs : https://github.com/coreos/flannel/blob/master/README.md
|
||||
# flannel_backend_type: "vxlan"
|
||||
# flannel_vxlan_vni: 1
|
||||
# flannel_vxlan_port: 8472
|
||||
# flannel_vxlan_direct_routing: false
|
||||
@@ -0,0 +1,57 @@
|
||||
---
|
||||
|
||||
# geneve or vlan
|
||||
kube_ovn_network_type: geneve
|
||||
|
||||
# geneve, vxlan or stt. ATTENTION: some networkpolicy cannot take effect when using vxlan and stt need custom compile ovs kernel module
|
||||
kube_ovn_tunnel_type: geneve
|
||||
|
||||
## The nic to support container network can be a nic name or a group of regex separated by comma e.g: 'enp6s0f0,eth.*', if empty will use the nic that the default route use.
|
||||
# kube_ovn_iface: eth1
|
||||
## The MTU used by pod iface in overlay networks (default iface MTU - 100)
|
||||
# kube_ovn_mtu: 1333
|
||||
|
||||
## Enable hw-offload, disable traffic mirror and set the iface to the physical port. Make sure that there is an IP address bind to the physical port.
|
||||
kube_ovn_hw_offload: false
|
||||
# traffic mirror
|
||||
kube_ovn_traffic_mirror: false
|
||||
|
||||
# kube_ovn_pool_cidr_ipv6: fd85:ee78:d8a6:8607::1:0000/112
|
||||
# kube_ovn_default_interface_name: eth0
|
||||
|
||||
kube_ovn_external_address: 8.8.8.8
|
||||
kube_ovn_external_address_ipv6: 2400:3200::1
|
||||
kube_ovn_external_dns: alauda.cn
|
||||
|
||||
# kube_ovn_default_gateway: 10.233.64.1,fd85:ee78:d8a6:8607::1:0
|
||||
kube_ovn_default_gateway_check: true
|
||||
kube_ovn_default_logical_gateway: false
|
||||
# kube_ovn_default_exclude_ips: 10.16.0.1
|
||||
kube_ovn_node_switch_cidr: 100.64.0.0/16
|
||||
kube_ovn_node_switch_cidr_ipv6: fd00:100:64::/64
|
||||
|
||||
## vlan config, set default interface name and vlan id
|
||||
# kube_ovn_default_interface_name: eth0
|
||||
kube_ovn_default_vlan_id: 100
|
||||
kube_ovn_vlan_name: product
|
||||
|
||||
## pod nic type, support: veth-pair or internal-port
|
||||
kube_ovn_pod_nic_type: veth_pair
|
||||
|
||||
## Enable load balancer
|
||||
kube_ovn_enable_lb: true
|
||||
|
||||
## Enable network policy support
|
||||
kube_ovn_enable_np: true
|
||||
|
||||
## Enable external vpc support
|
||||
kube_ovn_enable_external_vpc: true
|
||||
|
||||
## Enable checksum
|
||||
kube_ovn_encap_checksum: true
|
||||
|
||||
## enable ssl
|
||||
kube_ovn_enable_ssl: false
|
||||
|
||||
## dpdk
|
||||
kube_ovn_dpdk_enabled: false
|
||||
@@ -0,0 +1,64 @@
|
||||
# See roles/network_plugin/kube-router//defaults/main.yml
|
||||
|
||||
# Enables Pod Networking -- Advertises and learns the routes to Pods via iBGP
|
||||
# kube_router_run_router: true
|
||||
|
||||
# Enables Network Policy -- sets up iptables to provide ingress firewall for pods
|
||||
# kube_router_run_firewall: true
|
||||
|
||||
# Enables Service Proxy -- sets up IPVS for Kubernetes Services
|
||||
# see docs/kube-router.md "Caveats" section
|
||||
# kube_router_run_service_proxy: false
|
||||
|
||||
# Add Cluster IP of the service to the RIB so that it gets advertises to the BGP peers.
|
||||
# kube_router_advertise_cluster_ip: false
|
||||
|
||||
# Add External IP of service to the RIB so that it gets advertised to the BGP peers.
|
||||
# kube_router_advertise_external_ip: false
|
||||
|
||||
# Add LoadBalancer IP of service status as set by the LB provider to the RIB so that it gets advertised to the BGP peers.
|
||||
# kube_router_advertise_loadbalancer_ip: false
|
||||
|
||||
# Adjust manifest of kube-router daemonset template with DSR needed changes
|
||||
# kube_router_enable_dsr: false
|
||||
|
||||
# Array of arbitrary extra arguments to kube-router, see
|
||||
# https://github.com/cloudnativelabs/kube-router/blob/master/docs/user-guide.md
|
||||
# kube_router_extra_args: []
|
||||
|
||||
# ASN number of the cluster, used when communicating with external BGP routers
|
||||
# kube_router_cluster_asn: ~
|
||||
|
||||
# ASN numbers of the BGP peer to which cluster nodes will advertise cluster ip and node's pod cidr.
|
||||
# kube_router_peer_router_asns: ~
|
||||
|
||||
# The ip address of the external router to which all nodes will peer and advertise the cluster ip and pod cidr's.
|
||||
# kube_router_peer_router_ips: ~
|
||||
|
||||
# The remote port of the external BGP to which all nodes will peer. If not set, default BGP port (179) will be used.
|
||||
# kube_router_peer_router_ports: ~
|
||||
|
||||
# Setups node CNI to allow hairpin mode, requires node reboots, see
|
||||
# https://github.com/cloudnativelabs/kube-router/blob/master/docs/user-guide.md#hairpin-mode
|
||||
# kube_router_support_hairpin_mode: false
|
||||
|
||||
# Select DNS Policy ClusterFirstWithHostNet, ClusterFirst, etc.
|
||||
# kube_router_dns_policy: ClusterFirstWithHostNet
|
||||
|
||||
# Array of annotations for master
|
||||
# kube_router_annotations_master: []
|
||||
|
||||
# Array of annotations for every node
|
||||
# kube_router_annotations_node: []
|
||||
|
||||
# Array of common annotations for every node
|
||||
# kube_router_annotations_all: []
|
||||
|
||||
# Enables scraping kube-router metrics with Prometheus
|
||||
# kube_router_enable_metrics: false
|
||||
|
||||
# Path to serve Prometheus metrics on
|
||||
# kube_router_metrics_path: /metrics
|
||||
|
||||
# Prometheus metrics port to use
|
||||
# kube_router_metrics_port: 9255
|
||||
@@ -0,0 +1,6 @@
|
||||
---
|
||||
# private interface, on a l2-network
|
||||
macvlan_interface: "eth1"
|
||||
|
||||
# Enable nat in default gateway network interface
|
||||
enable_nat_default_gateway: true
|
||||
@@ -0,0 +1,64 @@
|
||||
# see roles/network_plugin/weave/defaults/main.yml
|
||||
|
||||
# Weave's network password for encryption, if null then no network encryption.
|
||||
# weave_password: ~
|
||||
|
||||
# If set to 1, disable checking for new Weave Net versions (default is blank,
|
||||
# i.e. check is enabled)
|
||||
# weave_checkpoint_disable: false
|
||||
|
||||
# Soft limit on the number of connections between peers. Defaults to 100.
|
||||
# weave_conn_limit: 100
|
||||
|
||||
# Weave Net defaults to enabling hairpin on the bridge side of the veth pair
|
||||
# for containers attached. If you need to disable hairpin, e.g. your kernel is
|
||||
# one of those that can panic if hairpin is enabled, then you can disable it by
|
||||
# setting `HAIRPIN_MODE=false`.
|
||||
# weave_hairpin_mode: true
|
||||
|
||||
# The range of IP addresses used by Weave Net and the subnet they are placed in
|
||||
# (CIDR format; default 10.32.0.0/12)
|
||||
# weave_ipalloc_range: "{{ kube_pods_subnet }}"
|
||||
|
||||
# Set to 0 to disable Network Policy Controller (default is on)
|
||||
# weave_expect_npc: "{{ enable_network_policy }}"
|
||||
|
||||
# List of addresses of peers in the Kubernetes cluster (default is to fetch the
|
||||
# list from the api-server)
|
||||
# weave_kube_peers: ~
|
||||
|
||||
# Set the initialization mode of the IP Address Manager (defaults to consensus
|
||||
# amongst the KUBE_PEERS)
|
||||
# weave_ipalloc_init: ~
|
||||
|
||||
# Set the IP address used as a gateway from the Weave network to the host
|
||||
# network - this is useful if you are configuring the addon as a static pod.
|
||||
# weave_expose_ip: ~
|
||||
|
||||
# Address and port that the Weave Net daemon will serve Prometheus-style
|
||||
# metrics on (defaults to 0.0.0.0:6782)
|
||||
# weave_metrics_addr: ~
|
||||
|
||||
# Address and port that the Weave Net daemon will serve status requests on
|
||||
# (defaults to disabled)
|
||||
# weave_status_addr: ~
|
||||
|
||||
# Weave Net defaults to 1376 bytes, but you can set a smaller size if your
|
||||
# underlying network has a tighter limit, or set a larger size for better
|
||||
# performance if your network supports jumbo frames (e.g. 8916)
|
||||
# weave_mtu: 1376
|
||||
|
||||
# Set to 1 to preserve the client source IP address when accessing Service
|
||||
# annotated with `service.spec.externalTrafficPolicy=Local`. The feature works
|
||||
# only with Weave IPAM (default).
|
||||
# weave_no_masq_local: true
|
||||
|
||||
# set to nft to use nftables backend for iptables (default is iptables)
|
||||
# weave_iptables_backend: iptables
|
||||
|
||||
# Extra variables that passing to launch.sh, useful for enabling seed mode, see
|
||||
# https://www.weave.works/docs/net/latest/tasks/ipam/ipam/
|
||||
# weave_extra_args: ~
|
||||
|
||||
# Extra variables for weave_npc that passing to launch.sh, useful for change log level, ex --log-level=error
|
||||
# weave_npc_extra_args: ~
|
||||
335
kubespray/contrib/terraform/openstack/variables.tf
Normal file
335
kubespray/contrib/terraform/openstack/variables.tf
Normal file
@@ -0,0 +1,335 @@
|
||||
variable "cluster_name" {
|
||||
default = "example"
|
||||
}
|
||||
|
||||
variable "az_list" {
|
||||
description = "List of Availability Zones to use for masters in your OpenStack cluster"
|
||||
type = list(string)
|
||||
default = ["nova"]
|
||||
}
|
||||
|
||||
variable "az_list_node" {
|
||||
description = "List of Availability Zones to use for nodes in your OpenStack cluster"
|
||||
type = list(string)
|
||||
default = ["nova"]
|
||||
}
|
||||
|
||||
variable "number_of_bastions" {
|
||||
default = 1
|
||||
}
|
||||
|
||||
variable "number_of_k8s_masters" {
|
||||
default = 2
|
||||
}
|
||||
|
||||
variable "number_of_k8s_masters_no_etcd" {
|
||||
default = 2
|
||||
}
|
||||
|
||||
variable "number_of_etcd" {
|
||||
default = 2
|
||||
}
|
||||
|
||||
variable "number_of_k8s_masters_no_floating_ip" {
|
||||
default = 2
|
||||
}
|
||||
|
||||
variable "number_of_k8s_masters_no_floating_ip_no_etcd" {
|
||||
default = 2
|
||||
}
|
||||
|
||||
variable "number_of_k8s_nodes" {
|
||||
default = 1
|
||||
}
|
||||
|
||||
variable "number_of_k8s_nodes_no_floating_ip" {
|
||||
default = 1
|
||||
}
|
||||
|
||||
variable "number_of_gfs_nodes_no_floating_ip" {
|
||||
default = 0
|
||||
}
|
||||
|
||||
variable "bastion_root_volume_size_in_gb" {
|
||||
default = 0
|
||||
}
|
||||
|
||||
variable "etcd_root_volume_size_in_gb" {
|
||||
default = 0
|
||||
}
|
||||
|
||||
variable "master_root_volume_size_in_gb" {
|
||||
default = 0
|
||||
}
|
||||
|
||||
variable "node_root_volume_size_in_gb" {
|
||||
default = 0
|
||||
}
|
||||
|
||||
variable "gfs_root_volume_size_in_gb" {
|
||||
default = 0
|
||||
}
|
||||
|
||||
variable "gfs_volume_size_in_gb" {
|
||||
default = 75
|
||||
}
|
||||
|
||||
variable "master_volume_type" {
|
||||
default = "Default"
|
||||
}
|
||||
|
||||
variable "node_volume_type" {
|
||||
default = "Default"
|
||||
}
|
||||
|
||||
variable "public_key_path" {
|
||||
description = "The path of the ssh pub key"
|
||||
default = "~/.ssh/id_rsa.pub"
|
||||
}
|
||||
|
||||
variable "image" {
|
||||
description = "the image to use"
|
||||
default = ""
|
||||
}
|
||||
|
||||
variable "image_gfs" {
|
||||
description = "Glance image to use for GlusterFS"
|
||||
default = ""
|
||||
}
|
||||
|
||||
variable "ssh_user" {
|
||||
description = "used to fill out tags for ansible inventory"
|
||||
default = "ubuntu"
|
||||
}
|
||||
|
||||
variable "ssh_user_gfs" {
|
||||
description = "used to fill out tags for ansible inventory"
|
||||
default = "ubuntu"
|
||||
}
|
||||
|
||||
variable "flavor_bastion" {
|
||||
description = "Use 'openstack flavor list' command to see what your OpenStack instance uses for IDs"
|
||||
default = 3
|
||||
}
|
||||
|
||||
variable "flavor_k8s_master" {
|
||||
description = "Use 'openstack flavor list' command to see what your OpenStack instance uses for IDs"
|
||||
default = 3
|
||||
}
|
||||
|
||||
variable "flavor_k8s_node" {
|
||||
description = "Use 'openstack flavor list' command to see what your OpenStack instance uses for IDs"
|
||||
default = 3
|
||||
}
|
||||
|
||||
variable "flavor_etcd" {
|
||||
description = "Use 'openstack flavor list' command to see what your OpenStack instance uses for IDs"
|
||||
default = 3
|
||||
}
|
||||
|
||||
variable "flavor_gfs_node" {
|
||||
description = "Use 'openstack flavor list' command to see what your OpenStack instance uses for IDs"
|
||||
default = 3
|
||||
}
|
||||
|
||||
variable "network_name" {
|
||||
description = "name of the internal network to use"
|
||||
default = "internal"
|
||||
}
|
||||
|
||||
variable "use_existing_network" {
|
||||
description = "Use an existing network"
|
||||
type = bool
|
||||
default = "false"
|
||||
}
|
||||
|
||||
variable "network_dns_domain" {
|
||||
description = "dns_domain for the internal network"
|
||||
type = string
|
||||
default = null
|
||||
}
|
||||
|
||||
variable "use_neutron" {
|
||||
description = "Use neutron"
|
||||
default = 1
|
||||
}
|
||||
|
||||
variable "port_security_enabled" {
|
||||
description = "Enable port security on the internal network"
|
||||
type = bool
|
||||
default = "true"
|
||||
}
|
||||
|
||||
variable "force_null_port_security" {
|
||||
description = "Force port security to be null. Some providers does not allow setting port security"
|
||||
type = bool
|
||||
default = "false"
|
||||
}
|
||||
|
||||
variable "subnet_cidr" {
|
||||
description = "Subnet CIDR block."
|
||||
type = string
|
||||
default = "10.0.0.0/24"
|
||||
}
|
||||
|
||||
variable "dns_nameservers" {
|
||||
description = "An array of DNS name server names used by hosts in this subnet."
|
||||
type = list(string)
|
||||
default = []
|
||||
}
|
||||
|
||||
variable "k8s_master_fips" {
|
||||
description = "specific pre-existing floating IPs to use for master nodes"
|
||||
type = list(string)
|
||||
default = []
|
||||
}
|
||||
|
||||
variable "bastion_fips" {
|
||||
description = "specific pre-existing floating IPs to use for bastion node"
|
||||
type = list(string)
|
||||
default = []
|
||||
}
|
||||
|
||||
variable "floatingip_pool" {
|
||||
description = "name of the floating ip pool to use"
|
||||
default = "external"
|
||||
}
|
||||
|
||||
variable "wait_for_floatingip" {
|
||||
description = "Terraform will poll the instance until the floating IP has been associated."
|
||||
default = "false"
|
||||
}
|
||||
|
||||
variable "external_net" {
|
||||
description = "uuid of the external/public network"
|
||||
}
|
||||
|
||||
variable "supplementary_master_groups" {
|
||||
description = "supplementary kubespray ansible groups for masters, such kube_node"
|
||||
default = ""
|
||||
}
|
||||
|
||||
variable "supplementary_node_groups" {
|
||||
description = "supplementary kubespray ansible groups for worker nodes, such as kube_ingress"
|
||||
default = ""
|
||||
}
|
||||
|
||||
variable "bastion_allowed_remote_ips" {
|
||||
description = "An array of CIDRs allowed to SSH to hosts"
|
||||
type = list(string)
|
||||
default = ["0.0.0.0/0"]
|
||||
}
|
||||
|
||||
variable "master_allowed_remote_ips" {
|
||||
description = "An array of CIDRs allowed to access API of masters"
|
||||
type = list(string)
|
||||
default = ["0.0.0.0/0"]
|
||||
}
|
||||
|
||||
variable "k8s_allowed_remote_ips" {
|
||||
description = "An array of CIDRs allowed to SSH to hosts"
|
||||
type = list(string)
|
||||
default = []
|
||||
}
|
||||
|
||||
variable "k8s_allowed_egress_ips" {
|
||||
description = "An array of CIDRs allowed for egress traffic"
|
||||
type = list(string)
|
||||
default = ["0.0.0.0/0"]
|
||||
}
|
||||
|
||||
variable "master_allowed_ports" {
|
||||
type = list(any)
|
||||
|
||||
default = []
|
||||
}
|
||||
|
||||
variable "worker_allowed_ports" {
|
||||
type = list(any)
|
||||
|
||||
default = [
|
||||
{
|
||||
"protocol" = "tcp"
|
||||
"port_range_min" = 30000
|
||||
"port_range_max" = 32767
|
||||
"remote_ip_prefix" = "0.0.0.0/0"
|
||||
},
|
||||
]
|
||||
}
|
||||
|
||||
variable "bastion_allowed_ports" {
|
||||
type = list(any)
|
||||
|
||||
default = []
|
||||
}
|
||||
|
||||
variable "use_access_ip" {
|
||||
default = 1
|
||||
}
|
||||
|
||||
variable "master_server_group_policy" {
|
||||
description = "desired server group policy, e.g. anti-affinity"
|
||||
default = ""
|
||||
}
|
||||
|
||||
variable "node_server_group_policy" {
|
||||
description = "desired server group policy, e.g. anti-affinity"
|
||||
default = ""
|
||||
}
|
||||
|
||||
variable "etcd_server_group_policy" {
|
||||
description = "desired server group policy, e.g. anti-affinity"
|
||||
default = ""
|
||||
}
|
||||
|
||||
variable "router_id" {
|
||||
description = "uuid of an externally defined router to use"
|
||||
default = null
|
||||
}
|
||||
|
||||
variable "router_internal_port_id" {
|
||||
description = "uuid of the port connection our router to our network"
|
||||
default = null
|
||||
}
|
||||
|
||||
variable "k8s_masters" {
|
||||
default = {}
|
||||
}
|
||||
|
||||
variable "k8s_nodes" {
|
||||
default = {}
|
||||
}
|
||||
|
||||
variable "extra_sec_groups" {
|
||||
default = false
|
||||
}
|
||||
|
||||
variable "extra_sec_groups_name" {
|
||||
default = "custom"
|
||||
}
|
||||
|
||||
variable "image_uuid" {
|
||||
description = "uuid of image inside openstack to use"
|
||||
default = ""
|
||||
}
|
||||
|
||||
variable "image_gfs_uuid" {
|
||||
description = "uuid of image to be used on gluster fs nodes. If empty defaults to image_uuid"
|
||||
default = ""
|
||||
}
|
||||
|
||||
variable "image_master" {
|
||||
description = "uuid of image inside openstack to use"
|
||||
default = ""
|
||||
}
|
||||
|
||||
variable "image_master_uuid" {
|
||||
description = "uuid of image to be used on master nodes. If empty defaults to image_uuid"
|
||||
default = ""
|
||||
}
|
||||
|
||||
variable "group_vars_path" {
|
||||
description = "path to the inventory group vars directory"
|
||||
type = string
|
||||
default = "./group_vars"
|
||||
}
|
||||
9
kubespray/contrib/terraform/openstack/versions.tf
Normal file
9
kubespray/contrib/terraform/openstack/versions.tf
Normal file
@@ -0,0 +1,9 @@
|
||||
terraform {
|
||||
required_providers {
|
||||
openstack = {
|
||||
source = "terraform-provider-openstack/openstack"
|
||||
version = "~> 1.17"
|
||||
}
|
||||
}
|
||||
required_version = ">= 0.12.26"
|
||||
}
|
||||
Reference in New Issue
Block a user