Kubernetes with CoreOS on AWS

This article provides reproducible steps for quick start of kubernetes cluster on AWS. The kube-aws part is roundup of CoreOs official Guide for kube-aws and first address for more details on that.

Let start...

Configure kube-aws

I'm using kube-aws on linux and version 0.8.3 of it, which results in

wget https://github.com/coreos/coreos-kubernetes/releases/download/v0.8.3/kube-aws-linux-amd64.tar.gz tar zxvf kube-aws-linux-amd64.tar.gz sudo mv linux-amd64/kube-aws /usr/local/bin

Installed by that. Don't forget to verifyyour binaries.

AWS Credentials

If you don't have it installed, please follow the Official Guide

To check if you are authenticated use aws ec2 describe-instances . If not authenticates so fare, you need to provide auth informations and AWS Region you want use. AWS Access Key ID and AWS Secret Access Key are Security Credentials of your user that has enough right to setup that all.

Here is example of how it might look like

aws configure AWS Access Key ID [None]: AKID1234567890 AWS Secret Access Key [None]: MY-SECRET-KEY Default region name [None]: us-west-2 Default output format [None]: text Preconfigure AWS Resources

Following AWS Resources are mandatory, so you need to prepare them in your AWS as well.

EC2 key pair that will be used. KMS key (Encryption key) External DNS name EC2 Key pair

Existing Key pairs can be found under EC2 section of AWS Web UI. Alternatively you can create new KeyPair by:

aws ec2 create-key-pair --key-name MyKeyPair

Of course you can upload existing public key as well.

AWS KMS

Now KMS. Probably you've not used encryption key before or want to use separate one for kubernetes. You can create one by:

aws kms --region=<your-region> create-key --description="kube-aws assets"

which will give you the arn of the new key.

DNS name for cluster access

You need to define DNS hostname where the cluster API will be accessible. This hostname will be used to provision the TLS certificate for the API server

When the cluster is created, the controller will expose the TLS-secured API on a public IP address. You will need to create an A record for the selected DNS hostname you want to point to this IP address.

Initialize an asset directory

Now everything is ready to create assets.

kube-aws init \ --cluster-name=kube-dev \ --external-dns-name=kube.mydomain.com \ --region=eu-central-1 \ --availability-zone=eu-central-1a \ --key-name=id_aws \ --kms-key-arn="arn:aws:kms:eu-central-1:588652342245:key/xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"

This will create cluster.yaml file based on initial values.

Check default values maybe you want to improve some of them. Unfortunately at the moment they are noo that good documented.

Generate contents of the asset directory

Now you can generate assets based on cluster.yaml file.

$kube-aws render WARNING: The generated client TLS CA cert expires in 3650 days and the server and client cert expire in 365 days. It is recommended that you create your own TLS infrastructure for revocation and rotation of keys before using in prod Success! Stack rendered to stack-template.json. Next steps: 1. (Optional) Validate your changes to cluster.yaml with "kube-aws validate" 2. (Optional) Further customize the cluster by modifying stack-template.json or files in ./userdata. 3. Start the cluster with "kube-aws up".

As you see i've generated assets using new random CA, it's Ok for trying but maybe not that convinient for production, so consider to use your own PKI stack.

However validate results.

$kube-aws validate Validating UserData... UserData is valid. Validating stack template... Validation Report: { Capabilities: ["CAPABILITY_IAM"], CapabilitiesReason: "The following resource(s) require capabilities: [AWS::IAM::Role]", Description: "kube-aws Kubernetes cluster kube-dev" } stack template is valid. Validation OK! Create the instances in the AWS

With

kube-aws up

Resource are starting beeing rendered in the AWS Cloud. It took about 5 minutes in my case.

Testing new cluster

To start use new cluster you need to setuo kubectl tool first.

Getting kubectl

Since it part of kubernetes release we need to dowload it first from here , like e.g.:

wget https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG.md#downloads-for-v145

then unzip and setup

tar xzf kubernetes-client-linux-amd64.tar.gz sudo cp kubernetes/client/bin/kubectl /usr/local/bin/kubectl sudo chmod +x /usr/local/bin/kubectl Running first tests

Now kubectl is installed, being in the directory of the cluster data you can use it to print node list of the cluster:

$kubectl --kubeconfig=kubeconfig get nodes NAME STATUS AGE ip-10-0-0-125.eu-central-1.compute.internal Ready 1h ip-10-0-0-50.eu-central-1.compute.internal Ready,SchedulingDisabled 1h Destroy

You can destroy your kubernetes cluster by executing.

kube-aws up Summary

My expirience with aws-kube is kind of bifid.

Pro Like a lot CoreOS as linux distribution for containers. Works well with defaults Contra Didn't used my existing VPC. (Should be possible meanwhile, but didn't found out) I miss a list of reources beeing created by Cloudformation. It would be nice to see them all as a list Documentetion is not on level i've expect it to be for production readiness But also feature set seem not to be on production readiness so far. While destroying kluster, some resources could not be deleted including Security groups, IAM roles and VPC as well Hard to reuse preconfigured resources, not possible or not documented.

Therefore kube-aws do not feel production ready at the moment. Looks like the best way in the long run is to install kubernetes manually and be aware of all the details.