DocumentationAPI ReferenceTerraform ProviderStatus PageSchedule a DemoFree Trial

Self Hosted Kubernetes Agent

Suggest Edits

When hosting your own env0 Kubernetes Agent, you can choose an image that is leaner than the one used in env0 SaaS. You might not need all of the extra CLIs and capabilities that exist in our main image. So in such a case, you can use a "lean" image - This image will only contain the bare minimum of what is necessary to run env0, and its size will be drastically smaller than the main image

Using the lean Docker image in your agent

For each main image we build, we also create a "lean" image. If the main image is tagged ghcr.io/env0/deployment-agent:<image_tag>, then the lean image would be tagged ghcr.io/env0/deployment-agent-lean:<image_tag>

Simply follow this guide for using a custom image, and pick the lean image you would like to use

👍

Using the lean Docker image on ARM server architecture

To use the lean Docker image on an arm64 (AKA aarch64) server, look for arm64 image tags in our container registry. The latest tag is ghcr.io/env0/deployment-agent-lean:prod-arm64.

Extending the lean image

You might like to add some CLIs or other capabilities to your lean image. For example, you can add Infracost to the lean image, in order to support cost estimation for your Terraform plans.

You can follow this guide to see how you can extend your image, and add anything you'd like to it

Here are some examples of how to extend the image:

Adding AWS CLI v2

# default parameters 
ARG BASE_LEAN_IMAGE=ghcr.io/env0/deployment-agent-lean:latest
ARG AWS_CLI_VERSION=2.15.50

FROM python:3.11-alpine3.19 as builder

RUN apk add --no-cache git unzip groff build-base libffi-dev cmake
RUN git clone --single-branch --depth 1 -b ${AWS_CLI_VERSION} https://github.com/aws/aws-cli.git

RUN cd aws-cli \
    && ./configure --with-install-type=portable-exe --with-download-deps \
    && make \
    && make install

# reduce image size: remove autocomplete and examples
RUN rm -rf \
    /usr/local/lib/aws-cli/aws_completer \
    /usr/local/lib/aws-cli/awscli/data/ac.index \
    /usr/local/lib/aws-cli/awscli/examples
RUN find /usr/local/lib/aws-cli/awscli/data -name completions-1*.json -delete
RUN find /usr/local/lib/aws-cli/awscli/botocore/data -name examples-1.json -delete
RUN (cd /usr/local/lib/aws-cli; for a in *.so*; do test -f /lib/$a && rm $a; done)

# copy aws v2 cli to the final image
FROM ${BASE_LEAN_IMAGE} as final-image
COPY --from=builder /usr/local/lib/aws-cli/ /usr/local/lib/aws-cli/v2
RUN ln -s /usr/local/lib/aws-cli/v2/aws /usr/local/bin/awsv2
RUN ln -s /usr/local/lib/aws-cli/v2/aws /usr/local/bin/aws

RUN aws --version

USER root
 
# install glibc for aws cli dependencies
ARG GLIBC_VER=2.35-r1
RUN apk add --no-cache \
        binutils \
        groff \
&& curl -sL https://alpine-pkgs.sgerrand.com/sgerrand.rsa.pub -o /etc/apk/keys/sgerrand.rsa.pub \
&& curl -sLO https://github.com/sgerrand/alpine-pkg-glibc/releases/download/${GLIBC_VER}/glibc-${GLIBC_VER}.apk \
&& curl -sLO https://github.com/sgerrand/alpine-pkg-glibc/releases/download/${GLIBC_VER}/glibc-bin-${GLIBC_VER}.apk \
&& curl -sLO https://github.com/sgerrand/alpine-pkg-glibc/releases/download/${GLIBC_VER}/glibc-i18n-${GLIBC_VER}.apk \
&& apk add --no-cache --force-overwrite \
        glibc-${GLIBC_VER}.apk \
        glibc-bin-${GLIBC_VER}.apk \
        glibc-i18n-${GLIBC_VER}.apk \
&& /usr/glibc-compat/bin/localedef -i en_US -f UTF-8 en_US.UTF-8 \
&& rm -rf \
        glibc-*.apk \
        /var/cache/apk/*

# Make sure there is NO ENTRYPOINT in this file
# If you want to use CMD, make sure to add `&& npm start` at the end of it

Adding Infracost

# Use the env0 lean image as the base image
FROM ghcr.io/env0/deployment-agent-lean:<image_tag>

# Install infracost - check https://github.com/infracost/infracost/releases for latest version
ARG INFRACOST_VERSION=0.10.29
RUN curl -sL https://github.com/infracost/infracost/releases/download/v${INFRACOST_VERSION}/infracost-linux-amd64.tar.gz | tar xz  \
    && mv infracost-linux-amd64 /opt/infracost
    
# Make sure there is NO ENTRYPOINT in this file
# If you want to use CMD, make sure to add `&& npm start` at the end of it

🚧

Using infracost in a self hosted agent

Note that in order to run cost estimation on your env0 agent, you'd need to install the agent with your own infracost API key. See infracostApiKeyEncoded under self hosted agent optional configuration

Adding Terratag

# Use the env0 lean image as the base image
FROM ghcr.io/env0/deployment-agent-lean:<image_tag>

# Install Terratag
ARG TERRATAG_VERSION=0.2.4
RUN npm -g install "@env0/terratag@$TERRATAG_VERSION" \
    && mv /usr/local/lib/node_modules/@env0/terratag/terratag /opt/terratag \
    && chown -R node /opt/terratag \
    # Verifies terratag installation
    && /opt/terratag -h

📘

Terratag

You should use Terratag if you wish to use cost monitoring and custom resource tagging.

Adding tgenv

# Use the env0 lean image as the base image
FROM ghcr.io/env0/deployment-agent-lean:<image_tag>

# Install and configure tgenv
ARG TGENV_VERSION=0.0.9
RUN git clone --branch v${TGENV_VERSION} https://github.com/env0/tgenv.git /opt/tgenv
ENV PATH="/opt/tgenv/bin:${PATH}"
RUN chown -R node /opt/tgenv

Modifying the UID

env0 uses a default Node user with UID 1000; in some k8s clusters you may have policies that restrict the runAsUser to a different UID. 

# Modify user ID User
RUN apk add --no-cache -U shadow
RUN usermod -u 2077 node
RUN apk del -U shadow

Note: You will also need to include in your customer.values.yaml a runAsUser to match the UID used in your Dockerfile.

Updated 4 days ago