Introduction

When having multiple servers you have to do the same thing on its nice with automation - luckily you dont have to invent the wheel yourself.

I decided to use ansible to enable cephfs on all my kubernetes nodes.

This is a simple playbook that can be used by others if required.

Assumptions

To use the playbook verbatim you have to have a working ceph.conf and a working keyring for your user located in the same directoy as the ansible play book.

I am using ceph quincy - if you want to use another, you have to change the baseurls for the ceph repositories that is being added.

You have a hosts.yaml file with the hosts you want to deploy this to - I have named my group k8s.

Step by step description

The playbook first starts by installing/verifying prerequisites.

These prerequisites is that the ceph repositories are install and that the EPEL repository is installed.

When that has been satisfied, either because they were already installed - or they are being installed, then it proceeds to install the ceph-common package.

When the package has been installed, then it downloads the ceph.conf and the ceph.client.kubedata.keyring into the directory /etc/ceph on the destination host.

And as the last bit, it ensures that the cephfs filesystem is added to /etc/fstab and mounted - so the cephfs filesystem is ready for use.

Very simple if you know what is required and how to write the ansible playbook.

Ansible playbook

---
- name: Enable ceph on kubernetes hosts
  gather_facts: true
  hosts: k8s
  tasks:
    - name: EPEL repository
      ansible.builtin.yum:
        name: epel-release
        state: present
      become: true
    - name: Ceph repository
      ansible.builtin.yum_repository:
        name: Ceph-noarch
        description: Ceph noarch
        file: ceph
        baseurl: https://download.ceph.com/rpm-quincy/el8/noarch
        gpgcheck: yes
        gpgkey: https://download.ceph.com/keys/release.gpg
      become: true
    - name: Ceph repository $basearch
      ansible.builtin.yum_repository:
        name: Ceph
        description: Ceph $basearch
        file: ceph
        baseurl: https://download.ceph.com/rpm-quincy/el8/$basearch
        gpgcheck: yes
        gpgkey: https://download.ceph.com/keys/release.gpg
      become: true
    - name: Install ceph
      ansible.builtin.dnf:
        name: ceph-common
        state: latest
      become: true
    - name: Download ceph config
      ansible.builtin.copy:
        src: '{{playbook_dir}}/ceph.conf'
        dest: /etc/ceph/ceph.conf
        owner: root
        group: root
        mode: '0664'
      become: true
    - name: Download ceph user key
      ansible.builtin.copy:
        src: '{{playbook_dir}}/ceph.client.kubedata.keyring'
        dest: /etc/ceph/ceph.client.kubedata.keyring
        owner: root
        group: root
        mode: '0664'
      become: true
    - name: update fstab
      ansible.posix.mount:
        fstype: ceph
        path: /mnt/kubedata
        state: mounted
        src: [email protected]=/
        opts: mon_addr=192.168.210.10:6789,rw,noatime,_netdev
      become: true

Running the playbook

To run the playbook - cd into the directory where you have the playbook and the configuration files and run the following:

ansible-playbook ./enable-cephfs-kubedata.yml -i hosts.yml --ask-become-pass

Assuming you copied the contents of the playbook into the file enable-cephfs-kubedata.yml.

The status of the run of the playbook might look like the one below, which is from my homelab.

TASK [Gathering Facts] *******************************************************************************************************************************************************************************************************
ok: [wyse1.root.dom]
ok: [wyse2.root.dom]
ok: [wyse3.root.dom]
ok: [wyse4.root.dom]
ok: [wyse5.root.dom]

TASK [EPEL repository] *******************************************************************************************************************************************************************************************************
ok: [wyse4.root.dom]
ok: [wyse3.root.dom]
ok: [wyse2.root.dom]
ok: [wyse1.root.dom]
ok: [wyse5.root.dom]

TASK [Ceph repository] *******************************************************************************************************************************************************************************************************
ok: [wyse2.root.dom]
ok: [wyse3.root.dom]
ok: [wyse5.root.dom]
ok: [wyse4.root.dom]
ok: [wyse1.root.dom]

TASK [Ceph repository $basearch] *******************************************************************************************************************************************************************************************************
ok: [wyse2.root.dom]
ok: [wyse3.root.dom]
ok: [wyse1.root.dom]
ok: [wyse4.root.dom]
ok: [wyse5.root.dom]

TASK [Install ceph] **********************************************************************************************************************************************************************************************************
ok: [wyse4.root.dom]
ok: [wyse2.root.dom]
ok: [wyse1.root.dom]
ok: [wyse3.root.dom]
ok: [wyse5.root.dom]

TASK [Download ceph config] **************************************************************************************************************************************************************************************************
ok: [wyse4.root.dom]
ok: [wyse2.root.dom]
ok: [wyse5.root.dom]
ok: [wyse3.root.dom]
ok: [wyse1.root.dom]

TASK [Download ceph user key] ************************************************************************************************************************************************************************************************
ok: [wyse1.root.dom]
ok: [wyse4.root.dom]
ok: [wyse3.root.dom]
ok: [wyse2.root.dom]
ok: [wyse5.root.dom]

TASK [update fstab] **********************************************************************************************************************************************************************************************************
ok: [wyse2.root.dom]
ok: [wyse1.root.dom]
ok: [wyse4.root.dom]
ok: [wyse3.root.dom]
ok: [wyse5.root.dom]

PLAY RECAP *******************************************************************************************************************************************************************************************************************
wyse1.root.dom             : ok=8    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0
wyse2.root.dom             : ok=8    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0
wyse3.root.dom             : ok=8    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0
wyse4.root.dom             : ok=8    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0
wyse5.root.dom             : ok=8    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

I hope you enjoyed this post and if you spot errors, please let me know in the comments below on on email directly.