.NET ZFS REST API

Sun, Oct 2, 2022 5-minute read

Introduction

ZFS is a highly scalable and resilient filesystem and many people use it as part of NAS systems like TrueNAS, napp-it and many others.

These NAS systems usually have a UI to manage ZFS and thats great - but if you want to install ZFS on a linux server and do not want a full blown NAS solution, then it gets tricky, since you cannot just install a UI to manage your ZFS installation.

I have wanted to roll my own NAS like solution for a very long time, so finally I decided to to some of the prequisite work.

Which means I have started the work of building a ZFS .NET library and a ZFS REST API.

Everything is still early days, but I have basic functionality which means you can inspect and manipulate certain parts of a pool.

Building blocks

To make a REST API there are many ways to do, but I decided to split the work into two distinct areas.

One is a ZFS library built on .NET 6. It interfaces with the zfs, zpool and other linux binaries by starting a process and reading the response. Similar to what you would do if you wanted to script something with a bash script or similar.

The advantage of having it as a library, is that I can encapsulate all the nitty gritty details of invoking processes, reading output, parsing lines etc in this library.

Any API that is being built on top of this just have a simple interface and requires no knowledge of how ZFS responds to the different commands.

On top of this library I have built a .NET 6 ASP.NET REST API that exposes a REST interface that utilizes the ZFS Library.

This means the actual ZFS knowledge required inside the API is fairly small, since all the zfs logic is encapsulated inside the library.

So the API is more or less just a thin wrapper around the ZFS library, with authentication, serialization etc on top.

ZFS Library

The ZFS library uses another library I have made, which contains the logic for making processes and getting the response.

This has the obvious advantage that the ZFS library only have to care about ZFS and not making processes etc - again to encapsulate logic where it belongs and not have it scattered around.

The library has a single public interface that exposes everything via member properties

/// <summary>
/// Public interface for the ZFS library.
/// Exposes functionality to manipulate a zfs installation via the binaries 'zfs' and 'zpool'
/// </summary>
public interface IZfs
{
    /// <summary>
    /// Contains snapshot related functionality
    /// </summary>
    ISnapshots Snapshots { get; }
        
    /// <summary>
    /// Contains dataset related functionality
    /// </summary>
    IDataSets DataSets { get; }
        
    /// <summary>
    /// Contains property related functionality
    /// </summary>
    IProperties Properties { get; }
        
    /// <summary>
    /// Contains pool related functionality
    /// </summary>
    IZPool Pool { get; }

    /// <summary>
    /// Gets or set the maximum wait time to wait for a command to process fully
    /// Current default timeout is set to 30 seconds.
    /// When the timeout is reached, the command is aborted - if possible and an exception is raised.
    /// There are no guarantees that the command will be aborted - it is aborted on best effort.
    /// </summary>
    TimeSpan CommandTimeout { get; set; }

    /// <summary>
    /// Gets information about zfs version from the underlying OS.
    /// </summary>
    VersionInfo GetVersionInfo();

    /// <summary>
    /// Lists the disks in the system
    /// </summary>
    IEnumerable<DiskInfo> ListDisks();
}

It is used like this:

var zfs = new Zfs();
var datasets = zfs.DataSets.GetDataSets();
foreach(var dataset in datasets)
{
  var atime = zfs.Properies.GetProperty(dataset.Name,"atime");
  Console.WriteLine(atime.Value);
}

The interface is still work in progress - I might move things around when its more finalized, but in general terms, I think its fairly easy to use.

The source code also contains both unit tests and integration tests that calls to a zfs installation.

These tests are an excellent example of how to use the library.

Test coverage is not full yet, but I am to have coverage as high as possible.

ZFS REST API

The REST API is built using ASP.NET controllers with routes defined for the different http methods supported.

I have used swagger to generate an API test application that is embedded into the solution. An example of the swagger UI is:

Zfs Api Swagger

I aim to provide a REST api as pure as possible, so its real resources that gets manipulated and not just a “REST” like RPC api.

Right now data is being returned as json and the api also requires json when posting or putting data, but in the future I plan to add real support the the Accept header, so a client can send

Accept: text/xml and the server would respond with xml.

The API uses basic authentication, since its simple to implement. In the future more advanced authenticaion mechanisms will be implemented if someone requests it.

The actual authentication happens via Linux PAM, where the username/password is sent to the underlying linux server and authenticated there.

The authentication is pluggable, so if someone wants to build another authentication module, its possible.

In the future the logged in user can also be used to invoking the actual calls to the zfs binaries - but right now its the API user that is running the actual zfs commands.

Other usage

Having built a .NET ZFS Library also opens up for the possiblity that someone else can build something utilizing that.

It could be a “fat” application that can manage a zfs installation or a web application and so forth.

Had I bundled everything together into the ZFS API it would be more work to make something else, since people would first have to disect and extract the required bits to access ZFS.

Source code

I have put the source code on github, licensed under MS-PL

ZFS Library Source code

ZFS API Source Code

Shared support library

Comments

Any comments, ideas or whatever, feel free to add a comment below or PM me.

I could use help for some of the work, so if you think its an interesting project and you want to participare, please get in touch.