Linux FUSE Internals

In this blog we will go through the code architecture and internal working of linux FUSE driver, user space callbacks etc.

Many good links are present on the internet for FUSE workings.

Few of these :
https://libfuse.github.io/doxygen/fast17-vangoor.pdf
https://github.com/libfuse/libfuse/wiki/Fsnotify-and-FUSE

These guys dont say how to form a fuse and codewise details of FUSE.
Lets try to address the same in this blog:

Using the FUSE.
There is a good implementation of fuse filesystem at : https://www.cs.nmsu.edu/~pfeiffer/fuse-tutorial.

The BBFS discussed here just use one call to launch a fuse daemon :

    fuse_stat = fuse_main(argc, argv, &bb_oper, bb_data);

Here only fuse_main function is used to register the bb_oper user space operations to fuse and open the fuse daemon.

fuse_main function implementation can be found in libfuse
libfuse : https://github.com/libfuse/libfuse

Lets see the code flow of fuse_main :
#define fuse_main(argc, argv, op, private_data) \
fuse_main_real(argc, argv, op, sizeof(*(op)), private_data)

libfuse has the function calls from the fuse_main_real :
fuse_main_real
fuse_parse_cmdline
fuse_new
fuse_mount
fuse_daemonize
fuse_loop

Lets see what fuse_new function does :
    fs = fuse_fs_new(op, op_size, user_data);
        f->se = fuse_session_new(args, &llop, sizeof(llop), f);

Next call is fuse_mount(g_fuse, g_mountpoint);
    Inside function calls are like fuse_kern_mount
    fuse_mount_sys
    mount

As the mount is done as per fuse filesystem any write on the dir will result in VFS write: 

static const struct file_operations fuse_file_operations = {
.llseek = fuse_file_llseek,
.read_iter = fuse_file_read_iter,
.write_iter = fuse_file_write_iter,
.mmap = fuse_file_mmap,
.open = fuse_open,
.flush = fuse_flush,
.release = fuse_release,
.fsync = fuse_fsync,
.lock = fuse_file_lock,
.flock = fuse_file_flock,
.splice_read = generic_file_splice_read,
.unlocked_ioctl = fuse_file_ioctl,
.compat_ioctl = fuse_file_compat_ioctl,
.poll = fuse_file_poll,
.fallocate = fuse_file_fallocate,
};

These reads/writes of the FUSE filesystem shall call the user space part of the FUSE. Lets see how the control moves to the user space :

static ssize_t fuse_file_write_iter(struct kiocb *iocb, struct iov_iter *from)
{
struct file *file = iocb->ki_filp;
struct address_space *mapping = file->f_mapping;
..
written_buffered = fuse_perform_write(iocb, mapping, from, pos);
..

num_written = fuse_send_write_pages(req, iocb, inode,
    pos, count);
...
..
}

fuse_send_write_pages calls fuse_send_write
fuse_send_write calls __fuse_request_send

This function does queue_request
queue_request adds the request to fiq->pending queue
(&req->list, &fiq->pending)

So we see that the request is added to FUSE Queue called fuse_iqueue.

Readers are woken up : Then it wakes up waitq of FUSE : wake_up_locked(&fiq->waitq);

Lets see how many readers are present in fiq->waitq.
As how many readers are added to this queue they will be woken up when request is added to this list.

The readers are present here: And they are reading at /dev/fuse char device.
const struct file_operations fuse_dev_operations = {
.owner = THIS_MODULE,
.open = fuse_dev_open,
.llseek = no_llseek,
.read_iter = fuse_dev_read,
.splice_read = fuse_dev_splice_read,
.write_iter = fuse_dev_write,
.splice_write = fuse_dev_splice_write,
.poll = fuse_dev_poll,
.release = fuse_dev_release,
.fasync = fuse_dev_fasync,
.unlocked_ioctl = fuse_dev_ioctl,
.compat_ioctl   = fuse_dev_ioctl,
};
EXPORT_SYMBOL_GPL(fuse_dev_operations);

static struct miscdevice fuse_miscdevice = {
.minor = FUSE_MINOR,
.name  = "fuse",
.fops = &fuse_dev_operations,
};

So from the user space if someone does read on the /dev/fuse then the call flow is
fuse_dev_read
    fuse_dev_do_read
        ....
        err = wait_event_interruptible_exclusive_locked(fiq->waitq,
                                !fiq->connected || request_pending(fiq));
        ....
        Now the requests are taken out from the fiq->pending queue
        req = list_entry(fiq->pending.next, struct fuse_req, list);
        Copy is done from the request
        err = fuse_copy_one(cs, &in->h, sizeof(in->h));
        now the request is put in the processing queue
        list_move_tail(&req->list, &fpq->processing);


Now we understand that a read in the userspace on /dev/fuse takes the data using the fuse iqueue from the kernel.


Who is calling the read on /dev/fuse ?
Lets recall that once we call fuse_main these functions gets called :
libfuse has the function calls from the fuse_main_real :
fuse_main_real
fuse_parse_cmdline
fuse_new
fuse_mount
fuse_daemonize
fuse_loop

The functions getting launched. One of these functions establishes session:
fuse_mount
fuse_session_mount
fuse_kern_mount  -- opens a channel:
This function calls fuse_mount_sys to open the "/dev/fuse" file
This file descriptor is saved in session->fd rather se->fd

Now when the fuse_loop is called :
fuse_loop 
fuse_session_loop(f->se)
fuse_session_receive_buf_int(se, &fbuf, NULL);
read(ch ? ch->fd : se->fd, buf->mem, se->bufsize);