[BUG] Cannot properly create and destroy jails on separate UFS volumes/mountpoints

[michael-o]

[MANDATORY] Describe the bug [MANDATORY]
When operating on UFS only with separate volumes bastille does not properly detect that and tries either do delete the mountpoint or says it already exits and the jail cannot be created.

[MANDATORY] Bastille and FreeBSD version (paste bastille -v && freebsd-version -kru output)
0.9.20220714
12.4-STABLE
12.4-STABLE
12.4-STABLE

[MANDATORY] How did you install bastille? (port/pkg/git)
port

[optional] Steps to reproduce?
Disclamer: Setup has been reproduced in a bhyve VM, but the actual setup is on a server with UFS/gvinum which I cannot and will not change for the years the server will live.

mounts:

# mount
/dev/vtbd0p3 on / (ufs, local, journaled soft-updates)
devfs on /dev (devfs, local, multilabel)
fdescfs on /dev/fd (fdescfs)
procfs on /proc (procfs, local)
/dev/gvinum/bastille on /usr/local/bastille (ufs, local, journaled soft-updates)
/dev/gvinum/deblndw013x10v1j on /usr/local/bastille/jails/deblndw013x10v1j (ufs, local, journaled soft-updates)
map -hosts on /net (autofs)
map automounts on /net (autofs)

# sh -x /usr/local/share/bastille/create.sh -T -V deblndw013x10v1j 12.4-RELEASE 146.254.218.211 vtnet0
...
+ [ -d /usr/local/bastille/jails/deblndw013x10v1j ]
+ error_exit 'Jail: deblndw013x10v1j already created.'
+ error_notify Jail: deblndw013x10v1j already created.

after fiddling creating the jail I want to destroy it:

# bastille destroy force deblndw013x10v1j
Deleting Jail: deblndw013x10v1j.
rm: /usr/local/bastille/jails/deblndw013x10v1j/.sujournal: Operation not permitted
rm: /usr/local/bastille/jails/deblndw013x10v1j: Device busy

So basically I cannot create separate volumes to have control of the jails, but need to maintain a single FS for them, unfortunately.

gvinum setup:

# gvinum l
1 drive:
D s1                    State: up       /dev/vtbd1p1    A: 16381/40957 MB (39%)

2 volumes:
V bastille              State: up       Plexes:       1 Size:       4096 MB
V deblndw013x10v1j      State: up       Plexes:       1 Size:         20 GB

2 plexes:
P bastille.p0         C State: up       Subdisks:     1 Size:       4096 MB
P deblndw013x10v1j.p0  C State: up      Subdisks:     1 Size:         20 GB

2 subdisks:
S bastille.p0.s0        State: up       D: s1           Size:       4096 MB
S deblndw013x10v1j.p0.s0 State: up      D: s1           Size:         20 GB

[michael-o]

Maybe bastille could run the following: df /usr/local/bastille/jails/deblndw013x10v1j | tail -n +2 | cut -w -f 6 and then compare the output to the jail basedir?

[tschettervictor]

@michael-o Is this still relevant?

I don't use UFS but wondering...

[michael-o]

@michael-o Is this still relevant?

I don't use UFS but wondering...

It is

[tschettervictor]

Currently when destroying a jail, and mount points are detected, Bastille will show an error and not continue. Would this help or is it irrelevant?

I guess because I don't use UFS I'm not exactly sure what you're trying to do.

[michael-o]

Currently when destroying a jail, and mount points are detected, Bastille will show an error and not continue. Would this help or is it irrelevant?

I guess because I don't use UFS I'm not exactly sure what you're trying to do.

I think the approach could be simple: If you haven't created the mountpoint, don't try to destroy it?

I need to create the root FS mountpoints upfront with UFS to have a reserved amount of space out of the gvinum volume.

[tschettervictor]

Given your first post, it seems you have created the jail directory already. Is that correct? Bastille checks for that and will error if it is already created.

[michael-o]

Given your first post, it seems you have created the jail directory already. Is that correct? Bastille checks for that and will error if it is already created.

Yes, correct because Batille will never k own how to create the UFS mount. That magic only works with ZFS.

[tschettervictor]

Oh I see what you're doing.

So basically, Bastille sees the existing directory (UFS mount) and fails to create the jail, so it destroys it, and that tries to destroy the mount?

Or does the jail actually get created?

[michael-o]

Oh I see what you're doing.

So basically, Bastille sees the existing directory (UFS mount) and fails to create the jail, so it destroys it, and that tries to destroy the mount?

Or does the jail actually get created?

It doesn't even destroy. It fails, but I need to test again what the actual behavior is. Need to test again. Will take a bit of time since I cannot touch the prod system and my test system is newer and runs on ZFS on purpose.