LinearTapeFileSystem
ltfs file backend does not work with a format and type different from LTO5
Describe the bug
Even tho you can change the tape type and density using cart_type and density_code, if you use any other format besides LTO5 it will not format.
To Reproduce
-
mkdir tape -
mkltfs -e file -d tape -
cd tape - Change the backend configuration to use any density and type different from LTO5, e.g.
<?xml version="1.0" encoding="UTF-8"?>
<filedebug_cartridge_config>
<dummy_io>false</dummy_io>
<emulate_readonly>false</emulate_readonly>
<capacity_mb>204800</capacity_mb>
<cart_type>L6</cart_type>
<density_code>5A</density_code>
<delay_mode>None</delay_mode>
<wraps>40</wraps>
<change_direction_us>2000000</change_direction_us>
<change_track_us>10000</change_track_us>
<threading_sec>0</threading_sec>
</filedebug_cartridge_config>
-
mkltfs -f -e file -d tape
LTFS15000I Starting mkltfs, LTFS version 2.5.0.0 (Prelim), log level 3.
LTFS15041I Launched by "mkltfs -f -t -e file -d tape0".
LTFS15042I This binary is built for Linux (x86_64).
LTFS15043I GCC version is 11.4.1 20231218 (Red Hat 11.4.1-3).
LTFS17087I Kernel version: Linux version 5.14.0-427.13.1.el9_4.x86_64 ([email protected]) (gcc (GCC) 11.4.1 20231218 (Red Hat 11.4.1-3), GNU ld version 2.35.2-43.el9) #1 SMP PREEMPT_DYNAMIC Wed Apr 10 10:29:16 EDT 2024 i386.
LTFS17089I Distribution: NAME="Red Hat Enterprise Linux".
LTFS17089I Distribution: Red Hat Enterprise Linux release 9.4 (Plow).
LTFS17089I Distribution: Red Hat Enterprise Linux release 9.4 (Plow).
LTFS15025D Validating command line options.
LTFS15037D Command line options are valid.
LTFS15003I Formatting device 'tape0'.
LTFS15004I LTFS volume blocksize: 524288.
LTFS15005I Index partition placement policy: None.
LTFS11337I Update index-dirty flag (1) - NO_BARCODE (0x0x55738f543c50).
LTFS15006D Opening the device.
LTFS17085I Plugin: Loading "file" tape backend.
LTFS30000I Opening a device through generic file driver (tape0).
LTFS30003I Opening a directory through generic file driver (tape0).
LTFS12023D Reserving device.
LTFS12028D Unlocking medium.
LTFS30061E Cannot unlock medium: unit not ready.
LTFS17160I Maximum device block size is 4194304.
LTFS11330I Loading cartridge.
LTFS30048I Loading a directory through generic file driver (tape0).
LTFS30086I Cartridge is unsupported (LTO5, 0x68).
LTFS11331E Failed to load the cartridge (ltfs_load_tape).
LTFS17157I Changing the drive setting to write-anywhere mode.
LTFS15007D Device opened.
LTFS15049I Checking the medium (load).
LTFS30198D Backend readpos: (0, 0) FM = 0.
LTFS15010I Creating data partition b on SCSI partition 1.
LTFS15011I Creating index partition a on SCSI partition 0.
LTFS17254E This cartridge cannot be reformatted in the drive (0x68, 5).
LTFS15012E Failed to format the medium.
LTFS12028D Unlocking medium.
LTFS12025D Releasing device.
LTFS15023I Formatting failed.
Expected behavior It should format to any other format available
Desktop (please complete the following information):
- OS: RHEL 9.4
Additional context
The reason is becase it's hardcoded in filedebug_tc.c:411
state->product_id = "ULTRIUM-TD5";
But also it tries to get the drive_type from the serial of the tape wich is weird.
for (i = 0; i < (int)strlen(state->serial_number); i++) {
if (state->serial_number[i] == '.') {
state->serial_number[i] = '\0';
state->product_id = &(state->serial_number[i+1]);
break;
}
}
In this part gets the first compatible drive from the ibm_supported_drives variable, wich is an ULT3580-TD5 that is not compatible with any other tape.
while(*d_cur) {
if((! strncmp(IBM_VENDOR_ID, (*d_cur)->vendor_id, strlen((*d_cur)->vendor_id)) ) &&
(! strncmp(state->product_id, (*d_cur)->product_id, strlen((*d_cur)->product_id)) ) ) {
state->drive_type = (*d_cur)->drive_type;
break;
}
d_cur++;
}
I'm currently working in a new option for the xml file to specify the drive type
I found out that the tape is hardcoded in filedebug_tc.c:411, but also that it uses the barcode to get the drive_type wich is a weird way to put it really, it should not be that way and it should be specified in the documentation
This works as designed. If there is no tape spec XML in a directory, LTO5 cartridge type is selected by default. Bu tif it is, cartridge type in the XMl is used. So you need to prepare tape spec XML before formatting a tape. It is really natural because real tape cartridge already fixed a cartridge type before format.
The file backend works as real world tape emulation. So this is intentional design.
This works as designed. If there is no tape spec XML in a directory, LTO5 cartridge type is selected by default.
Even if filedebug_tc_conf.xml is already made in the folder the command stills fails, because the emulated drive that the file backend uses is only compatible with lto5, you could try to make a file backend lto8 tape if you want.
Bu tif it is, cartridge type in the XMl is used. So you need to prepare tape spec XML before formatting a tape. It is really natural because real tape cartridge already fixed a cartridge type before format.
The file backend works as real world tape emulation. So this is intentional design.
I dont get it, if there is a different behaviour that I'm not aware of could you point it out to me? If there is that I could add it to the documentation because there's nothing that helps me in the docs and in the man pages
Even if filedebug_tc_conf.xml is already made in the folder the command stills fails, because the emulated drive that the file backend uses is only compatible with lto5, you could try to make a file backend lto8 tape if you want.
See https://github.com/LinearTapeFileSystem/ltfs/wiki/The-file-backend. In LTO, cartridge type and density code shall be matched in both file backend env and real env.
And you need to understand about tape device well because you are an IBM tape team...
Even if filedebug_tc_conf.xml is already made in the folder the command stills fails, because the emulated drive that the file backend uses is only compatible with lto5, you could try to make a file backend lto8 tape if you want.
See https://github.com/LinearTapeFileSystem/ltfs/wiki/The-file-backend. In LTO, cartridge type and density code shall be matched in both file backend env and real env.
And you need to understand about tape device well because you are an IBM tape team...
I know, in my example they are matched, I've checked in code and in the doc.
"L6": LTO6 LTO6: 0x5A
<cart_type>L6</cart_type> <density_code>5A</density_code>
But as I said the drive type is hardcoded to be L5.
LTFS30086I Cartridge is unsupported (LTO5, 0x68).
OK. It looks, the code might be included when IBM implemented the cartridge compatibility check in LE.
What do you want to do? If you just want to describe it in the doc, please add that in the Wiki. Or do you have any good idea? The lone you pointed out is related into the file backend for LE in IBM proprietary code.
OK. It looks, the code might be included when IBM implemented the cartridge compatibility check in LE.
Tomorrow I will check if LE version of mkltfs works as intended, but as right now I'm testing only the OSS version.
What do you want to do? If you just want to describe it in the doc, please add that in the Wiki. Or do you have any good idea? The lone you pointed out is related into the file backend for LE in IBM proprietary code.
I could probably remove the condition that hardcodes the drive_type in filedebug_tc.c:402 and get a compatible drive for the specified cart_type, or I could add a new option that lets you specify your drive_type in the xml.
if (ret == 0 && S_ISDIR(d.st_mode)) {
ltfsmsg(LTFS_INFO, 30003I, name);
state->dirname = strdup(name);
if (!state->dirname) {
ltfsmsg(LTFS_ERR, 10001E, "filedebug_open: dirname");
free(state);
return -EDEV_NO_MEMORY;
}
state->product_id = "ULTRIUM-TD5";
I could probably remove the condition that hardcodes
It's not a good idea. But you can override it after that line if it is not specified by LE.
I reminded why this behavior is implemented.
Actually, file backend make quite same behavior even if cartridge type and density code is changed from LTO5 in SDE. So I decided not to implement the complete function for SDE. In other words, this is really trivial and low priority feature to us.
Can I ask you why you want to work on this? I believe this is no impact for SDE.
But LE is different, because LE module need to select an appropriate drive to make I/O.
I reminded why this behavior is implemented.
Actually, file backend make quite same behavior even if cartridge type and density code is changed from LTO5 in SDE. So I decided not to implement the complete function for SDE. In other words, this is really trivial and low priority feature to us.
Can I ask you why you want to work on this? I believe this is no impact for SDE.
I was trying to solve an issue regarding the file backend in LE and EE. The VTL emulator has his own share of problems so I was checking of the file backend in SDE could help me to understand the problem and I got this problem.
But LE is different, because LE module need to select an appropriate drive to make I/O.
I know, this issue is not high priority but I think fixing this could help us and also can be used to test the code in different drives and tapes even tho the behaviour of the process does not change we still can use it to check if the messages and strings are correct(in automated tests). Is nice to have in general really
OK. It looks, the code might be included when IBM implemented the cartridge compatibility check in LE.
Tomorrow I will check if LE version of
mkltfsworks as intended, but as right now I'm testing only the OSS version.
It seems that the LE vesion of the tape_backend=file does not work, the VTL emulator (changer_backend=file) does work with a fair share of problems also generated by this code. Fixing this helps the LE version of the code.
It seems that the LE vesion of the tape_backend=file does not work, the VTL emulator (changer_backend=file) does work with a fair share of problems also generated by this code. Fixing this helps the LE version of the code.
I don't think so. At least, I used so many times while I was developing the LE....
Do you use -o changer_backend=file -o tape_backednd=file options and correct format of the changer backend directory?