format-udf

Bash script to format a block device (hard drive or Flash drive) in UDF. The output is a drive that can be used for reading/writing across multiple operating system families: Windows, macOS, and Linux. This script should be capable of running in macOS or in Linux.

Features

• Formats a block device (hard drive or Flash drive) in Universal Disk Format (UDF)
  • UDF revision 2.01 used for maximal compatibility (see note on Linux support below)
• Resulting file system can be read/written across multiple operating system families (Windows, macOS, and Linux)
• Runs on any OS having a Bash environment
• Optionally wipes device before formatting
• Ability to override detected device block size
• Option to force non-interactive mode (useful for scripting)
• Writes a fake MBR for added compatibility on Windows (optionally disabled)

For the advanced user, format-udf is also capable of formatting a single existing partition, without modifying the partition table. Beware that using this method will render the newly formatted UDF partition unusable on macOS (but still usable on Linux and Windows). (See #24 for caveats.) Because of this limitation, the recommendation is to format the entire device.

Why?

format-udf was created to address some OS-specific quirks that prevent a naively-formatted UDF device from working across various operating systems. Here are some of the complicating factors, which format-udf aims to abstract away:

• Different operating systems support different versions of the UDF specification. Some OS versions only offer read-only support.
• Windows seems to only mount UDF devices if the file system block size matches the device's logical block size
• Different operating systems (like Windows XP) only attempt mounting UDF file systems with a hard-coded block size
• Windows does not support hard disks without a partition table. (This is strange because Windows does not apply the same limitation to flash drives.)
• macOS seems to only mount UDF file systems that utilize the full disk (not just a partition)

At first glance, these constraints appear to be in partial conflict. The solution, as suggested by Pieter, is to place a fake partition table (via MBR) in the first block of the drive, which lists a single entire-disk partition. This works because UDF (perhaps intentionally) doesn't utilize the first block. Unfortunately, there has been no easy way to do this, while juggling all of the other variables (such as device logical block size). format-udf writes such a fake MBR for added compatibility on Windows. If this is not what you desire, you can disable the MBR with -p none.

The goal of this project is to provide access to a cross-platform file system with modern features, in such a way that is:

1. Easy to use for the average user
2. Maximally compatible across operating systems
3. As compliant as is reasonable with the UDF specification
4. Maximally flexible to help users with uncommon needs

UDF OS Support

Not all operating systems support UDF. The following tables detail operating system support for UDF. Data was adapted from https://en.wikipedia.org/wiki/Universal_Disk_Format#Compatibility (as retrieved on 2017-06-16).

Natively Supported

Both read/write are supported unless otherwise listed below.

Operating System	Read-only	Note
Windows XP, Server 2003	Read-only	Write support available with third party utilities
Windows Vista, 7, 8, 10		Referred to by Microsoft as "Live File System"; Requires fake full-disk partition
Mac OS 9
Mac OS X 10.5 through 10.11
macOS 10.12+
Linux 2.6+, 3.x	UDF revisions 2.01 and before have read/write. After UDF revision 2.01, read-only.
AIX 5.2, 5.3, 6.1
BeOS, magnussoft ZETA, Haiku
DosBox
eComStation, OS/2		Additional-fee drivers on OS/2
NetBSD 5.0
Solaris 8, 9, 10

Supported with Third-Party Utilities

Operating System	Note
Windows 95 OSR2+, 98	Utilities include DLA and InCD
Windows 2000, ME

Not Supported

Operating System	Note
DOS, FreeDOS, Windows 3.11 or older	File systems that have an ISO9660 backward compatibility structure can be read

4K Drive Support

Not all operating systems support 4K drives (Advanced Format). If you operating system supports UDF, but not your 4K drive, you still may encounter issues using format-udf.

Windows 4K Drive Support

The following tables detail Windows support for 4K drives. Data was adapted from the Microsoft support policy for 4K sector hard drives in Windows (as retrieved on 2017-06-16). Overlaid into this table are testing results from the format-udf community. (Special thanks to @pali for his testing on XP.)

Size / OS	512-byte native	512 emulation(AKA "512e")	4K native(AKA "4Kn")
Logical block size	512 bytes	512 bytes	4096 bytes
Physical block size	512 bytes	4096 bytes	4096 bytes
Maximum UDF filesystem capacity	2 TiB	2 TiB	16 TiB
Windows XP	Supported;Works	Unsupported;Doesn't work	Unsupported;Doesn't work
Windows XP Pro x64,Server 2003,Server 2003 R2	Supported;Likely works but untested	Unsupported	Unsupported
Windows Vista,Server 2008	Supported;Likely works but untested	Supported;Likely works but untested	Unsupported
Windows 7,Server 2008 R2	Supported;Likely works but untested	Supported;Likely works but untested	Unsupported
Windows 8,Server 2012	Supported;Likely works but untested	Supported;Likely works but untested	Supported;Likely works but untested
Windows 8.1,Server 2012 R2	Supported;Likely works but untested	Unsupported	Unsupported
Windows 10,Server 2016	Supported;Likely works but untested	Unsupported	Supported;Likely works but untested

If you have conducted testing and would like to update this table to benefit future users of format-udf, please send a pull request. Please include a link to your raw data or testing results.

Environment

• Any OS having a Bash environment
• The following tools must be installed, executable, and in the PATH:
  • printf
  • xxd
  • One of the following: blockdev, ioreg
  • One of the following: blockdev, diskutil
  • One of the following: lsblk, diskutil
  • One of the following: umount, diskutil
  • One of the following: mkudffs, newfs_udf

Prerequisites

To install necessary prerequisites on Ubuntu:

sudo apt-get install udftools coreutils vim-common

Installation

format-udf is a self-contained script. Simply copy format-udf.sh to a directory of your choosing. Don't forget to make it executable:

chmod +x format-udf.sh

Usage

Bash script to format a block device (hard drive or Flash drive) in UDF.
The output is a drive that can be used for reading/writing across multiple
operating system families: Windows, macOS, and Linux.
This script should be capable of running in macOS or in Linux.

Usage:  ./format-udf.sh [-b BLOCK_SIZE] [-f] [-p PARTITION_TYPE] [-w WIPE_METHOD] device label
        ./format-udf.sh -v
        ./format-udf.sh -h

    -b BLOCK_SIZE
        Block size to be used during format operation.
        If absent, defaults to value reported by blockdev/diskutil.
        This is an expert-only option.  Please consult the README for details.

    -f
        Forces non-interactive mode.  Useful for scripting.
        Please use with caution, as no user confirmation is given.

    -h
        Display help information and exit.

    -p