HDIUTIL(1) | General Commands Manual | HDIUTIL(1) |
hdiutil
—
manipulate disk images (attach, verify, create,
etc)
hdiutil |
verb [options] |
hdiutil
uses the DiskImages framework to
manipulate disk images. Common verbs include attach,
detach, verify,
create, convert, and
compact.
The rest of the verbs are currently: help, info, burn, checksum, chpass, erasekeys, imageinfo, isencrypted, mountvol, unmount, plugins, udifrez, udifderez, resize, segment, makehybrid, and pmap.
Disk images are data containers that emulate disks. Like disks, they can be partitioned and formatted. Many common uses of disk images blur the distinction between the disk image container and its content, but this distinction is critical to understanding how disk images work. The terms "attach" and "detach" are used to distinguish the way disk images are connected to and disconnected from a system. "Mount" and "unmount" are the parallel filesystems options.
For example, when you double-click a disk image in the macOS Finder, two separate things happen. First, the image is "attached" to the system just like an external drive. Then, the kernel and Disk Arbitration probe the new device for recognized file structures. If any are discovered that should be mounted, the associated volumes will mount and appear on the desktop.
When using disk images, always consider whether an operation
applies to the blocks of the disk image container or to the (often
file-oriented) content of the image. For example,
hdiutil
verify verifies that the
blocks stored in a read-only disk image have not changed since it was
created. It does not check whether the filesystem stored within the image is
self-consistent (as diskutil verifyVolume would). On the
other hand, hdiutil
create
-srcfolder
creates a disk image container, puts a
filesystem in it, and then copies the specified files to the new
filesystem.
The following option descriptions apply to all verbs:
-verbose
-quiet
hdiutil
's
exit status to indicate success or failure. No /dev entries or mount
points will be printed. -debug
and
-verbose
disable
-quiet
.-debug
-debug
enables -verbose
.Many hdiutil
verbs understand the
following options:
-plist
hdiutil
are expected to use
-plist
rather than try to parse the human-readable
output. The usual output is consistent but generally unstructured.-puppetstrings
hdiutil
is performing an operation that will take
an indeterminate amount of time to complete. Any program trying to
interpret hdiutil
's progress should use
-puppetstrings
.-srcimagekey
key=value-imagekey
is normally a synonym)-tgtimagekey
key=value-imagekey
is only a synonym if there is no input
image).-encryption
[AES-128|AES-256]-stdinpass
-stdinpass
replaces
-passphrase
which has been deprecated.
-passphrase
is insecure because its argument
appears in the output of ps(1) where it is visible to
other users and processes on the system. See EXAMPLES.-agentpass
-pubkey
to create an image protected by both a
passphrase and a public key.-recover
keychain_file-certificate
when the image was
created.-certificate
cert_file-pubkey
PK1,PK2,...,PKn-cacert
cert--capath
and --cacert
in curl(1).-insecurehttp
-shadow
[shadowfile]hdiutil
verbs taking
images as input accept -shadow
,
-cacert
, and
-insecurehttp
.Verbs that create images automatically append the correct extension to any filenames if the extension is not already present. The creation engine also examines the filename extension of the provided filename and changes its behavior accordingly. For example, a sparse bundle image can be created without specifying -type SPARSEBUNDLE simply by appending the .sparsebundle extension to the provided filename.
Each verb is listed with its description and individual arguments. Arguments to the verbs can be passed in any order. A sector is 512 bytes.
hdiutil
verb
-help
will provide basic usage information for
that verb.By default, the system applies additional mount options to filesystems backed by untrusted devices like disk images: options like nosuid and quarantine. PERMISSIONS VS. OWNERS explains the behavior of such filesystems and EXAMPLES shows how to override some of the default behavior.
The output of attach has been stable since Mac OS X 10.0 (though it was called hdid(8) then) and is intended to be program-readable. It consists of the /dev node, a tab, a content hint (if applicable), another tab, and a mount point (if any filesystems were mounted). Because content hints are derived from the partition data, GUID Partition Table types may leak through. Common GUIDs such as "48465300-0000-11AA-AA11-0030654" are mapped to their human-readable counterparts (here "Apple_HFS").
Common options: -encryption
,
-stdinpass
, -recover
,
-imagekey
, -shadow
,
-puppetstrings
, and
-plist
.
Options:
-readonly
-readwrite
-readwrite
can be used to modify the HFS+
filesystem on a HFS+/ISO hybrid CD image.-nokernel
-kernel
-notremovable
-notremovable
.
-mount
required|optional|suppressed-nomount
-mount
suppressed.-mountroot
path-mountrandom
path-mountroot
, but mount point directory
names are randomized with mkdtemp(3).-mountpoint
path-nobrowse
-owners
on|off-drivekey
key=value-section
subspecThe following options have corresponding elements in the com.apple.frameworks.diskimages preferences domain and thus can be rendered in both the positive and the negative to override any existing preferences.
-[no]verify
hdiutil
attach attempts to intelligently verify images that
contain checksums before attaching them. If
hdiutil
can write to an image it has verified,
attach will store an attribute with the image so
that it will not be verified again unless its timestamp changes. To
maintain backwards compatibility, hdid(8) does not
attempt to verify images before attaching them.
-[no]ignorebadchecksums
-[no]autoopen
hdiutil
defaults to
-noautoopen
.-[no]autoopenro
-[no]autoopenrw
-[no]autofsck
-force
]eject
’ ioctl). If Disk Arbitration
is not running, it may be necessary to unmount the filesystems with
umount(8) before detaching the image.
eject is a synonym for detach. In
common operation, detach is very similar to
diskutil(8)'s eject.
Options:
-force
-encryption
, -stdinpass
,
-srcimagekey
,
-puppetstrings
, and
-plist
.-ov
must be specified or create will fail. To make a
cross-platform CD or DVD, use makehybrid instead. See
also EXAMPLES below.
The size specified is the size of the image to be created.
Filesystem and partition layout overhead (80 sectors for the default
GPTSPUD layout on Intel machines) may not be available for the
filesystem and user data in the image.
Size specifiers:
-size
??b|??k|??m|??g|??t|??p|??e-sectors
sector_count-megabytes
size-srcfolder
source-fs
. Other
size specifiers, such as -size
, will override
the default size calculation based on the source content, allowing for
more or less free space in the resulting filesystem.
-srcfolder
can be specified more than once, in
which case the image volume will be populated at the top level with a
copy of each specified filesystem object.
-srcdir
is a synonym.-srcdevice
device-srcdevice
and
-srcfolder
can run into errors if there are
bad blocks on a disk. One way around this problem is to write over the
files in question in the hopes that the drive will remap the bad
blocks. Data will be lost, but the image creation operation will
subsequently succeed. Filesystem options (like
-fs
, -volname
,
-stretch
, or -size
)
are invalid and ignored when using -srcdevice
.
With APFS, imaging from a device that is an individual APFS volume is invalid. To create a valid APFS disk image, device needs to be an APFS container or contain an APFS container partition.
Common options: -encryption
,
-stdinpass
,
-certificate
, -pubkey
,
-imagekey
, -tgtimagekey
,
-puppetstrings
, and
-plist
.
-imagekey
di-sparse-puma-compatible=TRUE and
-imagekey
di-shadow-puma-compatible=TRUE will create,
respectively, sparse and shadow images that can be attached on Mac OS X
10.1.
General options:
-align
alignment-type
UDIF|SPARSE|SPARSEBUNDLE-type
is particular to create
and is used to specify the format of empty read/write images. It is
independent of -format
which is used to
specify the final read-only image format when populating an image with
pre-existing content.
UDIF is the default type. If specified, a UDRW of the specified size will be created. SPARSE creates a UDSP: a read/write single-file image which expands as is is filled with data. SPARSEBUNDLE creates a UDSB: a read/write image backed by a directory bundle.
By default, UDSP images grow one megabyte at a time.
Introduced in 10.5, UDSB images use 8 MB band files which grow as
they are written to. -imagekey
sparse-band-size=size can be used to specify
the number of 512-byte sectors that will be added each time the
image grows. Valid values for SPARSEBUNDLE range from 2048 to
16777216 sectors (1 MB to 8 GB).
The maximum size of a SPARSE image is 128 petabytes; the maximum for SPARSEBUNDLE is just under 8 exabytes (2^63 - 512 bytes). The amount of data that can be stored in either type of sparse image is additionally bounded by the filesystem in the image and by any partition map. compact can reclaim unused bands in sparse images backing APFS or HFS+ filesystems. resize will only change the virtual size of a sparse image. See also USING PERSISTENT SPARSE IMAGES below.
-fs
filesystem-help
. -fs
causes a
filesystem of the specified type to be written to the image. The
default file system is APFS. If -partitionType and/or -layout are
specified, but -fs is not specified, no file system will be created.
-fs
may change the partition scheme and type
appropriately. -fs
will not make any size
adjustments: if the image is the wrong size for the specified
filesystem, create will fail.
-fs
is invalid and ignored when using
-srcdevice
.-volname
volnameuntitled
’.
-volname
is invalid and ignored when using
-srcdevice
.-uid
uidunknown
’ user (see PERMISSIONS
VS. OWNERS).-gid
gidunknown
’.-mode
mode-srcfolder
is
specified, in which case the default mode is derived from the
specified filesystem object.-[no]autostretch
-stretch
max_stretch-stretch
initializes HFS+ filesystem data such
that it can later be stretched on older systems (which could only
stretch within predefined limits) using
hdiutil
resize or by
asr(8). max_stretch is
specified like -size
.
-stretch
is invalid and ignored when using
-srcdevice
.-fsargs
newfs_args-fs
. As an example with HFS+,
newfs_hfs(8) has a number of options that can
control the amount of space used by the filesystem's data
structures.-layout
layout‘SPUD
’ causes a DDM
and an Apple Partition Scheme partition map with a single entry to
be written. ‘GPTSPUD
’ creates
a similar image but with a GUID Partition Scheme map instead. When
attached, multiple /dev entries will be created, with either slice 1
(GPT) or slice 2 (APM) as the data partition. (e.g. /dev/disk1,
/dev/disk1s1, /dev/disk1s2).
Unless overridden by -fs
, the
default layout is ‘GPTSPUD
’
(PPC systems used ‘SPUD
’ prior
to Mac OS X 10.6). Other layouts include
‘MBRSPUD
’ and
‘ISOCD
’.
create -help
lists all
supported layouts.
-library
bundle-partitionType
partition_type-ov
-attach
-fs
, the attach will fail per the default
attach -mount
required behavior.Image from source options (for
-srcfolder
and
-srcdevice
):
-format
formatOptions specific to -srcdevice
:
-segmentSize
size_spec-size
conventions).Options specific to -srcfolder
:
-[no]crossdev
-[no]scrub
-[no]anyowners
hdiutil
can't
ensure correct file ownership for the files in the image.-skipunreadable
-[no]atomic
-copyuid
user-anyowners
or
-skipunreadable
must be used to prevent the
operation from failing.By default, create
-srcfolder
attempts to maintain the permissions
present in the source directory. It prompts for authentication if it
detects an unreadable file, a file owned by someone other than the user
creating the image, or a SGID file in a group that the copying user is
not in.
-format
format -o
outfile
As with create, the correct filename extension will be added only if it isn't part of the provided name. Format is one of:
In addition to the compression offered by some formats, the
UDIF read-only format skips unused space in HFS, APFS, ExFAT, and MS-DOS
(FAT, FAT32) filesystems. For UDZO, -imagekey
zlib-level=value allows the zlib compression level
to be specified a la gzip(1). The default compression
level is 1 (fastest).
Common options: -encryption
,
-stdinpass
,
-certificate
,
-srcimagekey
,
-tgtimagekey
, -shadow
and related, -puppetstrings
, and
-plist
.
Other options:
-align
alignment-pmap
-segmentSize
-segmentSize
is specified alone is 2*1024*1024
(1 GB worth of sectors) for UDTO images and 4*1024*1024 (2 GB
segments) for all other image types. size_spec
can also be specified ??b|??k|??m|??g|??t|??p|??e like
create's -size
flag.-tasks
task_count-shadow
and related,
-srcimagekey
, -encryption
,
-puppetstrings
, and
-stdinpass
.
Other options:
-device
-list
.-testburn
-anydevice
-[no]eject
-[no]verifyburn
-[no]addpmap
-[no]skipfinalfree
-[no]optimizeimage
-[no]forceclose
-nounderrun
-[no]synthesize
-speed
x_factormax
’
max
’ will cause the
burn to proceed at the maximum speed of the drive.
‘max
’ is the default speed.
Slower speeds can produce more reliable burns. The speed factor is
relative to the media being burned (e.g.
-speed
2 has a different
data rate when used for a DVD burn vs. a CD burn). Note that some
drives have a minimum burn speed in which case any slower speed
specified will result in a burn at the drive's minimum speed.
-sizequery
-erase
-fullerase
-erase
.-list
-device
.-o
image
sourcesource can either be a directory or a disk image. The generated image can later be burned using burn, or converted to another read-only format with convert. By default, the filesystem will be readable on most modern computing platforms. The generated filesystem is not intended for conversion to read/write, but can safely have its files copied to a read/write filesystem using ditto(8).
hdiutil
supports generating El
Torito-style bootable ISO9660 filesystems, which are commonly used for
booting x86-based hardware. The specification includes several emulation
modes. By default, an El Torito boot image emulates either a 1.2MB,
1.44MB, or 2.88MB floppy drive, depending on the size of the image. Also
available are "No Emulation" and "Hard Disk
Emulation" modes, which allow the boot image to either be loaded
directly into memory, or be virtualized as a partitioned hard disk,
respectively. The El Torito options should not be used for data CDs.
Filesystem options:
-hfs
-iso
-joliet
-udf
By default, if no filesystem is specified, the image will be created with all four filesystems as a hybrid image. When multiple filesystems are selected, the data area of the image is shared between all filesystems, and only directory information and volume meta-data are unique to each filesystem. This means that creating a cross-platform ISO9660/HFS+ hybrid has a minimal overhead when compared to a single filesystem image.
Other options (most take a single argument):
-hfs-blessed-directory
-bootinfo
to create a valid
BootX
file. (HFS+ only).-hfs-openfolder
-openfolder
option in
bless(8) (HFS+ only).-hfs-startupfile-size
-abstract-file
-bibliography-file
-copyright-file
-application
-preparer
-publisher
-system-id
-keep-mac-specific
-eltorito-boot
-no-emul-boot
or
-hard-disk-boot
must be used to enable
"No Emulation" or "Hard Disk Emulation" mode,
respectively (ISO9660/Joliet).-hard-disk-boot
-no-emul-boot
-boot-load-size
and execute it, without emulating any devices (ISO9660/Joliet).-no-boot
-boot-load-seg
-boot-load-size
-eltorito-platform
-eltorito-specification
-eltorito-specification
is
provided in addition to the normal El Torito command-line options, the
specification will be used to populate secondary non-default boot
entries.-udf-version
-default-volume-name
-hfs-volume-name
-iso-volume-name
-joliet-volume-name
-udf-volume-name
-hide-all
-hide-hfs
-hide-iso
-hide-joliet
will also be needed to
hide the file from mount_cd9660(8).-hide-joliet
-hide-joliet
effectively supersedes
-hide-iso
when the resulting filesystem is
mounted as ISO on OS X.-hide-udf
-only-udf
-only-iso
-only-joliet
-print-size
-plistin
If a disk image was specified for source, the image will be attached and paths will be evaluated relative to the mountpoint of the image. No absolute paths can be used in this case. If source is a directory, all argument paths should point to files or directories either via an absolute path, or via a relative path to the current working directory.
The volume name options, just like files in the
filesystems, may need to be mapped onto the legal character set for a
given filesystem or otherwise changed to obey naming restrictions. Use
drutil(1) as drutil
filename
myname to see how a given string would be
remapped.
The -abstract-file
,
-bibliography-file
, -and
-copyright-file
must exist directly in the
source directory, not a sub-directory, and must have an 8.3 name for
compatibility with ISO9660 Level 1.
Options:
-batteryallowed
-sleepallowed
Common options: -encryption
,
-stdinpass
,
-srcimagekey
, -shadow
and related, -puppetstrings
, and
-plist
.
hdiutil
info accepts -plist
.-type
type
Common options: -shadow
and related,
-encryption
, -stdinpass
,
-srcimagekey
,
-puppetstrings
, and
-plist
.
type is one of:
Common options: -recover
and
-srcimagekey
. The options
-oldstdinpass
and
-newstdinpass
allow, in the order specified, the
null-terminated old and new passwords to be read from the standard input
in the same manner as with -stdinpass
.
Common options: -plist
and
-quiet
.
Common options: -encryption
,
-stdinpass
,
-srcimagekey
, and
-shadow
and related.
-plist
. Note that mountvol
(rather than mount, though it often works in Mac OS X
10.5 and later) is the correct way to remount a volume after it has been
unmounted by unmount.
Prior to Mac OS X 10.5, mount/attach would treat a /dev entry as a disk image to be attached (creating another /dev entry). That behavior was undesirable.
-force
]hdiutil
mountvol (or
diskutil mount) will remount a volume that has been
unmounted by hdiutil
unmount.
Options:
-force
Options are any of:
Common options: -encryption
,
-stdinpass
,
-srcimagekey
, -shadow
and related, and -plist
.
Common options: -plist
.
Common options: -plist
.
hdiutil
resize cannot resize
filesystems other than HFS+ and its variants.
resize can shrink an image so that its HFS+
partition can be converted to CD-R/DVD-R format and still be burned.
hdiutil
resize will not
reclaim gaps because it does not move data.
diskutil(8)'s resize can move
filesystem data which can help hdiutil
resize create a minimally-sized image.
-fsargs
can also be used to minimize filesystem
gaps inside an image.
resize is limited by the disk image
container format (e.g. UDSP vs. UDSB), any partition scheme, the hosted
filesystem, and the filesystem hosting the image. In the case of HFS+
inside of GPT inside of a UDRW on HFS+ with adequate free space, the
limit is approximately 2^63 bytes. Older images created with an APM
partition scheme are limited by it to 2TB. Before Mac OS X 10.4,
resize was limited by how the filesystem was created
(see hdiutil
create
-stretch
).
hdiutil
burn does
not burn Apple_Free partitions at the end of the devices, so an image
with a resized filesystem can be burned to create a CD-R/DVD-R master
that contains only the actual data in the hosted filesystem (assuming
minimal data fragmentation).
Common options: -encryption
,
-stdinpass
,
-srcimagekey
, -shadow
and related, and -plist
.
Size specifiers:
-size
??b|??k|??m|??g|??t|??p|??e-sectors
sector_count | minOther options:
-imageonly
-partitiononly
-partitiononly
will fail if the new size won't
fit inside the image. On APM, shrinking a partition results in an
explicit Apple_Free entry taking up the remaining space in the
image.-partitionID
partitionID-nofinalgap
-limits
-limits
does not modify the
image.segment -o
firstSegname -segmentCount
#segs image [opts]
segment -o
firstSegname -segmentSize
size image [opts]
segment an UDIF disk image. Segmented images work around limitations in
file size which are sometimes imposed by filesystems, network protocols,
or media. Note: whether or not the segments are encrypted is determined
by the options passed to segment and not by the state
of the source image.
Common options: -encryption
,
-stdinpass
,
-srcimagekey
,
-tgtimagekey
,
-puppetstrings
, and
-plist
.
Options:
-segmentCount
segment_count-segmentCount
or
-segmentSize
will be honored.-segmentSize
segment_size-segmentCount
or
-segmentSize
will be honored. Segmenting
read/write (UDRW) images is not supported (as of Mac OS X 10.3).
-firstSegmentSize
segment_size-segmentSize
. Used for multi-CD restores.-restricted
-ov
Common options: -encryption
,
-stdinpass
,
-srcimagekey
, and
-shadow
and related.
-simple
-standard
-complete
-diagnostic
-endoffsets
-nofreespace
-shims
.-shims
-uuids
You must specify one of the following options:
-xml
file-replaceall
Options:
Common options: -encryption
,
-stdinpass
, and
-srcimagekey
.
Verifying:
hdiutil verify
myimage.img
Converting:
hdiutil convert master.dmg -format
UDTO -o master
hdiutil convert /dev/disk1 -format
UDRW -o devimage
hdiutil convert image.dmg -o
image.sparsebundle
hdiutil convert files.sparsebundle -format UDZO \ -imagekey zlib-level=5 -o files
hdiutil convert stuff.dmg -format
UDZO -encryption -o stuff-enc
Burning:
hdiutil burn myImage.dmg
hdiutil burn myRawImage.cdr
-noverifyburn -noeject
Creating a 50 MB read/write encrypted image:
hdiutil create -encryption -size 50m
e.dmg -fs HFS+J
Creating a 50 MB read/write encrypted image protected with public key only:
hdiutil create -encryption -size 50m e.dmg -fs HFS+J \ -pubkey F534A3B0C2AEE3B988308CC89AA04ABE7FDB5F30
Creating a 50 MB read/write encrypted image protected with public key and password:
hdiutil create -encryption -size 50m e.dmg -fs HFS+J -agentpass \ -pubkey F534A3B0C2AEE3B988308CC89AA04ABE7FDB5F30
Note that these two -pubkey usage examples assume a certificate corresponding to this public key is currently in the user's keychain or smart card. For additional information on smart card authorization setup see sc_auth(8).
Creating an encrypted single-partition image without user interaction:
printf pp|hdiutil create -encryption
-stdinpass -size 9m sp.dmg
Creating a "1 GB" SPARSE image (a 1 GB filesystem in a growable file):
hdiutil create -type SPARSE -size 1g
-fs HFS+J growableTo1g
Creating a "1 GB" SPARSEBUNDLE (a 1 GB filesystem in a growable bundle):
hdiutil create -type SPARSEBUNDLE
-size 1g -fs HFS+J growableTo1g
Creating a new mounted volume backed by an image:
hdiutil create -volname Dick -size
1.3m -fs HFS+ -attach Moby.dmg
Attaching an image on a web server to the system, with any writes going to a local file:
hdiutil attach https://my.webserver.com/master.dmg \ -shadow /tmp/mastershadowfile
Using a shadow file to attach a read-only image read/write to modify it, then convert it back to a read-only image. This method eliminates the time/space required to convert a image to read/write before modifying it.
hdiutil attach -owners on Moby.dmg -shadow /dev/disk2 Apple_partition_scheme /dev/disk2s1 Apple_partition_map /dev/disk2s2 Apple_HFS /Volumes/Dick ditto /Applications/Preview.app /Volumes/Dick hdiutil detach /dev/disk2 hdiutil convert -format UDZO Moby.dmg -shadow
Creating a RAM-backed device and filesystem:
NUMSECTORS=128000 # a sector is 512 bytes mydev=`hdiutil attach -nomount ram://$NUMSECTORS` newfs_hfs $mydev mkdir /tmp/mymount mount -t hfs $mydev /tmp/mymount
Using makehybrid to create cross-platform data with files overlapping between filesystem views, containing these files:
albumlist.txt song2.wma song4.m4a song6.mp3 song8.mp3 song1.wma song3.m4a song5.mp3 song7.mp3
hdiutil makehybrid -o MusicBackup.iso Music -hfs -iso -joliet \ -hide-hfs 'Music/*.wma' -hide-joliet 'Music/{*.m4a,*.mp3}' \ -hide-iso 'Music/*.{wma,m4a}'
will create an image with three filesystems pointing to the same blocks. The HFS+ filesystem, typically only visible on Macintosh systems, will not include the .wma files, but will show the .m4a and .mp3 files. The Joliet filesystem will not show the .m4a and .mp3 files, but will show the .wma files. The ISO9660 filesystem, typically the default filesystem for optical media on many platforms, will only show the .mp3 files. All three filesystems will include the "albumlist.txt" files.
Image from directory:
hdiutil create -srcfolder mydir
mydir.dmg
Image from directory using an intermediate sparse bundle:
hdiutil create -srcfolder mydir
-format UDSB mydir.sparsebundle
hdiutil convert mydir.sparsebundle
-format UDZO -o mydir.dmg
Manually changing ownership settings of a read-only disk image:
hdiutil attach myimage.dmg ... /dev/disk1s2 Apple_HFS /Volumes/myVolume diskutil unmount disk1s2 mkdir /Volumes/myVolume sudo mount -r -t hfs -o owners /dev/disk1s2 /Volumes/myVolume # -o owners is the default for manual mounts
Forcing a known image to attach:
hdiutil attach -imagekey
diskimage-class=CRawDiskImage myBlob.bar
The following environment variables affect
hdiutil
and DiskImages:
com_apple_hdid_verbose
-verbose
behavior for
attach.com_apple_hdid_debug
-debug
behavior for
attach.com_apple_hdid_nokernel
-nokernel
but works even with, for
example, create -attach
.com_apple_hdid_kernel
-kernel
was passed. In Mac OS X 10.4.x, in-kernel was the default behavior for
UDRW and SPARSE images. In Mac OS X 10.5 and later, these and other
kernel-compatible images again default to attaching with a user process.
If an image is not "kernel-compatible" and in-kernel mounting is
specified, the attach will fail. WARNING: ram:// images use wired memory
when attached in-kernel.com_apple_diskimages_insecureHTTP
-insecurehttp
does. Useful for clients of
DiskImages such as asr(8) which don't support a similar
command line option.DiskImages uses many frameworks and can encounter many error
codes. In general, it tries to turn these error numbers into localized
strings for the user. For background, intro(2) is a good
explanation of our primary error domain: the BSD errno values. For
debugging, -verbose
should generally provide enough
information to figure out what has gone wrong. The following is a list of
interesting errors that hdiutil may encounter:
hdiutil
attach means that no
filesystems could be recognized or mounted after the disk image was
attached. The default behavior in this case is to detach the disk image.
See attach for options modifying this behavior. This
error can occur if the disk image or contained filesystem is corrupt. It
can also occur if an image was created from a block device containing a
mounted, journaled filesystem (in which case the image contains a dirty
journal that can't be replayed without making the image read/write, such
as with attach -shadow
).ENXIO
]EINVAL
]hdiutil
's arguments are subtly non-sensical
(e.g. an invalid layout name passed to create
-layout
).EFBIG
]EAUTH
]-insecurehttp
and -cacert
for more information about verification of SSL peers.EBUSY
]EAGAIN
]EAGAIN
is returned if the appropriate read or write lock can't be obtained.EACCES
vs.
EPERM
As of Mac OS X 10.5, a more reliable, efficient, and scalable sparse format, UDSB (SPARSEBUNDLE), is recommended for persistent sparse images as long as a backing bundle (directory) is acceptable. Mac OS X 10.5 also introduced F_FULLFSYNC over AFP (on client and server), allowing proper journal flushes for HFS+J-bearing images. Critical data should never be stored in sparse disk images on file servers that don't support F_FULLFSYNC.
SPARSE (UDSP) images and shadow files were designed for intermediate use when creating other images (e.g. UDZO) when final image sizes are unknown. Generally speaking, SPARSE images are not recommended for persistent storage, though they are relatively safe on Mac OS X 10.3.2 and later. On versions earlier than 10.3.2, SPARSE should be avoided in favor of UDRW images and resize. On Mac OS X 10.5 and later, the more robust and faster SPARSEBUNDLE type is preferred.
Note that both sparse formats, UDSP and UDSB, are growable only up to a limit: the size parameter specified when they were created. They will take up less space on the hosting filesystem if they contain less data than their created size, and grow up to that size as data is added.
If more space is needed than is referenced by the hosted
filesystem, hdiutil
resize or
diskutil(8) resize can help to grow or
shrink the filesystem in an image. compact reclaims unused
space in sparse images. Though they request that hosted HFS+ filesystems use
a special "front first" allocation policy, beware that sparse
images can enhance the effects of any fragmentation in the hosted
filesystem.
To prevent errors when a filesystem inside of a sparse image has more free space than the volume holding the sparse image, HFS+ volumes inside sparse images will report an amount of free space slightly less than the amount of free space on the volume on which image resides. The image filesystem currently only behaves this way as a result of a direct attach action and will not behave this way if, for example, the filesystem is unmounted and remounted. Moving the image file to a different volume with sufficient free space will allow the image's filesystem to grow to its full size.
Since any /dev entry can be treated as a raw disk image, it is worth noting which devices can be accessed when and how. /dev/rdisk nodes are character-special devices, but are "raw" in the BSD sense and force block-aligned I/O. They are closer to the physical disk than the buffer cache. /dev/disk nodes, on the other hand, are buffered block-special devices and are used primarily by the kernel's filesystem code.
It is not possible to read from a
/dev/disk node while a filesystem is mounted from
it, but anyone with read access to the appropriate
/dev/rdisk node can use
hdiutil
verbs such as fsid or
pmap with it. Beware that information read from a raw
device while a filesystem is mounted may not be consistent because the
consistent data is stored in memory or in the filesystem's journal.
The DiskImages framework will attempt to use authopen(1) to open any device which it can't open (due to EACCES) for reading with open(2). Depending on session characteristics, this behavior can cause apparent hangs while trying to access /dev entries while logged in remotely (an authorization panel is waiting on console).
Generally, the /dev/disk node is preferred
for imaging devices (e.g. convert or
create -srcdevice
operations),
while /dev/rdisk is usable for the quick
pmap or fsid. In particular, converting
the blocks of a mounted journaled filesystem to a read-only image will
prevent the volume in the image from mounting (the journal will be
permanently dirty).
Some filesystems support permissions including users and groups. While important for security on a managed filesystem, users and groups ("owners") pose challenges for unmanaged, shared filesystems such as those typically present in disk images. macOS's solution to this problem is to make owners optional, both while creating files and enforcing permissions.
By default, unknown HFS+ filesystems on "external" devices (including disk images) mount with their owners ignored (mount -o noowners). Normally when owners are ignored, the system uses a special _unknown user and group to dynamically substitute the current user's identity for any owners recorded in the filesystem. These _unknown owners are even written to the volume when creating new files. The new files will continue to have "floating" ownership when mounted with owners honored. The net result is that shared volumes behave as expected regardless of how they are accessed.
The behavior is different when disk images are attached. With disk images, the owner of all files in a filesystem mount for which owners are ignored is the user attaching the disk image. The attaching owner is also used when creating new files.
On modern macOS systems, root (UID 0) can "see through" the "owners ignored" user mappings. Thus
sudo ls -l
/Volumes/imageVol
Unlike owners, permissions are never optional. A non-writable file will not be writable just because owners are ignored. However, a file that is writable by its owner will be writable by everyone if _unknown is the effective owner of the file for that file. Because anyone accessing an owners-ignored file is treated as the owner, everyone is effectively the owner. Because the default behavior for disk image filesystems is for all files to be owned by the user attaching the disk image, other users will be treated per the 'group' (if applicable) and 'other' permission modes.
diskutil(8)'s enableOwnership or the Finder's Get Info window can be used to configure a system to respect the on-disk owners for a filesystem in the future.
The DiskImages framework supports a variety of image formats, including read/write, read-only, and read-only compressed (which are decompressed in small chunks as I/O requests are made). It is capable of mounting most images directly from http:// URLs. Because DiskImages can make many requests over a single connection, responsiveness can be improved by modifying HTTP server settings such as apache's MaxKeepAliveRequests and KeepAliveTimeout.
Mac OS X 10.0 supported the disk images of Disk Copy 6 on Mac OS 9. OS X 10.1 added sparse, encrypted, and zlib-compressed images. These images will not be recognized on Mac OS X 10.0 (or will attach read/write, possibly allowing for their destruction). As the sparse, shadow, and encrypted formats have evolved, switches have been added to facilitate the creation of images that are compatible with older OS versions (at the expense of the performance and reliability improvements offered by the format enhancements). In particular, sparse images should not be expected to attach on versions of OS X older than that which created them.
With Mac OS X 10.2, the most common image formats went "in-kernel" (i.e. the DiskImages kernel extension served them without a helper process), image meta-data began being stored both as XML and in the embedded resource fork, and the default Disk Copy.app "compressed" format became UDZO (breaking compatibility with 10.0). Mac OS X 10.4 introduced bzip2 compression in the UDBZ format which provides smaller images (especially when combined with makehybrid) at the expense of backwards compatibility, some performance, and kernel compatibility.
In Mac OS X 10.4.7, the resource forks previously embedded in UDIF images were abandoned entirely to avoid metadata length limitations imposed by resource fork structures. As a result, UDIF images created on 10.4.7 and later will not, by default, be recognized by either Mac OS X 10.1 or Mac OS X 10.0. flatten can be used to customize the type of metadata stored in the image.
Mac OS X 10.5 introduced sparse bundle images which compact quickly but are not recognized by previous OS versions. Mac OS X 10.6 removed support for attaching SPARSEBUNDLE images from network file servers that don't support F_FULLFSYNC, although this requirement was relaxed in macOS 10.12. OS X 10.7 removed double-click support for images using legacy metadata; these can be rehabilitated using flatten and unflatten, or simply convert.
OS X 10.11 introduced lzfse compression in the ULFO format, providing faster, more efficient compression and smaller images compared to UDZO. These images are also supported in-kernel, but will not work on any earlier versions of the OS.
macOS 10.12 included a pre-release version of the Apple File System called APFS which was meant for evaluation and development purposes only. Files stored in APFS-based images may not be accessible in future releases of macOS, and won't work in past ones. All data to be stored in APFS volumes should be backed up prior to using APFS and regularly backed up while using APFS.
macOS 10.15:
macOS 11.0:
macOS 12.0:
macOS 13.0:
Disk images were first invented to electronically store and transmit representations of floppy disks for manufacturing replication. These images of floppies are typically referred to as 'Disk Copy 4.2' images, in reference to the application that created and restored them to floppy disks. Disk Copy 4.2 images were block-for-block representations of a floppy disk, with no notion of compression. DART is a variant of the Disk Copy 4.2 format that supported compression.
NDIF (New Disk Image Format) images were developed to replace the Disk Copy 4.2 and DART image formats and to support images larger than a floppy disk. With NDIF and Disk Copy version 6, images could be "attached" as mass storage devices under Mac OS 9. Apple Data Compression (ADC) -- which carefully optimizes for fast decompression -- was used to compress images that were typically created once and restored many times during manufacturing.
UDIF (Universal Disk Image Format) device images picked up where NDIF left off, allowing images to represent entire block devices and all the data therein: DDM, partition map, disk-based drivers, etc. For example, it can represent bootable CDs which can then be replicated from an image. To ensure single-fork files (NDIF was dual-fork), it began embedding its resource fork in the data fork. UDIF is the native image format for OS X.
Raw disk images from other operating systems (e.g. .iso files)
will be recognized as disk images and can be attached and mounted if macOS
recognizes the filesystems. They can also be burned with
hdiutil
burn.
macOS 10.15 added ULMO format images compressed with lzma. These images are smaller than comparable ULFO images compressed with lzfse. These images are not supported in-kernel, and are not usable on earlier OSes.
macOS 10.12 introduced the pre-release APFS for evaluation (see
COMPATIBILITY above). 10.12 also added an option to disable atomic copying
during image from folder operations, -noatomic
,
which may result in slightly faster image creation. pmap
added a new switch, -diagnostic
, which captures
troubleshooting information for Boot Camp configurations.
OS X 10.11 added ULFO format images compressed with lzfse. These images are more efficient and smaller than comparable UDZO images compressed with zlib, and retain kernel compatibility, but are not usable on earlier OSes.
OS X 10.10 quadrupled the default UDIF chunk size without affecting backward compatibility. UDIF images created or converted on 10.10 will benefit from smaller metadata and more efficient compression for UDZO and especially UDBZ formats.
OS X 10.7 added the ability to quickly render encrypted images inaccessible using the new erasekeys verb, which saves time versus securely overwriting the entire image.
In Mac OS X 10.6, pmap was rewritten to use
MediaKit's latest reporting routines so that it can properly support GPT
partition maps. Also -debug
now implies
-verbose
for all verbs.
Mac OS X 10.5 changed the behavior of attach when run on an existing image or /dev node: if the image was attached but no volume was mounted, the volume would be mounted. Prior systems would return the /dev without mounting the volume. This change effectively removes the ability to create a second /dev node from an existing one.
diskutil(8), asr(8), ioreg(8), hfs.util(8), apfs.util(8), msdos.util(8), exfat.util(8), authopen(1), ditto(8), drutil(1), diskarbitrationd(8).
09 Dec 2020 | macOS |