Dd (Unix)

src: i.ytimg.com

dd is a command line utility for Unix and Unix-like operating systems whose primary purpose is to convert and copy files.

On Unix, device drivers for hardware (such as hard disk drives) and special device files (such as/dev/zero and/dev/random) appear in file systems like normal files; dd can also read and/or write from/to these files, as long as they are implemented in their respective drivers. As a result, dd can be used for tasks such as backing up the boot sector of the hard drive, and obtaining a fixed amount of random data. The dd program can also convert data when copied, including exchange byte sequences and conversions to and from ASCII and EBCDIC text coding.

The name dd is the reference to the DD statement found in the IBM Job Control Language (JCL), where the initials stand for "Data Definition". The command syntax resembles a JCL statement more than any other Unix command does, so the syntax is probably a joke.

Originally intended to convert between ASCII and EBCDIC, dd first appeared in Version 5 Unix. The dd command is determined by IEEE Std 1003.1-2008, which is part of the Single UNIX Specification.

Video Dd (Unix)

Usage

The dd command syntax is different from many other Unix programs, it uses the option = value syntax because of a Command-line option rather than options - option value or - = more standard format., exceptions eg Plot the command dd 9, using the standard command line style options. By default, dd reads from stdin and writes to stdout, but this can be changed by using if (input file) and of (output file) selection.

Usage varies across different operating systems. Also, certain features of dd will depend on the capabilities of the computer system, such as dd capabilities to implement the option for direct memory access. SIGing a SIGINFO signal (or a USR1 signal in Linux) to a running dd process makes it print I/O statistics to standard error once and then resume copying. dd can read standard input from keyboard. When the end-of-file (EOF) is reached, dd will exit. Signals and EOF are determined by the software. For example, Unix tools moved to Windows vary as in EOF: Cygwin uses Ctrl D (Ordinary EOF Unix) and MKS Toolkit using ctrl z (ordinary Windows EOF).

Maps Dd (Unix)

Output messages

The GNU dd variant as provided with GNU coreutils does not reflect the message format displayed in the standard output at completion. However, this is explained by other implementations, e.g. it's with BSD.

Each line "Recorded in" and "Record out" indicates the number of complete blocks that are transferred partial block number, e.g. because the physical media ends before the complete block is read, or a physical error prevents the reading of the complete block.

src: i.ytimg.com

Block size

A block is a unit that measures the number of bytes read, written, or converted at a time. Command line options can specify different block sizes for input/read ( ibs ) compared to output/writing ( obs ), despite block size ( bs ) option will overwrite ibs and obs . The default values ​​for input and output block sizes are 512 bytes (traditional disk block size, and POSIX mandate size of "block"). The count option to copy is measured in blocks, as the jump counts for the reading and finds counts for writing. Conversion operations are also affected by "block conversion size" ( cbs ).

The value provided for the block size option is interpreted as a decimal number (base 10) integer. It can also contain suffixes to indicate that block size is an integer number of units larger than bytes. The suffix w (word) means multiplication with 2, lowercase b (block) means 512, lowercase k (kibibytes) means 1024, then capital letters M (Mebibytes) means 1024 ÃÆ'â € "1024, G (Gibibytes) means 1024 ÃÆ'â €" 1024 ÃÆ'â € "1024, and so on for Tebibytes, Exbibytes, Pebibytes , Zebibytes, and Yobibytes. Some implementations also understand the suffix suffix B to show SI units such as kB (kilobytes) for 1000 bytes or MB (Megabytes) for 1,000,000 bytes. Thus bs = 16M indicates a block of 16 mebibytes (16,777,216 bytes), or bs = 3kB assigns 3,000 bytes.

In addition, some implementations understand the x character as the multiplication operator for block sizes and calculation parameters. For example, bs = 2x80x18b is interpreted as 2 ÃÆ'â € "80 ÃÆ'â €" 18 ÃÆ'â € "512 = 1474560 bytes, the exact size of the 1440 KiB floppy disk.

For some uses of the dd command, block size has an effect on performance. Doing lots of small readings or writing is often slower than doing smaller ones. Using large blocks requires more RAM and can complicate error recovery. When dd is used with variable-block-size devices such as tape drives or networks, block sizes can determine the size of the recorded recording or packet size, depending on the network protocol used.

src: slideplayer.com

Usage

The dd command can be used for various purposes.

Data transfer

dd can duplicate data across files, devices, partitions and volumes. Data can be input or output to and from all of this; but there is an important difference regarding the output when going to the partition. Also, during transfer, data can be modified using the conv option to match the medium.

Attempts to copy an entire disk using cp can remove the last block if the length is unexpected; while dd might work. The source and destination disks must have the same size.

The noerror option means to continue if there is an error, while the sync option causes the output block to be padded.

Record backup master record and restore

You can repair the master boot record. These can be transferred to and from the repair files.

To duplicate the first two sectors of the floppy drive:

 dd if =/dev/fd0 from = MBRboot.img bs = 512 count = 2 

To create an image of all x86 master boot notes (including MS-DOS partition tables and MBR byte magic):

 dd if =/dev/sda from = MBR.img bs = 512 count = 1 

To make the image only boot code from the master boot record (without the partition table and without the magic bytes required to boot):

 dd if =/dev/sda from = MBR_boot.img bs = 446 count = 1 

Modify data

dd can modify data in place. For example, this overwrites the first 512 bytes of a file with null bytes:

 dd if =/dev/nol = path/to/file bs = 512 count = 1 conv = notrunc 

The notrunc conversion option means not cutting the output file - that is, if the output file already exists, just replace the specified byte and leave the rest of the output file alone. Without this option, dd will make the output file 512 bytes long.

To duplicate disk partitions as disk image files on different partitions:

 dd if =/dev/sdb2 from = partition.image bs = 4096 conv = noerror 

Disk wipe

For security reasons, it is sometimes necessary to remove disks from discarded devices.

To delete a disk by writing zero, dd can be used this way:

 dd if =/dev/nol =/dev/sda bs = 16M 

Another approach could be to remove a disk by writing random data for it:

 dd if =/dev/urandom =/dev/sda bs = 16M 

When compared to the above data modification example, the notrunc conversion option is not necessary because it does not matter when the dd output file is a block device.

The bs = 16M option makes dd read and write 16 Mebibytes at a time. For modern systems, larger block sizes may be faster. Note that filling the drive with random data may take longer than centralizing the drive, because random data must be generated by the CPU, while zeroing up very quickly. On modern hard-disk drives, centralizing the drive will keep most of the data it contains permanently irreversible. However, with other types of drives such as flash memory, much data can still be recovered by remanent data.

Modern hard disk drives contain the Secure Erase command that is designed to permanently and securely erase any accessible and accessible parts of the drive. It can also work for some Solid-state drives (flash drives). By 2017, it does not work on USB flash drives or on Secure Digital flash memory. If available, this is faster than using dd, and more secure. On a Linux machine can be accessed via the - security-erase-enhanced command option.

The corrupted program offers several patches as well as the removal of more secure individual files.

Data recovery

The initial history of open-source software for data recovery and recovery of files, drives and partitions including GNU dd , whose copyright notice began in 1985, with one block size per dd process, and no recovery algorithm other than the user's interactive session running one form dd after the other. Then, the C program called dd_rescue was written in October 1999, has two block sizes in the algorithm. However, the 2003 shell script writer dd_rhelp , which improves the data recovery algorithm dd_rescue , recommends GNU ddrescue , unrelated data recovery programs up to dd which was originally released in 2004.

To help distinguish newer GNU programs from older scripts, alternate names are sometimes used for GND ddrescue , including addrescue (names in freecode.com and freshmeat.net), gddrescue (Debian package name), and gnu_ddrescue (package name openSUSE). Another open-source program called savehd7 uses sophisticated algorithms, but also requires the installation of its own programming language interpreter.

Drive performance benchmark

To create a benchmark test drive and analyze the performance of a sequential (and usually single-threaded) read and write system for 1024-byte blocks:

 dd if =/dev/zero bs = 1024 count = 1000000 of = file_1GB  dd if = file_1GB =/dev/null bs = 1024 

Generate files with random data

To create a file from 100 random bytes using random kernel drivers:

 dd if =/dev/urandom = myrandom bs = 100 count = 1 

Convert file to uppercase

To convert a file to uppercase:

 dd if = filename = filename1 conv = ucase, notrunc 

src: i.ytimg.com

Limitations

As stated in the documentation section provided by Seagate, "Certain disk utilities, such as DD, that rely on low-level disk access may not support 48-bit LBA until updated". Using ATA hard disk drives of more than 128 GB in size requires 48-bit LBA system support; However, under Linux, dd uses the kernel to read or write to raw device files rather than accessing hardware directly. At the same time, support for 48-bit LBA has been present since version 2.4.23 of the kernel, released in 2003.

src: i.ytimg.com

Dcfldd

dcfldd is a dd fork, a version developed by Nick Harbor, who at that time worked for the US Department of Defense Computer Forensics Laboratory. Compared to dd, dcfldd allows for more than one output file, supports multiple simultaneous checksum calculations, provides verification mode for file matching, and can display the percentage of progress of an operation.

src: wallup.net

See also

• Back up
• Disk cloning
• Copy Disk
• Disk image
• .img (file name extension)
• List of Unix programs
• ddrescue version of GNU that copies data from corrupted files

src: i.ytimg.com

Note

src: slideplayer.com

References

src: i.ytimg.com

External links

Source of the article : Wikipedia