UDI format

From Sinclair Wiki
Jump to navigation Jump to search


UDI Disk Image Format

The UDI format was created for storing faithful disk images for emulation.

This format is member of the Complex Disk Image Formats and contains almost all information about a disk.

UDI file structure

UDI images can store any type of floppy disk or even Microdrive tape data.

UDI store raw track data: bytes and clock marks, as recorded on the disk. Because gap and mark bytes are recorded and data are not interpreted (e.g. splitted up sectors, etc), we can build a very faithful disk layer emulation based on UDI image format.

File Header

UDI files start with UDI! signature (magic bytes).

Header contain data about number of sides and number of tracks.

Data

Track data follow the “alt” schema:

  • Side: 0 Track: 0
  • Side: 1 Track: 0
  • Side: 0 Track: 1

...

  • Side: 0 Track: last
  • Side: 1 Track: last

Track data contain information about track type (e.g. MF or MFM coded, weak data, etc.), track length and a CRC check.

Further details can be found in the following specification.

UDI: Ultra Disk Image file format Version 1.0

UDI file structure

Offset Length Comment
0x0000 16 UDI file header
0x0010 EXTHDL Extended header
0x0010 + EXTHDL TRSL Tracks data
0x0010 + EXTHDL + TRSL 4 CRC32 
   CRC32 of the file


UDI file header

Offset Length Comment
0x0000 4 Signature "UDI!"
0x0004 4 File size - 4 (last 4 bytes is checksum)
0x0008 1 Version 
   0x00 - Version 1.0
0x0009 1 Number of cylinders - 1 
    e.g. 0x4F for 80 cylinders on disk
0x000A 1 Number of sides - 1 
    0x00 Single sided disk
    0x01 Double sided disk
    0x02..0xFF reserved
0x000B 1 Unused (should be 0)
0x000C 4 EXTHDL - Extended header size


Extended header

Offset Length Comment
0x0000 EXTHDL Unspecified data

Tracks data

Offset Length Comment
0x0000 TRLEN0 1st. track data
0x0000 + TRLEN0 TRLEN1 2nd. track data
0x0000 + TRLEN0 + TRLEN1 TRLEN2 3rd. track data
… … …
0x0000 + TRLEN0 + TRLEN1 + … + TRLENLAST-1 TRLENLAST Last track data
TRSL = TRLEN0 + TRLEN1 + … + TRLENLAST
Track data order in file
Single sided disk Double sided disk
Track No Side Cylinder No Side Cylinder No
0 0 or A 0 0 or A 0
1 0 or A 1 1 or B 0
2 0 or A 2 0 or A 1
3 0 or A 3 1 or B 1
… … … … …
Last 0 or A Last 1 or B Last

Track data format

Offset Length Comment
0x0000 1 Track type 
 0x00 - MFM data
 0x01-0xff reserved
0x0001 2 TLEN 
  Track data size in bytes include MARKs GAPs etc.
0x0003 TLEN Data bytes
0x0003 + TLEN CLEN Track clock marks 
where CLEN = TLEN / 8 + ( TLEN % 8 + 7 ) / 8
One bit corresponds with one byte of raw track data
0 - byte with normal CLK (normal data)
1 - byte with mark CLK (for index label 0xA1 ¨ 0xC2)
For example:
 { 0xA1,0xA1,0x00,0xA1,0xFE,0x00,0x00,0x01 }
 where 0xA1 - MFM markers
 for this raw data, bits array store as one byte - 0x0B
 unused last bits should be 0.
TRLENN = 3 + TLENN + CLENN


Algorithm of calculation CRC32 for UDI file

  DWORD fileSize; // Size of file 
  BYTE *buf;      // Pointer to file data 
  long CRC = 0xffffffff; 
  for( DWORD i = 0; i < fileSize; i++ ) 
  { 
    DWORD temp; 
    CRC ^= -1 ^ *(((unsigned char*)buf)+i); 
    for( BYTE k = 8; k--; ) 
    {
      temp = -(CRC & 1); CRC >>= 1; CRC ^= 0xedb88320 & temp;
    } 
    CRC ^= -1; 
  }

Appendix

If the file signature is udi! (written in small letters) instead of UDI!, then data is compressed. The compression algorithm has not yet been determined but likely will be similar to TELEDISK's compression algorithm.

Author of UDI file format

Author of UDI file format is Alex Makeev.
This description based on UDI file format Version 1.0 final! Revision 24 March, 2002
Original doc: http://www.zxmak.narod.ru/docs.htm

Notes

All calculations made with integer arithmetic (e.g. 15 / 8 gives 1 )
Operator % means modulus of, as used in C language
All multibyte values in Little Endian order, (e.g. TLEN = TLEN0 + 256 × TLEN1)

UDI: Ultra Disk Image file format Version 1.1

This is an 'smooth' extension of original UDI specification.

This version extend only the 'track data format' section.

This version drop the specification of the compresed UDI files idea with udi! (written in small letters) signature, but add a compressed track container type for similar purpose. So all v1.1 UDI files start with UDI! (written in capital letters) signature.

Whit this extension the UDI format can store floppy disk (and microdrive) data more faithfully then other disk image formats.

Track data format

Offset Length Comment
0x0000 1 Track type 
 0x00 - MFM data
 0x01 - FM data
 0x02 - mixed MFM/FM data
 0x80 - 'floating' or 'weak' MFM data
 0x81 - 'floating' or 'weak' FM data
 0x82 - 'floating' or 'weak' mixed MFM/FM data
 0x83 - 'floating' or 'weak' data with multiple read
 0xe0 - microdrive 'track'
 0xf0 - zlib 'compressed' track container
 0x03-0x7f and 0x81-0xef reserved
 0xf1-0xff reserved for other compressions (e.g. bzip2 and lzma)
0x0001 TDATAL Track data as described later
TRLENN = 1 + TDATALN

Track data for type 0x00 (MFM) and 0x01 (FM)

Offset Length Comment
0x0000 2 TLEN 
  Track data size in bytes include MARKs GAPs etc.
0x0002 TLEN Data bytes
0x0002 + TLEN CLEN Track clock marks 
where CLEN = TLEN / 8 + ( TLEN % 8 + 7 ) / 8
One bit corresponds with one byte of raw track data
0 - byte with normal CLK (normal data)
1 - byte with mark CLK (for index label 0xA1 ¨ 0xC2)
For example:
 { 0xA1,0xA1,0x00,0xA1,0xFE,0x00,0x00,0x01 }
 where 0xA1 - MFM markers
 for this raw data, bits array store as one byte - 0x0B
 unused last bits should be 0.
TDATALN = 2 + TLENN + CLENN


Track data for type 0x02 (mixed MFM/FM)

Offset Length Comment
0x0000 2 TLEN 
  Track data size in bytes include MARKs GAPs etc.
0x0002 TLEN Data bytes
0x0002 + TLEN CLEN Track clock marks 
where CLEN = TLEN / 8 + ( TLEN % 8 + 7 ) / 8
One bit corresponds with one byte of raw track data
0 - byte with normal CLK (normal data)
1 - byte with mark CLK (for index label 0xA1 ¨ 0xC2)
For example:
 { 0xA1,0xA1,0x00,0xA1,0xFE,0x00,0x00,0x01 }
 where 0xA1 - MFM markers
 for this raw data, bits array store as one byte - 0x0B
 unused last bits should be 0.
0x0002 + TLEN + CLEN MLEN MF marks 
where MLEN = CLEN
One bit corresponds with one byte of raw track data
0 - byte coded with MFM
1 - byte code with FM
unused last bits should be 0.
TDATALN = 2 + TLENN + 2×CLENN


Track data for type 0x80 (MFM) or 0x81 (FM) "floating" or "weak" data chunk

Offset Length Comment
0x0000 2 TLEN 
  Track data size in bytes include MARKs GAPs etc.
0x0002 TLEN Data bytes
0x0002 + TLEN CLEN Track clock marks 
where CLEN = TLEN / 8 + ( TLEN % 8 + 7 ) / 8
One bit corresponds with one byte of raw track data
0 - byte with normal CLK (normal data)
1 - byte with mark CLK (for index label 0xA1 ¨ 0xC2)
For example:
 { 0xA1,0xA1,0x00,0xA1,0xFE,0x00,0x00,0x01 }
 where 0xA1 - MFM markers
 for this raw data, bits array store as one byte - 0x0B
 unused last bits should be 0.
0x0002 + TLEN + CLEN FLEN Float marks 
where FLEN = CLEN
One bit corresponds with one byte of raw track data
0 - byte not 'floating' ( not 'weak')
1 - byte 'floating' ('weak')
unused last bits should be 0.
TDATALN = 2 + TLENN + 2×CLENN


Track data for type 0x82 (mixed MFM/FM) "floating" or "weak" data chunk

Offset Length Comment
0x0000 2 TLEN 
  Track data size in bytes include MARKs GAPs etc.
0x0002 TLEN Data bytes
0x0002 + TLEN CLEN Track clock marks 
where CLEN = TLEN / 8 + ( TLEN % 8 + 7 ) / 8
One bit corresponds with one byte of raw track data
0 - byte with normal CLK (normal data)
1 - byte with mark CLK (for index label 0xA1 ¨ 0xC2)
For example:
 { 0xA1,0xA1,0x00,0xA1,0xFE,0x00,0x00,0x01 }
 where 0xA1 - MFM markers
 for this raw data, bits array store as one byte - 0x0B
 unused last bits should be 0.
0x0002 + TLEN + CLEN MLEN MF marks 
where MLEN = CLEN
One bit corresponds with one byte of raw track data
0 - byte coded with MFM
1 - byte code with FM
unused last bits should be 0.
0x0002 + TLEN + 2 * CLEN FLEN Float marks 
where FLEN = CLEN
One bit corresponds with one byte of raw track data
0 - byte not 'floating' ( not 'weak')
1 - byte 'floating' ('weak')
unused last bits should be 0.
TDATALN = 2 + TLENN + 3×CLENN


Track data for type 0x83 ("floating" or "weak" data chunk with multiple 'read')

Offset Length Comment
0x0000 2 TLEN + (NUM - 1) 
  Track data size in bytes include MARKs GAPs etc.
  TLEN must be aligned to 8, so the lowest 3 bit
  always 0. NUM must be between 1 and 8.
0x0002 2 OFFS / 8 
  OFFSet from the start of the previouse Track
  OFFS aligned to 8.
0x0004 TLEN First alternate, Data bytes
0x0004 + TLEN CLEN First alternative, Track clock marks 
where CLEN = TLEN / 8
One bit corresponds with one byte of raw track data
0 - byte with normal CLK (normal data)
1 - byte with mark CLK (for index label 0xA1 ¨ 0xC2)
For example:
 { 0xA1,0xA1,0x00,0xA1,0xFE,0x00,0x00,0x01 }
 where 0xA1 - MFM markers
 for this raw data, bits array store as one byte - 0x0B
 start and length are 8 bit aligned, so there are no
 unused bits.
0x0004 + TLEN + CLEN TLEN Second alternative, Data bytes
0x0004 + 2 × TLEN + CLEN CLEN Second alernative, Track clock marks
... continues until all data chunk ...
TDATALN = 4 + TLENN + CLENN
where TLEN = (TLEN + (NUM - 1)) & 0xfff8
where NUM = ((TLEN + (NUM - 1)) & 0x0007) + 1
This track data is represent "alternative" read of the given part (OFFS, TLEN) of the previouse track (type: 0x00, 0x01 or 0x02)
This track must be preceeded by a full track (type: 0x00, 0x01 or 0x02). We can define several not overlapped alternative read areas. e.g.:
TrackN data (0x00); 3 × TrackN,offs1,len1; 2 × TrackN,offs2,len2; TrackN+1 data(0x00); e.t.c.
####### TrackN data (type: 0x00) ########
## TrackN,offs1,len1-1 ##
## TrackN,offs1,len1-2 ##
## TrackN,offs1,len1-3 ##
## TrackN,offs2,len2-1 ##
## TrackN,offs2,len2-2 ##
####### TrackN+1 data (type: 0x00) ########


Track data for type 0xe0 Microdrive cartridge track

Offset Length Comment
0x0000 2 TLEN / 8 
  Track data size in bytes include GAPs etc.
0x0002 TLEN Data bytes
0x0002 + TLEN BLEN Track bad bytes marks 
where BLEN = TLEN / 8
One bit corresponds with one byte of raw track data
0 - byte usable
1 - byte unusable (bad)
TDATALN = 2 + TLENN
Note: on Microdrive tapes the data recorded in two paralell tracks (with a standard stereo
tape head arrangement), but the Interface I hardware deal about multiplexing/demultiplexing
data so, ZX Spectrum never see the two track separately.
Note2: the track length is always 8 byte aligned, so if your data cannot fit on an 8 byte boundary, you must pad with unusable bytes. This is not unnatural because where the microdrive endless tape spliced there are several bad bytes.


Data for type 0xf0 ("compressed" track data)

Offset Length Comment
0x0000 2 DLEN - 1 
  Compressed data size minus one.
0x0002 1 Track type 
  This is the track type of uncompressed data,
  so emulators can calculate uncompressed length and
  make decision about the track without uncompressing
  the data (e.g. discard weak track data if it not
  supported, etc...)
0x0003 2 TLEN 
  This is the TLEN of uncompressed data,
  so emulators can calculate uncompressed length
0x0005 DLEN Data bytes. 
  If uncompress this data with zlib it gives track data
  as defined in Tracks data without Track type and TLEN.
TDATALN = 6 + DLENN
This track is a container which hold headerless track data in a compressed way.
So after uncompression, we get a whole track without Track type and TLEN
as defined in Tracks data.
Only 0x00, 0x01, 0x02, 0x80, 0x81, 0x82 and 0x83 are the valid Track types, so
compressed tracks cannot be nested.
Compression method is the zlib deflate algorithm with standard zlib header.
Uncompressed data length can be calculated from Track type and TLEN
according to section Track data for type 0x00 (MFM) and 0x01 (FM)
For e.g. if Track Type is 0x00 (MFM track), then track length is
TDATAL = 3 + TLEN + TLEN / 8 + (TLEN % 8 + 7) / 8 and uncompressed data
length is TDATAL - 3 (because compressed data does not include Track type and TLEN.
Note: The compressed size maximum is 65536 byte, so if a track compressed size greater than
65536 it can be recorded only as uncompressed track. We have to note that: the highest capacity
(2.88MB/ED) floppy disk's raw track size is about 50KB, so this limit looks acceptable.
Retrieved from ‘https://sinclair.wiki.zxnet.co.uk/w/index.php?title=UDI_format&oldid=2360’