Description

A library of routines for reading and writing in Level 5 MAT-File format.

Author

Ivan Raikov

Version

Requires

Usage

(require-extension mat5-lib)

Download

mat5-lib.egg

Documentation

mat5-lib is a library of routines for reading and writing in Level 5 MAT-File format. It provides functions to read and write Scheme arrays and vectors in the MAT-File format, as well as Scheme representations of MAT-File cells, structures, and objects.

MAT-File data types and records

datatype: (define-datatype MAT5:data-type MAT5:data-type? ...)

Representation of MAT-File data types:

(miINT8) 8-bit, signed
(miUINT8) 8-bit unsigned
(miINT16) 16-bit, signed
(miUINT16) 16-bit unsigned
(miINT32) 32-bit, signed
(miUINT32) 32-bit, unsigned
(miSINGLE) IEEE 754 single precision
(miDOUBLE) IEEE 754 double precision
(miINT64) 64-bit, signed
(miUINT64) 64-bit, unsigned
(miMATRIX) MAT-File array
(miCOMPRESSED) Compressed data
(miUTF8) Unicode UTF-8 encoded character data
(miUTF16) Unicode UTF-16 encoded character data
(miUTF32) Unicode UTF-32 encoded character data

record: (define-record MAT5:header magic text subsys version eport)

A record type that represents MAT-file Level 5 headers.

text header text (byte vector of size 116)
subsys subsystem-specific (byte vector of size 8)
version unsigned 16-bit version number (always 0x0100)
eport an endian port (see module endian-port)

record: (define-record MAT5:data-element type bytes data)

A record type that represents MAT-file Level 5 data elements.

type is of type MAT5:data-type
bytes is the size of the data element in bytes
data is the data element object -- either a fixnum/flonum or a MAT5:array object

datatype: (define-datatype MAT5:array MAT5:array? ...)

A representation of the different types of MAT-file arrays:

(MAT5:num-array name data-type dims real imag)

A MAT-file numeric array. A numeric array is represented as either an SRFI-47 array, or an SRFI-40 stream in which every element is an SRFI-4 column vector:

name array name (string)
data-type array element type (must be one of the numeric types)
dims array dimensions (a list of positive integers)
real real part (an SRFI-47 array or a stream of column vectors)
imag imaginary part (the same representation as the real part, or #f)

(MAT5:sparse-array name data-type dims row-index col-index real imag)

A MAT-file numeric sparse array. A numeric sparse array is represented as either an SRFI-47 array, or an SRFI-40 stream in which every element is an SRFI-4 column vector:

name array name (string)
data-type array element type (must be one of the numeric types)
dims array dimensions (a list of positive integers)
row-index row indices of non-zero elements
col-index column indices of non-zero elements
real real part (an SRFI-47 array or a stream of column vectors)
imag imaginary part (the same representation as the real part, or #f)

(MAT5:cell-array name dims cell)

A MAT-file cell array:

name array name (string)
dims array dimensions (a list of positive integers)
cell a MAT5:cell object in which every element is a MAT5:data-element

(MAT5:structure name dims field-names fields)

A MAT-file structure is represented by a MAT5:cell object in which every element is an association list that contains the fields of the structure:

name array name (string)
dims array dimensions (a list of positive integers)
class-name class name (string)
field-names the names of the class fields (a list of strings)
fields a MAT5:cell object in which every element is an association list of fields

(MAT5:object name dims class-name field-names fields)

A MAT5 object data element has the same subelements as a structure, plus a class name subelement:

name array name (string)
dims array dimensions (a list of positive integers)
class-name class name (string)
field-names the names of the class fields (a list of strings)
fields a MAT5:cell object in which every element is an association list of fields

record: (define-record MAT5:cell dims data)

A representation of MAT-File cells.

dims cell dimensions
data an R5RS non-homogenous array that contains elements of type MAT5:array.

MAT-File routines

Predicates and routines for handling MAT-file datatypes and structures
procedure: MAT5:numeric-type?:: MAT5:DATA-TYPE -> BOOLEAN

Returns true if the given type is an atomic numeric type.

procedure: MAT5:array-type?:: MAT5:DATA-TYPE -> BOOLEAN

Returns true if the given type is an array type.

MAT5 cell routines
procedure: init-MAT5:cell:: UINTEGER * ... -> MAT5:CELL

Create a new MAT5:cell object with the specified dimensions. The dimensions must all be positive integers, and at least one dimension must be specified.

procedure: MAT5:cell-ref:: MAT5:CELL * UINTEGER ... -> VALUE

Given a MAT5:cell object and an index, returns the value found at that index in the cell. The index must be a list of positive integers, and it must be within the bounds of the cell dimensions.

procedure: MAT5:cell-set!:: MAT5:CELL * VALUE * UINTEGER ... -> UNDEFINED

Given a MAT5:cell object and an index, destructively replaces the element at the given index of the cell with the given value.

procedure: vector->MAT5:cell:: VECTOR * UINTEGER LIST [* ORDER] -> MAT5:CELL

Given a vector, creates a new MAT5:cell object that consists of the elements of the vector. The vector must be of length equal to the total size of the array, and its elements are used to initialize the cell in either row-major order (left to right and top to bottom), or in column-major order (top to bottom and then left to right).

The optional argument ORDER specifies the initialization order and can be either 'row-major or 'col-major. The default is 'row-major.

MAT5 array routines
procedure: MAT5:array-foldi:: (INDEX * VALUE * AX -> AX) [* ORDER] -> MAT5:ARRAY * AX -> AX

Iterator function for non-homogenous MAT5:array objects; that is, objects that are either MAT5:cell-array (MAT-File cell) or MAT5:structure (MAT-File structure).

Analogous to the list iterator, this procedure repeatedly applies the given function to each element of the array, and accumulates the return value. The order of iteration is specified by the optional argument ORDER, which can be 'row-major (left to right and top to bottom) or 'col-major(top to bottom and then left to right). The default is 'row-major.

MAT-File routines for reading
procedure: MAT5:read-header:: ENDIAN-PORT -> MAT5:HEADER

Reads a MAT-File header from the given endian port. Returns a MAT5:header record.

procedure: MAT5:read-data-element:: ENDIAN-PORT -> MAT5:DATA-ELEMENT

Reads the next MAT-File data element from the given endian port. Returns a MAT5:data-element record.

procedure: MAT5:read-stream:: ENDIAN-PORT -> STREAM

Creates an SRFI-40 stream that has the MAT-File header from the given endian port as its first element, followed by all MAT-File data elements that can be read from the endian port. Any elements that are numeric arrays, or contain numeric arrays will be read lazily; i.e. the corresponding DATA field will be another SRFI-40 stream containing the column vectors of the numeric array.

procedure: MAT5:stream-read-data-element:: ENDIAN-PORT -> STREAM

Reads the next data element from the MAT-File. If the element is a numeric array, or if it contains numeric arrays, its numeric data will be read lazily; i.e. the corresponding DATA field will be an SRFI-40 stream containing the column vectors of the numeric array.

MAT-File routines for writing
procedure: MAT5:write-header:: ENDIAN-PORT * TEXT * SUBSYS -> UNDEFINED

Writes a MAT-File header to the given endian port. Arguments TEXT and SUBSYS are strings. If they are longer than their maximum permitted lengths (116 and 8, respectively), they will be truncated.

procedure: MAT5:write-data-element:: ENDIAN-PORT * VALUE -> BYTES

Writes a data element to the given endian port. VALUE can be one of the following types:

  • MAT5:array?
  • string?
  • s8vector?
  • u8vector?
  • s16vector?
  • u16vector?
  • s32vector?
  • u32vector?
  • f32vector?
  • f64vector?

procedure: MAT5:write-stream:: ENDIAN-PORT * TEXT * SUBSYS * STREAM -> UNDEFINED

Given an endian port, and an SRFI-40 stream consisting of MAT5:data-element objects, this routines writes a MAT-File header and all elements in the stream to the endian port. If any numeric arrays are contained in the input stream, it is expected that their data elements are encoded as SRFI-40 streams containing the column vectors of the array.

procedure: MAT5:stream-write-data-element:: ENDIAN-PORT * VALUE -> BYTES

Writes a MAT-File data element to the given endian port. If the input element is an array, any numeric array data must be encoded as an SRFI-40 stream, where each element of the stream is a column vector of the array.

Examples

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;
;; Example from the MAT-File documentation
;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
(let* ((eport+path (let-values (((fd temp-path) (file-mkstemp "/tmp/mat5-test.XXXXXX")))
			       (let ((temp-port (open-output-file* fd)))
				 (cons (port->endian-port temp-port) temp-path))))
       (eport (car eport+path))
       (temp-path (cdr eport+path)))

  (let ((aa (let ((real (make-array (ar32 0) 2 2))
		  (imag (make-array (ar32 0) 2 2)))
	      
	      (array-set! real 1.0  0 0)
	      (array-set! real 3.0  0 1)
	      (array-set! real 2.0  1 0)
	      (array-set! real 4.0  1 1)
	      
	      (array-set! imag 1.1  0 0)
	      
	      (MAT5:num-array  "my_array" (miSINGLE) '(2 2) real imag))))
       
    ;; Write MAT5 file header
    (MAT5:write-header eport "MATLAB 5.0 MAT-file" "")

    ;; Write test array aa
    (MAT5:write-data-element eport aa)

    ;; Close endian port for writing
    (close-endian-port eport)
    
    ;; Reopen endian port for reading
    (let ((eport (open-endian-port 'read temp-path)))

      ;; Read header from file
      (print (MAT5:read-header eport))

      ;; Read array from file
      (print (MAT5:read-data-element eport))

      ;; Close endian port for reading
      (close-endian-port eport))))


;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;
;; Reading and writing via the stream interface
;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
(let* ((eport+path (let-values (((fd temp-path) (file-mkstemp "/tmp/mat5-test.XXXXXX")))
			       (let ((temp-port (open-output-file* fd)))
				 (cons (port->endian-port temp-port) temp-path))))
       (eport (car eport+path))
       (temp-path (cdr eport+path)))

  (let ((aa (let ((av (stream (stream (list->f64vector '(2.0 5.0))
				      (list->f64vector '(1.0 3.0))
				      (list->f64vector '(4.0 6.0)))
			      
			      (stream (list->f64vector '(7.0  10.0))
				      (list->f64vector '(13.0 16.0))
				      (list->f64vector '(19.0 22.0)))
			      
			      (stream (list->f64vector '(8.0  11.0))
				      (list->f64vector '(14.0 17.0))
				      (list->f64vector '(20.0 23.0)))
			      
			      (stream (list->f64vector '(9.0  12.0)))
			      (list->f64vector '(15.0 18.0))
			      (list->f64vector '(21.0 24.0)))))
		  (MAT5:num-array  "aa2" (miDOUBLE) '(2 3 ??) av  #f))))

    ;; Write MAT5 file header 
    (MAT5:write-header eport "MATLAB 5.0 MAT-file" "")
    
    ;; Write test array aa
    (MAT5:stream-write-data-element eport aa)

    ;; Close endian port for writing
    (close-endian-port eport)

    ;; Reopen endian port for reading
    (let* ((eport (open-endian-port 'read temp-path))
	   ;; Create read stream
	   (stream (MAT5:read-stream eport))
	   ;; First stream element
	   (header (stream-car stream))
	   ;; Second stream element
	   (el2 (stream-car (stream-cdr stream))))
      
      ;; "Contents of second stream element"
      (for-each
       (lambda (x) 
	 (if (MAT5:array-type? (MAT5:data-element-type x))
	     (cases MAT5:array (MAT5:data-element-data x)
		    (MAT5:num-array (name data-type dims real imag)
				    (stream-for-each (lambda (x) (print "column: " x)) real))
		    (else (print data)))
	     (else (print x))))
       el2))
   
   ;; Close endian port for reading
    (close-endian-port eport)))

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;
;; Structure example from the MAT-File documentation
;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
(let* ((eport+path (let-values (((fd temp-path) (file-mkstemp "/tmp/mat5-test.XXXXXX")))
			       (let ((temp-port (open-output-file* fd)))
				 (cons (port->endian-port temp-port) temp-path))))
       (eport (car eport+path))
       (temp-path (cdr eport+path)))


  (let ((aa (let ((av (init-MAT5:cell 1 1))
		  (w  (make-array (ar64 0) 1 1))
		  (y  (make-array (ar64 0) 1 1))
		  (z  (make-array (ar64 0) 1 1)))
	      
	      (array-set! w 1.0  0 0)
	      (array-set! y 2.0  0 0)
	      (array-set! z 3.0  0 0)
	      
	      (MAT5:cell-set! av 
			      `(,(MAT5:num-array "w" (miDOUBLE) '(1 1) w #f) 
				,(MAT5:num-array "y" (miDOUBLE) '(1 1) y #f)
				,(MAT5:num-array "z" (miDOUBLE) '(1 1) z #f))
			      0 0)
	      
	      (MAT5:structure  "aa1" '(1 1) '("w" "y" "z")  av))))

    ;; Write MAT5 file header
    (MAT5:write-header eport "MATLAB 5.0 MAT-file" "")

    ;; Write test structure aa
    (MAT5:write-data-element eport aa)

    ;; Close endian port for writing
    (close-endian-port eport)
    
    ;; Reopen endian port for reading
    (let ((eport (open-endian-port 'read temp-path)))

      ;; Read header from file
      (print (MAT5:read-header eport))

      ;; Read array from file
      (print (MAT5:read-data-element eport))

      ;; Close endian port for reading
      (close-endian-port eport))))
    

License

Copyright 2005-2008 Ivan Raikov and the Georgia Institute of Technology

This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or (at
your option) any later version.

This program is distributed in the hope that it will be useful, but
WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
General Public License for more details.

A full copy of the GPL license can be found at
<http://www.gnu.org/licenses/>.