file module utils: Gwyddion Application Library Reference Manual

## file module utils file module utils — Utility functions for file modules |

## Functions

gboolean | gwy_get_gboolean8 ()
gint16 | gwy_get_gint16_le ()
gint16 | gwy_get_gint16_be ()
guint16 | gwy_get_guint16_le ()
guint16 | gwy_get_guint16_be ()
gint32 | gwy_get_gint32_le ()
gint32 | gwy_get_gint32_be ()
guint32 | gwy_get_guint32_le ()
guint32 | gwy_get_guint32_be ()
gint64 | gwy_get_gint64_le ()
gint64 | gwy_get_gint64_be ()
guint64 | gwy_get_guint64_le ()
guint64 | gwy_get_guint64_be ()
gfloat | gwy_get_gfloat_le ()
gfloat | gwy_get_gfloat_be ()
gdouble | gwy_get_gdouble_le ()
gdouble | gwy_get_gdouble_be ()
gdouble | gwy_get_pascal_real_le ()
gdouble | gwy_get_pascal_real_be ()
gboolean | gwy_app_channel_check_nonsquare ()
gboolean | gwy_app_channel_title_fall_back ()
gboolean | gwy_app_xyz_title_fall_back ()
guint | gwy_app_channel_remove_bad_data ()
GwyDataField * | gwy_app_channel_mask_of_nans ()
GQuark | gwy_text_header_error_quark ()
GHashTable * | gwy_text_header_parse ()
const gchar * | gwy_text_header_context_get_section ()
guint | gwy_text_header_context_get_lineno ()
void | gwy_file_channel_import_log_add ()
void | gwy_file_volume_import_log_add ()
void | gwy_file_xyz_import_log_add ()
void | gwy_file_curve_map_import_log_add ()

## Types and Values

#define | GWY_TEXT_HEADER_ERROR
enum | GwyTextHeaderError
| GwyTextHeaderParser
| GwyTextHeaderContext

## Includes

    #include <app/gwymoduleutils-file.h>

## Description

Functions `gwy_app_channel_check_nonsquare()` and `gwy_app_channel_title_fall_back()` perform common tasks improving the imported of channels from foreign data files. Typically one calls them on all imported channel ids after storing the data fields the the container, if they are useful for a given file type.

The group of functions `gwy_get_gint16_le()`, `gwy_get_gint16_be()`, etc. is intended to portably read packed binary data structures that are commonly found in SPM files. They all work identically: the binary data value is read from the buffer, converted if necessary, and the provided buffer pointer is moved to point after the value to faciliate sequential reading.

As no buffer size is passed, obviously no buffer size checking is performed. The caller has to ensure the buffer is large enough -- it is expected the caller checks the total buffer size before starting to parse it.

For example to portably read the following packed struct stored in big-endian byte order:

    1
    2
    3
    4
    5
    struct {
        guint16 xres;
        guint16 yres;
        gfloat measure;
    } header;

one can do (after checking the buffer size):

    1
    2
    3
    4
    const guchar *p = buffer;
    header.xres    = gwy_get_guint16_be(&p);
    header.yres    = gwy_get_guint16_be(&p);
    header.measure = gwy_get_gfloat_be(&p);

and `p` will point after `measure` in `buffer` after this snippet is finished.

The data types used in `header` do not matter (provided they are large enough to hold the values), the exact types are determined by the functions used. Therefore the reading would work identically if `header` was defined using common types:

    1
    2
    3
    4
    5
    struct {
        gint xres;
        gint yres;
        gdouble measure;
    } header;

## Functions

### gwy_get_gboolean8 ()

    gboolean
    gwy_get_gboolean8 (const guchar **ppv);

Reads a boolean value stored as a signle byte from a binary data buffer, moving the buffer pointer to point just after the value.

#### Parameters

ppv | Pointer to a pointer to boolean (stored as a signle byte) in a memory buffer. |

#### Returns

The gboolean value read from the buffer.

Since: 2.3

### gwy_get_gint16_le ()

    gint16
    gwy_get_gint16_le (const guchar **ppv);

Reads a signed 16bit integer value from a little-endian binary data buffer, moving the buffer pointer to point just after the value.

#### Parameters

ppv | Pointer to a pointer to a little-endian signed 16bit integer in a memory buffer. |

#### Returns

The gint16 value read from the buffer.

Since: 2.3

### gwy_get_gint16_be ()

    gint16
    gwy_get_gint16_be (const guchar **ppv);

Reads a signed 16bit integer value from a big-endian binary data buffer, moving the buffer pointer to point just after the value.

#### Parameters

ppv | Pointer to a pointer to a big-endian signed 16bit integer in a memory buffer. |

#### Returns

The gint16 value read from the buffer.

Since: 2.3

### gwy_get_guint16_le ()

    guint16
    gwy_get_guint16_le (const guchar **ppv);

Reads an unsigned 16bit integer value from a little-endian binary data buffer, moving the buffer pointer to point just after the value.

#### Parameters

ppv | Pointer to a pointer to a little-endian unsigned 16bit integer in a memory buffer. |

#### Returns

The guint16 value read from the buffer.

Since: 2.3

### gwy_get_guint16_be ()

    guint16
    gwy_get_guint16_be (const guchar **ppv);

Reads an unsigned 16bit integer value from a big-endian binary data buffer, moving the buffer pointer to point just after the value.

#### Parameters

ppv | Pointer to a pointer to a big-endian unsigned 16bit integer in a memory buffer. |

#### Returns

The guint16 value read from the buffer.

Since: 2.3

### gwy_get_gint32_le ()

    gint32
    gwy_get_gint32_le (const guchar **ppv);

Reads a signed 32bit integer value from a little-endian binary data buffer, moving the buffer pointer to point just after the value.

#### Parameters

ppv | Pointer to a pointer to a little-endian signed 32bit integer in a memory buffer. |

#### Returns

The gint32 value read from the buffer.

Since: 2.3

### gwy_get_gint32_be ()

    gint32
    gwy_get_gint32_be (const guchar **ppv);

Reads a signed 32bit integer value from a big-endian binary data buffer, moving the buffer pointer to point just after the value.

#### Parameters

ppv | Pointer to a pointer to a big-endian signed 32bit integer in a memory buffer. |

#### Returns

The gint32 value read from the buffer.

Since: 2.3

### gwy_get_guint32_le ()

    guint32
    gwy_get_guint32_le (const guchar **ppv);

Reads an unsigned 32bit integer value from a little-endian binary data buffer, moving the buffer pointer to point just after the value.

#### Parameters

ppv | Pointer to a pointer to a little-endian unsigned 32bit integer in a memory buffer. |

#### Returns

The guint32 value read from the buffer.

Since: 2.3

### gwy_get_guint32_be ()

    guint32
    gwy_get_guint32_be (const guchar **ppv);

Reads an unsigned 32bit integer value from a big-endian binary data buffer, moving the buffer pointer to point just after the value.

#### Parameters

ppv | Pointer to a pointer to a big-endian unsigned 32bit integer in a memory buffer. |

#### Returns

The guint32 value read from the buffer.

Since: 2.3

### gwy_get_gint64_le ()

    gint64
    gwy_get_gint64_le (const guchar **ppv);

Reads a signed 64bit integer value from a little-endian binary data buffer, moving the buffer pointer to point just after the value.

#### Parameters

ppv | Pointer to a pointer to a little-endian signed 64bit integer in a memory buffer. |

#### Returns

The gint64 value read from the buffer.

Since: 2.3

### gwy_get_gint64_be ()

    gint64
    gwy_get_gint64_be (const guchar **ppv);

Reads a signed 64bit integer value from a big-endian binary data buffer, moving the buffer pointer to point just after the value.

#### Parameters

ppv | Pointer to a pointer to a big-endian signed 64bit integer in a memory buffer. |

#### Returns

The gint64 value read from the buffer.

Since: 2.3

### gwy_get_guint64_le ()

    guint64
    gwy_get_guint64_le (const guchar **ppv);

Reads an unsigned 64bit integer value from a little-endian binary data buffer, moving the buffer pointer to point just after the value.

#### Parameters

ppv | Pointer to a pointer to a little-endian unsigned 64bit integer in a memory buffer. |

#### Returns

The guint64 value read from the buffer.

Since: 2.3

### gwy_get_guint64_be ()

    guint64
    gwy_get_guint64_be (const guchar **ppv);

Reads an unsigned 64bit integer value from a big-endian binary data buffer, moving the buffer pointer to point just after the value.

#### Parameters

ppv | Pointer to a pointer to a big-endian unsigned 64bit integer in a memory buffer. |

#### Returns

The guint64 value read from the buffer.

Since: 2.3

### gwy_get_gfloat_le ()

    gfloat
    gwy_get_gfloat_le (const guchar **ppv);

Reads a single-precision IEEE float value from a little-endian binary data buffer, moving the buffer pointer to point just after the value.

#### Parameters

ppv | Pointer to a pointer to a little-endian single-precision IEEE float in a memory buffer. |

#### Returns

The gfloat value read from the buffer.

Since: 2.3

### gwy_get_gfloat_be ()

    gfloat
    gwy_get_gfloat_be (const guchar **ppv);

Reads a single-precision IEEE float value from a big-endian binary data buffer, moving the buffer pointer to point just after the value.

#### Parameters

ppv | Pointer to a pointer to a big-endian single-precision IEEE float in a memory buffer. |

#### Returns

The gfloat value read from the buffer.

Since: 2.3

### gwy_get_gdouble_le ()

    gdouble
    gwy_get_gdouble_le (const guchar **ppv);

Reads a double-precision IEEE float value from a little-endian binary data buffer, moving the buffer pointer to point just after the value.

#### Parameters

ppv | Pointer to a pointer to a little-endian double-precision IEEE float in a memory buffer. |

#### Returns

The gdouble value read from the buffer.

Since: 2.3

### gwy_get_gdouble_be ()

    gdouble
    gwy_get_gdouble_be (const guchar **ppv);

Reads a double-precision IEEE float value from a big-endian binary data buffer, moving the buffer pointer to point just after the value.

#### Parameters

ppv | Pointer to a pointer to a big-endian double-precision IEEE float in a memory buffer. |

#### Returns

The gdouble value read from the buffer.

Since: 2.3

### gwy_get_pascal_real_le ()

    gdouble
    gwy_get_pascal_real_le (const guchar **ppv);

Reads a six-byte Pascale Real value from a little-endian binary data buffer, moving the buffer pointer to point just after the value.

#### Parameters

ppv | Pointer to a pointer to a little-endian six-byte Pascal Real in a memory buffer. |

#### Returns

The floating point value read from the buffer as a gdouble.

Since: 2.3

### gwy_get_pascal_real_be ()

    gdouble
    gwy_get_pascal_real_be (const guchar **ppv);

Reads a six-byte Pascale Real value from a big-endian binary data buffer, moving the buffer pointer to point just after the value.

#### Parameters

ppv | Pointer to a pointer to a big-endian six-byte Pascal Real in a memory buffer. |

#### Returns

The floating point value read from the buffer as a gdouble.

Since: 2.3

### gwy_app_channel_check_nonsquare ()

    gboolean
    gwy_app_channel_check_nonsquare (GwyContainer *data,
                                     gint id);

Sets `realsquare' for a channel highly non-square pixels.

The threshold for highly non-square is somewhat arbitrary. Fortunately, most files encoutered in practice have the measure ratio either very close to 1, larger or equal than 2.

#### Parameters

data | A data container. |
id | Data channel id. |

#### Returns

`TRUE` if the channel was found to have highly non-square pixels and `realsquare' was set (otherwise it was unset).

Since: 2.3

### gwy_app_channel_title_fall_back ()

    gboolean
    gwy_app_channel_title_fall_back (GwyContainer *data,
                                     gint id);

Adds a channel title based on data field units.

The guess is very simple, but probably better than `Unknown channel' in most cases. If there already is a title it is left intact, making use of this function as a fall-back easier.

#### Parameters

data | A data container. |
id | Data channel id. |

#### Returns

`TRUE` if the title was set (either by this function or before).

Since: 2.3

### gwy_app_xyz_title_fall_back ()

    gboolean
    gwy_app_xyz_title_fall_back (GwyContainer *data,
                                 gint id);

Adds a XYZ data title based on surface value units.

The guess is very simple, but probably better than `Unknown channel' in most cases. If there already is a title it is left intact, making use of this function as a fall-back easier.

#### Parameters

data | A data container. |
id | XYZ surface data id. |

#### Returns

`TRUE` if the title was set (either by this function or before).

Since: 2.45

### gwy_app_channel_remove_bad_data ()

    guint
    gwy_app_channel_remove_bad_data (GwyDataField *dfield,
                                     GwyDataField *mfield);

Replaces bad data points with some neutral values.

Since Gwyddion has no concept of bad data points, they are usually marked with a mask and replaced with some neutral values upon import, leaving the user to decide how to proceed further. This helper function performs such replacement.

#### Parameters

dfield | A data field. The values of bad data points are ignored and might be even left uninitialized. |
mfield | A mask field containing 1.0 in place of good data points, 0.0 in place of bad points. It will be inverted to become the mask of bad points. |

#### Returns

The number of bad data points replaced. If zero is returned, all points are good and there is no need for masking.

Since: 2.14

### gwy_app_channel_mask_of_nans ()

    GwyDataField *
    gwy_app_channel_mask_of_nans (GwyDataField *dfield,
                                  gboolean removebad);

Creates a mask corresponding to not-a-number and infinite values in a data field.

NaNs and infinities are not normally allowed in Gwyddion so this function should be used upon import if the source file format can represent such values.

The invalid values can be automatically removed using the same method as `gwy_app_channel_remove_bad_data()`.

#### Parameters

dfield | A data field possibly containing NaNs and infinities. |
removebad | Automatically remove the invalid values. |

#### Returns

A newly created data field with 1.0 in pixels corresponding to a NaN or infinity in `dfield` . If no such pixels are present, `NULL` is returned.

Since: 2.38

### gwy_text_header_error_quark ()

    GQuark
    gwy_text_header_error_quark (void);

Returns error domain for expression parsin and evaluation.

See and use `GWY_TEXT_HEADER_ERROR`.

#### Returns

The error domain.

Since: 2.18

### gwy_text_header_parse ()

    GHashTable *
    gwy_text_header_parse (gchar *header,
                           const GwyTextHeaderParser *parser,
                           gpointer user_data,
                           GError **error);

Parses a line-oriented text header into a hash table.

See GwyTextHeaderParser for details of memory and error handling.

Lines consisting only of whitespace are ignored.

#### Parameters

header | Text header to parse. It must be `NULL`-terminated and writable. Its contents will be modified to directly embed the hash keys and/or values. It must not be freed while the returned hash table is in use. |
parser | Parser specification. |
user_data | User data passed to parser callbacks. |
error | Error to set on fatal errors. |

#### Returns

A newly created hash table with values indexed by they keys found in the header.

Since: 2.18

### gwy_text_header_context_get_section ()

    const gchar *
    gwy_text_header_context_get_section (const GwyTextHeaderContext *context);

Gets the currently open section.

This function may be called if no sectioning is defined. It simply returns `NULL` then.

#### Parameters

context | Header parsing context. |

#### Returns

The name of the currently open section, `NULL` if there is none.

Since: 2.18

### gwy_text_header_context_get_lineno ()

    guint
    gwy_text_header_context_get_lineno (const GwyTextHeaderContext *context);

Gets the current header line.

#### Parameters

context | Header parsing context. |

#### Returns

The current line number, starting from zero.

Since: 2.18

### gwy_file_channel_import_log_add ()

    void
    gwy_file_channel_import_log_add (GwyContainer *data,
                                     gint id,
                                     const gchar *filetype,
                                     const gchar *filename);

Logs the import of a channel from third-party file.

This is a convenience wrapper for `gwy_app_channel_log_add()`. The source id will be set to -1. The file name will be added to function arguments.

#### Parameters

data | A data container. |
id | Data channel id. |
filetype | File type, i.e. the name of the function importing the data (without any "file::" prefix). Since 2.38 it is possible to pass `NULL` to fill the name of the currently running file type function automatically. |
filename | Name of the imported file. If it is not valid UTF-8, it will be converted to UTF-8 using `g_filename_to_utf8()`. Failing even that, non-ASCII characters will be escaped. |

Since: 2.35

### gwy_file_volume_import_log_add ()

    void
    gwy_file_volume_import_log_add (GwyContainer *data,
                                    gint id,
                                    const gchar *filetype,
                                    const gchar *filename);

Logs the import of volume data from third-party file.

This is a convenience wrapper for `gwy_app_volume_log_add()`. The source id will be set to -1. The file name will be added to function arguments.

#### Parameters

data | A data container. |
id | Volume data id. |
filetype | File type, i.e. the name of the function importing the data (without any "file::" prefix). Since 2.38 it is possible to pass `NULL` to fill the name of the currently running file type function automatically. |
filename | Name of the imported file. If it is not valid UTF-8, it will be converted to UTF-8 using `g_filename_to_utf8()`. Failing even that, non-ASCII characters will be escaped. |

Since: 2.35

### gwy_file_xyz_import_log_add ()

    void
    gwy_file_xyz_import_log_add (GwyContainer *data,
                                 gint id,
                                 const gchar *filetype,
                                 const gchar *filename);

Logs the import of xyz data from third-party file.

This is a convenience wrapper for `gwy_app_xyz_log_add()`. The source id will be set to -1. The file name will be added to function arguments.

#### Parameters

data | A data container. |
id | XYZ surface data id. |
filetype | File type, i.e. the name of the function importing the data (without any "file::" prefix). It is possible to pass `NULL` to fill the name of the currently running file type function automatically. |
filename | Name of the imported file. If it is not valid UTF-8, it will be converted to UTF-8 using `g_filename_to_utf8()`. Failing even that, non-ASCII characters will be escaped. |

Since: 2.45

### gwy_file_curve_map_import_log_add ()

    void
    gwy_file_curve_map_import_log_add (GwyContainer *data,
                                       gint id,
                                       const gchar *filetype,
                                       const gchar *filename);

Logs the import of volume data from third-party file.

This is a convenience wrapper for `gwy_app_curve_map_log_add()`. The source id will be set to -1. The file name will be added to function arguments.

#### Parameters

data | A data container. |
id | Volume data id. |
filetype | File type, i.e. the name of the function importing the data (without any "file::" prefix). Since 2.38 it is possible to pass `NULL` to fill the name of the currently running file type function automatically. |
filename | Name of the imported file. If it is not valid UTF-8, it will be converted to UTF-8 using `g_filename_to_utf8()`. Failing even that, non-ASCII characters will be escaped. |

Since: 2.60

## Types and Values

### GWY_TEXT_HEADER_ERROR

    #define GWY_TEXT_HEADER_ERROR gwy_text_header_error_quark()

Error domain for text header parsing. Errors in this domain will be from the GwyTextHeaderError enumeration. See GError for information on error domains.

Since: 2.18

### enum GwyTextHeaderError

Error codes returned by text header parsing.

Some errors, in particular `GWY_TEXT_HEADER_ERROR_KEY` and `GWY_TEXT_HEADER_ERROR_VALUE` are expected to be raised by user callbacks (they are not restricted to these codes though).

#### Members

GWY_TEXT_HEADER_ERROR_SECTION_NAME | Section name is invalid. It is raised by the parser only for an empty section name. |
GWY_TEXT_HEADER_ERROR_SECTION_END | Section ended when a different section or no section was open. Note that `gwy_text_header_context_get_section()` returns the section being closed at the time this error is raised. |
GWY_TEXT_HEADER_ERROR_SECTION_START | Section started before the previous ended. This is raised only if `endsection_template` is set. |
GWY_TEXT_HEADER_ERROR_PREFIX | Line lacks the mandatory prefix. |
GWY_TEXT_HEADER_ERROR_GARBAGE | Line cannot be parsed into a key-value pair. |
GWY_TEXT_HEADER_ERROR_KEY | Key name is invalid, namely empty. |
GWY_TEXT_HEADER_ERROR_VALUE | Value is invalid. This is never raised by the parser. |
GWY_TEXT_HEADER_ERROR_TERMINATOR | The text header has ended without being terminated by specified terminator. |

Since: 2.18

### GwyTextHeaderParser

    typedef struct {
        const gchar *comment_prefix;
        const gchar *section_template;
        const gchar *endsection_template;
        const gchar *section_accessor;
        const gchar *line_prefix;
        const gchar *key_value_separator;
        const gchar *terminator;
        gboolean (*item)(const GwyTextHeaderContext *context,
                         GHashTable *hash,
                         gchar *key,
                         gchar *value,
                         gpointer user_data,
                         GError **error);
        gboolean (*section)(const GwyTextHeaderContext *context,
                            const gchar *name,
                            gpointer user_data,
                            GError **error);
        gboolean (*endsection)(const GwyTextHeaderContext *context,
                               const gchar *name,
                               gpointer user_data,
                               GError **error);
        void (*end)(const GwyTextHeaderContext *context,
                    gsize length,
                    gpointer user_data);
        gboolean (*error)(const GwyTextHeaderContext *context,
                          GError *error,
                          gpointer user_data);
        GDestroyNotify destroy_key;
        GDestroyNotify destroy_value;
    } GwyTextHeaderParser;

Text header parser specification.

Memory considerations: In general, the parser attempts to reuse the contents of `header` directly for the hash keys and values. There are two cases when it cannot: sectioning implies that keys must be constructed dynamically and the use of `item` callback means the parser has no control on what is inserted into the hash table.

This means that the `item` callback must free `key` if sectioning is used and it is not going to actually use it as the hash table key. And, of course, suitable `destroy_key` and `destroy_value` functions must be set.

#### Members

const gchar *`comment_prefix`; | Prefix denoting comment lines. It is possible to specify multiple prefixes by separating them with newline ("\n"). |
const gchar *`section_template`; | Section start template. It must contain the character "\x1a" in the place where the section name apprears. For example, "[Section \x1a]". |
const gchar *`endsection_template`; | Section end template. It may or may not contain the substitute character "\x1a" depending on whether the section end markers contain the section name. It is invalid to set |
const gchar *`section_accessor`; | Glue to put between the section name and key when constructing hash table keys. It is invalid to set `section_accessor` without setting `section_template` . Typically, "::" is used. |
const gchar *`line_prefix`; | Mandatory prefix of each line. Note this applies only to key-value lines. The templates, such comments, sections and terminators, must still include `line_prefix` if they start with it. |
const gchar *`key_value_separator`; | The string separating keys and values on each line. Typically, "=" or ":" is used. When left `NULL`, whitespace plays the role of the separator. Of course, keys cannot contain whitespace then. |
const gchar *`terminator`; | Line that marks end of the header. It is mandatory if specified, `GWY_TEXT_HEADER_ERROR_TERMINATOR` is raised when the header ends sooner than `terminator` is found. |
`item` () | Callback called when a key-value pair is found. If set it is responsible for inserting the pair to the hash table with `g_hash_table_replace()`. It is free to insert a different pair or nothing. It must return `FALSE` if it raises an error. |
`section` () | Callback called when a section starts. It must return `FALSE` if it raises an error. |
`endsection` () | Callback called when a section end. It must return `FALSE` if it raises an error. |
`end` () | Callback called when the parsing finishes successfully, `length` contains the length of parsed text in bytes. |
`error` () | Callback called when an error occurs, including errors raised by other user callbacks. If it returns `TRUE` the error is considered fatal and the parsing terminates immediately. If it is left unset no errors are fatal hence no errors are reported to the caller. |
GDestroyNotify `destroy_key`; | Function to destroy keys, passed to `g_hash_table_new_full()`. It is invalid to set `destroy_key` if `item` callback is not set. |
GDestroyNotify `destroy_value`; | Function to destroy values, passed to `g_hash_table_new_full()`. It is invalid to set `destroy_value` if `item` callback is not set. |

Since: 2.18

### GwyTextHeaderContext

    typedef struct _GwyTextHeaderContext GwyTextHeaderContext;

GwyTextHeaderContext represents the parsing state. It is an opaque data structure and should be only manipulated with the functions below.

Since: 2.18