Hinterlib

Introduction

Hinterlib is one of the most heavily developed projects under the Aquefir namesake. As a libc surrogate, it exists to jumpstart ANSI C program development – therefore, it has a few broad ‘focus areas’:

1. Shorthanded type system
   • basic types like pointers and booleans
   • character types for ASCII and Unicode
   • integral types up to 128 bits
   • floating-point types up to double precision
   • SIMD vector types for both integrals and floating-points
   • modular memory type primitives
   • type decorators for bit-packing, aliasing and so on
2. General purpose I/O
   • streaming I/O a la FILE*
   • textual logging I/O
   • buffering facilities
3. Memory management facilities
   • software modular memory (SMM) toolkit, Himem
   • backing for textual data
   • provision for error stacking
   • stack allocation help
4. Non-generic general-purpose containers
   • uses SMM-capable algorithms
   • separation of algorithm from data structure
5. Operating system polyfills
   • Newline introspection
   • Current working directory
   • Command invocation a la system()
6. Textual/string manipulation tools
   • Unicode-centric alternative primitives
   • SMM-aware containers
   • ASCII/Unicode dichotomy
7. Comprehensive error handling
   • return value based code propagation
   • application-centric bookkeeping of codes
   • extensibility for the application level
   • pre-allocated error stack
8. General-purpose data handling utilities
   • checksum utilities
   • endian conversion routines
   • BigInt implementation
   • common mathematical functions
9. INI/config file parser
   • INI schema parser and validator

Shorthanded type system

Hinterlib offers shorthands for all of the common primitive types used in ANSI C programming. The following table enumerates them:

Vector shorthands

Hinterlib also offers a matrix of similarly-succinct shorthands for SIMD-aware vector types:

Software modular memory

The <hn/types/mem.h> header also provides matching shorthands for the software modular memory base types, known as knots, as well as for their complex aggregates, amalgams.

Decorators

Hinterlib provides the following decorators that expand appropriately in the presence of compiler support:

• HN_PACKED
• HN_TYPELESS (ergo may_alias)
• HN_NORETURN
• HN_NOTHROW
• HN_PURE
• HN_ALIGN(n) where n is the number of bits
• HN_DEPRECATED
• HN_VECSIZE(n) to create vector-aware array types

Definitions

Hinterlib makes out many important details about the dimensions of the target machine the code being compiled will run on. It exposes these as CPP macros to the user as follows:

• HN_CCVERSION version of the compiler in use
• Compiler identification macro
• ‘Platform’ identification macro
• Processor architecture identification macro
• HN_HAVE_I32 if true 32-bit integrals are available
• HN_HAVE_I64 if true 64-bit integrals are available
• HN_HAVE_I128 if compiler-based 128-bit integrals are available
• Endianness identification macros HN_LILENDIAN, HN_BIGENDIAN and HN_PDPENDIAN
• Pointer size identification macros HN_PTRSZ_16, HN_PTRSZ_32 and HN_PTRSZ_64

General-purpose I/O

Hinterlib provides a functionally congruent wrapper over the classic FILE*-based streaming I/O utilities of libc. This wrapper provides more clarity in its routine type signatures by employing an opaque structure pointer directly with no macro or typedef business, using shorthands for integrals in seek subroutine parameters, a pure data mode structure in place of libc’s string representation, and friendly utility functions for getting a hold of standard I/O handles and manipulating the internal standard I/O buffer. All of this is found in <hn/file.h>.

The library also provides a high-level logging utility with message categorisation, file descriptor redirection and intelligent nesting, all using pure data structures. All of this is found in <hn/log.h>.

Modular memory toolkit

Hinterlib provides a module called Himem for managing amalgams and knots a la software modular memory. This is used by other high level modules in Hinterlib and is to be used by downstream software to help simplify the abstractions involved without compromising opacity of the underlying data.

‘Modular memory’ is the semantic composition of large aggregations of memory—referred to primitively in the form of well-sized knots—stitched together into a complex but still mechanicalistic structure called amalgams. Hinterlib provides a container-like interface for conveniently mutating these amalgams, implementing methods that make it practical to handle them portably regardless of the underlying machine’s pointer size with minimal compromises on performance and without cutting out the transparency of the data structures that enable end users to create their own subroutines for modifying the amalgam’s underlying data. The interface provides the following methods for mutating the container-like plain old data objects, with their approximate C++ STL counterparts:

This technique is heavily relied upon by both Hinterlib internally and by downstream consumers of it to effectuate large memory management with small pointer sizes in banked memory hierarchies, such as those featured on the Intel 80286, Motorola 68000 and more.

Non-generic general-purpose containers

The chief in-tree consumer of the Himem software modular memory toolkit, Hinterlib’s container system aims to provide typeless, concrete data containerisation with SMM characteristics without compromising the transparency of the underlying data in the view of downstream code. This will allow data to have the greatest freedom of allocation lifetime, including even residing on the stack, as full user control of allocation is preserved by the interface design.

Instead of trying to supplicate a nonexistent abstract type system, it provides mechanicalist views of the underlying data’s dimensionality and the quantities and limits thereof, giving meaningful boundaries for users to ensure validity of data elements directly.

In addition to this concreteness, Hinterlib’s container toolkit tries to distinguish abstract data types from so-called data structures so that it is straightforward to select arbitrary algorithms that implement the former’s desired semantics regardless of the data type being targeted with it.

Operating system polyfills

Hinterlib of course must provide agnosticism of various eccentricities that differ between target operating systems, such as path particle separator strings and canonical newline forms. It will also provide an agnostic interface for introspecting about the running program’s location in the file system and the working directory it was invoked from. It will provide routines to modify the working directory and the program’s apparent environment variable set. Finally, it will provide a smarter interface for invoking other programs on the running system, similar in spirit to libc’s system() routine.

Textual string manipulation tools

Hinterlib’s container toolkit will be specialised to provide for textual data in two forms: ASCII and Unicode. Working copies of ASCII text will be 8-bit byte-oriented, while Unicode codepoint elements will be stored as UCS-4 in 32-bit unsigned integers. Separate algorithms will be implemented to provide ingress and egress of Unicode data from other encodings which may be used, like UTF-8.

Comprehensive error handling

Error code propagation is a problem Hinterlib aims to comprehensively solve by unifying two ideas: return codes and error stacks. The library facilitates two error code domains: the standard 8-bit POSIX error code list, and a wider 16-bit ‘extended error code’ domain which is application-specific.

Hinterlib library modules must maintain O(1) complexity of their own error code propagation so as to not need their own error stacks as state burdening downstream applications. However, downstream applications can (and should) use error stacks to effectively manage complex error states, especially when some or all of the error codes are non-fatal.

A legible interface will be provided to keep track of a downstream application’s extended error codes, their attributes and their strings.

General-purpose data handling utilities

As a matter of course, Hinterlib will help facilitate a great variety of checksum algorithms, endian conversion, checked integral arithmetic, a software arbitrary-precision integer (‘big int’) system, and various common mathematical functions.

INI/config file parser

Hinterlib provides C subroutines for parsing and serialising INI type configuration files with various compatibility options for both Microsoft Windows and Unix style config files. Additionally, it provides tools to validate INI files against so-called INI schemas, which also recognise said compatibility options in the course of running the validation algorithm.

Origin of the name

Razib Khan wrote about the genetic history of those inhabiting the Italian peninsula. In it he showed that during the height of imperial Rome, an intense cosmopolitanism attracted an incredible lineage of rulers from as far away as Syria around the time of its decline. Remarkably, when Rome depopulated in late antiquity to a mere 30,000 people, the inhabitants were the same genetically as those who had lived in Italy before the Roman Empire, as Etruscans. The rural hinterland proved more lasting in our genetics, as those cosmopolitan people lived and died in Rome leaving no trace of their lineage.

In the time between Hinterlib 2 and the original unilib, Hinterlib was actually one half of a hybrid fork called Hinterlib/Neopolitan. This was created to accommodate the now extraneous “Actually Portable Executable” file format, which provided more guarantees Neopolitan could then presume and pass on downstream that Hinterlib could not. Despite this fork being defunct, Neopolitan bore the name as a nod to the “Actually Portable Executable” format’s reference implementation, “Cosmopolitan”. In contrast to Neopolitan, regular unilib was renamed to Hinterlib.