Earthbound

Earthbound
Created by				Alexander Nicholi
Written in				POSIX sh
Licence				ASL 1.1

Earthbound is a network- and system-agnostic software procurement system, designed to obtain and consume software packages in various formats and manage them in concert together at once.

Background and purpose

Earthbound is part of the developmental quartet alongside Inbound, Outbound and Rebound. It arose out of the need to acquire and integrate upstream software sources in a way that presumes as little as practical about the file structure or network availability of the source. Initially, this meant upstream cryptography libraries for Hinterlib which cannot be brought in-tree since doing so would undermine the confidence of the routines' security.

Design

Earthbound only concerns itself with what we will call files of interest. These are files, often source code, that may come in from anywhere over the network, that were selected by the downstream developer to contain in good faith the data they need. Earthbound keeps SHA-3 hashes of these files of interest and uses them to verify the files' authenticity.

Beyond acquisition, Earthbound has a know-nothing approach, giving developers total discretion on what to do with these files after they have been downloaded and verified. This is true to its purpose of software procurement, which exalts the Unix philosophy of doing one thing and doing it well.

Configuration file

Earthbound uses a simple INI file structure to define a collection of files of interest along with their hashes and details on where they may be found. Observe:

# is used for comments, not ;
# must be at the start of a logical line
 # leading space is OK though
[file1.c]
sha2sum=70d181c0bebbac369cddb65e6304e55a8efdcef50589154faf30390fdc6a1427
# at least one is required, but multiple may be given
# if so, all hashes are checked
sha3sum=4e9e5320e30e44ff31463806d5578fbfdc9212f32d2e9dd41998d0a8aaf830c9
# all of these URLs are candidates and may be tried in any order
source=https://www.example.com/file1.c
source=https://www.example.de/file.c
source=ftp://ftp.example.ch/srv/file_1.c
# sourcing files in archives uses the # symbol like DOM IDs
source=https://cdn.example.com/package.tar.gz#/subdir/file.c

For Earthbound's INI files, the following schema rules apply:

1. all key–value pairs must appear under a section heading (there is no "global section")
2. only ASCII is permitted, except within comments, where the high bit may be set to passively allow UTF-8
3. all URIs must be valid UTF-8 and legalised into ASCII via percent-encoding
4. comments begin with the hash symbol #, not the semicolon ;
5. file names are specified as the contents of the section name in whole
6. file names cannot constitute paths; no directory component is permitted

Given an INI like the one above, Earthbound offers several benefits:

1. it will select the source URL using arbitrary algorithms which take advantage of the lack of ordering or precedence to choose the best download site
2. when multiple source files may download from the same URL sans the # suffix, the package only needs to be downloaded once