Spreadsheets2IATI

Combine and convert spreadsheets and IATI files into a single set of files for IATI activities and IATI organisations.

Setup notes

Workspace

Create a directory for the workspace if it does not exist.

Initialise the workspace, for instance with aida-s2i init

Create or copy a build.xml to have local build capabilities: for instance to get files from a folder synchronised with Google Drive.

Spreadsheets2IATI

The conversion process is initiated by a local call of ant in the workspace.

$ ant s2i-run

The process breaks down in a couple of components.

Figure 1. Sequence for s2i-run

The local build.xml in the workspace includes a library build file from the workbench. This makes it possible to extend or override the process in a particular workspace.

Get inputs

This step copies input files for the conversion

Gather input files

This step takes files from the input folder, and prepares them for processing, in a flat folder.

Excel files are converted into CSV.
Excel and CSV are then converted into an XML format.
Akvo and IATI XML files are copied.

Generate IATI partials

This step transforms prepared input files into "partial IATI" files. These intermediary files are not valid IATI yet, but contain the IATI representation for the particular input file.

Create IATI output

This step combines all "partial IATI files" into one IATI activities and one IATI organisations file. These files can contain activities that are not IATI schema-compliant.

With a paid Saxon license, it is possible to validate the file and then split it at the activity level. This will create one valid IATI file, and one file with activities that contain validation errors.

Postprocess IATI file

This step in the Dataworkbench applies known fixes to an IATI file. This can be anonymisation (replacing an organisation name), or known issues (like an identifier NL-KvK-… with lowercase "v").