Spreadsheets2IATI

Combine and convert spreadsheets and IATI files into a single set of files for IATI activities and IATI organisations.

Setup notes

Workspace

Create a directory for the workspace if it does not exist.

Initialise the workspace, for instance with aida-s2i init

Create or copy a build.xml to have local build capabilities: for instance to get files from a folder synchronised with Google Drive.

Spreadsheets2IATI

The conversion process is initiated by a local call of ant in the workspace.

$ ant s2i-run

The process breaks down in a couple of components.

Sequence for `s2i-run`
Figure 1. Sequence for s2i-run

The local build.xml in the workspace includes a library build file from the workbench. This makes it possible to extend or override the process in a particular workspace.

Get inputs

This step copies input files for the conversion

Diagram
Diagram
Diagram

Gather input files

This step takes files from the input folder, and prepares them for processing, in a flat folder.

  • Excel files are converted into CSV.

  • Excel and CSV are then converted into an XML format.

  • Akvo and IATI XML files are copied.

Diagram

Generate IATI partials

This step transforms prepared input files into "partial IATI" files. These intermediary files are not valid IATI yet, but contain the IATI representation for the particular input file.

Diagram

Create IATI output

This step combines all "partial IATI files" into one IATI activities and one IATI organisations file. These files can contain activities that are not IATI schema-compliant.

With a paid Saxon license, it is possible to validate the file and then split it at the activity level. This will create one valid IATI file, and one file with activities that contain validation errors.

Diagram

Postprocess IATI file

This step in the Dataworkbench applies known fixes to an IATI file. This can be anonymisation (replacing an organisation name), or known issues (like an identifier NL-KvK-…​ with lowercase "v").

Diagram

S2I publish

This step commits the produced IATI file to version control, and publishes it online on the dataworkbench.io website.

Diagram

Ant targets in the IATI Workbench

image$ant spreadsheet iati
Figure 2. The dependencies of Ant targets involved in spreadsheet-iati

IATI Summary

Creates spreadsheets with summary information based on XML files in the output folder.

To create those XML files in the output folder, we need to run a validation and then filter activities.

Twitter LinkedIn Github Gitlab Meetup RSS Miro Trello Zotero Hypothesis