Publish to HDX proposal
- David Megginson
Rationale
We are planning a "Publish to HDX" functionality from KoBo and HOT-OSM, and will be discussing with other partners as well (e.g. DHIS2). OSM will likely use the CKAN API, but for others, it would make sense to design a reusable framework where we can easily support publish buttons from any number of additional partners with little or no incremental effort on our part.
Proposal
The minimum viable product for a "Publish to HDX" link/button would be the ability to link to HDX's contribute form at https://data.humdata.org/contribute/new with parameters to prefill the form. For example, a button or HTML a element linking to
https://data.humdata.org/contribute/new?url=http%3A%2F%2Fexample.org%2Fapi%2F12345.csv
would open the HDX contribute form with http://example.org/api/12345.csv already filled in as the first resource URL. A more-complicated example could include many more parameters, such as the location, the org id, and the dataset title:
https://data.humdata.org/contribute/new?url=http%3A%2F%2Fexample.org%2Fapi%2F12345.csv&title=Sample%20dataset&org=unicef&location=SYR
We should accept submissions via GET or POST requests (with a redirect after the POST request, so that the user can safely refresh the form).
Client-side usage
On a Bootstrap3 site, the HTML markup for a publish button might look like this:
<a class="btn btn-success"
href="https://data.humdata.org/contribute/new?url=http%3A%2F%2Fexample.org%2Fapi%2F12345.csv">Publish to HDX</a>
or alternately, as a POST form:
<form method="post" action="https://data.humdata.org/contribute/new">
<input type="hidden" name="url" value="http://example.org/api/12345.csv"/>
<input type="hidden" name="title" value="Sample dataset"/>
<input type="hidden" name="org" value="foo_org"/>
<input type="hidden" name="location" value="SYR"/>
<button type="submit" class="btn btn-success">Publish to HDX</button>
We can publish samples that clients can easily copy and paste into their HTML templates. Since a contributor still needs to log in, any organisation could include this in their site, without even needing to contact us first.
Suggested HTTP parameters
The only required parameter will be url for the URL of the first resource, but a sophisticated client implementation could prefill many more fields.
| Parameter | Required? | Description |
|---|---|---|
| url | Yes | URL of first resource |
| name | No | Download filename for first resource |
| mimetype | No | Type of first resource |
| description | No | Long description of first resource |
| private | No | Flag for whether the dataset should be private (default: true) |
| title | No | Title for dataset |
| slug | No | Proposed slug for dataset (if the slug already exists, and the user has authorisation, open the form to update an existing dataset instead). |
| notes | No | Long description of dataset |
| subnational | No | Flag for subnational data (default: false) |
| dataset_source | No | Description of original source |
| org | No | HDX org id |
| maintainer | No | HDX user id for maintainer |
| date | No | Single date for dataset (excludes start_date and end_date) |
| start_date | No | Start date for dataset (excludes date) |
| end_date | No | End date for dataset (excludes date) |
| location | No | HDX location id (e.g. "SYR") |
| license | No | HDX license id |
| methodology | No | HDX methodology id |
| update_frequency | No | HDX update-frequency id |
| caveats | No | Text with caveats |
| tags | No | HDX tags, comma-separated |
Possible future features
- Allow a CKAN authorisation token to log the user in automatically.
- Allow real-time validation feedback on the dataset.
- Add options for creating instant previews.