The Humanitarian Data Exchange (HDX) is adding a new mandatory metadata field called Expected Update Frequency. It replaces a previous optional field, Update Frequency, and its purpose is to tell us how often datasets shared through the site are likely to be updated.
...
We are introducing a new set of features to HDX based on the concept of "Data Freshness". We are interested in assessing how current is the data within each dataset because we want the portal to become more useful for users of the data and : one important metric we can give them is how up to date are the datasets. Now that we have over 4,000 datasets on HDX, we want to Imagine a large walk in freezer in a restaurant. Delivery staff fill the freezer with new items akin to how contributors add new datasets to HDX. Cooks take products from the freezer and mix them in various tasty ways. Analogously, users find datasets in HDX and combine the data for analysis. Foodstuffs can be safely stored in the freezer for different periods of time. If noone checks, the caterers may use stale ingredients, so there needs to be a method to keep track of the items in the freezer and if any are beyond their safe storage date, to order replacements. Given the choice, chefs would like to use the freshest produce, and similarly we want users have access to the most up to date data in HDX, particularly since our "freezer" contains over 4,000 items! We want to help data providers oversee their data, particularly where update processes are manual, and make it easy for people to find data that is being actively maintained by contributors. For data providers, we will be able help ensure data remains current, particularly where update processes are manual.
How do we determine Data Freshness?
...
We are drawing on research being done on data freshness at Vienna University. Specifically, the researchers are looking at estimating the next change time for a resource based on previous update history and applying a Markov chain approach. The research is still ongoing but we hope to learn from their results to enhance HDX.
...
Let us know what you think of this approach. Send feedback to hdx@un.org. Watch this space! There will be more coming on the subject of Data Freshness.