Platforms for Open Data Projects

As part of the National Innovation and Science Agenda, Data61 is working with Australian government agencies on R&D projects that aim to increase the number and availability of high-value datasets between government agencies, and to the public. The projects under the Platforms for Open Data program will:

  • facilitate more use of public data by providing better search and discovery
  • make more high-value datasets available ensuring a high level of privacy protection, and
  • lead the use of public data by performing advanced analytics on data.

These projects use an iterative approach to develop new technologies based on the needs of the partner agency, ensuring that the technologies are applicable across multiple agencies. The end goal is to use these technologies to make more high-value government data available to more people while preserving privacy.

Opportunities to participate in future R&D data projects are managed via the Department of the Prime Minister and Cabinet

Platforms for Open Data Projects application form

Making more open data accessible

Making Australian Government Data Available (MAGDA)

Partner Agency: Digital Transformation Agency

Data61 is expanding the existing data.gov.au and NationalMap infrastructure to maximise the discoverability and reuse of high-value public data in government, industry and the community sectors. In 2017, we launched new software underpinning data.gov.au. The new architecture and interface makes searching easier and faster by moving from a data catalogue approach to a data search engine approach, and giving users the option to limit their search results by publisher, location, date-range and file format.

Making Australian Government Data Available (MAGDA) is a platform built to power a new generation of data portals. MAGDA is a solution for any problem that involves a collection or multiple collections of datasets that need to be searched, discussed and/or viewed in a single place. It doesn't matter what format the data is in, how well-formed the metadata is, where the data is stored or in how many places.

Regulation as a platform (RaaP)

Partner Agency: Department of Industry, Innovation and Science

This proof-of-concept platform aims to demonstrate how an open database of business regulation can operate as a digital platform to support advanced regulatory and compliance tools. The prototype platform will provide a computable version of Australian regulation encoded in a logic format that the software industry can leverage via open APIs. An open platform that can be maintained by government is intended to support an open market ecosystem for the development of business tools and enable the use of analytics and advanced computation to guide reform and improve whole of regulatory system design capability.

Historical Electoral Statistics Archive

Partner Agency: Australian Electoral Commission

Currently, it can be difficult for members of the public, researchers or even government agencies to access statistics relating to previous federal electoral events. Data61 is developing a prototype that will provide access to an historical electoral dataset in the public domain, as the authoritative source. The prototype will enable data linkages, data import and visualisations for some of the most popular queries.

Open Data Tiger Teams (completed in December 2017)

Partner Agency: Department of Prime Minister and Cabinet

Data61 Open Data Tiger Teams will work with government agencies to assist them in making their high-value datasets available from open data platforms such as data.gov.au  (while preserving privacy) and, where applicable, accessible from the NationalMap .

To improve the quality of public datasets available through data.gov.au , the Tiger Teams will help agencies use best practice standards, formats and workflows to produce best-in-class machine readable datasets, and develop methods for accessing, analysing and displaying these datasets. These workflows and artefacts, discoverable through data.gov.au , will share the knowledge openly with others and drive interest in datasets, building a portfolio of exemplar projects and approaches.

Making sensitive data more accessible while preserving privacy

Open Secure API to Public Data

Partner Agency: Australian Bureau of Statistics (and MADIP partner agencies)

Data61 is developing software that allows authorised users to safely query confidential unit-record datasets. Queries are confidentialised on-the-fly, and only aggregate and confidentialised data is returned to the user, which protects against the risk of re-identification of the data. The Protari API is being developed initially to provide secure access to the Multi-Agency Data Integration Project (MADIP) dataset . The software will make this important dataset available to other government agencies and researchers to inform policy and service delivery questions.

The Protari prototype software has been successfully deployed in the ABS test environment and we are currently working with ABS to improve functionality and move the prototype into external evaluation, making it available to more government staff and researchers.

Re-identification ready reckoner tool

Partner Agency: Department of Social Services

Data61 is developing a prototype tool that will help agencies understand the level of risk involved when making certain datasets publicly available. The tool allows a user to analyse and modify datasets, using built-in privacy risk assessment tools and privacy-preserving algorithms. By examining the outcome of the privacy risk metrics, the user will be able to find a reasonable balance between privacy and utility of their dataset, and understand the potential privacy implications of data release or sharing.

Search, Explore, Explain, Relate

Partner Agency: Department of Immigration and Border Protection

Data61 is developing a prototype self-service tool that will allow users to access datasets simply, visually and apply analysis to that data. The tool will decrease the manual intervention currently required to access datasets that include information on trade, travellers and migration domains. The data will be used to inform public policy and planning, from delivery of services such as health and education to city planning.

Designing datasets that preserve privacy

Partner Agency: Australian Public Service Commission

Data61 is developing prototype technology that will make Australian Public Service Commission (APSC) datasets more accessible whilst managing privacy of the data. Due to its richness and longevity, improved access and utility of the datasets will enable users to carry out a range of functions, including strategic workforce planning and evaluation of strategies and programs.

Sensitive Data Access (SenDA)

Partner Agencies: Department of Social Services (with Australian Institute of Health and Welfare) and the Australian Public Services Commission

Data61 is developing software and a user interface for data researchers to access large sets of de-identified government data at the unit record level, and to access longitudinal survey data. Researchers can find it difficult to access sensitive data for analysis and it can be expensive and time-consuming for data custodians to make the data available. The Sensitive Data Access (SenDA) software is being used to enable auditable data extraction and delivery into a secure environment for policy and research purposes by authorised users.

Significantly improving our understanding of Australian firms (completed in June 2017)

Partner Agency: Department of Industry, Innovation and Science

Data61 developed web-based data analytics software for unit record data while preventing access to the data itself. The software aimed to improve the accessibility of the Business Longitudinal Analytical Data Environment (BLADE), and thereby provide data to better understand economic activity in Australia, building up from firm-level data. The project involved research and technology development to enable mediated access to researchers and prevent spontaneous recognition of entities within the data.

Synthetic Social Security Payments Dataset (completed in June 2017)

Partner Agency: Department of Social Services

Data61 developed new algorithms to enable the safe release of a synthetic representation of a dataset covering 15 years of social security and family payments. The synthetic dataset removes the connection between real individuals and their data, alleviating confidentiality risks while retaining the overall characteristics of the original dataset. The synthetic dataset generated can be used as a time-series training dataset, or used by researchers to assess the utility of the dataset before going through an approval process to access the real dataset.

Getting value from spatial data

NationalMap

Partner Agency: Department of the Prime Minister and Cabinet (as part of the "Making Australian Government Data Available" project – see above)

NationalMap is a technology for map-based access to spatial data from Australian government agencies. NationalMap provides easy access to spatial data and provides an open framework of geospatial data services that supports commercial and community innovation.

For more information go to About NationalMap .

Extending the NationalMap platform to leverage Digital Earth Australia (previously called Australian Geoscience Data Cube)

Partner Agency: Geoscience Australia (with Dept. of Environment & Energy, National Computational Infrastructure and CSIRO Land & Water).

The NationalMap platform will be extended so that it can access data from Digital Earth Australia (DEA) to view 30 years of remote sensed imagery from the entire continent. The project will provide an open API to provide easy access to visualise data from the DEA. In July 2017 the new splitter and date picker feature was implemented, allowing users to make temporal comparisons of satellite imagery or analysed data over weeks, months or years.

Contact Data61

Your contact details

First name must be filled in

We'll need to know what you want to contact us about so we can give you an answer.

Contact Data61

Your contact details

First name must be filled in

We'll need to know what you want to contact us about so we can give you an answer.