As part of the National Innovation and Science Agenda, Data61 is working with Australian government agencies on R&D projects that aim to increase the number and availability of high-value datasets between government agencies, and to the public. The projects under the Platforms for Open Data program will:

  • facilitate more use of public data by providing better search and discovery
  • make more high-value datasets available ensuring a high level of privacy protection, and
  • lead the use of public data by performing advanced analytics on data.

These projects use an iterative approach to develop new technologies based on the needs of the partner agency, ensuring that the technologies are applicable across multiple agencies. The end goal is to use these technologies to make more high-value government data available to more people while preserving privacy.

Opportunities to participate in future R&D data projects are managed via the Department of the Prime Minister and Cabinet

Platforms for Open Data Projects application form [docx · 1mb]

Making more open data accessible

Agency MAGDA (Making Australian Government Data Available)

Partner Agencies: Digital Transformation Agency, Department of the Environment and Energy & Department of Agriculture and Water Resources

The Agency MAGDA project aims to provide government agencies with a platform to make data discoverable, available, re-usable and open. MAGDA will allow agencies to easily search for and maintain their own data, while making it safer and easier to publish.

The project will leverage the work already done by Data61, PM&C and the DTA in delivering the MAGDA PfOD project for data.gov.au. It will extend the capabilities of that platform, including developing additional functionality to allow access controls to be applied to metadata records and data, support more data sources and enable the platform to be used for publishing data in line with data management processes.

Longitudinal spine of government functions

Partner Agency: Department of Finance and the National Archives of Australia

Responsibility for government functions, such as specific policy areas, program delivery or regulatory authority, can move between government entities with Machinery of Government (MoG) changes. While information on each change is in the public domain, it is not structured in a way that enables a whole-of-government or time series view. This project will test whether a schema can be devised to better track changes in the association between entities and functions over time. It will take into account classification schemes currently  used in financial and non-financial reporting.

Regulation as Data

Partner Agency: Department of Human Services

The delivery and management of each payment and service is defined and guided by thousands of business rules that are embedded in a complex system. The current processes are manual, time intensive and unable to be tracked. This project will investigate how to accelerate the rule creation process to save time, or develop a traceability framework to demonstrate the relationship between any given legislation and rule.

Jobs and Education Data Integration Discovery Project

Partner Agency: Department of Jobs

Australia's labour market is changing rapidly due to factors such as advances in technology, demographic shifts and changes in the business environment. Job seekers, existing workers, business owners, education providers and policy makers need information and support to understand and adapt with this changing job and skill landscape, but it is difficult to identify and access these insights. The "Jobs and Education Data Integration" platform would be a medium to access government and commercial data on employment, skills and education options in Australia, and fed into other applications. This discovery project aims to scope the technical requirements associated with developing such a platform.

Making sensitive data more accessible while preserving privacy

Open Secure API to Public Data

Partner Agency: Australian Bureau of Statistics (and MADIP partner agencies)

Over the last two years, Data61 has been collaborating with the Australian Bureau of Statistics (ABS) on the development of the prototype 'Protari' (Protecting Against Re-Identification). Protari is an Application Programming Interface (API) that enables users to generate summary confidentialised statistics from unit record data 'on-the-fly'. The Protari API is being developed initially to provide secure access to the Multi-Agency Data Integration Project (MADIP) dataset . The software will make this important dataset available to other government agencies and researchers to inform policy and service delivery questions.

In the 2018-19 project Data61 and ABS will research the potential to enhance Protari's capabilities including enabling analysis over time using longitudinal datasets.

On Demand Linkage

Partner Agency: Department of Social Services

Government agencies need data from multiple sources to gain insight to improve social and economic outcomes and support citizens and organisations. Current data integration and data sharing efforts are manual, time and resource intensive and often result in partially integrated data products. This project will research and develop record linking mechanisms that focus on automation, privacy and accuracy of record linkage, with a view to develop a prototype "just-in-time" linkage tool.

Synthetic Australian Tax File

Partner Agency: Australian Taxation Office

Freely and publicly available data is valuable to many people, including administrators, planners, policy makers and researchers. This project aims to investigate and improve on the utility of privacy preserved datasets and to develop techniques suitable to longitudinal datasets, and for datasets with an ongoing data release schedule.

Improving availability of high-value datasets

Partner Agency: Australian Prudential Regulation Authority

APRA collects data from financial sector organisations via the Financial Sector (Collection of Data) Act on a monthly, quarterly and annual basis. APRA aims to reduce its reliance on manual processes for masking sensitive data prior to publication. This project aims to improve and automate (where feasible) the confidentialisation and risk management processes to improve the availability and quality of high value datasets.

Search, Explore, Explain, Relate

Partner Agency: Department of Immigration and Border Protection

Data61 is developing a prototype self-service tool that will allow users to access datasets simply, visually and apply analysis to that data. The tool will decrease the manual intervention currently required to access datasets that include information on trade, travellers and migration domains. The data will be used to inform public policy and planning, from delivery of services such as health and education to city planning.

Getting value from spatial data

Terria: Mapping as a Service

Partner Agencies: Digital Transformation Agency and Geoscience Australia

The project will enhance Australia’s NationalMap and Data61's open-source TerriaJS by building the foundations for turning the framework into "Software as a Service". Our vision is that governments, NGOs and industry will be able to create their own NationalMap, providing a user-friendly interface to their own geospatial data, without the need for software developers or geospatial experts.

Asbestos heat map and scorecard

Partner Agency: Asbestos Safety and Eradication Agency

It is estimated that one third of Australian homes contain asbestos. This project will investigate the potential of sourcing and using sparse data sets to develop a national probabilistic forecast of remaining asbestos in the built environment. The datasets will be used to produce a national 'heat-map' of remaining asbestos risks to inform government.

Completed projects

Making Australian Government Data Available (MAGDA) – project completed in June 2018

Partner Agency: Digital Transformation Agency

Data61 has expanded the existing data.gov.au and NationalMap infrastructure to maximise the discoverability and reuse of high-value public data in government, industry and the community sectors. The new architecture and interface makes searching easier and faster by moving from a data catalogue approach to a data search engine approach, and giving users the option to limit their search results by publisher, location, date-range and file format.

Making Australian Government Data Available (MAGDA) is a platform built to power a new generation of data portals. MAGDA is a solution for any problem that involves a collection or multiple collections of datasets that need to be searched, discussed and/or viewed in a single place. It doesn't matter what format the data is in, how well-formed the metadata is, where the data is stored or in how many places.

Australia's data portal

Further work continues in the Agency-MAGDA project.

Designing datasets that preserve privacy - – project completed in June 2018

Partner Agency: Australian Public Service Commission

Data61 developed prototype software to provide access to APSC datasets for decision making whilst maintaining privacy. Due to its richness and longevity, improved access and utility of the datasets will enable users to carry out a range of functions, including strategic workforce planning and evaluation of strategies and programs. The prototype developed (using Protari technology) includes multi-table output, auto-calculation of results, login capability for tracking queries and the ability to download files.

Re-identification ready reckoner tool – project completed in June 2018

Partner Agency: Department of Social Services

Data61 developed an automated tool to help agencies assess the level of risk in providing access to certain dataset variables. The tool allows a user to analyse and modify datasets, using built-in privacy risk assessment tools and privacy-preserving algorithms. By examining the outcome of the privacy risk metrics, the user can find a reasonable balance between privacy and utility of their dataset, and understand the potential privacy implications of data release or sharing.

Extending the NationalMap platform to leverage Digital Earth Australia (previously called Australian Geoscience Data Cube) – project completed in June 2018

Partner Agency: Geoscience Australia (with Dept. of Environment & Energy, National Computational Infrastructure and CSIRO Land & Water).

The prototype software developed in this project provides access to data from Digital Earth Australia making analysis and visualisation more readily available for the public, government and industry. Development of new capabilities including a splitter and date picker, provide improved user interaction with time series data.

Regulation as a platform (RaaP) – project completed in June 2018

Partner Agency: Department of Industry, Innovation and Science

This proof-of-concept platform demonstrates how an open database of business regulation can operate as a digital platform to support advanced regulatory and compliance tools. The prototype platform provides a computable version of Australian regulation encoded in a logic format that the software industry can leverage via open APIs.

More information at Regulation as a Platform

Your Data, Your Regions - project completed in June 2018

Partner Agency: Australian Bureau of Statistics

Data61 developed an accessible, open service that automatically transforms data from one geographic region to another. The new algorithms developed in the project are able to learn a predictive model tailored to each dataset, resulting in more accurate predictions. The new prototype web service also reports confidence intervals over a multitude of possible solutions, enabling the data users to quantify the uncertainty of their outputs.

Historical Electoral Statistics Archive – project completed in June 2018

Partner Agency: Australian Electoral Commission

Data61 created a relational database to provide the AEC with improved capacity to reuse data and publish historical election data from 1903-1972.

Sensitive Data Access (SenDA) - – project completed in June 2017

Partner Agencies: Department of Social Services (with Australian Institute of Health and Welfare) and the Australian Public Services Commission

Data61 developed software and a user interface for data researchers to access large sets of de-identified government data at the unit record level, and to access longitudinal survey data. The Sensitive Data Access (SenDA) software is being used to enable auditable data extraction and delivery into a secure environment for policy and research purposes by authorised users.

Significantly improving our understanding of Australian firms – project completed in June 2017

Partner Agency: Department of Industry, Innovation and Science

Data61 developed web-based data analytics software for unit record data while preventing access to the data itself. The software aimed to improve the accessibility of the Business Longitudinal Analytical Data Environment (BLADE), enabling mediated access to researchers and prevent spontaneous recognition of entities within the data.

Synthetic Social Security Payments Dataset – project completed in June 2017

Partner Agency: Department of Social Services

Data61 developed new algorithms to enable the safe release of a synthetic representation of a dataset covering 15 years of social security and family payments. The synthetic dataset removes the connection between real individuals and their data, alleviating confidentiality risks while retaining the overall characteristics of the original dataset. The synthetic dataset generated can be used as a time-series training dataset, or used by researchers to assess the utility of the dataset before going through an approval process to access the real dataset.

Open Data Tiger Teams – project completed in December 2017

Partner Agency: Department of Prime Minister and Cabinet

Data61 Open Data Tiger Teams worked with government agencies to assist them in making their high-value datasets available. Tiger Teams helped three agencies use best practice standards, formats and workflows to produce best-in-class machine readable datasets, and develop methods for accessing, analysing and displaying these datasets.

Contact Data61

Your contact details

First name must be filled in

We'll need to know what you want to contact us about so we can give you an answer.

Contact Data61

Your contact details

First name must be filled in

We'll need to know what you want to contact us about so we can give you an answer.