Infrastructure as Code

Our Data Engineering team strongly advocates the use of Infrastructure as Code (IaC) for cloud infrastructure deployment. IaC is the provisioning and managing of infrastructure through code (using tools such as TerraForm, Pulumi, and AWS CloudFormation) instead of manual processes. With IaC, it is substantially easier to deploy and maintain cloud infrastructure across multiple projects with similar use cases.

Such an approach makes it easier to plan a strategy of taking the most useful components from previous projects to slowly build up a repository of IaC templates which can be reused, accelerating future project development. Since the typical cloud infrastructure supporting data projects generally uses the same few components as illustrated below, it lends itself well to standardisation via IaC.

Data Infrastructure in a Box (DIAB)

The Data Engineering team has created a product called Data Infrastructure in a Box (DIAB) that provides standardised IaC templates for agencies in the Singapore government to deploy and accelerate the setup of data infrastructure in the Government Commercial Cloud (GCC).

You can find out more about DIAB here

Below are examples of cloud infrastructure spun up using our standardised IaC templates:

Data Acquisition

Data Transformation

Data Access