The X Data Engineering Strategy

The five components mentioned in the previous section are the driving factors that enable the effectiveness of data engineering within an agency along 2 axes, Organisation and Data State. Each component is arranged on the X Data Engineering Strategy and overlaps between both the Organisation and Data State axes, as illustrated in the figure below. To further improve on the data engineering initiative, agencies first need to focus on the components at the base of each axis, while taking into consideration any overlaps in the axes.

For instance, a team may want to initially consider the “People and Resources” component at the base of the Organisation axes, and the “Tools and Technologies” component at the base of the Data State axes. The team will also need to take into consideration the overlapping “Infrastructure and Architecture” component when making decisions at the base of each axis. Once these three components at the base of each axis have been taken into consideration, teams can focus on the components higher up in the Organisation/Data State axes such as “Sponsors and Use Cases”, and “Processes and Data”.

Throughout the rest of the playbook, we have organised the content around the components of a data engineering strategy referred. Each major section begins with an introduction of the component that the section is based upon. This sets the context for the importance of said component, before introducing readers to various subtopics and resources that we intend to expand on over time. Readers can then use these resources as reference for their various data engineering initiatives.