Aon

Since 2018, Aon has been focused on building its enterprise-level Data & Analytic Services Platform to provide insights that ensure continued success for its clients. This environment has become the home for hundreds of data engineers, data scientists, and data analysts who need early access to data from disparate internal and external systems, driving discovery analytics and ultimately the delivery of new datadriven solutions that reach their clients. A successful strategy for DataOps depended on three factors in particular:

1. Multi-tenancy: Aon needed to enable dozens of Data Engineering teams to build pipelines side-byside with each other, isolated from data through security controls, but sharing common compute clusters for economies of scale. With StreamSets, they were able to assign roles and permissions so each team could achieve this level of isolation, without impeding governed data sharing whenever appropriate. This represented a huge significant benefit in cost savings and operational scale since it eliminated the need to spin up separate environments for each team.
2. Connectors: Aon ingests data from multiple systems, requiring a DataOps platform that supports a wide range of destinations and analytics platforms, including Hadoop, SQL Server, Postgres, Oracle, and Snowflake. StreamSets has hundreds of prebuilt connectors for various origins and destinations that allows Aon the flexibility to build their data pipelines with ease.
3. Democratization of Data Engineering: Aon had a need to quickly onboard and skill-up new members across data engineering teams, helping them shorten the development time of their first pipelines – a need well suited to the low-code ETL tool. StreamSets Control Hub provides Aon’s data engineers with an intuitive design canvas to develop their pipelines, greatly increasing productivity compared to hand-coded solutions. This also helped ensure minimum design standards for new projects.

The StreamSets platform has enabled Aon to strengthen its DataOps practice to gain better insights from its market-leading proprietary data. By helping data engineers build pipelines to supply strategic client data of varying types and from various sources, StreamSets is helping Aon build its view of the client, including relevant data such as industry risk attributes, insurance programs, internal revenue, commercial claims, health benefits, investment portfolios, to name a few.

Furthermore, as Aon ingests data coming from various disparate sources, StreamSets pipelines help to clean and augment the data in flight. Not only has this helped the organization centralize the data, but they are also able to run, better connect, and expose it for analytics through Snowflake. StreamSets is a key component in establishing a mature development lifecycle and support posture within data engineering as they build, monitor, and manage their data pipelines via StreamSets Control Hub and adjacent monitoring tools provided within Aon.

Before adopting the StreamSets platform, lead time to acquire data from various sources for projects could take months, as the teams that needed that data would have to go through a project intake process, buy and install ETL tools, spin up SQL databases, and then begin their work. With StreamSets, not only has Aon’s time for data ingestion significantly reduced, but the organization now also has a team onboarding process that takes weeks or less. Aon has been able to remove the need to provision dedicated infrastructure to teams, which enables data engineers to land data quicker. This benefit is passed downstream as the time from ideation to first exploration of data by data scientists and data analysts is dramatically improved.