Customers can order a number of unique recipes, each week. Internally, a conversion from each recipe to a location on the production line is made. As this conversion differs each week, the Production team manually creates this source file. One week, one out of the thirty five available recipes was not defined in the file.

In a similar incident, a missing data point in the source file defining which ingredients belong to which recipe led to over 400.000 euros in damages and compensation.

In response to that incident, we created a tool which reads in, validates, and writes to our backend any relevant source files. Stakeholders can simply point the tool to the relevant source file, define any validations that need to be performed, and specify the location to which the validated data is written. Accordingly, no manual engineering work is required to add a new data source.

Due to our new pipeline, we caught the manual data error early. An automated warning was sent to the Production team, enabling them to update the source file to include the missing recipe. Accordingly, there was no customer impact.

Since then, our tool has grown to validating and pushing over 26 business-critical source files.

‍