Starting to Piece Azure Data Factory Together
After a couple of weeks of playing around in Azure, largely focusing on Azure Data Factory (ADF), Data Lakes, Synapse and other similar offerings from Microsoft, I feel that it is all starting to fall into place and make sense. Admittedly, the vague level of information provided did not help on areas to focus on but in the end, general curiousity has started to prevail.
As part of my future work building various pipelines within ADF, I have started to make notes on similarities and differences between existing tools that is used at work and Microsoft’s offering as I am aware that in the future I will more then likely be writing internal documentation (oh the joy!). Whilst doing this deep dive, I have come to love how linear and clean the different activities are laid out within the UI, but behind the clean UI and fairly simple design, is quite a powerhouse of ELTL tooling. This power is increased greatly once you realise how easy it is to integrate different services like Function and Logic apps.
Over the next few months, I am going to start to piece together and post here about solutions to problems that I come across whilst building Pipelines, as well as any useful tools and apps that I build along the way as it will be a long but fruitful journey.