Looking at this I have not written a blog post on here in over two years! I think it is time for a change. First of all to let you guys know what I have been up too and then too let you in what I will blogging about in the future:
What I have been up too
The past two years have been very interesting the company that I work for has seen a data explosion and unfortunately this meant that the DB2 estate that is needed to support this is quite expensive in terms of licensing and hardware costs compared to other offerings that are out there. Also we were struggling to find people with the requisite skills that supported our architecture of DB2 with an in house built ETL tools. It was rather unique solution in an area of the country were London sucks a lot of the people with DB2 skills.
Therefore we started looking at other platforms to house our data warehouse and power the businesses data future. We started looking at SQL server and all its offerings. This seemed like a winner for a time we started and building out the platform with an home built auto runner and other interesting features that I may blog about later. This solution was far cheaper in terms of licensing than DB2 (sorry IBM) and also came with a lot of features such as SSRS and SSIS and also a more readily available talent pool in the local area. This architecture still had one major problem hardware, we fill it up and stress it out far quicker than our normal 5 year planning caters for.
Then Amazon brought out Redshift. This is a winner it has features that will support our businesses data growth and has an eco system of products round it that will support the businesses data hunger. Although it does not come with an ETL tool like SSIS; in an effort to standardise opposed to our current scripted batch system we found Talend. AS a reporting solution found an awesome product called Looker. We are also looking at other Big Data solutions such as Hadoop, EMR, Databricks and many other things data related.
What will I be blogging about the future
So I now intend to get back to my blogging about my solutions to technical issues we face and solve, cool features we use and you can you to around Amazon Redshift, Looker, Talend and maybe occasionally DB2 as we wind down our usage. I will also be blogging more about the conferences I have attended, big data solutions we try out. So watch out for more content and hopefully interesting articles and findings.