This role paved my way into the field of data science. I was the first engineer hired in the data team for the product - Wishbone, Yarn etc. The apps were the top 10 iTunes apps in 2015. Most of the initial work was setting up data imports and streaming data. Since there is tons of data, ETL was the hardest part.
My biggest challenge was how to manage a warehouse with data growing at an exponential rate. This was the hardest part and we did it using a lot of compression techniques and moving the least used data to Redshift spectrum etc.
My biggest learning is how to mentally grow and cope with a high growth startup, with users reaching upto 10 million and wishbone cards hitting close to a billion.
Before starting the data role, I was involved in the backend infrastructure of the mobile ad company called ChirpAds and Playhaven. Playhaven is a popular ad game platform spinned out of UpSight.
In this role, my biggest challenge was the merging of two ad networks into one. During this process, we need to migrate data and make sure ads are running across both platforms.