Role and Responsibilities
- Design, develop and maintain optimal data pipeline architecture to support data sets.
- Accumulate large, complex structured and unstructured data sets that meet business requirements.
- Build, prepare requirement documents.
- Design, develop and implement process improvements, automating manual processes, optimizing real time data delivery.
- Design and build the infrastructure required for optimal data extraction, transformation, and loading of data from a wide variety of data sources using ‘big data’ technologies.
- Build analytics models and tools that utilize the data pipeline to provide actionable insights on operational efficiency and other key business performance metrics.
- Work with stakeholders including the Executive, Product, Data and Software Engineering teams internally and also with the customers to assist with data-related technical issues and support their data requirements.
- Build complex automated data pipelines for creating datasets, facilitate efficient data management.
- Should have discipline in writing clean and tested codes.
Requirements
- Good working SQL knowledge and experience working with relational databases, as well as working familiarity with a variety of databases and data sets/feeds.
- Strong analytical skills of manipulating, processing, and extracting value from large structured, unstructured datasets/feeds.
- Strong working experience with big data technologies and tools such as Hadoop, data lake, Spark, etc..
- 2-3 years of experience in a data engineering / data warehousing role.
- Experience in Dbt and a SaaS company highly regarded
- Experience in dealing with subscription based data sets and systems, CRM, product data in event-driven architecture
- Experience with databricks and AWS regarded but not essential
- Great English and communication will be helpful
- Anyone with growth mindset and who wants a challenge to work in a startup. We get chaotic and a bit unstructured sometimes