Role Responsibilities
- Design, develop and maintain optimal data pipeline architecture to support data sets.
- Accumulate large, complex structured and unstructured data sets that meet business requirements.
- Build, prepare requirement documents.
- Design, develop and implement process improvements, automating manual processes, optimizing real time data delivery.
- Design and build the infrastructure required for optimal data extraction, transformation, and loading of data from a wide variety of data sources using ‘big data’ technologies.
- Build analytics models and tools that utilize the data pipeline to provide actionable insights on operational efficiency and other key business performance metrics.
- Work with stakeholders including the Executive, Product, Data and Software Engineering teams internally and also with the customers to assist with data-related technical issues and support their data requirements.
- Build complex automated data pipelines for creating datasets, facilitate efficient data management.
- Should have discipline in writing clean and tested codes.
Requirements
- Good working SQL knowledge and experience working with relational databases, as well as working familiarity with a variety of databases and data sets/feeds.
- Experience building and optimizing data pipelines & architectures using data warehouse /data lake technologies.
- Extensive exposure working with data modeling technologies.
- 5+ years of experience as a Data Engineer.
- Strong analytical skills of manipulating, processing, and extracting value from large structured, unstructured datasets/feeds.
- Excellent verbal and written communication skills for effective communication in a multicultural environment with teams spread throughout the world.
- Strong working experience with big data technologies and tools such as Hadoop, data lake, Spark, etc..
- Strong working experience in languages such as Java, Python, R, etc…