Senior Consultant - Data Engineer

Job Description

Job Title: Senior ConsultantLocation: TRIL GTC ChennaiGCL: D1Role We are looking for a Data Engineer to help us build intelligent applications that make use of our structured and unstructured data to derive key insights. As part of the R&D Data Foundation engineering group, you will work together with ML engineers and data scientists to build the data foundations supporting R&D.We are building a global Competitive Intelligence platform that will provide industry-leading competitive intelligence across our R&D and Commercial organizations. As a member of our team, you will be primarily responsible for implementing ETL processes.You should be well-versed in the design and development of ETL and database developments for large data products, as well as maintaining and supporting production environments.Key Accountabilities

• Part of a DevOps team implementing and supporting ETL workflows. Data sources will be: structured, semi-structured and unstructured.
• Working with suppliers, data scientists, machine learning engineers, and platform teams to acquire and process data.
• Analyzing data requirements, source data, model the source, and determine the best methods in extracting, transforming and loading the data into the data lake and processing the data through the layers of the lake.
• Providing technical input around design, architecture, integration and support of the entire data sourcing platform with a focus on high availability, performance, scalability and maintainability.
• Act as the ETL technical liaison working with technical infrastructure teams to resolve problems and implement solutions to technical issues impacting application performance
• Managing data administration tasks such as scheduling jobs, troubleshooting job errors, identifying issues with job windows, assisting with backups, rollback and performance tuning.
• Test, document and quality assess new data solutions, to ensure they are fit for release.
• Communicate and coordinate with members of the development team to work across multiple projects. Explore, actively support and work on new technology initiatives that may be of interest to the organization.
• Manage automation of all ETL processes within a job workflow
• Documentation of data engineering workflows to support downstream use
• Testing of data in analytics applications, to ensure data validity and reconciliation to source systems
• Development of subject matter expertise in sub-domains of the Science & Enabling Unit portfolio understanding of the business process, data flows, data provenance, data restrictions and data use.

Highly Desirable Knowledge, Skills and Experience

• 6 years experience on data engineering ETL workflows
• B.Tech/ M.Tech/MSc in Computer Science.
• A strong understanding of databases and source systems, including experience with, RDBMS, NoSQL and Graph technologies.
• Experience writing ETL pipelines/orchestrations including code (Java, Python, C#).
• Good software development skills with demonstrable knowledge of Python and Java, and source control (GIT)
• Working knowledge of cloud environments (AWS preferred)
• Experience of semi-structured (XML, JSON) and unstructured data handling including extraction and ingestion via web-scraping and FTP/SFTP.

Additional skills and experience sought

• In addition, these are the bonus skills (not mandatory) for this position:
• Excellent communication and facilitation skills.
• Good written and verbal skills, fluent English.
• Experience of working with data scientists and their methods: understanding of how data needs to be prepared for use by data scientists.
• Experience of delivering solutions within IT projects delivered through Agile and Waterfall methodologies.
• Experience of working within a range of data architectures.
• Supporting a data-centric application.
• Working with APIs (support or development).

We are an equal opportunity employer and value diversity at our company. We do not discriminate on the basis of race, religion, colour, national origin, gender, sexual orientation, age, marital status, veteran status, or disability status.Date Posted27-Jun-2021Closing DateAstraZeneca embraces diversity and equality of opportunity. We are committed to building an inclusive and diverse team representing all backgrounds, with as wide a range of perspectives as possible, and harnessing industry-leading skills. We believe that the more inclusive we are, the better our work will be. We welcome and consider applications to join our team from all qualified candidates, regardless of their characteristics. We comply with all applicable laws and regulations on non-discrimination in employment (and recruitment), as well as work authorization and employment eligibility verification requirements.,

Keyskills :
sapcustomer relationsit projectsdata solutionsmachine learningbusiness processdata engineeringhigh availabilityunstructured datatechnical liaisoncorporate liaison