Data Engineer

Job Description

Data EngineerThis role has been designated as Edge , which means you will primarily work outside of an HPE office.This role is part of HP Enterprises Global IT across Americas, Europe, and Asia Pacific, and aligns with the Data and Analytics group, is a rapidly growing team and works in a fast-paced environment. Data and analytical solutions include multi PB data lake, batch and real-time data ingestion, data science, and visualization using open source software, custom home-grown systems, and 3rd party software. The team brings data management and engineering capabilities of systems analysis, data integration, and data governance, and hands-on skills for large scale data solutions, and makes data available for the use in applications and analytics.At HPE, we bring together the brightest minds to create breakthrough technology solutions and advance the way people live and work. Our legacy inspires us as we forge ahead dedicated to helping our customers make their mark on the world. We re solving the world s most complex challenges, and our people are at the forefront of progress.A professional in this position, works with the business users, technologists, analyses systems, and gathers how data is used in them. Determines methods and procedures to analyze source data, ingest the data into downstream environment, improve data quality, and make the data usable for consumption. Researches, designs, develops, configures, integrates, tests and maintains existing and new data and analytical solutions including databases through integration of technical and business requirements. Provides required documentation and participates in architecture reviews to ensure that the solutions comply with standards and use approved technologies. Typical customers are HP Enterprises end users and various functional areas such as Supply Chain, Research and Development, Marketing Finance.Responsibilities:

• Design, architect, and engineer big data solutions.
• Develop a modern data analytics lake.
• Propose and standardize ELT, BI, governance, and analytics.
• Publish enterprise ingress, development, transform, egress, consumption guidelines.
• Instrument monitoring, self-healing, and self-describing data pipelines to meet tier 1 OLAs/OpX: i.e., own operational outcomes.
• Implement continuous integration, continuous deployment, devops practice.
• Create, document, and manage data guidelines, governance, and lineage metrics.
• Technically lead, Design and develop distributed, high-throughput, low-latency, highly available
• Build monitoring tools for server-side components; work cohesively in globally distributed team
• Develop unit test cases to test your code and automate
• Understand business requirement to build reliable data infrastructure using big data technologies
• Work with the data engineering team to develop & maintain data pipelines for batch & stream processing.
• Build tools suitable to provide data 24x7 for a global team, thru acquiring, monitoring and root cause analysis of data issues
• Identify, design, and implement internal process improvements and tools to automate data processing and ensure data integrity while meeting data security standards
• Build tools for better discovery and consumption of data for various consumption models in the organization DataMarts, Warehouses, APIs, Adhoc Data explorations
• Architect and create data views from big data store to feed into analysis engines, visualization engines etc.
• Work with data scientist and business analytics team to assist in data ingestion and data-related technical issues

Education and Experience Required:

• Bachelor s degree in IT, Computer Science, Software Engineering, Business Analytics or equivalent
• 10 to 15 years of relevant experience in data warehousing / distributed system (e.g. Hadoop)

Knowledge and Skills:

• Experience with relational SQL and NoSQL DB
• Expert in building and optimizing big data data pipelines, architectures, and data sets
• Experience in data ingestion, cleaning and processing tools.
• Excellent experience in data processing using Scala/Python
• Proficient in Big data tools and ecosystem (e.g. HIVE, HBase, Kafka, Spark, .)
• Knowledge & practices of SDLC process and agile methodologies
• Strong in user requirements gathering, maintenance, and support
• Highly organized, self-motivated, pro-active, and able to plan
• Experience managing users and vendors is a plus

#GlobalITINJob:Information TechnologyJob Level:ExpertHewlett Packard Enterprise is EEO F/M/Protected Veteran/ Individual with Disabilities.HPE will comply with all applicable laws related to the use of arrest and conviction records, including the San Francisco Fair Chance Ordinance and similar laws and will consider for employment qualified applicants with criminal histories.,

Keyskills :
test casesdata warehousingsupply chainjavaopen source softwarepythonsqlbig datainformaticaroot cause analysisroot causeopen sourceresearch development