skillindiajobs
Register for Job
Hyderabad Jobs
Banglore Jobs
Chennai Jobs
Delhi Jobs
Ahmedabad Jobs
Mumbai Jobs
Pune Jobs
Vijayawada Jobs
Gurgaon Jobs
Noida Jobs
Oil & Gas Jobs
Banking Jobs
Construction Jobs
Top Management Jobs
IT - Software Jobs
Medical Healthcare Jobs
Purchase / Logistics Jobs
Sales
Ajax Jobs
Designing Jobs
ASP .NET Jobs
Java Jobs
MySQL Jobs
Sap hr Jobs
Software Testing Jobs
Html Jobs
IT Jobs
Logistics Jobs
Customer Service Jobs
Airport Jobs
Banking Jobs
Driver Jobs
Part Time Jobs
Civil Engineering Jobs
Accountant Jobs
Safety Officer Jobs
Nursing Jobs
Civil Engineering Jobs
Hospitality Jobs
Part Time Jobs
Security Jobs
Finance Jobs
Marketing Jobs
Shipping Jobs
Real Estate Jobs
Telecom Jobs

Web Scraping Engineer

3.00 to 8.00 Years   Pune   05 Oct, 2021
VITASTA CONSULTING PVT LTD
Job LocationPune
EducationNot Mentioned
SalaryNot Disclosed
IndustryIT - Software
Functional AreaApplication Programming / Maintenance
EmploymentTypeFull-time

Job Description

Roles & Responsibilities

  • As a Web Crawler, your role is to apply your knowledge set to fetch data from multiple online sources
  • Optimize the scraping capability to ensure the data is scrapped efficiently with minimum usage of server bandwidth. Scrape difficult websites by deploying anti-blocking and anti captcha tools.
  • Develop highly reliable web crawlers and parsers across various websites
  • Extract structured/unstructured data and store them into SQL/No SQL data store Work closely with Project/Business/Research teams to provide scrapped data for analysis
  • Develop frameworks for automating and maintaining constant flow of data from multiple sources.
  • Develop a deep understanding of the data sources on the web and know exactly how, when, and which data to scrap, parse and store this data
  • Active participation in troubleshooting and debugging.
  • Guide and mentor other data engineers.
  • Develop a Data Ingestion framework for automating and maintaining constant flow of data from multiple sources to the database.
  • Perform code reviews and suggest design changes.
  • Comply with coding standards and technical design.
  • Increase process efficiency by identifying repeatable jobs and automating them using appropriate tools and techniques
  • Creating efficient web crawlers. Create more/better ways to crawl relevant information
  • Familiarity with best practices and design patterns of programming languages
Eligibility
  • Bachelor/ Masters degree in computer science / Computer Engineering / Information Technology with minimum 5-year experience, of which 3 years has to be hands-on experience in crawling/scraping using frameworks such as Scrapy, Beautiful Soup, Selenium, APIs. Experience in Python of 3+ years is a must.
  • Experience in a fast paced start-up company or team lead experience at a start-up during the rapid team and business scaling phase.
  • Strong knowledge of scraping frameworks such as Scrapy, Beautiful Soup, HTQL, Jsoup, WebHarvest, URLlib and Selenium
  • Good to have Experience of complex crawling (like captcha, Mobile OTP based crawling, bypassing proxy)
  • Experience in various data extraction methods (like data extraction from PDF Files, web pages, etc.)
  • Good understanding of HTML DOM, CSS, Javascript, XPATH and RESTful web service
  • Familiarity with AWS, cloud-based technologies is a plus
  • Working knowledge in various SQL/NoSQL DBs , message queues & Web RestFul APIs
  • Familiarity with some ORM (Object Relational Mapper) libraries
  • Prior experience with team handling and people management.
  • Experience in Linux based OS, (ubuntu would be a plus)
Good to have Experience in cloud servers usage, in open source projects, lite web frameworks like flask, fastapi

Keyskills :
web scrapingdata extractionseleniumscrapy

APPLY NOW

Web Scraping Engineer Related Jobs

© 2020 Skillindia All Rights Reserved