Job Location: Pune
Responsibilities :
1. Develop the capability to efficiently scrape data from the web from multiple sources.
2. Scrape difficult websites by deploying anti-blocking and anti-captcha tools.
3. Develop frameworks for automating and maintaining the constant flow of data from multiple sources.
4. Optimize the scraping capability to ensure the data is scrapped efficiently with minimum usage of server bandwidth.
5. Gather and process raw data at scale (including writing scripts, web scraping, calling/create APIs, etc.) from the web/internet.
6. Automate software development processes, including build, deploy, and test.
7. Leverage cloud computing resources to optimally execute back-end processing.
8. Strong data analysis skills working with data quality, data consolidation and data wrangling projects.
Experience :
1. Experience in leading and mentoring a small team.
2. Minimum 5-year experience, of which 3 years have to be hands-on experience in crawling/scraping using frameworks such as Scrapy, Beautiful Soup, Selenium, APIs.
3. Experience of complex crawling like captcha and bypassing proxy, etc
4. Good troubleshooting and debugging skills.
5. Strong fundamental C.S. skills (Data structures, algorithms, multi-threading, etc.).
Qualification & Skills :
1. Bachelor’s or master’s degree in Computer Science or a related discipline
2. Selenium, Beautiful Soup, Java, Python, HTML/CSS, Scrapy, Jsoup, SQL, Azure/AWS Server Management, GitHub Management, Linux, Django Flask Framework
Submit CV To All Data Science Job Consultants Across India For Free
