An IT Consulting || Manpower Staffing || Managed Service Provider || Recruitment Process Outsourcing company.
About the Company
A leading provider of IT services and consulting, focused on delivering innovative, high-performance technology solutions. Specializing in data warehousing and big data environments, this company works with clients across various industries, providing expertise in implementing and optimizing complex systems.
About the Role
The Hadoop Production Support role requires a specialist to manage and support applications within the Hadoop data lake. The position will involve ensuring the reliable operation of production applications through 24/7 monitoring, troubleshooting, and remediation. A strong understanding of Hadoop tools, troubleshooting experience, and the ability to work in a fast-paced environment is essential.
Responsibilities
- Support multiple projects with competing deadlines while ensuring timely issue resolution.
- Monitor production applications 24/7 to ensure service-level agreements (SLAs) are met.
- Triage and resolve job failures promptly to ensure continuous operation.
- Collaborate with upstream and downstream applications to resolve delays and data issues.
- Generate and present status reports to senior management.
- Troubleshoot and resolve issues in Hadoop-related applications, including HDFS, MapReduce, Hive, Impala, Spark, Kafka, and other associated tools.
- Write and maintain Unix shell scripts to automate tasks and debug issues.
- Analyze and enhance existing shell scripts, Python, or Perl code as needed.
- Work with relational databases (SQL) and handle large SQL-based systems.
- Benchmark, debug, and optimize critical algorithms and software performance.
Required Skills
- Experience: 7+ years in data warehousing and at least 4 years in big data (Cloudera).
- Hadoop Ecosystem: Strong expertise in HDFS, MapReduce, Hive, Impala, Spark, and Kafka.
- Scripting: Proficient in Unix shell scripting and debugging.
- SQL: Solid understanding of relational databases and SQL.
- Problem-Solving: Strong analytical skills to debug and optimize code.
- Experience: Working in a production environment, particularly with Big Data.
- Knowledge: Strong understanding of distributed systems and their operations.
Preferred Qualifications
- Experience in the financial services industry.
- Familiarity with production support environments.
- Experience in business intelligence solutions and analytical environments.
- Exposure to industry trends and relevant application technologies.