About the Company
The company offers expertise in data science, digital transformation, business intelligence, app development, cloud services, and IT application support. A leader in providing innovative solutions, the firm serves clients in various industries, driving advancements in technology and business processes.
About the Role
A leading firm in the field of data science and digital transformation is seeking an experienced Big Data Developer with expertise in Python-based processing, Spark, and Hadoop. This role requires a hands-on developer with a deep understanding of Big Data technologies such as Hadoop, Hive, and Kafka. The ideal candidate will work within a collaborative environment, focusing on developing and optimizing data solutions for complex distributed systems.
Key Responsibilities
-
Develop and implement data processing solutions using Python, Spark, and Hadoop.
-
Apply functional programming constructs, utilizing Python and Scala to design and optimize Big Data solutions.
-
Perform data profiling using Pandas and Jupyter Notebooks.
-
Leverage Hadoop ecosystem components such as Hive, HBase, Impala, and Kafka to support business needs.
-
Troubleshoot and optimize performance for distributed systems, focusing on memory optimization.
-
Collaborate with cross-functional teams to design and deliver scalable data processing solutions.
-
Support CI/CD automation using Jenkins/Ansible to streamline deployments.
-
Work with Cloudera or Hortonworks technology stacks and contribute to improving system efficiency.
Required Skills
-
6-8 years of hands-on experience with Big Data technologies, including Hadoop, Hive, Spark, HBase, Kafka, NiFi, etc.
-
Proficient in Python, with knowledge of the “pythonic” ecosystem (Pandas, Jupyter Notebooks).
-
Strong experience in functional programming and data processing frameworks.
-
Solid SQL skills and expertise in data profiling using tools like Pandas.
-
Deep understanding of distributed systems, with extensive experience designing and implementing complex solutions.
-
Knowledge of CI/CD tools, such as Jenkins and Ansible, for automation.
-
Familiarity with Capital Markets and Regulatory Reporting business domains is a plus.
-
Excellent problem-solving, debugging, and communication skills.