GFT Polska
Senior Data Consultant (Hadoop, Spark)Senior Data Consultant (Hadoop, Spark)
Kraków, PL, 30-302
Type of contract: B2B contract
Salary range: 125-163 PLN net/h
What will you do?
You will work as a key member of a technical team alongside Engineers, Data Analysts and Business Analysts, contributing to a collaborative Agile development process while designing, developing and maintaining scalable data solutions in a dynamic DevOps environment.
Your tasks
- Define and contribute to software design and development using Pyspark
- Automate testing of new and existing components
- Promote development standards through code reviews and mentoring
- Provide production support and troubleshooting
- Implement tools and processes ensuring performance, scalability and monitoring
- Collaborate with Business Analysts to interpret and implement requirements
- Participate in planning, sprint reviews and retrospectives
- Contribute to system architecture and design
Requirements
- Experience with Pyspark or Scala development and design
- Experience using scheduling tools such as Airflow
- Knowledge of Hadoop ecosystem including Spark, Hive, YARN and ETL frameworks
- Strong SQL and RESTful services knowledge
- Experience working on Unix or Linux platforms
- Hands-on experience building data pipelines using Hadoop components
- Experience with Git, GitHub, Jenkins, Ansible and JIRA
- Understanding of big data modelling using relational and non-relational techniques
- Experience debugging code and communicating findings to development teams
- Openness to work 2 days a week from our client’s office (Kraków)
Nice to have
- Experience with Elasticsearch
- Experience developing Java APIs
- Experience in data ingestion processes
- Understanding of cloud design patterns
- Exposure to DevOps and Agile methodologies such as Scrum and Kanban
- Experience with Spark streaming
- Experience with Apache Airflow in production
- Experience with Hadoop ecosystem in enterprise environments
- Knowledge of Python backend services
- Experience with Scala for high performance systems
- Experience in data integration and ETL processes
- Knowledge of PL/SQL
- Experience with Linux and Unix system operations
#J-18808-Ljbffr