Category: Python
Dear Candidate,
Before submitting your resume, please pay attention to the location – we will not be able to review your resume and provide feedback if you are not (in fact) located in the location of the vacancy.
Thanks for understanding
This position requires analyzing tasks, understanding their logic and connections to other systems, and transforming them into Airflow Directed Acyclic Graphs (DAGs) on GCP, while documenting the process. Some familiarity with systems, networking, and infrastructure is preferred due to our hybrid setup of on-premise and cloud data systems
Responsibilities:
- Create & document efficient data pipelines (ETL/ELT).
- Write and optimize complex queries on large data sets.
- Transform data and map them to more valuable and understandable sets for consumption.
- Work with other teams (Big Data, Marketing, Customer Support, etc.) to gather requirements enable them to access datasets, and empower them to understand and contribute to our data processes
- Design and implement data retrieval, storage, and transformation systems at scale.
- Understanding and implementing data lineage, data governance, and data quality practices.
- Create tooling to help with day to day tasks.
- Troubleshoot issues related to data accuracy and maintaining an accurate source of truth.
- Introduce new technologies to the environment through research and POCs.
In terms of legacy tech stack:
- python scripts
- shell scripting
- SQL scripts and stored procedures
- understanding of systems / infrastructure deployment.
- Hadoop / hive / mapr / SQL Server / mysql
Newer tech stack:
- python
- airflow
- GCP
- kubernetes / working with docker containers
- messaging systems / RabbitMQ
And comfortable with :
- using terminal / ssh / git
- s/w development workflows
- CI/CD terminologies, tools
- infrastructure as code
Should understand concepts like:
- Handling of PII data
- data governance
- data lake, data warehouses, batch/stream processing etc.
Requirements:
- Advanced experience ( ~5 yr ) with Python and SQL.
- Experience with RabbitMQ and Kubernetes
- Experience working in hybrid environments( GCP / Azure / AWS / on-prem ) and, specifically with storage, database, and distributed processing services.
- Experience working with REST APIs and Python data tools (Pandas/SQLAlchemy)
- Experience building ETL and big data pipelines (Airflow/Dagster/Prefect)
- Ability to read and understand existing code/scripts/logic
- Experience with an IaC tool like Terraform.
- Comfortable using Terminal / ssh / Powershell / Bash / Code versioning, branching / git
- Experience managing a project backlog and working cross-functionally with multiple stakeholders.
- Ability to work effectively on a self-organizing team with minimal supervision.
- Initiative in communicating with co-workers, asking questions and learning.
- Excellent oral and written communication skills.
Bonus points for:
- Experience with dbt.
- Experience with CICD tools (Jenkins/Github/Gitlab).
- Experience with stream-processing systems (Storm/Flume//Kafka).
- Experience with schema design and dimensional data modeling.
Apply
Dear Candidate,
Before submitting your resume, please pay attention to the location – we will not be able to review your resume and provide feedback if you are not (in fact) located in the location of the vacancy.
Thanks for understanding
This position requires work hours alignment with UK/Lisbon time zone
About the project – №1 Travel platform in the world!
We believe that we are better together, and we welcome you for who you are. Our workplace is for everyone, as is our people powered platform. At company, we want you to bring your unique perspective and experiences, so we can collectively revolutionize travel and together find the good out there.
Product is the world’s largest travel site, operates at scale with over 500 million reviews, opinions, photos, and videos reaching over 390 million unique visitors each month. We are a data driven company that leverages our data to empower our decisions. Product is extremely excited to play a pivotal role in supporting our travelers.
Our data engineering team is focused on delivering Product’s first-in-class data products that serve all data users across the organization. As a member of the Data Platform Enterprise Services Team, you will collaborate with engineering and business stakeholders to build, optimize, maintain, and secure the full data vertical including tracking instrumentation, information architecture, ETL pipelines, and tooling that provide key analytics insights for business-critical decisions at the highest levels of Product, Finance, Sales, CRM, Marketing, Data Science, and more. All in a dynamic environment of continuously modernizing tech stack including highly scalable architecture, cloud-based infrastructure, and real-time responsiveness.
Product provides a unique, global work environment that captures the speed, innovation and excitement of a startup, at a thriving, growing and well-established industry brand.
We take pride in our data engineering and are looking for a talented and highly-motivated engineer with a passion for solving interesting problems to add to our high-performing team.
What you will do:
- Provide the organization’s data consumers high quality data sets by data curation, consolidation, and manipulation from a wide variety of large scale (terabyte and growing) sources
- Build first-class data products and ETL processes that interact with terabytes of data on leading platforms such as Snowflake and BigQuery
- Partner with our Analytics, Product, CRM, and Marketing teams
- Be responsible for the data pipelines’ SLA and dependency management
- Write technical documentation for data solutions, and present at design reviews
- Solve data pipeline failure events and implement anomaly detection
- Work with various teams from Data Science, Product owners, to Marketing and software engineers on data solutions and solving technical challenges
- Mentor junior members of the team
Who You Are:
- Bachelor’s degree in Computer Science or related field
- 6+ years experience in commercial data engineering or software development
- Experience with Big Data technologies such as Snowflake, Databricks, PySpark
- Expert level skills in writing and optimizing complex SQL; advanced data exploration skills with proven record of querying and analyzing large datasets
- Solid experience developing complex ETL processes from concept to implementation to deployment and operations, including SLA definition, performance measurements and monitoring
- Hands-on knowledge of the modern AWS Data Ecosystem, including AWS S3
- Experience with relational databases such as Postgres, and with programming languages such as Python and/or Java
- Knowledge of cloud data warehouse concepts
- Experience in building and operating data pipelines and products in compliance with the data mesh philosophy would be beneficial. Demonstrated efficiency in treating data, including data lineage, data quality, data observability and data discoverability
- Excellent verbal and written communication skills. Ability to convey key insights from complex analyses in summarized business terms to non technical stakeholders and also ability to effectively communicate with other technical teams
- Strong interpersonal skills and the ability to work in a fast-paced and dynamic environment
- Ability to make progress on projects independently and enthusiasm for solving difficult problems
Why join us:
- Ability to work remotely from anywhere in the world
- Close cooperation with the development team and client
- Opportunity to influence product development
- Professional growth: the certification preparation course is free for our specialists. The company pays for two attempts to pass the exam, regardless of the exam result
- We cover English classes (with a native speaker)
- Boost professional brand: you can participate in local conferences as a listener or as a speaker
- Regular team buildings: have fun with teammates
- Gifts for significant life events (marriage, childbirth)
- Tech and non-tech Zazmic Communities: support and share experience with each other
Apply
Dear Candidate,
Before submitting your resume, please pay attention to the location – we will not be able to review your resume and provide feedback if you are not (in fact) located in the location of the vacancy.
Thanks for understanding
About the project – №1 Travel platform in the world!
We believe that we are better together, and at company we welcome you for who you are. Our workplace is for everyone, as is our people powered platform. At company, we want you to bring your unique perspective and experiences, so we can collectively revolutionize travel and together find the good out there.
Our team is building out a next generation Machine Learning Platform for all data scientists and machine learning engineers across all of Company’s brands. Our mission is to empower data scientists to work independently and scale their productivity to enable broader and deeper utilization of machine learning techniques to help improve the business performance. Company hosts over 400 million monthly active visitors and operates across multiple cloud environments. Our data is at the petabyte scale, requiring a scalable, efficient, and reliable machine learning platform to support it.
We are seeking a talented, experienced, Senior Software Engineer to pave the way towards revolutionizing how we leverage data and Machine Learning!
We leverage Kubernetes and a variety of open source software (e.g. Kubeflow, Seldon, Istio, Mlflow) to train and deploy over one hundred models serving a billion requests per day.
What you will do:
- Develop across our evolving technology stack - we’re using Python, Java, Kubernetes, Apache Spark, Postgres, Redis, ArgoCD, Argo Workflow, Seldon, MLFlow and more. We are migrating into AWS cloud and adopting many services that are available in that environment.
- You will have the opportunity to learn many cutting edge technologies around Machine Learning Platform. You will push the boundaries, to test, develop and implement new ideas, technology and opportunities, and be well rewarded and recognized for doing so
- Take responsibility for all aspects of software engineering, from design to implementation, QA, maintenance, and support
- Touch code at every level – from the UI, backend microservices, database, big data processing, operations, to CD/CI automation
- Collaborate closely with data science teams to define requirements, discuss solutions, and develop high quality deliverables for our customers
Who You Are:
- Computer Science degree or equivalent experience
- At least 5 years’ experience of commercial software development.
- Willingness and ability to take on new projects and technologies
- Ability to break down complex problems into simple solutions
- Strong analytical skills and desire to write clean, correct and efficient code
- Sense of ownership, urgency and pride in your work.
- Experience with Python
- Experience with Java, Docker, Kubernetes, Argo, Spark and AWS cloud services are plus.
- Exposure to Machine Learning practices is a plus
Why join us:
- Ability to work remotely from anywhere in the world
- Close cooperation with the development team and client
- Opportunity to influence product development
- Professional growth: the certification preparation course is free for our specialists. The company pays for two attempts to pass the exam, regardless of the exam result
- We cover English classes (with a native speaker)
- Boost professional brand: you can participate in local conferences as a listener or as a speaker
- Regular team buildings: have fun with teammates
- Gifts for significant life events (marriage, childbirth)
- Tech and non-tech Zazmic Communities: support and share experience with each other
