1,542 Spark jobs in Singapore

Overview

We are seeking a skilled Data Engineer to join our team in Singapore. The ideal candidate will have strong expertise in SQL, Python, Spark, and Hive , with hands-on experience building scalable data pipelines, data models, and analytical solutions. You will work closely with data scientists, analysts, and business stakeholders to deliver high-quality data solutions that support business decision-making.

• Design, develop, and maintain robust, scalable data pipelines and ETL processes.
• Optimize data storage, retrieval, and processing using SQL, Spark, and Hive.
• Collaborate with cross-functional teams to gather requirements and translate them into technical solutions.
• Implement data quality, validation, and monitoring processes to ensure reliability.
• Support data modeling, warehousing, and integration for analytics and reporting.
• Write clean, efficient, and reusable Python code for automation and data processing.
• Ensure data security, compliance, and governance across all solutions.

• Bachelor’s degree in Computer Science, Engineering, or related field.
• 4–7 years of experience as a Data Engineer (or similar role).
• Strong proficiency in SQL, Python, Spark, and Hive .
• Experience with distributed data processing and big data ecosystems.
• Hands-on experience with data pipeline tools (e.g., Airflow, NiFi, or similar) is a plus.
• Familiarity with cloud platforms (AWS, Azure, or GCP) is advantageous.
• Strong problem-solving and communication skills.

Overview
We are looking for a proficient Data Engineer with strong skills in Spark and SQL to join Unison Group. The successful candidate will be responsible for designing, building, and managing data pipelines that facilitate real-time data processing. This role involves optimizing data flow and enhancing data quality to support the analytical needs of various teams within the organization.
Responsibilities
Develop and maintain robust data pipelines using Apache Spark to process large datasets efficiently.
Utilize SQL for data querying, manipulation, and management across various relational database systems.
Collaborate with data scientists and analytics teams to understand data requirements and deliver suitable solutions.
Implement data transformation and aggregation processes to optimize data for analysis.
Monitor and troubleshoot data pipelines, ensuring high availability and reliability.
Perform data quality checks and maintain data integrity throughout the data lifecycle.
Stay updated on industry trends and best practices in data engineering, particularly in Spark and SQL technologies.
Qualifications
Bachelor’s degree in Computer Science, Information Technology, or a related field.
Experience as a Data Engineer, with a focus on Spark and SQL.
Good to have experience in Banking/Finance domain.
Proficiency in Spark for distributed data processing and ability to write efficient Spark jobs.
Strong SQL skills, with experience working with relational databases (e.g., MySQL, PostgreSQL, Oracle).
Experience with data integration, ETL processes, and data warehousing concepts.
Knowledge of big data technologies and frameworks is a plus.
Strong analytical and problem-solving skills.
Excellent communication and teamwork capabilities.
Benefits
Salary: SGD 6000 - 8000
Great Work Culture
Includes Standard Annual and Sick Leave
#J-18808-Ljbffr

Overview
We are seeking a skilled
Data Engineer
to join our team in Singapore. The ideal candidate will have strong expertise in
SQL, Python, Spark, and Hive , with hands-on experience building scalable data pipelines, data models, and analytical solutions. You will work closely with data scientists, analysts, and business stakeholders to deliver high-quality data solutions that support business decision-making.
Key Responsibilities
Design, develop, and maintain robust, scalable data pipelines and ETL processes.
Optimize data storage, retrieval, and processing using SQL, Spark, and Hive.
Collaborate with cross-functional teams to gather requirements and translate them into technical solutions.
Implement data quality, validation, and monitoring processes to ensure reliability.
Support data modeling, warehousing, and integration for analytics and reporting.
Write clean, efficient, and reusable Python code for automation and data processing.
Ensure data security, compliance, and governance across all solutions.
Qualifications
Bachelor’s degree in Computer Science, Engineering, or related field.
4–7 years of experience as a Data Engineer (or similar role).
Strong proficiency in
SQL, Python, Spark, and Hive .
Experience with distributed data processing and big data ecosystems.
Hands-on experience with data pipeline tools (e.g., Airflow, NiFi, or similar) is a plus.
Familiarity with cloud platforms (AWS, Azure, or GCP) is advantageous.
Strong problem-solving and communication skills.
#J-18808-Ljbffr

Key Responsibilities

• Design, develop, and maintain robust data pipelines and ETL processes to support analytics and reporting needs.

• Collaborate with data scientists, analysts, and business stakeholders to understand data requirements and deliver high-quality solutions.

• Implement data integration solutions across structured and unstructured data sources.

• Ensure data quality, integrity, and security across all stages of the data lifecycle.

• Optimize data workflows for performance and scalability in cloud and on-premise environments.

• Support data migration and transformation initiatives for client projects.

• Monitor and troubleshoot data pipeline issues and provide timely resolutions.

Required Qualifications

• Bachelor's degree in Computer Science, Information Systems, Engineering, or related field.

• 3+ years of experience in data engineering or related roles.

• Proficiency in SQL and Python or Scala.

• Experience with data pipeline tools such as Apache Spark, Kafka, Airflow, or similar.

• Familiarity with cloud platforms (AWS, Azure, or GCP).

• Strong understanding of data warehousing concepts and tools (e.g., Snowflake, Redshift, BigQuery).

• Knowledge of data governance, security, and compliance standards.

Key Responsibilities
Design, develop, and maintain robust data pipelines and ETL processes to support analytics and reporting needs.
Collaborate with data scientists, analysts, and business stakeholders to understand data requirements and deliver high-quality solutions.
Implement data integration solutions across structured and unstructured data sources.
Ensure data quality, integrity, and security across all stages of the data lifecycle.
Optimize data workflows for performance and scalability in cloud and on-premise environments.
Support data migration and transformation initiatives for client projects.
Monitor and troubleshoot data pipeline issues and provide timely resolutions.
Required Qualifications
Bachelor’s degree in Computer Science, Information Systems, Engineering, or related field.
3+ years of experience in data engineering or related roles.
Proficiency in SQL and Python or Scala.
Experience with data pipeline tools such as Apache Spark, Kafka, Airflow, or similar.
Familiarity with cloud platforms (AWS, Azure, or GCP).
Strong understanding of data warehousing concepts and tools (e.g., Snowflake, Redshift, BigQuery).
Knowledge of data governance, security, and compliance standards.
#J-18808-Ljbffr

Join us to apply for the Senior Data Engineer (Big Data Processing) role at Centre for Strategic Infocomm Technologies (CSIT) .

CSIT develops products to advance Singapore's national security interests, including counter-terrorism and computer network defence. We seek talented senior data engineers to lead and develop mission-critical products that identify, analyse, and investigate threats.

As a senior data engineer, you will contribute to the development and maintenance of high-performance data pipelines and systems related to national security. Your work will involve handling complex technical projects to collect, parse, and process large datasets, transforming them into valuable insights. Join us to leverage cutting-edge technologies to protect the nation.

• Design and build scalable, robust, and maintainable data pipelines using advanced platforms and technologies.
• Collaborate with product managers, engineering managers, and stakeholders to deliver impactful data products.
• Optimize performance of large-scale data pipelines for low latency, high throughput, and reliability.
• Ensure data integrity, quality, and governance through robust design and operational monitoring.
• Troubleshoot and resolve complex issues in critical data pipelines.
• Implement best practices in data engineering, including testing, orchestration, and monitoring.
• Lead technical discussions to shape the future of data products.
• Mentor junior engineers and foster a strong data engineering culture.

• Bachelor's degree in computer science or related field.
• At least 5 years of experience in data engineering, focusing on large-scale data pipelines.
• Proven ability to write optimized ETL code for large datasets.
• Proficiency in Scala or Java.
• Experience with Spark, Kafka, Hadoop, Elastic stack, Kudu.
• Strong understanding of modern data architectures and modeling paradigms.
• Effective communication of technical details.
• Good interpersonal, analytical, and problem-solving skills in a dynamic environment.

• Purposeful and meaningful work.
• Work with talented, driven professionals.
• Use modern technologies and stacks.
• Excellent engineering culture and work-life balance.
• Aim for engineering and operational excellence.
• Encourage innovation.

Note: As CSIT is under the Ministry of Defence (Singapore), only Singapore Citizens are eligible.

• Seniority level: Mid-Senior level
• Employment type: Full-time
• Job function: Information Technology
• Industries: IT Services and Consulting

Join us to apply for the
Senior Data Engineer (Big Data Processing)
role at
Centre for Strategic Infocomm Technologies (CSIT) .
CSIT develops products to advance Singapore's national security interests, including counter-terrorism and computer network defence. We seek talented senior data engineers to lead and develop mission-critical products that identify, analyse, and investigate threats.
Role
As a senior data engineer, you will contribute to the development and maintenance of high-performance data pipelines and systems related to national security. Your work will involve handling complex technical projects to collect, parse, and process large datasets, transforming them into valuable insights. Join us to leverage cutting-edge technologies to protect the nation.
Responsibilities
Design and build scalable, robust, and maintainable data pipelines using advanced platforms and technologies.
Collaborate with product managers, engineering managers, and stakeholders to deliver impactful data products.
Optimize performance of large-scale data pipelines for low latency, high throughput, and reliability.
Ensure data integrity, quality, and governance through robust design and operational monitoring.
Troubleshoot and resolve complex issues in critical data pipelines.
Implement best practices in data engineering, including testing, orchestration, and monitoring.
Lead technical discussions to shape the future of data products.
Mentor junior engineers and foster a strong data engineering culture.
Requirements
Bachelor's degree in computer science or related field.
At least 5 years of experience in data engineering, focusing on large-scale data pipelines.
Proven ability to write optimized ETL code for large datasets.
Proficiency in Scala or Java.
Experience with Spark, Kafka, Hadoop, Elastic stack, Kudu.
Strong understanding of modern data architectures and modeling paradigms.
Effective communication of technical details.
Good interpersonal, analytical, and problem-solving skills in a dynamic environment.
Why Join Us?
Purposeful and meaningful work.
Work with talented, driven professionals.
Use modern technologies and stacks.
Excellent engineering culture and work-life balance.
Aim for engineering and operational excellence.
Encourage innovation.
Note: As CSIT is under the Ministry of Defence (Singapore), only Singapore Citizens are eligible.
Additional Details
Seniority level: Mid-Senior level
Employment type: Full-time
Job function: Information Technology
Industries: IT Services and Consulting
#J-18808-Ljbffr

Join to apply for the Data Processing Assistant role at Seatrium

Join to apply for the Data Processing Assistant role at Seatrium

• Understands the problem statement and requirement of the current processes from the relevant stakeholders
• Participates in the data collection, cleansing and matching and understand of the current set of data which required to be analysed
• Analyses the data both structured and unstructured set across different categories based on relevant stakeholders
• Develops automation scripts to perform the task efficiently for current systems which is operationalised
• To use AI tools to analyse the integrated data set for insights
• Develops AI/ML based solutions based on various IOT inputs including model trainings and detection for visual analytics
• Monitors the performance and ensures the analytics solutions deployed are executed optimally and support in the troubleshooting and refinement process
  
  

• Understands the problem statement and requirement of the current processes from the relevant stakeholders
• Participates in the data collection, cleansing and matching and understand of the current set of data which required to be analysed
• Analyses the data both structured and unstructured set across different categories based on relevant stakeholders
• Develops automation scripts to perform the task efficiently for current systems which is operationalised
• To use AI tools to analyse the integrated data set for insights
• Develops AI/ML based solutions based on various IOT inputs including model trainings and detection for visual analytics
• Monitors the performance and ensures the analytics solutions deployed are executed optimally and support in the troubleshooting and refinement process
  
  

• Power BI, Power Automate, MS Office Suites, Python, Unity, Data Fabric will be good skills to have
• Good Communication and Interpersonal Skills, Problem-Solving Abilities, Self-Motivated
• The candidate will be assigned mentor to learn these skills during the internship as well
  
  

• Seniority level Internship

• Employment type Full-time

• Job function Other
• Industries Shipbuilding

Referrals increase your chances of interviewing at Seatrium by 2x

Get notified about new Data Assistant jobs in Pioneer, West Region, Singapore .

We’re unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI.