Skip to main content

Job Description


Data Engineer

11-03-2024 23:33:16

5 - 10 years

  • Chennai, Tamil Nadu, India (CHN)

Roles and responsibilities:

  1. Data Integration: Designing, implementing, and maintaining data integration processes. Ensuring smooth data flow between different systems and databases. Managing ETL (Extract, Transform, Load) and ELT (Extract, Load and Transform) processes.
  2. Data Quality Management: Implementing and maintaining data quality standards. Identifying and resolving data quality issues. Collaborating with data stewards to improve data quality.
  3. Data Warehousing: Maintaining data warehouses. Optimizing data storage and retrieval processes. Managing and monitoring data warehouse performance.
  4. Data Governance: Establishing and enforcing data governance policies. Collaborating with stakeholders to define data ownership and access policies. Ensuring compliance with data privacy regulations.
  5. Data Security: Implementing and monitoring data security measures. Managing access controls and permissions. Conducting regular security audits and assessments.
  6. Data Monitoring and Troubleshooting: Monitoring data pipelines for errors and anomalies. Investigating and resolving data-related issues. Implementing proactive measures to prevent data problems.
  7. Performance Optimization: Tuning and optimizing database and data processing performance. Identifying and resolving bottlenecks in data pipelines. Implementing best practices for data performance.
  8. Collaboration: Collaborating with cross-functional teams business stakeholders. Participating in data-related projects and initiatives. Providing support to end-users.
  9. Documentation: Documenting data processes, workflows, and configurations. Creating and maintaining documentation for data-related policies and procedures.

Skills Required:

  • Expertise in Data store architectures like Modern data warehouse (MDW) and Lakehouse architecture.
  • Must be familiar with extract, load, transform (ELT) and extract, transform, load (ETL) tools.
  • Should have solid knowledge of data processing languages, including SQL and Python
  • Should be proficient in using the following, to create data processing solutions: SSIS, Azure Data Factory (ADF), Azure Synapse Analytics, Azure Event Hubs, Azure Data Lake Storage, Azure Databricks.
  • Strong analytical and critical-thinking skills.
  • Domain knowledge related to the industry they are working in.
  • Effective communication skills to convey complex data insights to non-technical stakeholders.