About the Role
Diggibyte is looking for an experienced Lead / Senior Data Engineer to design, build, and optimize modern data platforms that power business intelligence, analytics, and AI-driven solutions. The ideal candidate will have strong expertise in cloud-based data engineering, scalable ETL/ELT pipelines, and the Databricks Lakehouse Platform. If you are passionate about solving complex data challenges and delivering high-performance solutions, we’d love to hear from you.
Key Responsibilities
- Lead the design, development, and maintenance of scalable data engineering solutions on the Databricks Lakehouse Platform.
- Develop, optimize, and maintain high-performance data pipelines using Python and PySpark.
- Build reliable data transformation workflows and analytical models using DBT.
- Design scalable ETL/ELT pipelines to process structured and unstructured data efficiently.
- Work extensively with Databricks, PySpark, and SQL to deliver robust data solutions.
- Leverage cloud platforms including AWS, Microsoft Azure, or Google Cloud Platform (GCP) for modern data architectures.
- Implement data governance, metadata management, and security using Unity Catalog.
- Collaborate with data scientists, analysts, architects, and business teams to understand data requirements.
- Ensure data quality, performance optimization, and adherence to best engineering practices.
- Mentor junior engineers and contribute to technical leadership within the team.
Required Skills & Technologies
- Databricks Lakehouse Platform
- Python
- PySpark
- SQL
- DBT (Data Build Tool)
- AWS / Microsoft Azure / Google Cloud Platform
- Unity Catalog
- Data Warehousing & ETL/ELT
- Data Modeling
- Version Control (Git)
Preferred Qualifications
- Bachelor’s or Master’s degree in Computer Science, Information Technology, Data Engineering, or a related field.
- 4–12 years of professional experience in Data Engineering.
- Strong understanding of cloud-native data platforms and distributed data processing.
- Experience with performance tuning and optimization of large-scale data pipelines.
- Excellent analytical, problem-solving, and communication skills.
Good to Have
- Databricks Certified Data Engineer Associate or Professional certification.
- Experience with Databricks Auto Loader.
- Knowledge of Delta Live Tables (DLT) or Lakeflow Declarative Pipelines.
- Exposure to CI/CD practices for data engineering projects.
- Familiarity with data governance and security best practices.
Why Join Diggibyte?
- Work on cutting-edge cloud and data engineering projects.
- Collaborate with experienced technology professionals.
- Opportunity to work with modern data platforms and AI-ready architectures.
- Continuous learning and professional development.
- Competitive salary and excellent career growth opportunities.
- Flexible and collaborative work environment