| Data Architect Azure Databricks, Azure Data Lakehouse, and Medallion Architecture || 100% interview || hybrid in kentucky at Remote, Remote, USA |
| Email: [email protected] |
|
https://jobs.nvoids.com/job_details.jsp?id=1952798&uid= From: hanshika, vyze [email protected] Reply to: [email protected] Job Description - Data Architect Azure Databricks, Azure Data Lakehouse, and Medallion Architecture 6 months Louisville, KY Onsite - Initially the candidate needs to go on site 3-4 weeks. They can do remote working after. Video/Phone Job Summary We are seeking an experienced Data Architect with expertise in Azure Databricks, Azure Data Lakehouse, and Medallion Architecture. The ideal candidate will have strong implementation experience, including the integration of Delta Lake tables, and a background in working with Customer Data Platforms (CDP). This role involves architecting scalable, high-performing data platforms to support advanced analytics, machine learning, and business intelligence solutions. Key Responsibilities 1. Design and Implement Data Architecture o Lead the design and development of scalable, cloud-based data architectures using Azure Data Lakehouse and Medallion Architecture principles. 2. Azure Databricks and Delta Lake o Architect and implement data pipelines and ETL processes with Azure Databricks. o Ensure seamless integration with Delta Lake for ACID transactions, time travel, and optimized data storage. 3. Medallion Architecture Implementation o Build Bronze, Silver, and Gold data layers to ensure efficient data processing, aggregation, and enrichment for analytics and reporting. 4. Customer Data Platform (CDP) Integration o Design and manage data architecture to integrate CDPs with Azure data ecosystems. o Enable personalized and customer-centric analytics. 5. Data Lakehouse Management o Implement and optimize Azure Data Lake Storage (ADLS) for structured, semi-structured, and unstructured data. o Ensure efficient querying and data retrieval. 6. Data Governance and Security o Define and implement data governance and security practices using tools like Azure Data Catalog, Azure Purview, and Azure Key Vault. o Ensure compliance with data privacy regulations (GDPR, CCPA). 7. Performance Optimization o Optimize data pipelines for performance, scalability, and cost-efficiency. o Ensure SLAs are met for downstream analytics and reporting systems. 8. Collaboration o Work closely with data engineers, business analysts, and data scientists to define data requirements. 9. Real-Time Data Integration o Architect real-time data ingestion pipelines with APIs, event hubs, and other sources. o Streamline real-time analytics using Azure Databricks and Delta Lake. 10. Documentation and Best Practices o Establish data architecture best practices. o Provide detailed documentation for data pipelines and architectures. o Ensure knowledge sharing across the organization. ________________________________________ Required Qualifications 5+ years of experience in data architecture and engineering. Expertise in Azure Databricks, Azure Data Lakehouse, and Delta Lake. Proven hands-on experience with Medallion Architecture, including Bronze, Silver, and Gold layers. Strong experience with Azure Data Lake Storage (ADLS) and Delta Lake integration. Knowledge of Customer Data Platforms (CDP) and their integration with cloud data architectures. Proficiency in data governance, security, and compliance using Azure Purview, Data Catalog, and Key Vault. Experience in ETL/ELT development and real-time data ingestion pipelines. Proficiency in SQL, Python, and Spark for data processing and pipeline development. Familiarity with data modeling techniques, big data technologies, and cloud data warehousing. Excellent communication and collaboration skills. ________________________________________ Preferred Qualifications Experience with machine learning and advanced analytics using Azure Databricks. Familiarity with API integration and real-time data streaming solutions (e.g., Azure Event Hubs, Azure Stream Analytics). Knowledge of DevOps for data, including automated CI/CD pipelines. ________________________________________ Education Bachelors or Masters degree in Computer Science, Information Systems, or a related field. Keywords: continuous integration continuous deployment golang Kentucky Data Architect Azure Databricks, Azure Data Lakehouse, and Medallion Architecture || 100% interview || hybrid in kentucky [email protected] https://jobs.nvoids.com/job_details.jsp?id=1952798&uid= |
| [email protected] View All |
| 03:24 AM 21-Nov-24 |