Home

Gopi m - Sr. Data Engineer

[email protected]

Location: Maryland Line, Maryland, USA

Relocation: Yes

Visa: Green Card

Resume file: Gopi_Sr.Data_Engineer_1748352990438.docx Please check the file(s) for viruses. Files are checked manually and then made available for download.

Senior Data Engineer with Around 11 years of experience in designing, developing, and managing cloud-based data warehousing, big data solutions, ETL/ELT pipelines, and business intelligence applications across various industries. Expertise in AWS (S3, Redshift, DynamoDB, EMR, Lambda, Kinesis, RDS, ECS, CloudFormation) and Azure (Data Factory, Synapse Analytics, Data Lake, Cosmos DB, Databricks) for scalable and high-performance data solutions. Hands-on experience in Hadoop Ecosystem, Cloudera, Apache Spark (PySpark/Scala), Hive, Kafka, and HDFS, optimizing large-scale data processing. Strong database management skills across SQL (PostgreSQL, MySQL, MSSQL Server, Oracle) and NoSQL (Cassandra, MongoDB, DynamoDB) with expertise in query optimization, indexing, and data modeling. Experienced in designing and automating ETL & Data Pipelines using Apache Airflow, DBT, Informatica PowerCenter, Talend, and DataStage. Led data platform modernization initiatives using Snowflake, Databricks, and DBT, enhancing data processing, analytics, and reporting for enterprise-scale solutions. Deep knowledge of data modeling & warehousing, including fact/dimension modeling, star and snowflake schemas, and data lake architectures. Skilled in Python (Pandas, NumPy, SciPy, SQLAlchemy), Shell Scripting, Spark RDD, DataFrame API, and Spark Streaming for data processing and workflow automation. Proficient in DevOps & CI/CD using Terraform, Ansible, Docker, Kubernetes, and Git-based pipelines for seamless deployment and infrastructure automation. Strong understanding of data governance & security, including IAM roles, encryption, GDPR/CCPA compliance, and cloud security best practices. Expertise in business intelligence & data visualization with Tableau, Power BI, QlikView, and SAS for creating impactful reports and dashboards. Optimized SQL queries and Spark jobs for complex data transformations in Parquet and Delta formats, leading to improved job execution time and resource utilization. Skilled in software development & API engineering, developing REST APIs with Django & Flask for data integration and transformation. Experienced in implementing master data management (MDM) solutions using Semarchy xDM, ensuring data consistency, integrity, and governance across enterprise systems. Experience working in Agile & SDLC methodologies, leading data-driven projects from design to deployment. Hands-on expertise in AI-driven data solutions, integrating AI-powered analytics into data pipelines for predictive insights and automation. Strong problem-solving skills in performance optimization & scalability, including Spark tuning, query optimization, and efficient data processing. Passionate about architecting robust data solutions, streamlining workflows, and leveraging AI to drive business impact. Keywords: continuous integration continuous deployment artificial intelligence business intelligence sthree database