| Hadoop Architect Loc : Remote at Remote, Remote, USA |
| Email: [email protected] |
|
http://bit.ly/4ey8w48 https://jobs.nvoids.com/job_details.jsp?id=930971&uid= From: bhumika, Adifice Technologies [email protected] Reply to: [email protected] Title : Hadoop Architect Location : Remote MOI : Skype Visa : USC and GC Only Key Responsibilities: 1. Migration Planning and Execution: Lead the migration of one of the more complex clusters, collaborating with the Team to develop repeatable procedures for internal team adoption. Coordinate the transition from MapR to PowerStore's Hadoop emulation capabilities, ensuring smooth data migration and minimal downtime. Address infrastructure requirements, including worker nodes, edge nodes, and storage configurations, focusing on the Warehouse and UPC clusters. 2. Infrastructure Optimization: Optimize the existing Hadoop infrastructure, currently comprising 250-300 servers, with a plan for scalability. Collaborate with the Team to implement PowerScale-based solutions, supporting multiple storage protocols, including HDFS and NFS for other POSIX clients. 3. Integration of Services: Ensure the integration of services that act on migrated data, focusing on Hive and Sparc. Collaborate with internal development teams for any MapReduce code migration/refactoring. Lead the integration efforts for Identity and Access, aligning with OpenLDAP in a joint effort. 4. Cluster Setup and Configuration: Finalize the configuration of racked hosts and storage, paying attention to worker nodes, edge nodes, and storage specifics. 5. Data Migration: Oversee the data migration process using DistCP, preserving current permissions during transfers. Develop creative solutions for potential challenges in data migration, such as local transfers before transportation to the Chicago Data Center. Work with the customer to gather more details about bandwidth limitations during transfers. 6. Timeline and Collaboration: Ensure the migration is targeted for completion by the end of January or February, aligning with customer expectations. Collaborate with onshore or offshore workers as needed, adhering to U.S. hours and coordinating with external support from Exceldata for Apache Hadoop. Qualifications: Proven experience as a Hadoop Architect with a focus on large-scale data migration projects. In-depth knowledge of Apache Hadoop, Apache Ambari, and related technologies. Experience with PowerScale, PowerStore, and UCS servers. Proficiency in integrating Hadoop services, particularly Hive and Sparc. Strong communication skills and the ability to collaborate with internal and external teams. Familiarity with DistCP and creative problem-solving for complex data migration challenges. Understanding of Identity and Access integration, particularly with OpenLDAP. Ability to work within tight timelines and adhere to project schedules. Keywords: green card http://bit.ly/4ey8w48 https://jobs.nvoids.com/job_details.jsp?id=930971&uid= |
| [email protected] View All |
| 09:43 PM 12-Dec-23 |