Lead Data Engineer for Big Data Infrastructure

20 hours ago


Reston, Virginia, United States Starcom consulting limited Full time
About the Role

We are seeking an experienced Lead Data Engineer to join our team at Starcom Consulting Limited. As a key member of our data engineering team, you will be responsible for designing, building, and maintaining large-scale data infrastructure on cloud and on-premise platforms.

Job Description

The successful candidate will have a strong background in big data technologies, including Hadoop, Spark, and NoSQL databases. They will be responsible for architecting and implementing data pipelines, ETL processes, and data warehousing solutions that meet the needs of our business stakeholders.

Responsibilities
  • Design and develop data pipelines and ETL processes using Ab Initio and other tools
  • Develop and maintain large-scale data warehouses and data lakes
  • Collaborate with cross-functional teams to gather requirements and deliver high-quality solutions
  • Ensure data quality, integrity, and security throughout the data lifecycle
  • Stay up-to-date with emerging trends and technologies in big data and analytics
Requirements
  • Bachelor's degree in Computer Science, Information Technology, or a related field
  • Total of 10+ years of IT experience predominantly in Data Integration/ Data Warehouse area
  • At least 5 years of ETL Design and Development experience using Ab Initio
  • 1-2 years of Data Integration project experience on Hadoop Platform, preferably Cloudera
  • AbInitio CDC (Change Data Capture) experience in a Data Integration/ETL project setting is great to have
  • Working knowledge of HDFS, Hive, Impala, and other related Hadoop technologies
  • Sound understanding of SQL and ability to write well-performing SQL queries
  • Good knowledge of OLTP and OLAP data models and other data warehouse fundamentals
  • Rigor in high code quality, automated testing, and other engineering best practices
  • Ability to unit test the code thoroughly and to troubleshoot issues in production environments
  • Must have some working experience with Unix/Linux shell scripting
  • Must be able to work independently and support other junior developers as needed
What We Offer
  • A competitive salary of $120,000 - $180,000 per year
  • Opportunities for professional growth and development
  • A dynamic and collaborative work environment


  • Reston, Virginia, United States Fannie Mae Full time

    Fannie Mae is a leading player in the housing finance industry, and we are currently seeking an experienced Data Infrastructure Expert to join our team. In this role, you will have the opportunity to work on developing data infrastructure and pipelines to capture, integrate, organize, and centralize data, ensuring it is readily accessible and usable.Job...


  • Reston, Virginia, United States Fannie Mae Full time

    Fannie Mae is a leading provider of financial services, and we're seeking a skilled Data Infrastructure Architect to join our team.About the RoleAs a Data Infrastructure Architect, you will play a key role in designing and implementing data infrastructure solutions that enable our business to make informed decisions. This is a critical position that requires...

  • Data Engineer

    19 hours ago


    Reston, Virginia, United States SOSi Full time

    About the Role:SOSi is seeking a Data Engineer to join our analytics team working on an innovative MLOps workload leveraging cutting-edge technologies and supporting a government customer in Huntsville, Alabama.This role will be responsible for delivering automation to key national security missions interacting with petabyte-scale data on supercomputing...


  • Reston, Virginia, United States Starcom consulting limited Full time

    About Starcom Consulting LimitedWe are a leading consulting firm seeking a highly skilled Data Infrastructure Architect to join our team.


  • Reston, Virginia, United States SOS International LLC Full time

    SOS International LLC seeks a highly skilled Data Engineer to join our MLOps Team in Huntsville, Alabama. As a key member of the team, you will be responsible for designing, building, and maintaining efficient data workflows leveraging cutting-edge technologies.The ideal candidate will have significant experience as a Data Engineer or in an advanced...


  • Reston, Virginia, United States Hexaware Technologies Full time

    Hexaware Technologies seeks an Enterprise Data Architect Lead to drive digital transformation and innovation for businesses worldwide.We are a leading global IT Services company, dedicated to providing comprehensive AI empowered services including IT Consulting, Application Development, Infrastructure and Cloud Management, and Business Process services.The...


  • Reston, Virginia, United States ABBTECH Professional Resources Full time

    A highly competitive salary of $240,000 - $264,000 per year is available for this position in the Reston area.Job SummaryWe are seeking an experienced Chief Data Infrastructure Specialist to join our team at ABBTECH Professional Resources. This role involves assisting with the maintenance of existing legacy applications and transitioning to a new system...

  • Data Flow Architect

    5 days ago


    Reston, Virginia, United States DAn Solutions Inc Full time

    Overview:DAn Solutions Inc is seeking a highly skilled Data Flow Architect to join our team. As a key member of our engineering group, you will be responsible for designing, implementing, and maintaining complex data flows using Apache NiFi.About the Job:The successful candidate will work closely with our clients to understand their requirements and deliver...


  • Reston, Virginia, United States BAE Systems USA Full time

    Job OverviewBAE Systems USA is seeking a highly skilled Data Engineer to support our government customer in designing, implementing, and operating data management systems for intelligence needs. As a Data Engineer, you will play a critical role in developing and maintaining large-scale data management systems that store, access, use, integrate, and manage...


  • Reston, Virginia, United States TransUnion Risk and Alternative Data Solutions Full time

    Company OverviewTransUnion Risk and Alternative Data Solutions is a leading provider of data-driven marketing solutions. Our team thrives in a collaborative and innovative environment that encourages creativity and growth.Job SummaryWe are seeking an experienced Senior Industry Executive to join our sales team. This role will be responsible for driving new...


  • Reston, Virginia, United States McAfee Full time

    {"title": "Data Ingestion Engineer", "section": [{"label": "About the Role", "content": "We are seeking an analytical leader to join our Enterprise Data Platform team as the Capability Lead, Data Ingestion, reporting to the Enterprise Data Platform Lead."}, {"label": "Key Responsibilities", "content": "Leading the Enterprise Data Platform Ingestion...


  • Reston, Virginia, United States Fannie Mae Full time

    Job OverviewFannie Mae is a flexible hybrid company, embracing flexibility for our employees to work where they choose. As a Sr. Data Encryption Engineer, you will contribute to developing data infrastructure and pipelines to capture, integrate, organize, and centralize data.The Sr. Data Encryption Engineer role offers the flexibility to make each day your...


  • Reston, Virginia, United States ICF Full time

    We are seeking a skilled Transformation Data Engineer to join our team at ICF, a global advisory and technology services provider. This role will play a pivotal part in shaping the future of data engineering, working closely with cross-functional teams to deliver high-quality solutions that drive informed decision-making.Job Description:In this role, you...


  • Reston, Virginia, United States CoSolutions, Inc. Full time

    Job Description:We are seeking a highly skilled Senior Data Scientist to join our team at CoSolutions, Inc. This is a senior-level position that requires a strong background in data analysis and engineering. The ideal candidate will have experience with big data processing, machine learning, and advanced analytics.The salary for this position is estimated to...


  • Reston, Virginia, United States ASCENDING Full time

    Job OverviewASCENDING, an Advanced AWS partner, specializes in cloud migrations, custom web app development, data analytics/big data, and DevOps. We are seeking a Software Engineer II / AWS Cloud Engineer to join our consulting services group.Key ResponsibilitiesThe ideal candidate will have 3+ to 5 years of experience in software engineering, working with...


  • Reston, Virginia, United States CapTech Consulting Full time

    CapTech Consulting is seeking a seasoned Senior Cloud Data Engineer Leader to join our team. This role offers an exceptional opportunity to drive innovative data solutions and lead cross-functional teams in delivering high-quality, cloud-based data engineering projects.About the RoleAs a Senior Cloud Data Engineer Leader, you will be responsible for...


  • Reston, Virginia, United States nTech Workforce Full time

    About the JobnTech Workforce is seeking a highly skilled Data Dog Implementation Specialist to support our DataDog implementation. The successful candidate will be responsible for managing and maintaining the Data Dog tool on Linux platforms, configuring centralized logging of all logs from different sources, and creating required dashboards with data...


  • Reston, Virginia, United States Fannie Mae Full time

    About the RoleFannie Mae seeks a seasoned Technology Engineer to join our team in Reston, VA. As a Cloud Infrastructure Lead Associate, you will play a pivotal role in designing and developing advanced solutions for information technology (IT) infrastructure environments.ResponsibilitiesMonitor advanced performance and capacity metrics for technology...


  • Reston, Virginia, United States SilverEdge Full time

    About the RoleWe are seeking an experienced Cloud Data Engineer to join our team at SilverEdge Government Solutions. This is a unique opportunity to work with top technology talent and make a meaningful impact on solving some of the world's most challenging problems.Job SummaryThe successful candidate will be responsible for architecting, deploying, and...

  • Senior Data Analyst

    4 days ago


    Reston, Virginia, United States CoSolutions Full time

    We are seeking a highly skilled Senior Data Analyst to join our team at CoSolutions. As a key member of our team, you will be responsible for analyzing large datasets and extracting valuable insights to inform business decisions.The estimated salary range for this position is between $120,000 and $180,000 per year, depending on experience.**Job...