Senior Data Engineer, Commercial Software

Job DescriptionHybrid: This role is categorized as hybrid. This means the successful candidate is expected to report to (insert specific location where employee is expected to report) three times per week, at minimum (or other frequency dictated by the business if more than 3 days)Overview:The Commercial Software division at GM is seeking talented individuals across various software engineering roles to join us. Our mission? Driving forward all digital solutions for commercial fleet owners and drivers, catering to businesses of all sizes, from small and medium businesses to large enterprises.In today's dynamic landscape, the demand for intelligent, safe, and eco-friendly transportation solutions is skyrocketing. GM is at the forefront of this transformative journey, championing connected software-defined vehicles that redefine the driving experience. At the heart of this paradigm shift lies software – enabling seamless communication, bolstering security, facilitating updates, processing data, and ultimately, delivering an unparalleled user experience. These capabilities extend beyond consumer benefits, holding even greater significance for business owners.Recognizing this pivotal moment, GM has recruited a significant number of Silicon Valley engineering, product, and leaders to lead the GM Software & Services organization. As a leading OEM with a vast fleet of vehicles spanning the globe, we're uniquely positioned to leverage our comprehensive control over in-vehicle and cloud software. Our aim? Seamlessly integrating solutions such as fleet management, energy optimization, transportation logistics, safety systems, and more, all geared towards enhancing efficiency and sustainability.To realize our vision, we're actively expanding our software teams in Silicon Valley. We seek individuals who can blend a passion for technology and sustainability with curiosity, rigorous thinking and a strong customer centric approach. This is an exceptional opportunity where you can leverage the scale of GM and make a significant impact and find fulfillment in crafting innovative software solutions. We invite you to join us on this exciting journey toward a better future.Role Summary:As a Senior Data Engineer, you will be responsible for designing, developing, and maintaining efficient data workflows and associated cloud infrastructure that support GM Commercial Software’s core analytic products and services. A key focus of this role will be on developing and optimizing streaming data pipelines, ideally utilizing Spark or similar technologies. You will work closely with our data scientists, DevOps, and software engineers to automate pipelines that process streaming vehicle telemetry data, ensuring real-time data processing and transformation. Additionally, you will transform data within our data lakehouse into deployable data models that power our automated fleet insights, visualizations, and emerging machine learning, optimization, and AI applications. The ideal candidate will have extensive experience in setting up ELT pipelines to handle massive volumes of spatiotemporal data, designing enterprise data warehouse models, implementing robust data quality tests, and leveraging data pipeline-as-code integration systems. Your expertise in streaming data technologies will be crucial for ensuring the smooth and efficient orchestration of the workflows that power our advanced analytics platform.Responsibilities:Design, implement, and maintain streaming data pipelines using spark or analogous technologies.Design, implement, and maintain ELT pipelines in our cloud-native data lakehouse platform.Work with data scientists to turn exploratory analyses into production data transformation workflows within our multihop data lakehouse architecture.Support enterprise-wide use of BI tools and assist in developing internal and external analytics products (e.g., dashboards).Generate and deploy feature engineering pipelines to power internal and customer-facing machine learning-based products.Maintain data quality testing and monitoring tools.Work with information security, DevOps, and DataOps to maintain data classification, auditing, and access and cost control policies.QualificationsRequired:Bachelor's degree (Master’s preferred) or equivalent experience in computer science, data science, engineering, or related quantitative field5+ years of industry experience developing, implementing, and maintaining solutions for Big Data or data warehousing systems3+ years of industry experience working in a cloud environment (Azure preferred)3+ years experience working with SQL query authoring for automated data transformation (familiarity with dbt preferred, but not required).2+ years of experience developing streaming data processing pipelines (use of Spark/pySpark preferred)Basic understanding of machine learning/statistical learning principlesExperience implementing and maintaining data workflow orchestration and integration tools (e.g. Airflow/Astro, Prefect, dbt cloud, etc.)Understanding of and experience with application of data quality tools integrated with CI/CD automation frameworks in functional deployment environments (e.g., Github Actions/Azure DevOps pipelines).Familiarity with data quality testing frameworks (Great Expectations, Deequ)Self-driven with an interest in on-the-job learning.Preferred:Familiarity with enterprise warehouse data modeling techniques (e.g., Kimball)Experience integrating simulation systems with distributed, data-intensive processing or analytics applicationsDesired:Working familiarity with terraformDomain knowledge in automotive systemsEngagement with modern data stack community (open source and commercial)High degree of attention to software craftsmanship and professionalismExperience working with containerization technologies and orchestration platforms (specifically Docker and Kubernetes)Compensation: The compensation information is a good faith estimate only. It is based on what a successful applicant might be paid in accordance with applicable state laws. The compensation may not be representative for positions located outside of New York, Colorado, California, or Washington.The salary range for this role is $152,100.00 - 232,00.00. The actual base salary a successful candidate will be offered within this range will vary based on factors relevant to the position.Bonus Potential: An incentive pay program offers payouts based on company performance, job level, and individual performance.Benefits: GM offers a variety of health and wellbeing benefit programs. Benefit options include medical, dental, vision, Health Savings Account, Flexible Spending Accounts, retirement savings plan, sickness and accident benefits, life insurance, paid vacation & holidays, tuition assistance programs, employee assistance program, GM vehicle discounts and more.This job may be eligible for relocation benefits.About GM:Our vision is a world with Zero Crashes, Zero Emissions and Zero Congestion and we embrace the responsibility to lead the change that will make our world better, safer and more equitable for all.Why Join Us:We aspire to be the most inclusive company in the world. We believe we all must make a choice every day – individually and collectively – to drive meaningful change through our words, our deeds and our culture. Our Work Appropriately philosophy supports our foundation of inclusion and provides employees the flexibility to work where they can have the greatest impact on achieving our goals, dependent on role needs. Every day, we want every employee, no matter their background, ethnicity, preferences, or location, to feel they belong to one General Motors team.Benefits Overview:The goal of the General Motors total rewards program is to support the health and well-being of you and your family. Our comprehensive compensation plan includes, the following benefits, in addition to many others:Paid time off including vacation days, holidays, and parental leave for mothers, fathers and adoptive parents;Healthcare (including a triple tax advantaged health savings account and wellness incentive), dental, vision and life insurance plans to cover you and your family;Company and matching contributions to 401K savings plan to help you save for retirement;Global recognition program for peers and leaders to recognize and be recognized for results and behaviors that reflect our company values;Tuition assistance and student loan refinancing;Discount on GM vehicles for you, your family and friends.Diversity Information:General Motors is committed to being a workplace that is not only free of discrimination, but one that genuinely fosters inclusion and belonging. We strongly believe that workforce diversity creates an environment in which our employees can thrive and develop better products for our customers. We understand and embrace the variety through which people gain experiences whether through professional, personal, educational, or volunteer opportunities. GM is proud to be an equal opportunity employer.We encourage interested candidates to review the key responsibilities and qualifications and apply for any positions that match your skills and capabilities.Equal Employment Opportunity Statements:GM is an equal opportunity employer and complies with all applicable federal, state, and local fair employment practices laws. GM is committed to providing a work environment free from unlawful discrimination and advancing equal employment opportunities for all qualified individuals. As part of this commitment, all practices and decisions relating to terms and conditions of employment, including, but not limited to, recruiting, hiring, training, promotion, discipline, compensation, benefits, and termination of employment are made without regard to an individual's protected characteristics. #J-18808-Ljbffr